DSXchange

Performance will decrease? So? Isn't the goal to get a working solution? I've already posted how you can get your desired output, 1 column or 50. And I really don't see the addition of a number of columns as any kind of performance killer. And please stop repeating your example. Repetition != helpin...

<click>

Awesome. Another "example" with made up data that doesn't match what really needs to be done. Would it have killed you to have mentioned that in the original post? And repeating everything with just one column as the example doesn't help nor change anything, my answer is still my answer. :...

New problem = new post.

Please take the text that I italicized in your post and start a new post with it. Minus the italics of course.

That dummy column buys you nothing. Instead, hardcode the three column names and then pivot. Meaning, turn this input: EMPNO|ENAME|SALARY 101|AAA|10000 102|BBB|20000 103|CCC|30000 Into this: EMPNO|ENAME|SALARY|EMPNO_NM|ENAME_NM|SALARY_NM 101|AAA|10000|EMPNO|ENAME|SALARY 102|BBB|20000|EMPNO|ENAME|SAL...

I'm not sure you should be worrying about that. I very much doubt that your hardware will support 900 concurrent Parallel jobs, let alone any sort of a higher number.

So, which part isn't working - the sort, the transformer or the remove duplicates? Seems to me you'll need to hardcode all three values, pivot then the concatenation and finally remove duplicates.

A quick search turned this up, see if it helps.

http://www-01.ibm.com/support/docview.w ... wg21593492

That's not the way a direct path load works. It bypasses the whole transaction logging mechanism which is where you get your speed - no transaction logging means no way to undo anything. When the load aborts, you get whatever you've loaded up to that point and you get whatever state the table is in ...

Technically, you are able to source it, it just doesn't like what you've told it to do. That error needs to go to your SA and you need to figure out what specific line that is coming from.

DataStage is just another client so it obeys all access privileges the same way any other tool like sqlplus or Toad would. So I wouldn't look at DataStage directly but rather the way the grants are setup in the database and how you are referencing the table in the stage - are you including the owner...

This seems to me like something you are better off doing in C++, say in a BuildOp for example.

Or engage IBM, as Eric suggested in the post I linked to.

As noted, doesn't really matter as nothing about this has changed.

Hadn't gotten to that part yet. The rub here is the comparison to reference data and an open number of occurrences too, I'd wager. Hadn't had much of a chance to ponder today how you would work that... perhaps a lookup and transformer looping to check each occurance? And then some sort of aggregatio...