DSXchange

Why not just have your Mainframe folks stop providing you with an invalid file? Meaning, correct the delimiters so you don't have to screw around like that.

It's documented and the name is closer to Key Change Column rather than cluster anything, from what I recall. It will contain a 1 for the first record in the 'key group' or a 0 otherwise.

Or add checkpoints so it can restart 'from the middle' if it failed at that point. Or copy the Sequence job and remove the parts you don't want to run.

The records need to be deleted and thats the requirement and we don't want to have a table exclusively to remove the dups. The technique I mentioned is pretty basic stuff and it is used when the number of records to be deleted is greater than the number of records to be saved. Is that the case here...

True.

... or add the APT variable that shuts down sneaky tsort insertions, yes?

... and when I said you could run it, I in no way meant to imply it was the right thing to do or that it stood any chance of working correctly. Just, it's a script and scripts can be run "from the UNIX server".

Again, as noted - use dsjob.

OK, first off - my bad. For some reason I missed the whole "in a routine" part even though it was staring me in the face and was thinking of the Execute Command stage after seeing the sqlplus command. So, as noted, you have to remove them from the DSExecute result but yes you can do it ins...

But that really isn't involved with your problem. Still seems to me you need to find where your 'shared library' path environment variable is getting reset.

Your parallel processes are deadlocking. This is where you would need to override the auto partitioning and control it so it doesn't deadlock. Probably hash partitioned over the keys involved would help, I'd have a chat with your DBA and confirm.

Where did you create this sample file - on the Mainframe? If so, how exactly did you transfer it to the DataStage server?

Yes, with a specific Service Pack level from what I recall. Should all be in the docs somewhere.

All that means is the date is in May. Of any year. That really work for you?

There's no "filter" stage in Server. What DSGuru is hinting at is how would you do the comparison without using "between"? It's certainly not required, just... easier.

I would think that this would be more correcter: