Transformer erasing predecessor sort information

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
zulfi123786
Premium Member
Premium Member
Posts: 730
Joined: Tue Nov 04, 2008 10:14 am
Location: Bangalore

Transformer erasing predecessor sort information

Post by zulfi123786 »

Hi,

I am reading a sorted sequential file and have a couple of transformers which are using keychange detection functions. I have added a dummy sort over sort keys to mark that the data is sorted but it looks like the transfomer is erasing this information on the output link.

This makes sense as a column can be modified inside transformer, is there any way to avoid inserting dummy sort stages following every transformer ?

Thanks
- Zulfi
priyadarshikunal
Premium Member
Premium Member
Posts: 1735
Joined: Thu Mar 01, 2007 5:44 am
Location: Troy, MI

Post by priyadarshikunal »

It should not get lost unless you have some operation on keys, are you re-partitioning etc in transformer, is the partitioning set to propagate etc. Check the score to see if its partitioning or doing something unnoticed from job design.
Priyadarshi Kunal

Genius may have its limitations, but stupidity is not thus handicapped. :wink:
zulfi123786
Premium Member
Premium Member
Posts: 730
Joined: Tue Nov 04, 2008 10:14 am
Location: Bangalore

Post by zulfi123786 »

Nothing done on the keys, Dont think the score would report information of sort order preservance across operators.
- Zulfi
priyadarshikunal
Premium Member
Premium Member
Posts: 1735
Joined: Thu Mar 01, 2007 5:44 am
Location: Troy, MI

Post by priyadarshikunal »

score will report re-partition if happening before any transformer as you say. As it will partition data after reading from seq file anyways. So you have to have one sort operator just to mark that data is already sorted.
Priyadarshi Kunal

Genius may have its limitations, but stupidity is not thus handicapped. :wink:
Post Reply