Performance between stages

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
abc123
Premium Member
Premium Member
Posts: 605
Joined: Fri Aug 25, 2006 8:24 am

Performance between stages

Post by abc123 »

Would a sort/filter combination yield better performance than a transformer just by itself?
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

I can see how a Transform stage can act like a filter, but the sort portion of the question is unclear to me as a transform stage doesn't have anything like that functionality. Any job without a sort is going to be faster than one with.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

The answer for Filter vs Transformer (no sort in either case) will depend on the version of DataStage you are running. In 7.5.1 and later - all else being equal, that is you are not having the Transformer do any other processing - the difference will be negligible.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
abc123
Premium Member
Premium Member
Posts: 605
Joined: Fri Aug 25, 2006 8:24 am

Post by abc123 »

Thanks Ray and Arndw. I was looking at a post where the poster was getting an error with just filter stage. However, when he used a sort first and then did filter, it worked. That's why I was wondering whether filter/sort will be better than a parallel transformer on 7.5.1A.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

"Better" in what way? Probably no significant difference.

The Filter stage does not require sorted input - without knowing precisely which post you are referring to I can not comment on that "solution".
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply