Memory space for the stages

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
skp
Premium Member
Premium Member
Posts: 135
Joined: Wed Dec 26, 2007 1:56 am
Location: India

Memory space for the stages

Post by skp »

Hi,

Which stage among the four stages below consumes less memory at the time of joining two inputs data --- Merge/Funnel/Lookup/Transformerstage? Could some one explain with little bit explanation please?
priyadarshikunal
Premium Member
Premium Member
Posts: 1735
Joined: Thu Mar 01, 2007 5:44 am
Location: Troy, MI

Post by priyadarshikunal »

funnel and transformer are not for joining data. Unless you taking server jobs as well in to the consideration. What you exactly want to know for these 4 stages.

Out of curiosity, is it an interview question?
Priyadarshi Kunal

Genius may have its limitations, but stupidity is not thus handicapped. :wink:
skp
Premium Member
Premium Member
Posts: 135
Joined: Wed Dec 26, 2007 1:56 am
Location: India

Post by skp »

THanks Priya.

Sorry the four stages are Join/Funnel/Lookup/Merge stage. Yes it is interview question but it does not matter i believe.
priyadarshikunal
Premium Member
Premium Member
Posts: 1735
Joined: Thu Mar 01, 2007 5:44 am
Location: Troy, MI

Post by priyadarshikunal »

First of all Funnel does not perform join operation, it is similar to "Union all" in SQL. It does matter if its a interview question as we here in dsxchange strongly believe that the interview questions should be answered based on ones experience rather than re-iterating someone else's answer. But for your truthfulness below are the pointers.

The other 3 Join/Lookup/Merge stages can perform tasks of SQL joins. Please read parallel job developer guide where they have sections like Join Versus Lookup etc which should give a proper idea on their functionality and Memory uses.

To start with the lookup uses RAM to store data coming from reference link while others use disk to store the same.
Priyadarshi Kunal

Genius may have its limitations, but stupidity is not thus handicapped. :wink:
Post Reply