Inermediate link in join stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
memrinal
Participant
Posts: 74
Joined: Wed Nov 24, 2004 9:13 pm

Inermediate link in join stage

Post by memrinal »

I need to join three files together to get the desired file. currently i am doing this by doing a join twice like

File B File C
| |
| |
File A ---> Join-----> Join ---->Output File

both the joins are Left Outer. I need some fields from each file.
While going through the documentation of DataStage PX I read about the Intermediate Links and tried to use them. but the problem that i faced was that only those fields from input can be used to derive output file's fields, which are keys.
However this is not the case in my job. i have to derive several output file fields from input file fields which are not part of the key.
How an i use intermediate links in join stage. Is there any way i can use a single join stage to do this or i will have to stick with multiple joins.
Thanks in Advance

P.S. Data is very huge and Lookup stage cant be used.
Someswar Barkataki
Participant
Posts: 6
Joined: Tue Jul 13, 2004 7:40 am

Re: Inermediate link in join stage

Post by Someswar Barkataki »

Hi
I have used 14 files to join using intermediate Links and it works fine. I too used a left outer join and derived not only the key columns but other columns from each of the input files as well. However, my left most file is the master file that contains two key columns and all other files contain these two common key fields .

Hope that replies your question whether you can use a single join or not.
By the way I donot see any disadvantage using cascading join too since you need to join three files only.

Thanks,
Somesh
olgc
Participant
Posts: 145
Joined: Tue Nov 18, 2003 9:00 am

Re: Inermediate link in join stage

Post by olgc »

How these 14 files are brought to the join stage: after two links are added as left and right links, more links are added to the stage and will be treated as Intermediate links without any issue?

Thanks,
olgc
Participant
Posts: 145
Joined: Tue Nov 18, 2003 9:00 am

Re: Inermediate link in join stage

Post by olgc »

I Guess you can use merge stage also, don't you?
olgc
Participant
Posts: 145
Joined: Tue Nov 18, 2003 9:00 am

Re: Inermediate link in join stage

Post by olgc »

Can any one tell what the intermediate links are different with left and right links? For the types of join: inner, left / right outer and full, are they not applied to these intermediate links?

thanks,
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Do you have an actual issue you need help with or did you just decide to indulge in a little thread necromancy today?
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

DataStage does it the same way database servers do it: pairwise from the left. So, if you have three inputs for example, the left and intermediate inputs are joined then the result of that is joined with the right input.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply