which stage to use

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
prasad111
Premium Member
Premium Member
Posts: 173
Joined: Fri May 19, 2006 10:53 am

which stage to use

Post by prasad111 »

Hi All,
This is one of the requirement, need help

Code: Select all

Input data

col1|col2

M|abc
C|def
U|hji
M|ad
U|ryer
M|fntry
C|rtf
U|gsfdg


Ouput 
col1|col2|col3 |col4|col5|col6

M|abc|C|def|U|hji
M|ad|	|	|U|ryer
M|fntry|C|rtf|U|gsfdg
whenever an M starts the data has to be entered in a new row.

Regards
Prasad
meena
Participant
Posts: 430
Joined: Tue Sep 13, 2005 12:17 pm

Re: which stage to use

Post by meena »

Hi Prasad,
If col1 values are consistent( 'M','C' & 'U').

Use the transformer stage to filter the col1 using constarints and to get three output streams.Define a common dummy column in all three outstreams with the same constant value.Next use the merge stage to merge all the three streams.
Or you can do the same using filter,modify and merge stages.

Code: Select all

Input data

col1|col2

M|abc
C|def
U|hji
M|ad
U|ryer
M|fntry
C|rtf
U|gsfdg


Ouput 
col1|col2|col3 |col4|col5|col6

M|abc|C|def|U|hji
M|ad|	|	|U|ryer
M|fntry|C|rtf|U|gsfdg
[/quote]
Aruna Evoori
prasad111
Premium Member
Premium Member
Posts: 173
Joined: Fri May 19, 2006 10:53 am

Re: which stage to use

Post by prasad111 »

meena wrote:Hi Prasad,
If col1 values are consistent( 'M','C' & 'U').

Use the transformer stage to filter the col1 using constarints and to get three output streams.Define a common dummy column in all three outstreams with the same constant value.Next use the merge stage to merge all the three streams.
Or you can do the same using filter,modify and merge stages.

Code: Select all

Input data

col1|col2

M|abc
C|def
U|hji
M|ad
U|ryer
M|fntry
C|rtf
U|gsfdg


Ouput 
col1|col2|col3 |col4|col5|col6

M|abc|C|def|U|hji
M|ad|	|	|U|ryer
M|fntry|C|rtf|U|gsfdg
[/quote]

As you can see in the sample data shown M, C and U are not consistent, there might be
M and C, or
M and U or
M, C and U present.
As they are not consistent I guess we cannot use merge stage to merge the data.
vijayrc
Participant
Posts: 197
Joined: Sun Apr 02, 2006 10:31 am
Location: NJ

Re: which stage to use

Post by vijayrc »

prasad111 wrote:
meena wrote:Hi Prasad,
If col1 values are consistent( 'M','C' & 'U').

Use the transformer stage to filter the col1 using constarints and to get three output streams.Define a common dummy column in all three outstreams with the same constant value.Next use the merge stage to merge all the three streams.
Or you can do the same using filter,modify and merge stages.

Code: Select all

Input data

col1|col2

M|abc
C|def
U|hji
M|ad
U|ryer
M|fntry
C|rtf
U|gsfdg


Ouput 
col1|col2|col3 |col4|col5|col6

M|abc|C|def|U|hji
M|ad|	|	|U|ryer
M|fntry|C|rtf|U|gsfdg
As you can see in the sample data shown M, C and U are not consistent, there might be
M and C, or
M and U or
M, C and U present.
As they are not consistent I guess we cannot use merge stage to merge the data.[/quote]

A transformer will do the trick...You know at the sight of 'M' a new row is to be written...If you encounter a C or an U after a M, you can hold them in stage variable and write this row when you encounter the next 'M'
Helps ?? or confusing ??
vijayrc
Participant
Posts: 197
Joined: Sun Apr 02, 2006 10:31 am
Location: NJ

Re: which stage to use

Post by vijayrc »

prasad111 wrote:
meena wrote:Hi Prasad,
If col1 values are consistent( 'M','C' & 'U').

Use the transformer stage to filter the col1 using constarints and to get three output streams.Define a common dummy column in all three outstreams with the same constant value.Next use the merge stage to merge all the three streams.
Or you can do the same using filter,modify and merge stages.

Code: Select all

Input data

col1|col2

M|abc
C|def
U|hji
M|ad
U|ryer
M|fntry
C|rtf
U|gsfdg


Ouput 
col1|col2|col3 |col4|col5|col6

M|abc|C|def|U|hji
M|ad|	|	|U|ryer
M|fntry|C|rtf|U|gsfdg
As you can see in the sample data shown M, C and U are not consistent, there might be
M and C, or
M and U or
M, C and U present.
As they are not consistent I guess we cannot use merge stage to merge the data.[/quote]

A transformer will do the trick...You know at the sight of 'M' a new row is to be written...If you encounter a C or an U after a M, you can hold them in stage variable and write this row when you encounter the next 'M'
Helps ?? or confusing ??
us1aslam1us
Charter Member
Charter Member
Posts: 822
Joined: Sat Sep 17, 2005 5:25 pm
Location: USA

Post by us1aslam1us »

Follow this link and see whether that can move you in a proper direction:

viewtopic.php?t=99149&postdays=0&postor ... ge&start=0

sAM
I haven't failed, I've found 10,000 ways that don't work.
Thomas Alva Edison(1847-1931)
prasad111
Premium Member
Premium Member
Posts: 173
Joined: Fri May 19, 2006 10:53 am

Post by prasad111 »

vijayrc and us1aslam1us reply was very helpful.

Thanks a lot.............
Post Reply