Row generator and Column generator

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
asmita
Participant
Posts: 1
Joined: Tue Mar 26, 2013 6:14 am

Row generator and Column generator

Post by asmita »

Hi All,
i am new to this site. Can any one of you suggest me ,what i can do to improve performance tuning in Row generaor, Column generator and Peek stage. Please do guide me.

Thanks,
Asmita
Asmita Singh
- If you are stuck at something .... consider it an opportunity to think a new.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

There is very little that can be done to improve the performance of the row and column generators.

The Peek stage is used for debugging purposes and the only way to make it work faster is to have it log less information, in which case it probably isn't needed and should be removed from the job in question.

Row and Column generators are incredibly fast active stages. I can't think of any stage which can process as much data as these stages can produce.
Last edited by ArndW on Fri Jul 12, 2013 7:37 am, edited 1 time in total.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Start by posting in the correct forum. :wink: I moved your post.

Secondly, provide more details as I don't really see any of those stages requiring any 'performance tuning'. Did you have a specific job or job design in mind when posted this question? And since you started in a forum that doesn't require them, can you let us know what version you are running and your server platform - Windows or UNIX.
-craig

"You can never have too many knives" -- Logan Nine Fingers
rameshrr3
Premium Member
Premium Member
Posts: 609
Joined: Mon May 10, 2004 3:32 am
Location: BRENTWOOD, TN

Post by rameshrr3 »

Try toggling the eexecution mode from Sequential to Parallel and see if you get an improvement in performance. Its wise to post the context in which you are trying this . Other wise our answers are a shot in the dark . If you are on a Grid/MPP, check if any Node Map or node pool constraint is applied to it and try changing the constraint so that it runs on a node local to the conductor.
jwiles
Premium Member
Premium Member
Posts: 1274
Joined: Sun Nov 14, 2004 8:50 pm
Contact:

Post by jwiles »

Column Generator is probably already running in parallel.
Row Generator runs in sequential mode by default. You should understand that by setting it to parallel, you will have more records generated (number of nodes executing * number of records requested). You may not want to do that, or you may want to adjust the number of rows requested to a lower value when switching it to parallel.

Do you have a job in which you--or someone else--believe that these stages ARE performance bottlenecks (and why do you think so?), or perhaps has someone asked this as part of an interview?

Regards,
- james wiles


All generalizations are false, including this one - Mark Twain.
Post Reply