Concurrent Jobs Run

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
pradkumar
Charter Member
Charter Member
Posts: 393
Joined: Wed Oct 18, 2006 1:09 pm

Concurrent Jobs Run

Post by pradkumar »

Hi Everyone..

I have 40 sequence jobs that has to run concurrently .. when i try to run all the jobs,some of the jobs are aborting ..But when i run them individually all are finising successfully. I have searched the forum and found the information about the memory and cpu utilization . By using the top command i found that the cpu utilization is reaching 100 even when i ran 10 jobs concurrently. From the ds forum search i got the reason might be hardware is underpowered for the load being put on it or perhaps because some tunables are set too low.

My Dev server configuration is V490 2 x 2 USIV+ 1.5 proc 16 Gb RAM Sol9

So i decided not run all the jobs concurrently , either we increase the resources or we run a few of the jobs in parallel, and sequentially run another set of a few concurrent running jobs. The effect of the 2nd option will be to increase the elapsed time .

Anyone can suggest how to achieve this situation and what settings on the server side has to be tuned .

Thanks in Advance
kandyshandy
Participant
Posts: 597
Joined: Fri Apr 29, 2005 6:19 am
Location: Singapore

Post by kandyshandy »

You are on the right track. Reduce the number of concurrent running jobs to fit your resources.
Kandy
_________________
Try and Try again…You will succeed atlast!!
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Or, of course, increase the available resources (buy hardware). The experience with version 8 seems to be more and more that the development machine needs at least as much grunt as the production machine, particularly when there are multiple developers.

It's a simple supply and demand situation.

Use the resource estimation tool to get the figures you need to max out the machine without overloading it.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply