Server jobs vs parallel jobs

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
chandra4u
Participant
Posts: 13
Joined: Wed May 17, 2006 5:45 am
Location: Bangalore

Server jobs vs parallel jobs

Post by chandra4u »

One of our client wants to see the performance of server jobs as well as parallel jobs. They want us to run both type of jobs and show them the performance. Could you please help me how can I do that. I would like to use text file as input, do some transformation and then load them into Oracle. Please suggest me what are different stages I can use in server and parallel jobs, so that I can show them that the parallel jobs are faster than the server jobs.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Good luck. There are many situations where server jobs are faster (finish sooner) than parallel jobs implementing the same logic.

Parallel jobs will typically win if the volume of data to be processed is huge and can be processed truly in parallel (for example using Data Sets rather than Sequential Files).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chandra4u
Participant
Posts: 13
Joined: Wed May 17, 2006 5:45 am
Location: Bangalore

Post by chandra4u »

Any other suggestions !!!
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Where is your expertise - Server or Parallel? What part of this do you need help with?
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

chandra4u wrote:Any other suggestions !!!
I find that offensive.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

I did some comparison in a post DataStage server v enterprise: some performance stats that showed sort and aggregation was way ahead in parallel. My post DataStage Tip: Extracting database data 250% faster shows how good the parallel Enterprise stages are and I reviewed a recent French benchmark that compare server to parallel in ETL Benchmark Favours DataStage and Talend.
reachmexyz
Premium Member
Premium Member
Posts: 296
Joined: Sun Nov 16, 2008 7:41 pm

Post by reachmexyz »

chandra4u wrote:Any other suggestions !!!
you can make use of aggregator in Parallel.
Server: pull the records from flat file to aggregator(server) and load to oracle.
Do Parallel with the same stages Aggregator (parallel) and that will run faster. Bewate data should be of huge volume.

Another is try to run two jobs in server in which output of first is fed to input of second job. (in server use sequential file as output and input of 2nd and 1st job whereas in parallel use datasets. ) again the volume of the data should be high.

Whatever you do data volume should be high to show differences between server and parallel.
Post Reply