Server jobs vs parallel jobs
Moderators: chulett, rschirm, roy
Server jobs vs parallel jobs
One of our client wants to see the performance of server jobs as well as parallel jobs. They want us to run both type of jobs and show them the performance. Could you please help me how can I do that. I would like to use text file as input, do some transformation and then load them into Oracle. Please suggest me what are different stages I can use in server and parallel jobs, so that I can show them that the parallel jobs are faster than the server jobs.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Good luck. There are many situations where server jobs are faster (finish sooner) than parallel jobs implementing the same logic.
Parallel jobs will typically win if the volume of data to be processed is huge and can be processed truly in parallel (for example using Data Sets rather than Sequential Files).
Parallel jobs will typically win if the volume of data to be processed is huge and can be processed truly in parallel (for example using Data Sets rather than Sequential Files).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
I did some comparison in a post DataStage server v enterprise: some performance stats that showed sort and aggregation was way ahead in parallel. My post DataStage Tip: Extracting database data 250% faster shows how good the parallel Enterprise stages are and I reviewed a recent French benchmark that compare server to parallel in ETL Benchmark Favours DataStage and Talend.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
-
- Premium Member
- Posts: 296
- Joined: Sun Nov 16, 2008 7:41 pm
you can make use of aggregator in Parallel.chandra4u wrote:Any other suggestions !!!
Server: pull the records from flat file to aggregator(server) and load to oracle.
Do Parallel with the same stages Aggregator (parallel) and that will run faster. Bewate data should be of huge volume.
Another is try to run two jobs in server in which output of first is fed to input of second job. (in server use sequential file as output and input of 2nd and 1st job whereas in parallel use datasets. ) again the volume of the data should be high.
Whatever you do data volume should be high to show differences between server and parallel.