Parallel Config file for ISD job !

Dedicated to DataStage and DataStage TX editions featuring IBM<sup>®</sup> Service-Oriented Architectures.

Moderators: chulett, rschirm

Post Reply
allavivek
Premium Member
Premium Member
Posts: 211
Joined: Sat May 01, 2010 5:07 pm

Parallel Config file for ISD job !

Post by allavivek »

Hi All,

For the ISD jobs can we can config file more than one node ?

The way i see is, will the ISD server has the capability to detect multiple nodes in config file or not ? or it just submits to conductor node and the processing inside takes in parallel.

Thank you.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

When the job starts, whether from ISD or not, the conductor process constructs the score out of the generated OSH and the parallel execution configuration file. So, yes, jobs executed by ISD can run parallel.

But you must ask yourself whether it's worth it. What's the usual payload of a web client's request? The smallest amount of data moved by a parallel job is 32KB. So anything less than 32KB forced to run in parallel will simply waste all nodes but one.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
allavivek
Premium Member
Premium Member
Posts: 211
Joined: Sat May 01, 2010 5:07 pm

Post by allavivek »

Ray ,

I got your point on config file , but i didnt get when you said ,
"smallest amount of data moved by parallel job is 32 kb".

Thank you.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

32KB is the block size used in a Data Set, whether virtual or physical. This is the smallest amount of data moved within a DataStage parallel job. Typically it's more - remember that the default buffer sizes are 3MB.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

99.999 % of all cases with real time jobs, you are going to want a single node config...anything else is total overkill and complicates your support and debugging.

use multiple instances (set inside of ISD) if you want to increase the traffic.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

....that being said, ISD knows nothing about config files --- that is all abstracted from ISD....it simply starts the Job, and the ISD stages are run at the conductor node...beyond that it has no visibility. ...and as noted before, there is little chance that you need anything more than single node.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply