Page 1 of 1

Parallel Config file for ISD job !

Posted: Sun Dec 02, 2012 11:18 pm
by allavivek
Hi All,

For the ISD jobs can we can config file more than one node ?

The way i see is, will the ISD server has the capability to detect multiple nodes in config file or not ? or it just submits to conductor node and the processing inside takes in parallel.

Thank you.

Posted: Sun Dec 02, 2012 11:27 pm
by ray.wurlod
When the job starts, whether from ISD or not, the conductor process constructs the score out of the generated OSH and the parallel execution configuration file. So, yes, jobs executed by ISD can run parallel.

But you must ask yourself whether it's worth it. What's the usual payload of a web client's request? The smallest amount of data moved by a parallel job is 32KB. So anything less than 32KB forced to run in parallel will simply waste all nodes but one.

Posted: Sun Dec 02, 2012 11:34 pm
by allavivek
Ray ,

I got your point on config file , but i didnt get when you said ,
"smallest amount of data moved by parallel job is 32 kb".

Thank you.

Posted: Mon Dec 03, 2012 1:05 am
by ray.wurlod
32KB is the block size used in a Data Set, whether virtual or physical. This is the smallest amount of data moved within a DataStage parallel job. Typically it's more - remember that the default buffer sizes are 3MB.

Posted: Mon Dec 03, 2012 6:00 am
by eostic
99.999 % of all cases with real time jobs, you are going to want a single node config...anything else is total overkill and complicates your support and debugging.

use multiple instances (set inside of ISD) if you want to increase the traffic.

Ernie

Posted: Mon Dec 03, 2012 7:58 am
by eostic
....that being said, ISD knows nothing about config files --- that is all abstracted from ISD....it simply starts the Job, and the ISD stages are run at the conductor node...beyond that it has no visibility. ...and as noted before, there is little chance that you need anything more than single node.

Ernie