Multiple Instance Server Jobs / Load Balancing on Solaris

[StuartLeaf]

Hello all,

I believe I am experiencing an issue on the above topic and would like some advice. I have a simple DS Server Job ( Load OCI9 straight to Sequential File ) compiled in multiple instance mode. I have a Job Control job which wraps around it, and triggers a parameterised number of instances of the load job as required. Our development server is a 2 CPU machine.

The issue I have is this: when I trigger multiple instances of my job and observe the created processes via prstat at the OS level, it appears that the processes all compete with each other on the same CPU rather than spreading the load / running more than one instance in parallel.

The output from prstat shows that (with an example of 4 instances) one of the job processes will have a status of cpu0 or cpu2, and all the rest have a status of run. I have never seen more than one job instance process be allocated a CPU at a time in my tests; as a result, the total throughput of several job instances has never exceeded the performance of a single thread doing the same thing.

The purpose of my job is to allow partitioning of my source data for processing in many parallel threads. This behaviour is stopping me from yielding any performance benefits!

As a result I have questions...

1) Can anything be done at the time of launching instances of a job to influence how they are spread over CPUs? I can't think of anything!

2) Is it possible that some configuration of Solaris could change my results?

3) Is my understanding of the intended usage of Multiple Instance jobs awry?

4) Am I misinterpreting the output of prstat?

Comments are extremely welcome!!!

Regards,

Stuart.

[ogmios]

Hello all,
1) Can anything be done at the time of launching instances of a job to influence how they are spread over CPUs? I can't think of anything!

2) Is it possible that some configuration of Solaris could change my results?

3) Is my understanding of the intended usage of Multiple Instance jobs awry?

4) Am I misinterpreting the output of prstat?

Comments are extremely welcome!!!

Regards,

Stuart.

1) OS decides, you can't do anything about it. If the system is very loaded it may well be that what you see is normal.

2) A kind of resource scheduler could have been implemented to split resources evenly, check with your sysadmins.

3) I don't use too many multiple instance as they experience problems sometimes, but the behaviour should work for what you intend to do.

4) Use "top", in there you can see the cpu's which are currently executing which jobs. If the corresponds to prstat it's ok.

Ogmios

What do you plan to do with a little 2 CPU machine , the last Solaris server for DataStage I worked on had 32 CPU's

[mhester]

I don't know about your system, but I believe there are configurations that allocate processors for specific tasks. i.e., i/o, management, processing etc... and this may be what you are seeing. I also think the OS has some built-in smarts since it would not be too wise to saturate both cpu's, thereby degrading performance for all.

I have experienced on a Sun 4 cpu box where multi-instance jobs advantage of 2 processors and the other two are being used for other things. I don't believe I've ever seen all cpu's processing all jobs. I believe they swap around, but that would still balance the load.

I don't know if there is a way to "force" the issue - I really think this is left up to the OS and depends on many factors like what application/database software is installed and running.

Regards,

Michael Hester