Controller Problem with Multi Instance Jobs

Vishvas · Post by **Vishvas** » Tue Apr 04, 2006 8:35 am

We have 5 projects under one server. There are 7 jobs that are common for all the projects. These are multi instance jobs. They are designed in such a way that one calls the other in sequence. We have around 10 streams (i.e. 10 instances) in each project. So each of those 7 jobs will be triggered for each stream with different instance name. So at a time there may be 50 instances (10 instance/project) of the same job running on one server.

The problem is when we start all the jobs at the same time we are facing controller problem (-14). But if we start only 4 projects (any 4), then there is no problem. I don't know where there is any limitation on the number of jobs that a server can handle. Can anybody help us in solving this issue.

Vishvas

kumar_s · Post by **kumar_s** » Tue Apr 04, 2006 8:43 am

There are limitation for max number of processors involved for each user in Unix. You can speak to you admin to increase the value.

gateleys · Post by **gateleys** » Tue Apr 04, 2006 9:07 am

Whenever I have multiple job instances, I make sure that the number of instances running concurrently does not exceed the number of CPUs in my DataStage server.

gateleys

kumar_s · Post by **kumar_s** » Tue Apr 04, 2006 9:11 am

Try to execute the jobs with two or more different userid parallely. If this goes, you need to alter your default settings.

kumar_s · Post by **kumar_s** » Tue Apr 04, 2006 9:15 am

You administrator may use

Code: Select all

chdev -l sys0 -a maxuproc='nnn'

command to increase the max number of process per id.

Vishvas · Post by **Vishvas** » Tue Apr 04, 2006 10:06 am

Sorry I didn't mention about the users. All the five projects are started with different unix users and each user has the process limit of 8000. So its not a problem with the max no of process/user. Also DS server runs on a 16 CPU HP box.

Is there any limitation that the number of instance of a single job should not exceed the number of CPU's?

Vishvas

kduke · Post by **kduke** » Tue Apr 04, 2006 10:39 am

The problem is not in the number of processes per CPU or user. It is that you need to add delays in between the job starts. We the sever starts jobs it takes up a lot of resources. The server needs more time to recover. Add a sleep between starting these jobs. You can easily overwhelm a computer with type of processing.