Datastage sever job - Memory usage issue

ktv · Post by **ktv** » Mon Jul 28, 2008 3:03 am

I'm facing a memory problem in datastage ..

A server job(call it the control job) is used to call 17 jobs, one by one, through routines in the transformer stage. This server job is said to take up half the datastage server memory through out the whole process ie., until all the 17 jobs are run.

I need to know
1. How to track the usage of memory ?
2. The reason for the datastage server memory usage of the control job. Is it because the other jobs are executed in the same memory space ?? Is there any other reason ?

Help solicited ...

ArndW · Post by **ArndW** » Mon Jul 28, 2008 3:11 am

Memory usage can be monitored with system tools. The normal method of starting many other jobs is thorugh use of a job sequence. If you call the jobs from a transform they will most likely all get started at roughly the same time unless you build in a "-wait". How much memory is half of your server's memory?

ray.wurlod · Post by **ray.wurlod** » Mon Jul 28, 2008 3:01 pm

DataStage jobs allocate to themselves only the memory they require. Using one job to invoke another (presumably through UtilityRunJob() or a similar function) starts a child process, which should have an independent address space.

DataStage does make use of shared memory for lock tables and various other structures; if there were nothing else happening then you may see this charged against your first-started job, since this was the first process to attach to the shared memory. Although I would really expect to see that the DataStage resource service be charged with that memory on Windows platforms. Are you sure that you've identified the correct process (and, if so, how)?

ktv · Post by **ktv** » Tue Jul 29, 2008 11:02 pm

ArndW wrote:Memory usage can be monitored with system tools. The normal method of starting many other jobs is thorugh use of a job sequence. If you call the jobs from a transform they will most likely all get started at roughly the same time unless you build in a "-wait". How much memory is half of your server's memory?

Thanks for the reply.

The jobs are being called in a routine by passing the job name as the parameter and using DSRunJob. So for each job, the routine is invoked. DSWaitForJob is used after DSRunJob to wait for the job to complete.

As for how much is half the servers memory, i'm not sure. I will post it as soon as i get the info.