Scratch Space and Resource Disk in Datastage

RohanSharma · Posted: Tue Apr 03, 2007 5:11 am

Hello there...
Well I have this job running at the client end and it processes real heavy gusts of data...
Now some time back the jobs fails saying
"APT_CombinedOperatorController,0: write failed: Output file full, and no more output files
write failed: Output file full, and no more output files"

Now This message says that Output file is full but this does not indicate that whether it is Datasets or Some Temporary Files being full.

Doubt arises when we see the error message in the next run

APT_CombinedOperatorController,0: Fatal Error: Tsort merger aborting: Scratch space full

(there is some sorting logic in the job)

1)Now I need to know what is the Scratch Space
2) Also when I see the Environment settings of my project I see that
something called as resource disk also being used.

Can you tell me what exactly are Scratch Disk and the Resource Disk.

Also there is an all together different temporary directory being used in Datastage in the variable TMPDIR Shocked

This TMPDIR is not specific to our project it is standard to Datastage.
and can bee seen in the administrator's general Settings Tag

So if this TMPDIR is there then what is the need of Scratch Space to be used
[If my assumption that scratch space is a temporary space used while executing a job is correct ] Cool

ray.wurlod · Posted: Tue Apr 03, 2007 6:53 am

TMPDIR is used after scratch space fills.

RohanSharma · Posted: Tue Apr 03, 2007 10:23 pm

Thanks Ray
For the information,

But then what role DISK SPACE plays??

Also I tried to run the job again...
The job is failing again saying that
"APT_CombinedOperatorController,0: write failed: Output file full, and no more output files"

But when I check the corresponding UNIX directory I see only that 15% of the total disk space in TMPDIR location is used.
So Is there any Settings in datastage which limits the usage of the TMPDIR space.

Please let me know it is very important for me to know this to get out of the quagmire I'm currently trapped in
Razz

ray.wurlod · Posted: Tue Apr 03, 2007 11:14 pm

First off, disable operator combination so you can work out which operator is having the problem. Resource disk is used only for Data Sets and File Sets. If it turns out that the stage having th ...

What's this?

RohanSharma · Posted: Fri Apr 06, 2007 2:14 am

ray.wurlod wrote:

First off, disable operator combination so you can work out which operator is having the problem.

Resource disk is used only for Data Sets and File Sets. If it turns out that the stage having the ...

Ray can you please tell me how to Disable Oerator Combination and what exactly it is ??

ray.wurlod · Posted: Fri Apr 06, 2007 7:05 am

Operator combination is an optimization whereby multiple adjacent parallel operators are caused to execute in the same process, provided certain conditions are met. Disable it either using the AP ...

What's this?