DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
RohanSharma
Participant



Joined: 28 Jan 2007
Posts: 20
Location: Gurgaon
Points: 279

Post Posted: Tue Apr 03, 2007 5:11 am Reply with quote    Back to top    

DataStage® Release: 7x
Job Type: Parallel
OS: Unix
Additional info: Scratch Space Full
Hello there...
Well I have this job running at the client end and it processes real heavy gusts of data...
Now some time back the jobs fails saying
"APT_CombinedOperatorController,0: write failed: Output file full, and no more output files
write failed: Output file full, and no more output files"


Now This message says that Output file is full but this does not indicate that whether it is Datasets or Some Temporary Files being full.

Doubt arises when we see the error message in the next run

APT_CombinedOperatorController,0: Fatal Error: Tsort merger aborting: Scratch space full

(there is some sorting logic in the job)

1)Now I need to know what is the Scratch Space
2) Also when I see the Environment settings of my project I see that
something called as resource disk also being used.

Can you tell me what exactly are Scratch Disk and the Resource Disk.


Also there is an all together different temporary directory being used in Datastage in the variable TMPDIR Shocked
This TMPDIR is not specific to our project it is standard to Datastage.
and can bee seen in the administrator's general Settings Tag

So if this TMPDIR is there then what is the need of Scratch Space to be used
[If my assumption that scratch space is a temporary space used while executing a job is correct ] Cool
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 51699
Location: Canberra, Australia
Points: 280769

Post Posted: Tue Apr 03, 2007 6:53 am Reply with quote    Back to top    

TMPDIR is used after scratch space fills.

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong
currently hiring: Sydney and Melbourne
Rate this response:  
Not yet rated
RohanSharma
Participant



Joined: 28 Jan 2007
Posts: 20
Location: Gurgaon
Points: 279

Post Posted: Tue Apr 03, 2007 10:23 pm Reply with quote    Back to top    

Thanks Ray
For the information,

But then what role DISK SPACE plays??

Also I tried to run the job again...
The job is failing again saying that
"APT_CombinedOperatorController,0: write failed: Output file full, and no more output files"

But when I check the corresponding UNIX directory I see only that 15% of the total disk space in TMPDIR location is used.
So Is there any Settings in datastage which limits the usage of the TMPDIR space.

Please let me know it is very important for me to know this to get out of the quagmire I'm currently trapped in
Razz
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 51699
Location: Canberra, Australia
Points: 280769

Post Posted: Tue Apr 03, 2007 11:14 pm Reply with quote    Back to top    

First off, disable operator combination so you can work out which operator is having the problem. Resource disk is used only for Data Sets and File Sets. If it turns out that the stage having th ...

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong
currently hiring: Sydney and Melbourne
Rate this response:  
Not yet rated
RohanSharma
Participant



Joined: 28 Jan 2007
Posts: 20
Location: Gurgaon
Points: 279

Post Posted: Fri Apr 06, 2007 2:14 am Reply with quote    Back to top    

ray.wurlod wrote:
First off, disable operator combination so you can work out which operator is having the problem.

Resource disk is used only for Data Sets and File Sets. If it turns out that the stage having the ...




Ray can you please tell me how to Disable Oerator Combination and what exactly it is ??
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 51699
Location: Canberra, Australia
Points: 280769

Post Posted: Fri Apr 06, 2007 7:05 am Reply with quote    Back to top    

Operator combination is an optimization whereby multiple adjacent parallel operators are caused to execute in the same process, provided certain conditions are met. Disable it either using the AP ...

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong
currently hiring: Sydney and Melbourne
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours