Newbie question: resource disk and scratch disk management
Posted: Wed Sep 01, 2004 1:36 pm
Hi,
I'm just starting to work with DataStage PX (as a sys admin, not as a developer) and I haven't really seen the product working so far.
I have to answer some questionsand though I went through different documentations, some points remain obscure.
When configuring the PX Engine, one has to define a Configurations/config.apt file to define the available nodes and disk resources.
The disk resource are either scratch disk or disk.
The resource disk is meant to hold persistent data (how persistent ? during the execution of the project ? After the execution of the jobs of the project ?).
The scratch disk is for temporary files created by the PX engine (and so, unknown from the developper ?)
My concern is how to manage these resource directory in term of space, purge and backup.
I would think that the scratch disk probably doesn t need backup.
Does it need to be purged from time to time or does PX do this after the job completion ?
What about the resource disk directory ? It is called "Datasets" by default.
Does it handle only "Datasets" stage defined by the developpers in DS Designer or is there other kind of files stored there ?
Who is responsible for cleaning the files held in this directory ?
Does the developer have to code the deletion of these files ?
Are those file erased with a new content everytime the job is rerun ?
The above questions should help answering this following one:
does this resource disk directory need to be backuped ? How often ?
Concerning the size of these directory, how can it be determined ?
By the developer only or is there some PX Engine "overhead" ?
Thank your help !
Phil.
I'm just starting to work with DataStage PX (as a sys admin, not as a developer) and I haven't really seen the product working so far.
I have to answer some questionsand though I went through different documentations, some points remain obscure.
When configuring the PX Engine, one has to define a Configurations/config.apt file to define the available nodes and disk resources.
The disk resource are either scratch disk or disk.
The resource disk is meant to hold persistent data (how persistent ? during the execution of the project ? After the execution of the jobs of the project ?).
The scratch disk is for temporary files created by the PX engine (and so, unknown from the developper ?)
My concern is how to manage these resource directory in term of space, purge and backup.
I would think that the scratch disk probably doesn t need backup.
Does it need to be purged from time to time or does PX do this after the job completion ?
What about the resource disk directory ? It is called "Datasets" by default.
Does it handle only "Datasets" stage defined by the developpers in DS Designer or is there other kind of files stored there ?
Who is responsible for cleaning the files held in this directory ?
Does the developer have to code the deletion of these files ?
Are those file erased with a new content everytime the job is rerun ?
The above questions should help answering this following one:
does this resource disk directory need to be backuped ? How often ?
Concerning the size of these directory, how can it be determined ?
By the developer only or is there some PX Engine "overhead" ?
Thank your help !
Phil.