Temporary lookuptable.* files unexpectedly accumulating

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
leroden
Premium Member
Premium Member
Posts: 2
Joined: Tue Jan 27, 2009 3:57 am
Location: Brussels

Temporary lookuptable.* files unexpectedly accumulating

Post by leroden »

According to IBM documentation, lookups can create temporary datasets on the resource disk defined in the configuration file.
These lookups seem to be cleaned after a successfull execution, but remain if the job aborts.
One must then clean the files manually or with a script.
http://www-01.ibm.com/support/docview.w ... wg21441823

We have a number of jobs that are scheduled to run on a regular basis, in the same project, on the same server.
These jobs are all based on the same template and use the same settings (apparently...)
However, one and only one of these jobs generates 'lookuptable' dataset files, which are never deleted, even though that job never aborts.

16 files are created at each run.
The job has 8 lookup stages, normal or sparse, including a total of 21 reference links (16 reference links to normal lookups).
The project is configured to run on 1 node only, and each stage is set explicitely in 'sequential' mode.

Even if we are setting up a script to cleanup that directory, we would like to avoid the creation or the non-deletion of these files.

Does anyone know what is happening ?

1. Is it Possible to avoid the creation or the non-deletion of these files?
2. How to read the content of these files? (they are not plain text and the Data Set Manager returns 'unknown error reading record schema')
Denis Leroy
IT Specialist
Business Analytics and Optimization
ray.wurlod
Participant
Posts: 54595
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Welcome aboard. These tables in the scratch disk *should* be cleaned up automatically, as you surmise. That they aren't is something you should raise with your official support provider. In the meantime your cleanup script serves as a workaround.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
leroden
Premium Member
Premium Member
Posts: 2
Joined: Tue Jan 27, 2009 3:57 am
Location: Brussels

Post by leroden »

Thanks Ray.

If we get an answer from the support, I'll post it here.

But any other answer from anyone is still welcome !
Post Reply