Loading data from datasets to text file

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
dspxlearn
Premium Member
Premium Member
Posts: 291
Joined: Sat Sep 10, 2005 1:26 am

Loading data from datasets to text file

Post by dspxlearn »

Hello,

We have a requirement to load the dataset into a text file, archive the text file and delete the datasets due to the space constraints. We thought of loading the data from dataset to a text file through orchadmin utilities.

So, can anyone suggest if this is a good approach to use the orchadmin utility or to use a datastage job itself?

Note: We have many datasets each with different metadata and the text file will be used to load the data back to the datasets if needed.
Thanks and Regards!!
dspxlearn
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Ultimately using a DataStage job or orchadmin execute the same functionality.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

I would write a generic job to do the dataset -> text transfer. The job would declare no columns at all and use RCP, parameterizing the dataset file name and the output file. In the after-job routine I would call the shell command "orchadmin rm {path-and-name of dataset}" to delete the dataset.
If the text files are for archival, I would add a filter to your output sequential file such as "gzip -" to compress the archive while writing.
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

Have you compared the cost of adding disk space vs. all the extra labor?

Have you considered using the Compress and Expand stages?
Choose a job you love, and you will never have to work a day in your life. - Confucius
Post Reply