Moving 'dataset' data segment files to new file systems

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
PeteM2
Premium Member
Premium Member
Posts: 44
Joined: Thu Dec 15, 2011 9:17 am
Location: uk

Moving 'dataset' data segment files to new file systems

Post by PeteM2 »

Currently all our data segment files for datasets reside on the same file system. I would like to spread the segment files across a number of file systems to reduce the I/O bottleneck but have the file descriptor remain on the current file system.

What is the best method of achieving this?
thanks
jwiles
Premium Member
Premium Member
Posts: 1274
Joined: Sun Nov 14, 2004 8:50 pm
Contact:

Post by jwiles »

Copy the datasets using either the orchadmin command or a parallel DS job, using a configuration file which specifies the disk resources you wish to write the data segment files to.

Another possible option from the hardware perspective would be to replace the existing storage with a well-distributed RAID system mounted to the same location. Backup your existing filesystem, replace the storage and then restore the filesystem onto the new storage.

You cannot simply move the segment files to a new location.

Regards,
- james wiles


All generalizations are false, including this one - Mark Twain.
PeteM2
Premium Member
Premium Member
Posts: 44
Joined: Thu Dec 15, 2011 9:17 am
Location: uk

Post by PeteM2 »

Is there an option for orchadmin whereby the descriptor file remains in the same file location but the data segments are moved to the locations specified in the config file?
thanks
jwiles
Premium Member
Premium Member
Posts: 1274
Joined: Sun Nov 14, 2004 8:50 pm
Contact:

Post by jwiles »

No. If you need to preserve the filename of the descriptor, rename it, then perform the copy into the original name and then remove/delete (using orchadmin) the old dataset.

Alternatively, you can always recreate the datasets using the jobs which created them in the first place, overwriting the existing name.

Regards,
- james wiles


All generalizations are false, including this one - Mark Twain.
Post Reply