DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
srini.dw



Group memberships:
Premium Members

Joined: 18 Aug 2006
Posts: 183
Location: Chennai
Points: 1914

Post Posted: Wed Apr 20, 2011 6:04 am Reply with quote    Back to top    

DataStage® Release: 8x
Job Type: Parallel
OS: Unix
Hello,

The descriptor file is generated in path /is/DEV/datasets/PX_BEA.ds

The data file is generated in path is/node1/dataset

i.,e BEA.ds.dsadmin.DEV.0000.0000.0000.65e7.cf771bfe.0000.b324c6a7

My Question

1. Can we go ahead and delete the data file without deleting the original descriptor file.

2. What is the good method to remove the data file
orchadmin rm OR orchadmin delete command.

Thanks,
jwiles



Group memberships:
Premium Members

Joined: 14 Nov 2004
Posts: 1274

Points: 10406

Post Posted: Wed Apr 20, 2011 6:29 am Reply with quote    Back to top    

You seem to have a slight confusion on which files are what in a parallel dataset.

The dataset descriptor is the file which points to the dataset segments , describes the characteristics of the dataset and is typically named with the .ds extension. In your situation, this is the file /is/DEV/datasets/PX_BEA.ds

The dataset segments contain the actual data and have the names such as PX_BEA.ds.dsadmin.DEV.0000.0000.0000.65e7.cf771bfe.0000.b324c6a7

Please read Chapter 11: Managing data sets of the Parallel Job Developer Guide .

To answer your corrected questions:

1) Delete the dataset segments without deleting the dataset descriptor ? Technically you can, but for what reason? The dataset is no longer valid once you delete a part of it

2) What is a good method to remove a dataset segment ? Use the orchadmin command (rm and delete are synonymous). This will delete the entire dataset (segments and descriptor). Otherwise you must do the deletion manually.

A single question back to you: WHY?

Regards,

_________________
- james wiles


All generalizations are false, including this one - Mark Twain.
Rate this response:  
Not yet rated
srini.dw



Group memberships:
Premium Members

Joined: 18 Aug 2006
Posts: 183
Location: Chennai
Points: 1914

Post Posted: Wed Apr 20, 2011 6:47 am Reply with quote    Back to top    

Thanks for the reply.

In my case I got a requirement to delete dataset segments which has names such as PX_BEA.ds.dsadmin.DEV.0000.0000.0000.65e7.cf771bfe.0000.b324c6a7 located in is/node1/dataset.

Without knowing dataset descriptor file exist or do not exist in the seperate path can I go ahead and do the above task.

Thanks,
Rate this response:  
Not yet rated
zulfi123786



Group memberships:
Premium Members

Joined: 04 Nov 2008
Posts: 706
Location: Bangalore
Points: 5832

Post Posted: Wed Apr 20, 2011 7:01 am Reply with quote    Back to top    

if you are sure that the dataset PX_BEA.ds will never be referenced you can go ahead and delete the segment files corresponding to PX_BEA.ds

Having done that also delete the descriptor file as it will now be worthless
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 39347
Location: Denver, CO
Points: 200968

Post Posted: Wed Apr 20, 2011 7:04 am Reply with quote    Back to top    

srini.dw wrote:
In my case I got a requirement to delete dataset segments which has names such as PX_BEA.ds.dsadmin.DEV.0000.0000.0000.65e7.cf771bfe.0000.b324c6a7 located in is/node1/dataset.

A single question back to you: WHY?

_________________
-craig

The moments seemed lost in all the noise, a snow storm, a stimulating voice
Of warmth of the sky, of warmth when you die
Rate this response:  
Not yet rated
srini.dw



Group memberships:
Premium Members

Joined: 18 Aug 2006
Posts: 183
Location: Chennai
Points: 1914

Post Posted: Wed Apr 20, 2011 7:10 am Reply with quote    Back to top    

We have been facing space issues in Dev environment, hence the requirement.

"if you are sure that the dataset PX_BEA.ds will never be referenced you can go ahead and delete the segment files corresponding to PX_BEA.ds"

1 Question
What happens if I delete BEA.ds.dsadmin.DEV.0000.0000.0000.65e7.cf771bfe.0000.b324c6a7 and the PX_BEA.ds file still remain.

We would be getting some error.

Thanks,
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 39347
Location: Denver, CO
Points: 200968

Post Posted: Wed Apr 20, 2011 7:20 am Reply with quote    Back to top    

Yes, the dataset will become unusable. Deleting individual segments makes no sense. Your requirement should read: "if you are sure that the dataset PX_BEA.ds will never be referenced you can ...

_________________
-craig

The moments seemed lost in all the noise, a snow storm, a stimulating voice
Of warmth of the sky, of warmth when you die
Rate this response:  
Not yet rated
jwiles



Group memberships:
Premium Members

Joined: 14 Nov 2004
Posts: 1274

Points: 10406

Post Posted: Wed Apr 20, 2011 7:36 am Reply with quote    Back to top    

If you are consistently running into space problems within the development environment, you should begin implementing a reasonable management policy within the development teams.

Suggestions to start off with:

- No long-term storage of parallel datasets. Once you have completed the assignment using the datasets, delete them
- No production-level quantities/volumes for files/datasets. This is a development environment, not a performance testing environment. If you MUST, use and get rid of it as soon as reasonably possible.
- Delete datasets using the appropriate methods: orchadmin or the dataset administration tool in the GUIs (which uses orchadmin anyway). Manually delete segments and descriptor files only when necessary (corruption/missing files/etc.)

While you may not be in a position to implement and enforce such policies, you can at least recommend them to others and begin practicing them yourself.

The system admins may also need to consider increasing and/or moving dataset storage allocations.

Regards,

_________________
- james wiles


All generalizations are false, including this one - Mark Twain.
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 52091
Location: Sydney, Australia
Points: 282761

Post Posted: Wed Apr 20, 2011 4:29 pm Reply with quote    Back to top    

srini.dw wrote: In my case I got a requirement to delete dataset segments which has names such as PX_BEA.ds.dsadmin.DEV.0000.0000.0000.65e7.cf771bfe.0000.b324c6a7 located in is/node1/dataset. ...

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Sydney and Melbourne
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours