DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
Meem4m
Participant



Joined: 03 Oct 2007
Posts: 3
Location: India
Points: 34

Post Posted: Wed Oct 03, 2007 10:51 pm Reply with quote    Back to top    

DataStage® Release: 7x
Job Type: Parallel
OS: Windows
Hi All,
Can anyone advice me how to remove header and trailer from the Sequential file. The thing is, the source file has two header and two trailer. How to remove those two headers and trailers in Datastage???

Thanks in Advance,
Meem
ArndW

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup

Joined: 16 Nov 2004
Posts: 15926
Location: Germany
Points: 90069

Post Posted: Wed Oct 03, 2007 11:24 pm Reply with quote    Back to top    

DataStage jobs don't "read ahead", so they wouldn't know that they have reached a trailer record unless the format were different. I recommend that you use the head and tail commands, which ar ...

_________________

Image
Rate this response:  
Not yet rated
bkumar103
Participant



Joined: 25 Jul 2007
Posts: 214
Location: Chennai
Points: 1320

Post Posted: Thu Oct 04, 2007 2:28 am Reply with quote    Back to top    

You can achieve this through the sed.
Write a shell script. Run the shell script as the before job subroutine. The script may be like this

sed '1,1d'< inseqfile > outseqfile -- This can remove the first line from a file
sed '$d' < inseqfile > outseqfile -- This is to remove the last line from the file..

You can arrange the above script to achieve the same what you want.

You can try with Head and Tail stage also.
Rate this response:  
Not yet rated
JoshGeorge
Participant



Joined: 03 May 2007
Posts: 612
Location: Melbourne
Points: 3744

Post Posted: Thu Oct 04, 2007 2:59 am Reply with quote    Back to top    

If you have MKS Toolkit installed (Which Ships and installs with version 7.5x2) you can try the below described:

To remove header (2) and footer (2) from your file try this command in your filter option of sequential file satge:

Code:
 
sed 'N;$!P;$!D;$d' -e '1d' -e '2d'

_________________
Joshy George
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 39125
Location: Denver, CO
Points: 199793

Post Posted: Thu Oct 04, 2007 6:42 am Reply with quote    Back to top    

Won't PX do this 'automagically' because (typically) the metadata for the header/trailer records are different and thus they... reject? Confused

_________________
-craig

If you try and take a cat apart to see how it works, the first thing you have on your hands is a non-working cat. -- Douglas Adams
Rate this response:  
Not yet rated
ArndW

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup

Joined: 16 Nov 2004
Posts: 15926
Location: Germany
Points: 90069

Post Posted: Thu Oct 04, 2007 8:28 pm Reply with quote    Back to top    

Craig - yep, that is the way to do it if the formats of the header/trailer are different and you don't need the data in those rows. The header row is usually column names and will have the same number ...

_________________

Image
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 51925
Location: Sydney, Australia
Points: 281946

Post Posted: Thu Oct 04, 2007 9:25 pm Reply with quote    Back to top    

And one can always re-parse the raw string using Column Import stage. Idea

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Sydney and Melbourne
Rate this response:  
Not yet rated
ArndW

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup

Joined: 16 Nov 2004
Posts: 15926
Location: Germany
Points: 90069

Post Posted: Thu Oct 04, 2007 9:27 pm Reply with quote    Back to top    

Ray - are you sure? I tried to convert that raw type into something useable and failed - I'll look into the column import stage now.

_________________

Image
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours