Removing Header/Trailer from Seq File

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Post Reply
Meem4m
Participant
Posts: 3
Joined: Wed Oct 03, 2007 10:42 am
Location: India

Removing Header/Trailer from Seq File

Post by Meem4m »

Hi All,
Can anyone advice me how to remove header and trailer from the Sequential file. The thing is, the source file has two header and two trailer. How to remove those two headers and trailers in Datastage???

Thanks in Advance,
Meem
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

DataStage jobs don't "read ahead", so they wouldn't know that they have reached a trailer record unless the format were different. I recommend that you use the head and tail commands, which are available to you as you have PX under windows installed.
bkumar103
Participant
Posts: 214
Joined: Wed Jul 25, 2007 2:29 am
Location: Chennai

Post by bkumar103 »

You can achieve this through the sed.
Write a shell script. Run the shell script as the before job subroutine. The script may be like this

sed '1,1d'< inseqfile > outseqfile -- This can remove the first line from a file
sed '$d' < inseqfile > outseqfile -- This is to remove the last line from the file..

You can arrange the above script to achieve the same what you want.

You can try with Head and Tail stage also.
JoshGeorge
Participant
Posts: 612
Joined: Thu May 03, 2007 4:59 am
Location: Melbourne

Post by JoshGeorge »

If you have MKS Toolkit installed (Which Ships and installs with version 7.5x2) you can try the below described:

To remove header (2) and footer (2) from your file try this command in your filter option of sequential file satge:

Code: Select all

 
sed 'N;$!P;$!D;$d' -e '1d' -e '2d'
Joshy George
<a href="http://www.linkedin.com/in/joshygeorge1" ><img src="http://www.linkedin.com/img/webpromo/bt ... _80x15.gif" width="80" height="15" border="0"></a>
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Won't PX do this 'automagically' because (typically) the metadata for the header/trailer records are different and thus they... reject? :?
-craig

"You can never have too many knives" -- Logan Nine Fingers
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Craig - yep, that is the way to do it if the formats of the header/trailer are different and you don't need the data in those rows. The header row is usually column names and will have the same number of column separators as the data, but if you have non-string columns then the text column names will trigger the header row(s) to be rejected.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

And one can always re-parse the raw string using Column Import stage. :idea:
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Ray - are you sure? I tried to convert that raw type into something useable and failed - I'll look into the column import stage now.
Post Reply