Need to get all flat files into one flat file

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
reshma11
Premium Member
Premium Member
Posts: 25
Joined: Mon Oct 13, 2008 7:52 am

Need to get all flat files into one flat file

Post by reshma11 »

I have a requirement that I need to get all the output flat files into one flat file separated with file name. Can any one help me on this how to get this done.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

So... you need to generate it like that from the start or can you generate them separately and then combine them afterwards? The latter is just 'concatenation' in a loop and something you could script. The former... would have to ponder that. :?
-craig

"You can never have too many knives" -- Logan Nine Fingers
reshma11
Premium Member
Premium Member
Posts: 25
Joined: Mon Oct 13, 2008 7:52 am

Post by reshma11 »

I am generating them separately and I need to combine them afterwards. How can I do that.
nagarjuna
Premium Member
Premium Member
Posts: 533
Joined: Fri Jun 27, 2008 9:11 pm
Location: Chicago

Post by nagarjuna »

i think you can do it in sequential file stage . Read multiple files using a sequential file and use the option FileName column.
Nag
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

As noted, a script would be fairly simple. Do a 'for' loop thru the files, echo the filename to the output file and then cat the file itself. Use '>>' redirection in each case to append rather than overwrite.
-craig

"You can never have too many knives" -- Logan Nine Fingers
reshma11
Premium Member
Premium Member
Posts: 25
Joined: Mon Oct 13, 2008 7:52 am

can do it in sequential file stage

Post by reshma11 »

In sequential file stage options filter = awk ' BEGIN { print "#pHEADER#"; } { print; } END { print "#pTRAILER#"; } '

Thanks
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

Code: Select all

for fileName in `ls -1`
do
  echo $fileName >> yourFileName
  cat $fileName  >> yourFileName
done
Post Reply