DSXchange

Hello,

I'm using the Folder Stage to merge several files together. Between each file I'm getting a blank line. I'm working on a Unix server. Any suggestions of what to look for?

Mike

The end of file character (Ctrl-D in UNIX, Ctrl-Z in Windows).

It's probably impossible to check for when using the Folder stage. Alter your design so that empty lines are filtered out (for example, filter through grep -v '' in UNIX)

ray.wurlod wrote:The end of file character (Ctrl-D in UNIX, Ctrl-Z in Windows).

It's probably impossible to check for when using the Folder stage. Alter your design so that empty lines are filtered out (for example, filter through grep -v '' in UNIX)

Hi Ray,

Thanks for the feedback. I'm trying to keep all my file manipulations inside of datastage. But maybe I can clean the resulting file using the After-Job routine. I'll try it out.

Mike

You CAN do this within DataStage!
The Sequential File stage (version 6 and later) supports the idea of streaming the data through a filter command.
They had zip/unzip in mind when they first created it, but there's no restriction. The stdout of the filter command becomes the stream that is read by the Sequential File stage.
Check it out!

ray.wurlod wrote:You CAN do this within DataStage!
The Sequential File stage (version 6 and later) supports the idea of streaming the data through a filter command.
They had zip/unzip in mind when they first created it, but there's no restriction. The stdout of the filter command becomes the stream that is read by the Sequential File stage.
Check it out!

You know I looked at that originally, but didn't spend enough time with it. I ended up creating another job that strips all blank lines plus others out of the file. It's not pretty but it will be easy to support for others on my team. I will go back and play with that feature, it could have a lot more uses.

Thanks for your help Ray!

Mike