DSXchange

Hi
We just migrated our datastage installation from AIX to LINUX and we have run into an issue with the sequential file stage. The sequential file stage is used to read a file and store the fields in an oracle table.

The sequential file is reading files that may have CRLF or just LF as record delimited.

In AIX, the sequential file stage, with the FINAL_DELIMITER set to END, is able to strip both CRLF and LF from the last field before storing it into a DB.

But under LINUX, it is only stripping the LF from CRLF leaving the CR as part of the last field which then gets stored in the DB along with the field value as a non printable character.

Short of changing the jobs and the scripts, what else can we do to make LINUX behave more like AIX where without any prompting it is able to understand that CRLF or LF are both record terminators and need to be stripped.

You can get it right on a stage-by-stage basis (setting the Record Delimiter or Record Delimiter String property as appropriate).

... unless you are saying they can be different inside the same file? Or do you just mean it can change per file but all will be consistent within any given file?

Just trying to understand the scope of the issue.

Thanks for your responses guys but this is a strange problem

To answer your question Chulett, Every file will either have a CRLF or just LF. But we can't predict which file will have which line delimiter.

The issue is that when we had these jobs running in AIX environment, the sequential file stage, with its FINAL_DELIMITER=END, was somehow able to correctly process both kinds of files.

When a file had CRLF, it removed the CRLF from the last field and when it had LF, it removed the LF.

But now since we moved everything to LINUX, it is only removing LF from both kinds of files leaving CR untouched which is now ending up in the database as part of the last field.

Quesiton - What was making AIX handle both CRLF and LF correctly.
And how to we make LINUX behave in a similar manner.

AIX is... special. And I sincerely doubt you'll be able to make anything else behave like that. Perhaps some sort of "pre-processing" is in order?

Just a thought...

When you migrated from AIX to Linux did you also upgrade your DataStage version?

Newer versions of DataStage tend to get more strict, so it could be due to that rather than the OS.

Good point. I shiver in antici .... .... ... pation!