Line breaking in sequential file

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
maryonoid
Participant
Posts: 3
Joined: Thu Apr 09, 2009 5:55 pm

Line breaking in sequential file

Post by maryonoid »

Hi everybody!
As I'm new here at this community, maybe I haven't see all the topics for this question.
Even so, I'm facing this problem in a sequential file.
I've created two jobs, one to query a table in a DB2 database and creates a sequential file with some records. The other reads this file.
There is a DESCRIPTION field which has the description content where for some records it has breaking lines. This field is a CLOB type.
The issue I'm facing is, when the DS creates the seq file, it's also creating those braking lines.
When I run the other job to read the seq file, DS returns an error because it understands the breaking lines are the end of the records but not, they are the DESCRIPTION field continuation.
I've already tryed setting the record delimiter in the Sequential File Format stage with different characters, but not succeded.
If anyone has experienced this same issue I apreciate you're sharing to solve this.
Thanks and best regards.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Welcome. :D

How enamored are you / your users of these 'breaking lines'? I've found it far simpler to strip them from the field before writing to the file than to deal with them on the reading side. And typically there isn't an issue with zapping them during the process, at least in my experience.

If they are actually <CR><LF> pairs then you'll be seeing the <CR> in the data as well, so would want to strip those specific pairs from the field rather than just the line feed that is causing you grief.
-craig

"You can never have too many knives" -- Logan Nine Fingers
maryonoid
Participant
Posts: 3
Joined: Thu Apr 09, 2009 5:55 pm

Post by maryonoid »

hi Chullet!
Yeah, I thought like that! Removing the pairs. But I had not success on doing that in DataStage. I've tryed to use Convert() function in a Tranform stage, changing every char(13) and char(10) for a simple white space, but it continues creating that breaking lines in the description field in the seq file.
I had to use a perl script to treat that characters and rearrange the Description field in just one line.
It worked, but I'd like to do that by DS.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Convert should work fine... can you post the syntax you tried?
-craig

"You can never have too many knives" -- Logan Nine Fingers
maryonoid
Participant
Posts: 3
Joined: Thu Apr 09, 2009 5:55 pm

Post by maryonoid »

Sure!
I'm using this in a Transform stage:

If IsNull(LNK_EXEC_TASK.DESCRIPTION) THEN "-1" ELSE Convert(char(13)|char(10)," ", LNK_EXEC_TASK.DESCRIPTION)
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

The pipe is certainly a problem. Change it to a colon, which is the concatenation operator in DataStage:

Convert(char(13):char(10)," ", LNK_EXEC_TASK.DESCRIPTION)
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply