more than one final delimiter?

[abyss]

hi all
i have a really simple question, but just can't fix it!
how to specify multiple final delimiters in the sequential file!? for example I want both pipeline (|) OR 'end' characters are the finial delimiters for a record. what's the syntax!?
sorry, I am really frustrated right now, i read docs and google it, just can't find the answer!
thanks for the help
Abyss

[jerome_rajan]

Are you saying that your file could look like this ?

Code: Select all

col1,col2,col3
a,b,c|
l,m,n
x,y,z~

or like this?

Code: Select all

col1,col2,col3
a,b,c|~
l,m,n|~
x,y,z|~

[abyss]

like this:

Code: Select all

col1,col2,col3 
a,b,c| l,m,n|x,y,z|~
1,2,3|e,r,t
c,x,z

if i can set finial delimiter to end OR | then i can get all records

[Plagvreugd]

I don't know a way to *really* use several delimiters in one sequential file stage.

But perhaps you can cope with this by using a Filter in the sequential file stage. Here you can define a filter program that for example would change all pipebars in the file into end-characters, so instead of two delimiters you only have one.

But with large files I suspect that this is not a very fast/efficient solution.

Kind regards,
Johannes.

[ray.wurlod]

It's not clear (from your examples, anyway) which data belongs in which column.

You may be better off, once you've identified all the variants, to read each line as a single VarChar and do the parsing in a Transformer stage.

If nothing else, your parsing will be executing in parallel.

[abyss]

hi, thanks for the reply guys. sorry i didn't make the requirement clear. I was in the hurry.
problem is quite simple, as you know in sequential file we need to define "finial delimiter". it's normally end of line , tab or something else.
what's i want is to set BOTH end of line character AND pipeline (|) as finial delimiter.

for example:

Code: Select all

the input file may like this:
record1|record2|record3
record4|
record5

(each record use comma as delimiter)

the output should be:
record1
record2
record3
record4
record5

regards
abyss

[qt_ky]

Try Ray's suggestion to read each record as a single varchar column. You would also benefit from using the loop feature inside the Transformer stage. The product documentation has good examples.

[ray.wurlod]

Some other options:
(1) get "them" to supply cleaner data
(2) specify pipe as Record Delimiter property value and pre-filter the newline characters from the file (perhaps using tr -d command as the Filter command in the Sequential File stage)

[chulett]

what's i want is to set BOTH end of line character AND pipeline (|) as finial delimiter.

Seems to me you might mean OR rather than AND.

If so, there is no OR in delimiters.