Reading Sequential File including Null o Spaces

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Amin
Premium Member
Premium Member
Posts: 27
Joined: Fri Oct 24, 2014 10:02 am

Reading Sequential File including Null o Spaces

Post by Amin »

I have file whose 1st and last row is enter, NULL or space when I set sequential file stage formatting even nullfield to '' it's not inserting .
Let me know how i can read all records even empty one having space,enter or Null.
Formate as below
1.
2. 00001 T , N N MO 021698 DM UNFI
.
.
....
45050
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

First, you need to realise that "", " " and NULL are not the same thing, and that NULL (whatever that is) never occurs in text files. The NUL byte ('\0') may appear as a string terminator.

So you need to be very sure what these "rogue" lines contain, and deal with them.

One way is to have a Rejects link from the Sequential File stage that reads them. That way any line from the file that does not satisfy the metadata definition that you are using will be passed (as a single raw string) to the Rejects link for you to process as you will.

Another technique is to read the file specifying a single column of type VarChar (or Long VarChar if you must), and cleanse and parse in a downstream Transformer stage. This gives the benefit that the parsing step is occurring in parallel (unless, for some reason, you choose to force the Transformer stage to execute in sequential mode).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Amin
Premium Member
Premium Member
Posts: 27
Joined: Fri Oct 24, 2014 10:02 am

Post by Amin »

Here is my file contens
[CR LF]
00001 N N MO 021698 DM UNFI [CR LF]
00002 N N MO 021698 DM UNFI [CR LF]
00003 N N MO 021698 DM UNFI [CR LF]
[CR LF]
----------------------------EndFile
Total records are 5
But data stage sequential stage ignore [CR LF] and read only 3 records
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Do you have a reject link to capture your empty rows (since they do not match the metadata)? Are there warnings in the job log reporting that import failed for two rows?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Amin
Premium Member
Premium Member
Posts: 27
Joined: Fri Oct 24, 2014 10:02 am

Post by Amin »

Thanks for reply

1: No information or data passes to reject link. As, 0 rows output to reject link.
2: There is no specific warnings about this.
3: File import 100 percent shows in log with Number of records are 3.

:roll:
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

Choose a job you love, and you will never have to work a day in your life. - Confucius
Post Reply