DSXchange

O ok. So the first field is "PL1 200705" . I thought they were two seperate fields. Anywho, I change the metadata to exactly your sepcification. It still works at my end. Looks like there is a problem in your file. How did you import the metadata? Did you use the view button while importing metadata...

Yes. You will do all your fancy audit collection and then load it to a file (sequential or hashed).

Show us the metadata of your file. For all amout fields, I used Decimal(15,2). Rest I used varchar, exept for the second field. For that I used Integer. I am having no problem at all.
How did you import the metadata. Did you type it in? Please import the metadata, much better than typing it out.

There are several ways of doing it. One of the most famous ways is Kim's ETLStats. There are other ways, by using DataStage API functions. Search the archives. You will find loads of methods.
If you dont have audit tables, get them created. An alternative would be a flat file or a hashed file.

I presume you have 17 columns. I was able to read the file successfully using the following properties. Under Format Tab: Record Level ---> Final Delimiter = end ---> Record Delimiter = UNIX newline Field Defaults ----> Delimter = tab ----> Quote = none

Can you post some sample records? Also can you post what properties you have tried? Specifically, what is the Record Delimiter that you have set.

Does this column have CURRENT TIMESTAMP defined at the database level? If yes then you dont need to send in this column, it will automatically put the CURRENT TIMESTAMP in that column.

Can you live with an awk script to get this done. Its an awesome workaround for dynamic variables, such as dynamic columns. You are sitting on parallel engine on windoze. I presume you have MKS Toolkit installed and can run unix scripts, Yes?

Tell them Craig, man.

Then use the sample stage. Create three datasets from your dataset. Process each dataset seperately.
The sample stage can split your data based on percentage.

Is it first 5M rows? Look into the Head stage. Depending upon the number of nodes, specify how many rows to pick from each partition.
eg: If you have a two node config file then you need to specify 2.5M per node.

Remove duplicate stage requires sorted data, sort stage does not. Remove duplicate stage can retain first or last duplicate whereas sort stage only marks the first duplicate with 1 and the rest with 0. So its rather easy to retain last row using Remove Duplicate Stage. Both these stages do exactly w...

You still answered only half of my question. Did you reset the job? Also double click on the error message and post the entire content here.

For more specifics, its a better idea, to read the manual about these stages to get a clear understanding of when these stages to use when one stage takes precedecence over the other.

Do you still want to do it on unix or want a DataStage function to do this?
Can you post the exact sample data, not just type it out. I want to see where these words exist, as in, the only words in the line, or at the end of the line or the begining, etc.