DSXchange

See if the following custom routine helps you. FUNCTION SplitRecord(Arg1) FieldToSplit = TRIM(Arg1) Type = "" Year = "" tmpType = "" tmpYear = "" FLAG = @TRUE FieldLength = Len(FieldToSplit) FOR n = FieldLength To 1 STEP -1 asciiCode = ...

Congratulations Ray. Thats one
humongous
post count. Way to go.

In the field "CAR 3", is 3 the year? Or is it supposed to be 03 and you did not include the 0. Where I am getting to is that if the last two places are always going to be the year then you can use Right(in.Link, 2) for the year part and LEFT(in.Link, LEN(in.Link)-2) for the Type.

sort -u will eliminate duplicates and will retain only one per key. Read about sort in google to get its complete usage.

Externally, i.e; Sort it via the unix sort command

Are they both set as keys in all the links coming in and out of the link collector?

If you are indeed ftp'ing in Ascii format then the files get converted to Unix format. That means you need to specify 'Unix Format' in the sequential file stage.

Well, it can be done using EREPLACE() and count together. Use EREPELACE() to change '~~' to something that is most likely not going to show up in the data. Something like ! or & and do a count on that. In px, pxEreplace() can be used to replace the double tilda. NOTE: pxEreplace() is not an IBM ...

Whats there to discuss, you have the strategy right there as your second point. Use the process date to extract only the changed records. Pass it through to your datawarehouse. Now if these daily changes are going to be huge ( in millions) then you can bulk load the changes to a work table. Pass a s...

Knowledge of your job design should help.
Make sure your metadata is correct and no varchar field goes to any kind of numeric field.

Go to the temp folder, and keep checking for any file created while this job is running. Then see what happens to that file once the job aborts. A workaround would be to put either an IPC stage between the two transformers or a passive stage like a flat file.

What was the problem in the first place? Memory mismanagement?

In the filter command put in the following awk command.

This should take care of it.

How are you connection to DB2 mainframe? Or you are getting a flat file from the mainframe? If you are going directly to mainframe, try specifying a TO_CHAR for the date in the sql select and read it as varchar instead of date.

As for your first error. Look at your config file and see what paths have you specified their. As for your second error, it seems like there is NULL encountered in a non-nullable column. Please check the metadata and the records. If the column is infact not nullable, then handle nulls appropriately....