DSXchange

Too much. Your UniVerse stage can access the hashed file directly, provided its name is know in the VOC file. You can create a VOC pointer to a pathed hashed file using the SETFILE command (search the forum for syntax).

Do you have NLS enabled in DataStage or Unicode enabled in SAP (or both)?

I was never able to find any documentation. You can infer usage of these functions by inspection of the generated job control code in a job sequence.

Your Field() functions assume "," as the delimiter, but your sed output is more likely to have "\n" as the delimiter. You need to align these, perhaps convert each \n to a comma in the sed script.

How are you detecting duplicates?

Why not create a simple server job that detects them (HAVING COUNT(*) > 1) and use its user status area to communicate to a trigger in the job sequence that sends the appropriate email?

No idea, given the sparse information that you provided.

For example how - exactly - are you "starting the job from UNIX"?

With only six rows nothing you do will make a lot of difference. What partitioning (Auto) uses depends on what's upstream of the Filter stage. Hash partitioning may worsen performance if it causes your data to be skewed (you have, for example, many more NY than NJ). The Filter stage does not use the...

It does whatever you ask it to do. But "it" here is not DataStage - DataStage simply sends the data to the database in an "UPDATE" context. If the primary key is the same, the row is the same. This is stock standard SQL behaviour - nothing at all to do with DataStage and how it works. If the primary...

It's preferable to use the conversion function, so that you don't get the warning (alert) messages about implicit conversion in the job log.

Just as a test, can you read the file using the following record schema?
record
{final_delim=end, delim=none}
(
RecNum:string[18];
RecLength:string[max=3];
Other:string[max=50];
)

The value received by the job can be seen in the "Job starting" message in the job log.

DataStage is not a reporting tool. You would ordinarily use regular SQL to get these answers. An Aggregator stage can give you an average (mean). Rank is more difficult, particularly when you have not specified how you wish tied ranks to be handled. An ETL approach would typically involve sorted dat...

The number of logical nodes is the number of nodes named and defined in the configuration file. You should have more than one configuration file; one that can use all of the server machine's resources and some other, smaller ones, that can be used when not all of those resources are required (for sm...

No, you can use sparse lookup at any time, whether or not you have checked the memory. It is the least efficient option, and is intended for use only when the reference data set is known in advance to be too large to fit in virtual memory. Note that sparse lookup can not be selected dynamically, so ...