DSXchange

The one that reads one line from a file does not have to be, indeed if it were it would be a waste of resources.

A Search on DSXchange yielded seven hits (not counting this thread). A search on Google yielded more than 10,000 hits.

Yes, I can see that as a reading, and a variable-number-of-elements vector is probably a good solution in that case.

No need for vector/array. Just concatenate to the existing string if the indicator is greater than one, or use the new string if the indicator is 1.
The Remove Duplicates stage captures the last of each group. That, along with sorted data, is all you need.

DataStage does not create any locks.

But the database server does.

When View Data provides you with the opportunity to inspect the output of the command, accept the invitation. It will tell you what went wrong.

Just dream up appropriate WHERE clauses (probably with job parameters for the constant(s)) for extraction SQL.

Use a fork-join design with one side of the fork using an Aggregator to count the records with a given grouping key. After the Join, add the "(Duplicate record)" text to any record that has a count > 1. Or, if you have a source that is accessible via SQL, you could use a HAVING clause and ...

Posting the same question in more than one thread will not get you any more answers.

Please choose meaningful subjects. It will assist future searchers.

What do you think the message means? What nodes (fastnames) are mentioned in the configuration file that was selected for this job run?

The original poster specified "Teradata database" in the additional information field. I doubt that Teradata supports the TO_CHAR function.

Attention to detail is one of the vital attributes that anyone working in data quality and information management requires.

This is not the whole story. Only a certain amount of memory per node is available for sorting. Once this amount is exceeded, the sort must use disk files in the scratchdisk areas. This adds cost that you have not considered, probably more cost than the other steps (depending on the source of the ro...

Welcome aboard.

That's not a step - it's the log entry containing the configuration file that the job is using.

Clearly the job, being such a simple design, is waiting for *something* in Oracle. Enlist your DBA to learn what that is.

There is probably a mechanism for loading only the rows you will need into the lookup table. The technique is known as "bucket table": you generate a list of the keys from source, then join that result to the reference source as part of loading the lookup table. In the old days, the bucket...