DSXchange

And isn't that what I initially recommended?

That won't help. That is client inactivity. You are simply overloading your server. Stop doing that, or get a more powerful server.

Are these numbers in a single field (and, if so, how are they delimited) in the same record, in separate fields in the same record, or in the same field in multiple records (and, if so, how is the input sorted)?

Use an Aggregator stage. Use First as the aggregate function.

Who needs a routine?



OK, this answer hinges on knowledge about how the Fmt() function works, but it's all in the DataStage BASIC manual.

DSSentMail() is a routine, not a command. It's intended primarily to be invoked by a Notification activity in a job sequence, but you could call it from your own before/after subroutine. In the Repository you will find a DSSendMailTester routine. Examine its code to learn how to call DSSendMail().

The log message shows that the pathname is being generated successfully. Do you have a timeout set in the Wait For File activity? Do you have two output links from the Wait For File activity, the first with an OK trigger and the second (with an Otherwise or Failure trigger) to handle the fact that t...

Or just reorder the statements in the Job Control code so that, instead of Run Wait Run Wait Run Wait, you have Run Run Run Wait Wait Wait.

The job sequence approach is better, as it frees each job's resources as soon as each is finished, rather than waiting for all to have finished.

If you don't want the warnings, you must include logic to reject the row before an attempt is made to send it to the database. For example if the column in question is Char(16) then you need something like in a constraint expression.

You can not attach an already-running job if that job is already attached to another process (such as a job sequence). That might explain a hang on DSAttachJob().

To paraphrase Arnd's question: what happens if the number is too large for Decimal(10,6) ?

The first, and most likely candidates, are NLS_LANG (for Oracle) and LANG_C (for Solaris). Check with your Oracle DBA for the appropriate value for NLS_LANG.

The nice thing about PX is that you can simply throw more processing nodes at it. Unless there are specific order issues in the text file, you can use multiple readers (each reading a portion of the file) or pre-process the file to split it into multiple files. Research the Sequential File stage pro...

You first need to defined "performance". In batch-oriented ETL the key performance indicator is merely the ability to achieve the extraction and load time windows, ideally with a safety margin.

This post contains some information about DataStage architecture.

Are there any line terminator characters in your data? That is, is any of the "comments" multi-line in the source data?