DSXchange

What is this, an interview? You asked for suggestions, I gave you suggestions based on more than 12 years' experience, and you immediately respond with "any suggestions?". I resent that, I really do.

Do you add them to the stage using the parameter helper tool (Insert Job Parameter)? What do they look like when loaded into properties? Is the generated connection string consistent with what you've put into the individual properties? Are all the required "#" characters in place? You did not answer...

Start by not using rows/sec as a metric of "performance". Run the job with performance statistics gathering enabled for each Transformer stage (on the Tracing tab of Job Run Options dialog). Find out where these stages are spending most of their time. With these kinds of volumes you might contemplat...

That's TERRIBLE code!! You need to own the update lock before a Write to a hashed file, to prevent lost updates. You merrily proceed with your (unprotected) write even if the file could not be opened. FUNCTION ResetSDKSequence(SequenceName, ResetValue) * If SequenceName is not provided, rout...

Is it the routine or the calling of it?

Add two diagnostic calls to DSLogInfo() within the routine to report the start time and exit time.

Inspect the job control code generated by compiling the job sequence, see if you can spot anything there that might introduce such a delay.

The answer is NO. A Decimal number is a binary data type. Whether it has leading zeroes or not it is stored the same way. DecimalToString() generates the shortest possible valid string. If you want to add leading zeroes, then you must do so explicitly. For example: Right("000" : Decima...

Check that the appropriate quote characters have been used in SQL and that job parameter references are correctly encased in a pair of "#" characters wherever used in a passive stage.

Enrol in the DataStage Essentials class. The question indicates a lack of knowledge even at entry level.

Integers (int32) can be up to 10 digits long. Not all will fit in a Decimal data type with only eight digits of precision. DataStage alerts you to this fact.

If you insist upon using a Lookup stage, then the entire reference data set must fit in memory. Clearly in your case it doesn't, so allocate a lot more scratchdisk resource. Search for ways to make the extraction more efficient - do the views REALLY need to expose 100 million records? Can you, for e...

Are the fields AND their intermediate delimiters missing, or are all the delimiters there and the fields between them empty? You handle the two cases differently. In the first case read each line (record) in the file as a single VarChar field and parse it subsequently. In the second case adjust the ...

A number of Solaris sites I've worked on have had swap mounted on /tmp, which I always felt was an odd practice.

Do you get any errors/warnings when trying to schedule jobs? The crontab command will allow you to see whether the jobs have actually been queued. If they have not, ask your UNIX system administrator whether the permissions for cron and at commands have been tightened. You might also ask whether the...

Has someone specifically started an instance of dsrpcd intended to listen only to localhost (127.0.0.1)? If so, why?

Or it may have something to do with your specific tunnelling protocol - you'd have to ask your support provider about that.

The answer is NO, but it's not anything to do with DataStage: it has to do with the nature of a sequential file, in which you must read past every single byte to get to the next one.

Since you have to read every byte in the sequential file anyway, use the easier downstream methods for parsing.