DSXchange

Just extend your script, or the commands in the script, to pick up the files with the additional suffixes.

udayanguha wrote: Datastage will take care of the best partitioning method

Not quite true. DataStage will select a partitioning method that will always work. It may not be "best". It will be guaranteed to partition the data correctly for the stage in question.

Auto will give you Hash on the (entire) Sort key. It may be more efficient to specify explicitly under a couple of circumstances. If there is high cardinality on the first Sort key, you may prefer to partition on that key only. If the Sort key is an integer, then the Modulus algorithm will be more e...

It is possible in some UNIXes to have multiple processes write to the same file simultaneously.

Don't use a hashed file. Retrieve them directly from Oracle.

The CSV file was created on a Windows system (hence the 0x0d character at the end). Set the Record Delimiter String property value to "DOS style", or set the Final Delimiter property value to 013.

You could possibly use the looping capability in a Transformer stage to loop through the elements and suppress the DummyKey elements from being transferred to the output.

Otherwise, of course, a routine to do the same.

Is this XML a single string, or on multiple lines as shown? If the latter, simply use a filter (Filter stage or Transformer stage output link constraint) to prevent transfer of any line beginning with "<s:Dummy".

Use a Copy stage or a Transformer stage (or, indeed, pretty much any active stage type) to effect the column name changes during mapping from input to output.

chulett wrote:I thought practically all of a Sequence job's compilation 'effort' happened on the client.

Yes, but all of the objects to be compiled have to be transferred from the engine, and various resulting objects need to be transferred back to the engine. A slow network plays havoc with compilation time.

Seems to me that you want Duplicates to Retain=Last.

You can log an enhancement request through your support provider, or even through DSXchange!

Sequences are compiled differently than regular DataStage jobs, with some of the effort actually occurring on the client. Perhaps a faster connection between client and engine would help you.

It simply cannot generate signed numbers. Its output is defined to be an unsigned long integer (which most of the rest of the world uses for surrogate key values).

You can, of course, always access the Oracle sequence in custom SQL rather than via a Surrogate Key Generator stage.

It occurs to me that you're not using the right tool for the job. Have you investigated IBM InfoSphere Optim Data Privacy?