DSXchange

Sparse lookups should be used when you have a very large number of lookup rows and a small number of input rows. If you had less than 100,000 input rows per day a sparse lookup may be faster than a normal lookup as it saves you from having to pull the 28 million rows into a lookup table (enormous di...

Your main consideration is how the system will perform if you try and run the load of both servers on one server at the same time. For parallel jobs this means a lot of extra disk I/O, especially on the scratch space and temp space, do you have enough disk space allocated? Even if you run both sets ...

Different parts of your job are processing at different speeds which mixes up the order of your rows. You need to add a sequence number to the start of all header, detail and tail records using an @inrownum and send it down all three paths, in your collector sort by that sequence number.

The usage analysis function in version 7.x doesn't really work that well. It is better in version 8. You would hope that the table names in your Oracle stages are accurate even though they are superseded by the user-defined SQL. You could extract all your jobs to an XML output file and view it using...

You can upgrade to version 8 for QualityStage and it should be free if you are under a maintenance contract, but you will need to pay for the SOA license which the software catalog has at US$75,000 for a 2 CPU license. The good news is you wont need a DataStage license as QualityStage 8 gives you ac...

It is probably happening where you have a lookup stage followed by a transformer stage. They are being combined into a single operator, the lookup is producing a null value where a lookup is not found, the transformer is trying to run a derivation function on that field and this is causing the row t...

The only way to do it in version 7 is to use the QualityStage plugin for DataStage. DataStage can provide the input and output links for the QualityStage job then add the RTI input and output stages and the RTI pack can turn it into a web service.

I wrote about choosing the right job type or configuration file for small jobs in my blog entry DataStage tip: What do you mean I need to optimise small jobs? If you are doing any sorting or aggregation on large volumes a parallel job is a must, see DataStage server v enterprise: some performance st...

I wrote in some detail about trapping rejected rows from the Enterprise database stage in this blog post . The easiest way to work out what is going on in your reject link is to point it to a Peek stage and run the job. You should be able to see the database message fields and values in the Director...

You can now get DataStage version 8 for Windows, though like version 7.5.x2, it is only certified for Windows 2003 and you would probably hit installation problems on earlier versions.

I agree, $PROJDEF tends to be more reliable than $ENV as it is all centralised in one place (the Administrator and the DSPARAMS file) and it can use DataStage encryption for passwords. One of the first Director log messages is a list of environment variables for that instance of the job and it is us...

If you wanted just an integer ID (1 to 999999) you could use the surrogate key generator, but since you want varchar keys with leading 0s you are better off with the Transformer ID generator in the FAQ linked by DSguru2B. With multiple partitions you will probably get a small number of gaps in the s...

I'm dubious as to whether you can deliver effective training without having extensive product experience or having been on a training course yourself or having access to prepared training materials. The manuals do not provide example jobs or code or show short cuts and tips.

Sometimes the email function of e.services doesn't work. I suggest your ring IBM-Ascential support and ask for the patch and ask them to manually send you an e.services authentication message.

Best not to rush to conclusions.