DSXchange

Another way to do it is to use the After Subroutine to execute a DataStage Server job that runs the SQLs for you.

You should be able to achieve a cross-join by adding an extra column to each dataset - using a constant value on every row - and then join on that column.

try

The functionality of the Surrogate Key Stage is duplicated in a new "Surrogate Key" tab in the Parallel Transformer. It does nothing extra or special, it just permits you to have one less Stage on the design pane. My understanding of the Surrogate Key generator is that it allocates irrever...

It certainly shouldn't happen, but Director is just reading from a table to get its info. I could imagine a simple corruption in the table causing such a problem. We have a similar problem going on in V8 at the moment with multi-instance Server jobs. We have a number of very small server jobs (they ...

Do you need a -wait to wait for a result?

If you are using Oracle 9i or later, you don't need to use a Lookup dataset - you can generate the number list on the fly: SELECT CAST(colcnt AS NUMBER(5)) AS colcnt , CAST(LEVEL AS NUMBER(5)) AS colinst FROM ( SELECT LEVEL AS colcnt FROM DUAL CONNECT BY LEVEL <= 120 ) CONNECT BY colcnt = PRIOR colc...

Here's one possibility: Take your input (currently:) Key, Col1, Col2, Col3 123, ab, cd, ef and concatenate all of the non-key columns leaving just 2 columns: Key, NonKeyCol "123", "ab,cd,ef," Choose a delimiter that does not appear in the data. I have used comma (,). Make sure th...

<woops - see next post>

As described by the other responses, DataStage is not necessarily the best tool to do this type of thing. Personally, I like to get my data into at least 1NF before I start processing it with DS - there are other languages that are better at this than DS. If you do want to use DS, then clearly the l...

How about any CPU quotas? Your 5 day session would not have tripped them.

Also post the entire error-stack captured in the DS log.

That could be *a* solution. However, from what I recall the Server MERGE stage uses hashed structures under the covers to perform that operation so I doubt it will work much more better. And THAT's why he's a guru! I agree, the documentation bears you out on that as well. Unlike the Parallel Merge ...

rleishman wrote:The solution is not to use hashed files. The alternative would be to pre-sort your two sources and use a MERGE stage.

Did you read this?

Try running it with a smaller hashed file - just build it with (say) 5000 rows. Obviously your job won't work properly, but we're interested to see if the large hashed file is causing the problem. If it is a lot faster with the smaller hashed file, you can try setting the Minimum Modulus on the hash...

If you don't want to rebuild the indexes, then you can still do DIRECT=TRUE PARALLEL=FALSE providing you have no referential integrity constraints. If you do have RI constraints that you do not want to disable, then the fastest load method is also the slowest: conventional inserts. You can use an UP...