DSXchange

There's nothing to stop you using database sequences or SERIAL data types when loading the dimension tables. Whatever mechanism you use to load the dimension tables, you will need to build a "reverse lookup" mechanism when loading the fact tables, because you will need to map the "business keys" enc...

Not hard, just something you have to design for if you want it.

Murphy's fourth postulate: If it's stupid but it works, it's not stupid. Your technique is entirely valid in server jobs, because the Engine does not use data types - everything is a string with implicit conversion governed by context (for example arithmetic). This helps DataStage to be tolerant of ...

The point about parallel TMU ( rb_ptmu ) is that the indexes are updated by separate processes, thereby not slowing down the loading of rows into the main table. The main bottleneck is usually foreign key constraints (referential integrity). These must be checked by the process loading the main tabl...

The only Red Brick stage in DataStage is the bulk loader stage.

Says something important about purpose-designed DW databases, I feel.

So restarting is not automatic in DataStage. You can create designs so that restartability is automatic. Techniques used to be taught on the Ascential class "DataStage Best Practices" - is this still offered? There is a cost; you must create one, better two, staging areas as recovery points. You al...

I had replied to this earlier, just before the DSXChange server failed.

Read about the failure on the home page.

You have not informed us whether or not the dsrpcd process is running. Without that vital clue we are powerless to assist further. Please advise the result of executing the command suggested by Craig. If the dsrpcd process is not running, you will never be able to connect. In that case, we will assi...

SAP Extract PACK and SAP BW Load PACK must be separately purchased. The additional charge is largely the cost of certification from SAP. The software, if you have it, is on a separate CD. ODBC drivers for UNIX are specialized pieces of software provided by third-party vendors. The drivers that ship ...

It is absolutely vital that the UNIX kernel parameters referred to in the installation documentation are set to at least the advised values.
DataStage Engine relies heavily on shared memory and semaphores; the default UNIX values are simply insufficient. (Exception: AIX has a "self tuning kernel").

If the table has a key you can pre-load a hashed file with just the key values, and use that to check for existence (and therefore the insert versus update decision). This will run much faster than trying to do the lookups against the target table itself. If the key-only hashed file is sufficiently ...

Memory can also be a bottleneck if you're using memory-hungry stage types (Aggregator, Sort) on an underconfigured machine.

Other areas of performance tuning is choosing efficient algorithms; this is largely experience, and depends on your data, so there's no magic list of cures.

Probably just imps and gremlins (other users?) doing random inserts on just the page where your process wants to be.

Not really much information in your question. Are you doing a lot of computation (transformation) per row? How are you loading the Oracle table? If they are all new rows for Oracle, seriously consider an approach that uses sqlldr rather than INSERT. It is way faster. The best approach is to have you...

In my experience with DataStage another common reason for deadlocks is updating the same table you are selecting from, and shooting for huge transaction size (unlimited rows per transaction). Oof!