DSXchange

Since you're not using PX, you may need to consider doing manually partitioning of the load files. By creating a load-ready file for each partition, you can absolutely name the partition to load in the control file. Then, launch sqlldr for each file against each partition simultaneously. That is, of...

direct=FALSE on the sqlldr command invocation

Direct path load is the optimal loading method for high volume warehouse tables. You are going to have to use a careful indexing strategy, as indexing becomes trickier. For instance, bitmap indexes become fun, as they deal with low cardinality columns and sqlldr does not play nice. As a preference, ...

Sure, just as soon as you tell us how you are running the stored procedure...

ODBC stage, OCI stage, before/after routine call to shell, Batch job calling DSExecute, Command stage... Get my point?

A global routine concept is available, but not natively thru DataStage. If you want to create routine in one project, and have it visible in all projects and usable, you have to a do a lot of things outside the ordinary. You will have to override the local routine file in each project with a pointer...

You can have an ETL tool keep every row of data in the pipeline during processing tucked away somewhere in case of failure. But what are the data storage requirements of the ETL tool? That data has to be somewhere. For example, if you're sourcing from a table and writing to a target and the job fail...

You should contact Ascential because they will give you all the literature and information you need. You'll get some really good looking .pdf's to help in your presentation to customers. This forum is really for developers. It's like going to a car repair shop and talking to mechanics. You're going ...

By separating pure inserts from pure updates, you enable Oracle to skip a conventional load and go straight for a direct path load. Depending on the partitioning method and indexing strategy, you can either hurt or help load performance. If your table is partitioned in such a way that data is evenly...

DataStage's metadata of Universe and Unidata tables is not accurate of the physical data structure of those tables. Since dictionaries contain I-descriptors, T and F correlatives, then the data returned via the dictionary is not the same as the physical structure. Since the data is also multivalued,...

May I suggest you look over at [url]asktom.oracle.com[/url]for some background on the upsert method?

http://asktom.oracle.com/pls/ask/f?p=4950:8:14389124738426353420::NO::F4950_P8_DISPLAYID,F4950_P8_CRITERIA:6618304976523

Please describe your process. We can't even begin to tell you what to do.

The DSD.StageRun are the active stages within a job. However, you're missing teh DSD.RUN controlling process. Your job has tragically died, leaving threads out there (notice PPID is 1). You will have to kill these threads, if they're talking to a database have the DBA kill the connections and then m...

You need to do a "ps -ef |grep phantom" and see if any threads are running. If they are, then you need to tell us what the jobs are doing. Okay, we know you have a source file, but where is the data going? Could you be waiting on a big commit? Could your load have failed and you're waiting on a big ...

It's not unbelievable that a 20% row count increase has a 5X impact on the total runtime. The number of round trip lookups against the database is increasing in a non-linear fashion, as you are spending a lot more time asking the database a lot of times to go find data. This increase your presence i...

You are having scaling issues because your job is not designed to be scalable. It is designed to be easy to see everything happening. I bet next month your job never finishes in time. A scalable job design would utilize bulk loading, it would no use direct lookups against the database during transfo...