DSXchange

The Aggregator solution is the best one given the situation.

The MOD on @OUTROWNUM trick will not work. You will only get 5 rows output, a 0, 1, 2, 3, and 4 row. If you have 500, 50000, or 5000000 source rows, you'll only get 5 rows of output.

Guys, none of you have answered the original question. They have 8 servers that act in concert as both the servers for a DB multi-node database and PX. One of those 8 servers also acts as the "home" server for the DataStage installation. That "home" server also has the tasks of being the database co...

The MOD trick doesn't work. You need to create a new column to be a grouping key. Hopefully, your data is ordered so all you have to do is use a stage variables in a transformer. The stage variable is a GROUPKEY that starts at 1 and only increments every fifth row. Output to an aggregator and group ...

For all we know, the source SQL in the driving table is taking 75% of the runtime just to compute the result set, the other 25% is the spooling and reference lookup. To get the total compute and spool time, build a job of just OCI --> XFM --> SEQ using the driver table. See how long that takes. Then...

If you started from an empty table and things slow down over time doing straight inserts, it points to the underling table architecture. You can be encountering growth inefficiencies across the board. Your tables could be partitioned, and PX is loading them according to partition, which is optimal. ...

Please describe the nature of the tables, the nature of the data load process (inserts vs. updates), the hardware involved. I'm not even beginning to ask about competing processes on the server.

Data source empty is not an "error". If your metadata is bad, then DS may have trouble parsing the file and because your metadata says throw away any bad rows, you end up with an "empty" file, which just means none of the rows were usable. Here's a good suggestion: Take your sequential source file a...

No, your developer should have used parameters instead of hardcoded values. If your developer used parameters, and simply hardcoded defaults, then you can use any of the mass parameter update utilities available. Search for Parameter Manager offered on this sites companion tools section. If your dev...

CREATE TABLE ETL_JOBSTREAM_AUDIT ( BATCH_NBR NUMBER(14), JOBSTREAM VARCHAR2(255 BYTE), START_TIMESTAMP DATE, END_TIMESTAMP DATE, STATUS VARCHAR2(20 BYTE), BATCH_TYPE VARCHAR2(20 BYTE), USER_ID VARCHAR2(255 BYTE), COMMENTS VARCHAR2(255 BYTE), RUNTI...

but I was hoping to avoid it here and contain it all in DataStage Worthy goal and much cleaner. But, it's always in your interest to be able to measure inserts and updates and separate them as discrete acts. As data warehouses mature at a company, it becomes more critical to be able to audit your p...

This is a prime example of operational metadata. You need to store runtime information into a table somewhere that describes the execution activities of your jobstreams. From there you can retrieve for subsequent runs, any information required to satisfy that run. Other requirements in the future wi...

Here's what you are doing: You are attaching to a job, which gives you exclusive access to run a job. Then, you are setting the parameter value. Then, you are detaching the job. In essence, you have done nothing. If you were to have RAN the job after setting the parameter value, it would have used t...

what if 1)some jobs are scheduled through DataStage Schedular 2)some jobs are scheduled through CRONTAB Directly putting a job into DataStage Scheduler is the same as use Windoze AT or Unix cron. You are not using job control. Job control means controlling jobs. If you want to use DataStage Schedul...

If you use Sequencer jobs, then use the option to reset and then run. If you are using Batch jobs, add logic to get the job status and if necessary run with a reset mode, wait for it to finish, then run as normal. All of the APIs are documented in your DS BASIC manual.