Page 1 of 1
Jobs aborted in the middle of dataloading into warehouse
Posted: Wed Mar 15, 2006 4:23 am
by senani
Hi frens,
I do have the 10,000 rows of data and the job get aborted while loading into warehouse after loading 5000 rows. How to hadle this situation to load it from 5001 record...
Re: Jobs aborted in the middle of dataloading into warehouse
Posted: Wed Mar 15, 2006 4:52 am
by ogmios
senani wrote:Hi frens,
I do have the 10,000 rows of data and the job get aborted while loading into warehouse after loading 5000 rows. How to hadle this situation to load it from 5001 record...
There's nothing automatic in DataStage for that. You have to make your own restarting.
Some options:
- put the load in 1 transaction (for 10000 rows this would still work) by putting transation size to 0 in the target stage, upon aborting all changes will be rolled back. And you can rerun.
- include a generated key in the rows (generated in a stagging area), and load e.g. all rows smaller than the highest generated key in the target.
If you're just starting with datawarehousing I would suggest to run to the bookstore and buy all Kimball books you can find.
Ogmios
Posted: Wed Mar 15, 2006 8:03 am
by chulett
Another option similar to the second one offered by ogmios.
Assuming a stable input source such that 5001 is always the same row when rerunning, a job parameter used in the output link constraint. Set it to default to zero and your constraint to say '@INROWNUM > JobParameterName'. When it's time to restart, run the job with the value set to the last good row number loaded - 5000 - and it will pick up with 5001. Or use ">=" if you are more comfortable passing in 5001 as the value.