Jobs aborted in the middle of dataloading into warehouse

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
senani
Participant
Posts: 2
Joined: Fri Sep 09, 2005 1:20 am

Jobs aborted in the middle of dataloading into warehouse

Post by senani »

Hi frens,
I do have the 10,000 rows of data and the job get aborted while loading into warehouse after loading 5000 rows. How to hadle this situation to load it from 5001 record...
ogmios
Participant
Posts: 659
Joined: Tue Mar 11, 2003 3:40 pm

Re: Jobs aborted in the middle of dataloading into warehouse

Post by ogmios »

senani wrote:Hi frens,
I do have the 10,000 rows of data and the job get aborted while loading into warehouse after loading 5000 rows. How to hadle this situation to load it from 5001 record...
There's nothing automatic in DataStage for that. You have to make your own restarting.

Some options:
- put the load in 1 transaction (for 10000 rows this would still work) by putting transation size to 0 in the target stage, upon aborting all changes will be rolled back. And you can rerun.
- include a generated key in the rows (generated in a stagging area), and load e.g. all rows smaller than the highest generated key in the target.

If you're just starting with datawarehousing I would suggest to run to the bookstore and buy all Kimball books you can find.

Ogmios
In theory there's no difference between theory and practice. In practice there is.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Another option similar to the second one offered by ogmios.

Assuming a stable input source such that 5001 is always the same row when rerunning, a job parameter used in the output link constraint. Set it to default to zero and your constraint to say '@INROWNUM > JobParameterName'. When it's time to restart, run the job with the value set to the last good row number loaded - 5000 - and it will pick up with 5001. Or use ">=" if you are more comfortable passing in 5001 as the value.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply