Has anyone designed how they do Initial loads and Incremental's in Datastage. We have a requirement to bulk load the initial load and subsequently process delta's.
What I am try to stay away from is coding two jobs. My thought is to code one job and use a parameter to control which DB2 connector gets used. One being the Bulk loader and the other being the Insert then update.
I realize that the initial will be used minimal times. I am also not sure whether the Bulk loader will use up a connection in the job for nothing when not used
Has anyone used a different methodology?
Thanks
Coding Bulk and Inserts/Updates
Moderators: chulett, rschirm, roy
Coding Bulk and Inserts/Updates
Jim Stewart
We are working with 300+ million rows for the first initial load, subsequent deltas are manageable. It is the first load that causes problems due to slow inserts loading. We get 4 times more performance with the Bulk loader. I was just wondering if anyone has used one job with two DB2 connectors, one for initial load(bulk) and one for deltas(inserts). Processing controlled by the value of the parameter.
Jim Stewart
Re: Coding Bulk and Inserts/Updates
It will.Raftsman wrote:I am also not sure whether the Bulk loader will use up a connection in the job for nothing when not used
FWIW, in the very small number of times where I've had to differentiate between an initial and subsequent loads, I've created two jobs.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers