Percentage based rejectiontion handling

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
baba
Participant
Posts: 4
Joined: Sun Sep 30, 2007 12:51 am

Percentage based rejectiontion handling

Post by baba »

Hi all,
Can any body help me to know, How to handle if rejects are 40% during validation( ie validation based on Null,Duplicates,PK) , JOB sappose to be Aborted automatically and same record has to be sent to Client through Email
Sceranio: Simple Job Source to Staging 1
Types of Jobs used: Sequential file to Transformer to Oracle Stage

Thanks
Baba
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Welcome Aboard. :D

That's something you'll need to code in yourself and depending on when you enforce the %, you may need to know the total number of record you will be processing as well.

Do you 'need' to stop things immediately when it hits 40%? Pass the total in as a parameter and check as you go via a Constraint. Not sure how easily you could make that work for a multi-node job, however. :?

Rather then abort a running job (never really a good idea) why not check after the validation and not proceed onwards instead? An 'after' check would be easier to arrange - get total and reject counts in a Sequence job, compare the two and decide what to next.

Food for thought...
-craig

"You can never have too many knives" -- Logan Nine Fingers
baba
Participant
Posts: 4
Joined: Sun Sep 30, 2007 12:51 am

thanks Craig , i will try

Post by baba »

chulett wrote:Welcome Aboard. :D

That's something you'll need to code in yourself and depending on when you enforce the %, you may need to know the total number of record you will be processing as well.

Do you 'need' to stop things immediately when it hits 40%? Pass the total in as a parameter and check as you go via a Constraint. Not sure how easily you could make that work for a multi-node job, however. :?

Rather then abort a running job (never really a good idea) why not check after the validation and not proceed onwards instead? An 'after' check would be easier to arrange - get total and reject counts in a Sequence job, compare the two and decide what to next.

Food for thought...
Post Reply