Threshold for Reject Records

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
praburaj
Premium Member
Premium Member
Posts: 133
Joined: Thu Jun 30, 2005 1:26 am
Location: philippines

Threshold for Reject Records

Post by praburaj »

Hi,

My source file is having more than 10 lakhs records. When I execute my jobs some of the records are going to reject files. Now, my requirement Is If reject records are greater than 10% then I have to stop the job. I don't know how to achieve this?.

Please give me valuable Input for this
prabakaran.v
Sreedhar
Participant
Posts: 187
Joined: Mon Oct 30, 2006 12:16 am

Post by Sreedhar »

Hi,

If you are loading into a database then we can stop. But if your requirement is to abort the job while read from Sequential file, where the reject are more then 10%, then you have to do that before the job starts.

You can call that as a Preprocessing.
Regards,
Shree
785-816-0728
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Meaning, you'll need to get the count of records in the file before the job runs and pass that count in as a parameter so you can compute what 10% of that is at runtime... that or pass in the 10% threshold and use it directly.
-craig

"You can never have too many knives" -- Logan Nine Fingers
praburaj
Premium Member
Premium Member
Posts: 133
Joined: Thu Jun 30, 2005 1:26 am
Location: philippines

Post by praburaj »

yes!I need to do like that. Plz help me how to do this?
prabakaran.v
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

A Sequence job could use an Execute Command stage to count the number of lines in the file using wc -l and the result could then be passed as a job parameter to your processing job. It might take a little tweaking - using Field() to separate the count from the filename in the output, if I recall correctly - but you can certainly make that approach work.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply