eliminate certain records

[BSG1]

Hi all,

I have a data set file which has few key column records only. Now my requirement is process only those records from input in hich key is not matching to the key column in dataset. Lookup can't help. any other idea on how to accomplish it in job design? Input is Oracle table and records ot be rejected are in dataset.

[chulett]

Lookup can't help.

Why is that?

[BSG1]

Not able to fins how lookup would help. Dataset file has key1 which are to be rejected. source has all the key1 columns. Now if I use lookup with lookup from dataset file, what properties and condition to use to get unmatched records get to output?

[BSG1]

I think it can work. I should capture all unmatched record to reject link and use that reject link for further processing.

[SURA]

Yes you got it! It will work.

Consider the volume of the data. If it is huge, you can try some other stage.

[BSG1]

data at reference link is very less like few hundreds so everything is going to go to reject only. What other options i have?

[FranklinE]

Looks like a job for negative logic, something I try to avoid if at all possible.

A constraint is "open" where lookup key matching is not. I've seen reject links used for scenarios like yours, but a "negative" constraint using not-equal (<>) is more direct, and also let's you use more than one constraint if your data is more complex than just the one key.

[BSG1]

negative constraint is not working. I tried picking column values to put in lookup condition for <> but it gives column from source only and not from reference link.

[FranklinE]

Sorry, I meant my suggestion to be for a stage other than Lookup. Try a transformer.