Page 1 of 1

SCD detect if match was found for derivation

Posted: Thu Mar 26, 2020 12:44 pm
by thompsonp
I am using the SCD stage for a Type 2 slowly changing dimension.
One of the audit columns in the dimension is version number.
This should be set to 1 for the first occurence of a business key and then increment with each type 2 change.

The column is a Decimal(9,0) and not nullable.

I have not been able to specify a derivation in the SCD stage for this column that sets the frst record to 1.

The logic in the derivation needs to be "If no matching record found in dimension lookup then set to 1, else set to dimension lookup value + 1".

I tried testing the value from the lookup and also adding a found_flag column to the dimension lookup input which is nullable, but neither allows me to detect the absence of a match.

Is there a way to do this within the SCD stage?

Posted: Wed Apr 29, 2020 1:30 am
by ray.wurlod
Set up your Lookup stage with a reject link to capture the failed lookups, and hard-code the 1 somewhere on that stream before funnelling with the "lookup succeeded" stream.