DSXchange

Hi All,

Iam working on 8.7 version ISD jobs.

When we use lookup stage reference(Normal ), does for every request to service the data gets refreshed and loaded to memory or is it one time load ?

Also , how does the behaviour of aggregator stages work if the data is broke down to packets before sending to ISD input ? dont we get wrong results ?

Thank you.

Vivek

1. Use only "sparse" lookups for ISD based Jobs. Otherwise you will not get updates...there is no refresh, and you wouldn't want one because that could take a long time.

2. The aggregator works fine because it is all based on "end of wave"....meaning that each "request" is like a new job set....the aggregator will release its rows (assuming you are not telling it about sorting, etc.) at the end of each "request's" set of data (which could be a whole array), as though it was a batch job that was finishing.

The only exception I've seen to this behavior is @INROWNUM, which simply continues to increment for the life of the job instance. Perhaps @OUTROWNUM does the same thing, although I haven't tried it.

Ernie

Thanks Ernie for information.

Ernie ,

I have a question , When we are dealing with say 100k of records , does sparse lookup doesn't effect performance ?

Thank you.

Probably not, because an ISD job is probably only performing one lookup; the performance difference will be minor.