DSXchange

Sorry, but nobody mentioned stage variables. Read Jackie's post again.

Me, no idea. No Windows server = no MS OLEDB stage to even look at.

Permissions issue, it seems. Google for that message.

Ah... was wondering about the job handle. Still better to leverage DSPrepareJob IMHO.

(hey, it was 6AM and the coffee hadn't kicked in yet)

That's not huge. You need to determine your bottleneck, have you broken your job into logical pieces to see where the actual problem is? For example, replace Oracle with a flat file and check the speed.

Don't see anything obviously wrong.

Look into using DSPrepareJob instead of all that IF-THEN checking though, one stop shopping.

Sure and it has been discussed here a zillion times. Kim Duke has a complete set of utilities for doing this (and much more) freely available from his website - search there and here for discussions of ETLStats. Might be a little overwhelming, though. For just the requirements as you've stated, you'...

I guess so, because if you speed the job up you can say "Hello performance!".

I'd noticed the same thing - this somehow seems to be a Windows issue.

Simply build the expression using the "..." ellipsis, it will walk you thru it. Basically it would be: Activity Variable = Job Parameter.

Okay... has anyone looked into doing all that in a single query? Should be perfectly feasible.

Your "COUNTER" can simply be the $Counter variable from the Start Loop stage. How is "INDEX" defined - job parameter, hard-coded, user variable, ???

The 'static files' comment would be a Server merge operation.

Seeing as how there are two separate options for logging, I would think they are not mutually exclusive - it's just that they are typically used in that manner. So I would therefor think you could have both if you really wanted and could take the 'performance hit' that I assume that would bring. You...

If these "five large tables" are all in the same database then your join is a sql join in the source db stage. Otherwise, yes a hashed file can be used to effect a join. That 2GB limit is only for 32bit (default) hashed files, they can be created as 64bit so there is no such limit. Would n...