DSXchange

qt_ky wrote:A normal Lookup is doing something more behind the scenes that does not appear to be documented--as it reads data from the reference link, it writes it to some sort of memory mapped file that may or may not have an index built on the key columns (I don't know), and then finally it reads that file into memory.

Your recollection is correct. The reference data are loaded into a hash table in memory and an index to that is built. (That's why the Lookup stage is a composite operator: LUT_CreateTable and LUT_Lookup.) You can do this ahead of time by populating a Lookup File Set.

Aha! Perhaps an option for you... pre-build that before recycling the job?

And kids, don't try this one at home - have the 64bit version of PX installed everywhere EXCEPT production. Large lookups work fine up until that last critical elevation. That was a not fun situation to walk into.

Thank you, Ray. That gives us another avenue to explore.

Must the server engine's uvconfig file be modified from the default of:
64BIT_FILES 0
to:
64BIT_FILES 1
and regenerated/restarted to enable 64-bit hashed file capability?

Craig, could you elaborate or post a link to your reference about the 64bit version of PX? I'm not sure what you meant by that.

First answer: NO. That forces every hashed file in the project to be 64bit. You just need to pre-create a hashed file as 64bit, I don't recall the syntax for that but it has been mentioned here and I'm sure it's in the docs somewhere. Ah... discussed here.

I just meant that the Production environment there was running the 32bit version of PX but 64bit everywhere else. Bottom line was things like lookups and their "memory maps" that ran fine in all of the lower environments could fail in production unless they handled the partitioning more better.