Lookup Space

pradeep_nov18 · Post by **pradeep_nov18** » Sat Apr 17, 2010 2:33 am

Hi All

How to resolve the issue.

lookuptable.20100417.stsv13d": No space left on device.

Can we do any tuning in the job.since we cannot increase the resource space for this job alone

This job is having single lookup holding 5 reference.

Thanks for your help

keshav0307 · Post by **keshav0307** » Sat Apr 17, 2010 4:39 am

what are the reference link sources?
if they are tables, try sparse lookup.
Else try join

chulett · Post by **chulett** » Sat Apr 17, 2010 7:16 am

If you are running on multiple nodes, try hash partitioning your input stream and lookup data, doing that properly will minimize the space it requires.

ray.wurlod · Post by **ray.wurlod** » Sat Apr 17, 2010 4:08 pm

There is probably a mechanism for loading only the rows you will need into the lookup table.

The technique is known as "bucket table": you generate a list of the keys from source, then join that result to the reference source as part of loading the lookup table.

In the old days, the bucket table was actually created in the database where the reference data originated, and the join done there. In parallel jobs, of course, you can use a Join stage to good effect.