Optimal Heap Space Allocation

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
ian.e.armbruster
Participant
Posts: 2
Joined: Tue Jul 01, 2008 7:23 am

Optimal Heap Space Allocation

Post by ian.e.armbruster »

Hi all,

Currently we're running into a problem with heap space allocation, which we know how to fix. That being said we're now trying to find what the optimal space to allocate would be. My question is, when heap space is allocated,

Is each job then given that set amount of space? or is the total amount of space given to all processes running at one time?

Or, if each job recieves that allocation individually, does it take the full allocation each run, or is it taken only on an as needed basis?

I ask because currently we have 650+ sequences, and if we get many of these running at the same time and they are each taking the full allocation regardless of need, we're going to exceed our capacity very quickly.

Any input would be useful, please let me know if I've left anything out.
ray.wurlod
Participant
Posts: 54595
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Welcome aboard.

Each operator in each job takes only what it needs. You do not allocate any space; the requests are built in to the operators.

Nothing in DataStage takes "the full allocation regardless of need". Therefore you already have optimal allocation.

Your issue is one of total load being too great.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
ian.e.armbruster
Participant
Posts: 2
Joined: Tue Jul 01, 2008 7:23 am

Post by ian.e.armbruster »

Thanks ray,
Post Reply