Page 1 of 1
Help with DS.TOOLS Option 6
Posted: Wed Dec 01, 2004 9:30 pm
by chulett
6. Adjust job tunable properties
Looking for an explanation of the Aggregator stage properties I can change with this option. Defaults:
Code: Select all
1 = ReportAfterRows 100
2 = ReportAfterTime 15
3 = TableSize 8192
4 = ResizeAt 2
5 = ResizeBy 8
6 = Resize2Threshold 524288
Is there a DataStage / Universe pdf that explains how these work so I can decide if I can make any meaningful changes to them? Thanks!
Posted: Thu Dec 02, 2004 3:01 pm
by ray.wurlod
The short answers are:
(1) you can change any of the values on the list
(2) it's not documented anywhere (except in an advanced training class I wrote years ago but was canned by Informix USA under the "not invented here" rule)
The first two options are about forcing the Aggregator stage to update its status so that row counts can be monitored. Without these, the Aggregator (with unsorted input particularly) would just sit there gobbling up rows until they were all in, then blast the result rows out in a flatulent burst.
The remaining four are about memory management for the Aggregator stage. The TableSize (8192 bytes) figure is the initial amount of memory allocated by the Aggregator stage. This memory stores a set of rows corresponding to the output row definition; each grouping value and each aggregating value is represented.
As more and more rows arrive with different grouping values, more memory is required. If the input is sorted, a change in value of one of the sorted columns can trigger output of some rows, with that memory able to be freed.
Otherwise, there comes a point when the initial allocation of memory has all been consumed. At this point, the allocation is increased by multiplying its current size by the ResizeAt figure. So, by default, when the initial 8KB has been used, another 8KB is allocated; when that 16KB has been used another 16KB has been allocated.
And so on, until the size specified in the Resize2Threshold has been reached. When this 512KB has been used, the other multiplier (the ResizeBy figure is used), so the allocation is increased to 4MB.
The options are not really well named in my opinion.
Posted: Thu Dec 02, 2004 3:09 pm
by chulett
Excellent! Exactly the information I was looking for... thanks for taking to time to post all that here. :D