various parameters which affect the complexity of the job

ds123 · Post by **ds123** » Wed May 31, 2006 12:33 am

Hi ,

if we list the various parameters which affect the complexity of the job they are as follows:

1) number of rows to be processed

2)complexity of the formulas

3)number of look ups in the job

4)number of source tables

and 5) if any additional functionalities like mail notification ,purging of datasets etc..

Are the above parameters right ,

if there are any more parameters to be considered please let me know,

Thanks in advance

ArndW · Post by **ArndW** » Wed May 31, 2006 12:40 am

With PX jobs you have parallelism which might make jobs more complex (I'm not sure if you mean design-time run run-time complexity); with repartitioning and sorting. The same applies for aggregations and similar actions within the job. Job design spends a lot of time analyzing source data - that always takes longer than expected. Then the performance tuning part on the output side usually occupies quite a bit of time as well.

ds123 · Post by **ds123** » Wed May 31, 2006 12:44 am

ArndW wrote:With PX jobs you have parallelism which might make jobs more complex (I'm not sure if you mean design-time run run-time complexity); with repartitioning and sorting. The same applies for aggregations and similar actions within the job. Job design spends a lot of time analyzing source data - that always takes longer than expected. Then the performance tuning part on the output side usually occupies quite a bit of time as well.

Hi ArndW,

thanks for the reply

its the design time complexity,

here we are not taking performance tuning part ,which can be taken up later

Its basically the design time complexity...

like the question is

apart from the following parameters are there any other parameters to be considered:

1) number of rows to be processed

2)complexity of the formulas in tarnsformation

3)number of look ups in the job

4)number of source tables

and 5) if any additional functionalities like mail notification ,purging of datasets etc..

if so please let me know

Regards,

ray.wurlod · Post by **ray.wurlod** » Wed May 31, 2006 1:25 am

Don't forget the user requirements which, if you don't get signed agreement, will change just before you are ready to go into production. Even if you have signed agreement they'll still change, but now you are in a proper software development life cycle and can oblige them to submit a formal change request including justification.

The only time you don't have to worry about the goalposts shifting is when they haven't even worked out what game they're playing.