Performance issue
Posted: Tue Jan 11, 2005 9:13 am
Hi,
I have a job that uses 7 stages.4 DB2 UDB stages, 1 for source , 2 for reference links,1 hash file and 1 for target database.Input data is comming from 4 different sources into transformer and output of trasformer is give to an aggregator and from aggregator to target data base.I am grouping by two columns in aggregatorand calculating sum of another 2 columns.I ran this job for previous month and it took 1hr.I ran the same job for current month and it took 10hrs??????????
The number of rows in the source increased by 170,000 from previous month to this month and I see in performance statistics as 112 rows/sec for all the links.
935004 rows read from stream Link
935004 rows read from Reference Link1
935004 rows read from Reference Link2
744255 rows read from Reference Link3(hash File)
744255 rows written to Aggregator
730890 rows written to Target
I am already using user defined SQL to filter rows in source itself to improve performance.Can Anybody see any reason why this job is taking so long?
Plz respond to me if this make sense to anybody!
I have a job that uses 7 stages.4 DB2 UDB stages, 1 for source , 2 for reference links,1 hash file and 1 for target database.Input data is comming from 4 different sources into transformer and output of trasformer is give to an aggregator and from aggregator to target data base.I am grouping by two columns in aggregatorand calculating sum of another 2 columns.I ran this job for previous month and it took 1hr.I ran the same job for current month and it took 10hrs??????????
The number of rows in the source increased by 170,000 from previous month to this month and I see in performance statistics as 112 rows/sec for all the links.
935004 rows read from stream Link
935004 rows read from Reference Link1
935004 rows read from Reference Link2
744255 rows read from Reference Link3(hash File)
744255 rows written to Aggregator
730890 rows written to Target
I am already using user defined SQL to filter rows in source itself to improve performance.Can Anybody see any reason why this job is taking so long?
Plz respond to me if this make sense to anybody!