DSXchange

How to improve the performance of Java Transformer ?
Java Transformer throughput is 1 row/sec. I have to process huge data.
I have tried few thing like

1.Partitioned data into multiple links and used multiple Java transformer.
2.Reducing Java Code to minimum.

But these thing havn't improved performance much, still throughput is 6 rows/sec?

Your help is hightly appreciated.

Not sure...could be a lot of things. Here are some questions and also things to consider:

a) What are you doing inside your code? Are you calling out to somewhere? performing i/o? Doing heavy calculations? doing your own lookups into an rdbms?

b) how many columns? are you going thru a loop for each of them? Compare the same logic to another job that uses your class and only sends in one column (or dramatically fewer, assuming your logic can be adapted), but performs the same functionality. Yoiu may benefit by passing in one single large concatenated column.

c) Is this a Java Transformer? (input link and output link?)

d) What release, and are you at 9.1 or considering it soon? The 9.1 Java Integration Stage improves the performance architecture of calling java classes from within DataStage.....it is more like Connectors and vastly improves the performance for row handling.

Ernie

Yes ,Calling Address Doctor

Thanks

Might be worth timing a call to Address Doctor in isolation. If this takes close to 1 second there's clearly nothing you can do about the throughput of the Java Transformer stage that makes the same call.

...and also doing a test with a class that merely does a simple transformation, such as raising a string column's value to upper case....then compare that.

Ernie