Working with XML Output and Web Services Transformer

Dedicated to DataStage and DataStage TX editions featuring IBM<sup>®</sup> Service-Oriented Architectures.

Moderators: chulett, rschirm

Post Reply
dganeshm
Premium Member
Premium Member
Posts: 91
Joined: Tue Aug 11, 2009 3:26 pm

Working with XML Output and Web Services Transformer

Post by dganeshm »

Hi All,

I am loading data into IBM Maximo via web services transformer using WSDL call. In the current design if there are 100 records , it makes 100 calls to Web service and there by belting the server 100 times. (100 is just an example , records go as high as 2 million). So in those cases its really taxing the server.

So what I tried is to consolidate all records in a single xml and make just one call to the Web Service and load it to server. When I tried about 100 records it worked smoothly , but when I ran it for 7000 records the job is in hung state , still trying to load into MAXIMO. Not a DataStage issue but a MAXIMO limitation. What I currently want to do is to limit the number of records in a XML , so is there a way to limit it while creating XML from the XML output , if there are 1000 records , then 2 XMLs should be generated with 500 records in each of them.

Any help would be greatly appreciated.
Regards,
Ganesh
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Hi Ganesh....

Great work so far....often times sites don't have a web service that supports the sending of an "array" of rows --- fortunately your Maximo target appears to suppor that just fine.....smart idea also to break it up....even if Maximo didn't have a limit, you might eventually bump into DataStage's own upper limits............

Take a look at the "trigger" capability on the xml output stage. This allows you to select a column on the input (it doesn't have to be a column that will appear in your xml document) that "triggers" the release of XML on the output link. You will have to have several counters for the rows in an upstream stage, one that counts up to 500 and another that counts "units" of 500. Ultimately, that second counter will become your "trigger."

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
dganeshm
Premium Member
Premium Member
Posts: 91
Joined: Tue Aug 11, 2009 3:26 pm

Post by dganeshm »

Hi Ernie,

That worked like a charm , although instead of using two counters , I did a mod (@INROWNUM,500) = 0 then count +1 else count , to keep track of every 500 rows.. Thanks again Ernie..
Regards,
Ganesh
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Nice! ; )
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply