XML data in an XML tag as input to Web Service

Dedicated to DataStage and DataStage TX editions featuring IBM<sup>®</sup> Service-Oriented Architectures.

Moderators: chulett, rschirm

Post Reply
kennyapril
Participant
Posts: 248
Joined: Fri Jul 30, 2010 9:04 am

XML data in an XML tag as input to Web Service

Post by kennyapril »

Hello,

Have a question regarding the XML format of Web Service

As of now our source is as below through a table
Name(char)
Entity(integer)
Account(integer)
Tran_CLOB(XML data)
Prod_cd
Prod_type


Now we are planning to build a web service with the same fields but as WISD input and WISD output as below

WISD input

<NAME>John</Name>
<Entity>123</Entity>
<Account>1234</Account>
<Tran_clob> has all the transactions in XML format inside this around 10,000 as separate records</Tran_clob>
<Prod_cd>CD</Prod_cd>
<Prod_type>K</Prod_type>

So the above data will be the input to WISD and we parse the XML , do the transformations and insert these records into table and send error report to WISD output.

Is it a good option to follow this procedure in ETL to insert around 10,000 records on the fly when ever a new request is sent to the web service...

Thank you!
Regards,
Kenny
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

That would depend entirely on the web service and how it was designed.
-craig

"You can never have too many knives" -- Logan Nine Fingers
kennyapril
Participant
Posts: 248
Joined: Fri Jul 30, 2010 9:04 am

Post by kennyapril »

The web service will be designed with close to 20 stages with medium transformations and inserted into tables. Each request we expect 10,000 records in the XML tag.
Regards,
Kenny
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

That's a lot of data in a single soap request. But functionally, it should be fine.
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
kennyapril
Participant
Posts: 248
Joined: Fri Jul 30, 2010 9:04 am

Post by kennyapril »

Actually this was built through .net and we had issues when inserting records.
Now our plan is .net will trigger the ISD which will insert the records and send an error message to users.
Regards,
Kenny
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

As noted above, functionally it should be fine. You will receive a large variable "chunk" of xml in that clob (just make the column a large varchar) and then push it into an xmlInput or xml stage to parse it all out and do what you need to.....

A "more formal" way to do it would be to just code the Job for one single row and then tell ISD to make it into an "array". Then the calling application will let the underlying SOAP engine take care of packaging it correctly according to the wsdl that defines the incoming payload........but if the calling application is already written, that may not be an option.

Test it well.....that much data on the wire could lead to reliability issues, and also be sure that you aren't getting any truncation within the ds job after ISD receives it.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply