Passing repetitive elements from SOAP request to ISD stage

[Gopi Krishna]

Hi All,

In My current project we are calling the web service methods from Datastage which are implemented in JAVA.But before calling the webservice method i need to standardize the data which is coming as part of the SOAP request and then need to pass the standardized data to the web service method to load the data to tables.More over i need to implement this entire process in Real time. So we have used service director to make the job run on 24 X 7 basis.

My job structure is as follows:

ISD Input Stage(To read values in the request)----- > Standardize stage(Applying standardize rules)-----> trnasformr(filtering out necessary columns) ---- > xml output stage(Converting back to XML to call Web service method) ----- > webservice transformer(to call the java web methods) ----- > ISD Output stage(Sending response back)



I added this job to the service director and taken out the WSDL. and the WSDL is as follows.I used the SOAP over HTTP service while adding the job in Service Director.


<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soap="http://PIMServ1.PIM1.isd.ibm.com/soapoverhttp/">
<soapenv:Header/>
<soapenv:Body>
<soap:Dept>
<deptID>?</deptID>
<employeeID>?</employeeID>
<employeeStatus>?</employeeStatus>
<employeeName>?</employeeName>
<identifier>?</identifier>
<telnumber>?</telnumber>
<email>?</email>
</soap:Dept>
</soapenv:Body>
</soapenv:Envelope>

If i pass the data with out repetitive elements then the job is working fine. But as per my requirement a single employee may have multiple phone numbers and email ids. I need to design a job in such a way that it should handle multiple emails and phone numbers as below.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soap="http://PIMServ1.PIM1.isd.ibm.com/soapoverhttp/">
<soapenv:Header/>
<soapenv:Body>
<soap:Dept>
<deptID>?</deptID>
<employeeID>?</employeeID>
<employeeStatus>?</employeeStatus>
<employeeName>?</employeeName>
<identifier>?</identifier>
<telnumber>?</telnumber>
<telnumber>?</telnumber>
<telnumber>?</telnumber>
<email>?</email>
<email>?</email>
<email>?</email>
</soap:Dept>
</soapenv:Body>
</soapenv:Envelope>

I tried with accept array option which is available in service director while adding the job but it is not working properly to my requirement. if i pass 3 email ids as part of the request,ISD stage is taking the last entered email id and preparing the single record with all the other columns to process. but in my case if i pass 3 email ids the source request has to convert into 3 records first and on those 3 records i will apply standardize rules and then after i will prepare an XML with all the 3 records to call webservice methos to load the data.

Can anybody suggest me how to achieve it? do i need to change the job design or are there any options available in Service Director to solve this.

Thanks !

[eostic]

Set it up as an array in ISD (check box) and then YOU GIVE THEM (the developers writing the client app) the WSDL. It's that simple. Don't try to conform to some other wsdl --- just share the wsdl that ISD creates for you.

Ernie

[eostic]

Looking at this and comparing to your other threads... it's more clear now what you are trying to do.

ISD will easily support an array.....but in your example...it's not an array ---- it is "two" arrays. I suspect, based on the xml structure you've shown, that the emails and telephone numbers are separate lists?

Before we talk about possible techniques, let's confirm the structure....is the first telephone number related to the first email? ...or could there be 2 phone numbers and 5 emails or other unrelated combinations of email/phone numbers in the sets?

Ernie

[Gopi Krishna]

Hi Ernie

Thanks for the response.

Right now I don't have Premium Membership so I am not able to see what you are saying here. Can anybody help me in getting this.

Thanks !

[ray.wurlod]

There's a link on the DSXchange Home page to a place where you can get "this" (premium membership).

[Gopi Krishna]

Thanks Ray !

[Gopi Krishna]

Hi Ernie,

Thanks for the response.

As you said, there could be 2 phone numbers and 5 emails or other unrelated combinations of email/phone numbers in the sets. But all those emails and phone numbers will belong to one single organization. At a time we will have one single organization information in the request.The basic concept behind this is one organization may have multiple phone numbers and multiple emails. They have already designed the SOAP request like i said in my post. Now I have to read the data from the SOAP request to standardize the data. To standardize the data I have designed a job and added it to service director for this. But the WSDL which i created from the service director that is not able to handle multiple elements. need to design a wsdl which can handle multiple elements.

Thanks !

[eostic]

The WSDL generated from ISD is purely relational....it can have many rows (an array sent up or returned), but it's just "rows", as with any ETL.

There are two alternatives to you.....option 1, which is the simplest, is to build two services. One for emails and one for phone numbers. ISD is great for fine-grained services. You will have the jobs and services built very quickly and can do whatever transformation you need to on those to validate them or whatever it is that might be required.....then have the java clients do the coordination of those two services. They will use the two WSDL Operations that you create for each of those, instead of the one they are using today.

The second option is to code the ISD service to handle the WSDL that they are currently using. This is very do-able but it involves a whole lot of other skills.

You can use the payload generated from any other WSDL just fine, but the burden of parsing it becomes yours --- ISD will just pass it thru.

Get yourself VERY familiar with the SOAP payload that the client tools will be sending. Use a tool like SOAPui and expand the service with some data (SOAPui will let you do this even if there is no service yet). Check out the whole soap "envelope". Save it to disk. Write jobs that parse that SOAP envelope via the xml stages (reading content from a single large column [pasting the envelope into an upstream Derivation is usually the easiest way to prototype this]) and pull out the things that you need or want to transform.

Study the "return" envelope that is expected by the client tools. Get familiar with that. You will have to build it yourself. Practice using the xml stages to build that return. Learn how to have the xml stages send out a single large column downstream.

Put it all together in a Job that reads from the SOAP envelope and writes out a final xml that matches the return envelope.

Then put it together with ISD Stages, where the ISDInput Stage sends a single large varchar into the downstream stages, and the ISDOutput Stage accepts a single large varchar.

Now practice with the Text over HTTP binding of ISD. This was designed explicitly for this purpose. With Text over HTTP, ISD simply "passes" the whole envelope into DataStage where you can do whatever you need to ....when you send the response, ISD just forwards that response back to the client.

Any kind of web service can be used, but the work to parse and build the SOAP envelopes is yours. You need to learn about the xml stages, techniques for capturing soap envelopes, the Text over HTTP binding, and how to push strings around and into/out-of the xml stages.

There are pros and cons to both approaches.

Ernie