WSDI Problem with service with more than one column

Dedicated to DataStage and DataStage TX editions featuring IBM<sup>®</sup> Service-Oriented Architectures.

Moderators: chulett, rschirm

Post Reply
MT
Premium Member
Premium Member
Posts: 198
Joined: Fri Mar 09, 2007 3:51 am

WSDI Problem with service with more than one column

Post by MT »

Hi,
we created two WSDI services.

1)
One service is just for testing. For this service only one operation is implemented requesting and returning only one column. A job using this service is working and there are no problems importing its definition to a Webservice Client Stage via the Webservice Meta Data Importer.


2)
The other service is the one that should be implemented. For this service more than one operation is implemented. The operations are requesting and returning more than one column. For testing this service only one operation was activeted.

Input and output columns of the activated operation are grouped into a structure. Trying to import the definitions of these service/operation returns the following message:

"Cannot complete the mapping automatically. The messagy may define some recursive constructs. You need to procedd to a manual mapping using the XML Meta Data Importer."

We tried the solution suggest by viewtopic.php?t=137003
to import it meta data.

An import of the service/operation and its definitions to the Webservice Client Stage with the XML Meta Data Importer is possible with this workaround. We set up the stage as mentioned in the post. Trying to run the job using this WS client is not succesful. The jobs aborts with the following error message in the Director log:

Code: Select all

TestWS01..SF_WSD01WSDI.IDENT1: com.ascentialsoftware.wsclient.ServiceInvocationException: Service invocation exception: 

at com.ascentialsoftware.wsclient.WebService.callService(WebService.java:1602)

at com.ascentialsoftware.wsclient.WebService.callService(WebService.java:1544)

at com.ascentialsoftware.wspack.Transformer.process(Transformer.java:104)

at com.ascentialsoftware.jds.StageFactory.runStage(StageFactory.java:544)

at com.ascentialsoftware.jds.StageFactory.runSource(StageFactory.java:335)

Caused by: java.lang.NullPointerException

at com.ascentialsoftware.wsclient.xml.DerivationParser.match(DerivationParser.java:622)

at com.ascentialsoftware.wsclient.xml.DerivationParser.element(DerivationParser.java:367)

at com.ascentialsoftware.wsclient.xml.DerivationParser.segment(DerivationParser.java:271)

at com.ascentialsoftware.wsclient.xml.DerivationParser.segment(DerivationParser.java:293)

at com.ascentialsoftware.wsclient.xml.DerivationParser.segment(DerivationParser.java:293)

at com.ascentialsoftware.wsclient.xml.DerivationParser.segment(DerivationParser.java:208)

at com.ascentialsoftware.wsclient.xml.DerivationParser.expr(DerivationParser.java:171)

at com.ascentialsoftware.wsclient.xml.DerivationParser.parse(DerivationParser.java:138)

at com.ascentialsoftware.wsclient.xml.TreeBuilder.compileRules(TreeBuilder.java:84)

at com.ascentialsoftware.wsclient.MappingHandler.fromArray(MappingHandler.java:304)

at com.ascentialsoftware.wsclient.InputRequest.buildRequestBodyDocument(InputRequest.java:908)

at com.ascentialsoftware.wsclient.WebService.callServiceInternal(WebService.java:1625)

at com.ascentialsoftware.wsclient.WebService.callService(WebService.java:1591)

... 4 more

Thanks in advance for any help

best regards
Michael
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

The problem is not with ISD....it's with Web Services Pack. The WSPack can't easily handle services with arrays. There are ways to do it, but for the moment I will assume that your real future client is a java or .NET application somewhere that wants to use your ISD DataStage or QualityStage based service.

Test it with a tool like SOAPui, or Actional's Soap Tester (both easily found via google or other search). They understand arrays and will easily be able to import and test your service.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
MT
Premium Member
Premium Member
Posts: 198
Joined: Fri Mar 09, 2007 3:51 am

Post by MT »

eostic wrote:The problem is not with ISD....it's with Web Services Pack. The WSPack can't easily handle services with arrays. There are ways to do it, but for the moment I will assume that your real future client is a java or .NET application somewhere that wants to use your ISD DataStage or QualityStage based service.
...
Ernie
Hi Ernie,

thanks for your answer.
We have these things running in DS 7.5 and RTI and are now
testing the upgrade to 8.1.
Do I understand you right Ernie that this functionality - which we had in 7.5 - is not working in DS 8.1.
This seems to be a known problem - at least by you...
Can we wait for a fix ?
Should we open a PMR?

Any recommendations?

Thanks in advance
Michael
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

No..... the WSPack (Web Service Tranformer or WS Client Stages) in DataStage has never supported arrays. It isn't able. There are ways to get around this, but the WSPack itself is happiest with single row in, single row out services.

All I'm doing is trying to separate the issues.

Issue number one, as I read it above in your original entry, is "how do you validate that ISD is working with the new service you created, that has multiple operations....some with multiple fields and rows in the request and the response"....

To test that, deploy the service as you have done, and use a tool like SOAPui to validate and test it. WSPack is not a good tool or facility for that.

Second is WSTransformer... A related SOA technology but not in any way connected to ISD. It is simply importing a WSDL and it can't handle the wsdl that ISD is generating. RTI generated much more primitive WSDL, so that may be one of the reasons this worked earlier. But either way, let's separate the two issues and review them independently.

Is your ISD service working as you'd expect from a typical industry testing tool?

If not...ok...let's look.

If so....then let's talk about how you want to use it...and why, if from DataStage with WSPack, or from elsewhere....and if it makes sense to call this web service from DataStage itself, explore how that could be done.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
MT
Premium Member
Premium Member
Posts: 198
Joined: Fri Mar 09, 2007 3:51 am

Post by MT »

eostic wrote:No..... the WSPack (Web Service Tranformer or WS Client Stages) in DataStage has never supported arrays. Ernie
Hi Ernie,
I do not want to argue with you about it but it simply worked with RTI and the WebService Pack in 7.5.3.
Could you please point me to the location where it is stated that this is not supported - because this is important for us and the customer....

We are trying to get soapUI installed in our current environment to test it.
And yes you are right the final consumer of the WebService is outside of DataStage but we have built a complete TestSuite within DataStage to test our WebServices and other processes which I do not want to give up.
In my eyes it is not an abstract wish to have DataStage being able to consume it own WebServices - it is just a must have and no one would ever think it is not supported....

Do you have any further recommendations for us?
I think this could be a point to stop the migration to DS 8.1 for the customer.....

best regards
Michael
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

ok...let's clarify then.....is your Operation returning "more than one column".....or is the Operation returning "more than one ROW"......

...it's the "more than one row" that is the crucial piece of this....although it wouldn't surprise if the new "structure" in ISD also causes WSPack a problem.

As noted earlier, RTI was "primitive" and frankly, not as standard, in the way it generated WSDL....

...but....in either case, WSPack is able to consume such services --- just not as "easily" as you might expect. So we'll help you get it to work. It just might take some massaging because WSPack hasn't evolved as far with the SOA standards as ISD.

For simply unit testing of ISD, I absolutely recommend that you use something like SOAPui --- so get that working....

Independent of that, I wouldn't want you to lose your test suite....but it might require some alterations to the Jobs. Contact me offline... we can exchange the WSDL and I'll take a closer look at what techniques are needed within WSPack.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply