ISD with a Web services enabled Datastage Job

Dedicated to DataStage and DataStage TX editions featuring IBM<sup>®</sup> Service-Oriented Architectures.

Moderators: chulett, rschirm

Post Reply
yijuanl
Participant
Posts: 23
Joined: Fri Jun 24, 2005 8:32 am

ISD with a Web services enabled Datastage Job

Post by yijuanl »

I have created a WS enabled job in DS with Oracle stage as input stage, transformor and WSDL output with multiinstances and web enabled checked. And I compiled and ran it sucessfully in DS. Then In ISD, I connect to my datastage server without any problem then create a project -Aplication -service -binding as HTTP SOAP and REST - operation to linked to the DS job. But when I run the URL: http://servername:port/wisd-rest/[applicationName]/[serviceName]/[operationName] to test my work, I got error 500 like this: Caused by: javax.ejb.EJBException: nested exception is: java.rmi.UnmarshalException: error unmarshalling return; nested exception is: java.io.EOFException

Please help?
Thanks a lot in advance.
Jane Liu
yl
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Hard to say.....it's a fairly generic error. Here are some things to try:

Did the Job abort? Is there anything in the DS Director log?

Are you passing any Job Parameters? [is there any input message for this service?]....if so, they would need to be passed via url when using REST.

Get a copy of SOAPui, or Actional Soap Tester (both free or trial SOAP testing tools)....see what happens whey you try to use the SOAP over HTTP binding. REST is a slightly more complex binding to use, albeit simpler to invoke once it is set up correctly.

What are the data types in the Oracle table? Try it with just one very simple "varchar" column.

Check out the setting for pad characters (I can't recall the variable name at the moment, but it's something like APT_xxxxPAD_CHAR)...be sure it is set to a single blank.

Try testing with a Server Job. For a job this small and simple, and not "always on", it may have a faster execution time anyway.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
yijuanl
Participant
Posts: 23
Joined: Fri Jun 24, 2005 8:32 am

Post by yijuanl »

Thank you so much for your quick reply!
1) The job is not aborted when I ran it in DS and in Soapui and in browser.
2) I do not pass any parameters in the job.
3) When I ran it in SoapUI, I got an error like "javax.ejb.EJBException: nested exception is: javax.transaction.TransactionRolledbackException: CORBA TRANSACTION_ROLLEDBACK"
4) The job I created is a server job, there are three colums: empid (varchar(11)), fullname(varchar(100)) and firstname(varchar(30)) and last name (varchar(30)).
5) And I do not know where to check the setting for Pad Characters.

Thanks again!
Jane
yl
yijuanl
Participant
Posts: 23
Joined: Fri Jun 24, 2005 8:32 am

Post by yijuanl »

I modifed my DS server job query with 4 columes and two rows returned. It works with the testing in SOAPUI and REST URL. But when I changed the query to return around 7000 records in my DS job, I got the same error as I indecated before.

Any idea?
Tahnk you so much for your help!
Jane
yl
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

It's possible it's a volume issue, or it could also be a particular data item issue... (binary zeros or other strange characters have been known to blow up the SOAP processing).....

On another note, SOAP is probably not a good vehicle for returning a large quantity of rows. Try to qualify them in some fashion. 7000 doesn't seem like a lot, so I'm a bit surprised that it would blow up "yet", but it could depend on the size and structure of the columns, their lengths, etc.

I'd probably do testing with some "other" 7000 rows...perhaps from another table to see if it might be a data "value" issue.

These things aren't particularly easy to debug.

Be sure also to get your jobs working perfectly, writing to a flat file or similar, before ever deploying them as a service.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
yijuanl
Participant
Posts: 23
Joined: Fri Jun 24, 2005 8:32 am

Post by yijuanl »

Thank you very much for your advice!
1) I started with 100 rows returned and I did not get any error in REST URL.
2) When I jumped to 715 rows, I got the same error as indicated before. Then I go back to 100 rows -> 150 rows -> 200 rows ... -> 700 rows -> 715 rows this time I did not get any error
3) Then I jumped to 819 returned rows, I got error.
4) Now again I came back to 715 rows, without error. Then I increase the volumes little by little. When I hit 819 rows I did not get any error.
This pattern makes the debug really hard.
Any idea?
Thanks in advance!
Jane
yl
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Change the model of your execution..... retrieve only high level keys....then based on a selection from there, drill down further....then drill further...create 3 or 4 different REST services that are built for drilling down, so that none of them ever return a huge number of rows. Also consider reducing the size of the payload.....make the column names tiny (c1, c2, c3) and explore the use of custom style sheets to display them (or your front end application that is receiving them)...or have fewer columns, or shrink your columns (for instance, if many of them are long CHARs, you might be sending lots of worthless blanks if it is an EE job.....).

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply