Web Service

yijuanl · Post by **yijuanl** » Fri Jun 03, 2011 12:01 pm

Hi,
I have a web-enable job. In the Oracle Stage, its output link has 11003 records. When I run it in Datastage job, I get the error: dspipe_wait(7367): Writer timed out waiting for Reader to connect.

I deploy it as a web service in ISD. When the browser calls it, in Director, no error occurs. However in the web page, I only get 800 records instead of 11003.

I change the Free Buffer Run as '100', then I got 1017 reocrds in the web page.

What should I do in order to get all the data displaying in the web page.

Thanks!
Jane Liu

eostic · Post by **eostic** » Fri Jun 03, 2011 1:44 pm

Check to see if it is a volume issue and not an environment or Oracle issue......send the exact same data to a flat file.

...btw....what binding are you using? I will assume SOAP or XML for now...

Write another Job that reads from that flat file and goes to ISDoutput. Publish it as a Service.

I will assume you are already using the "return array" check box to ensure that that you will send back multiple rows.

Try a variety of client tools. Some of them may have limits on how much data they can receive via SOAP or REST/XML.

Ernie

yijuanl · Post by **yijuanl** » Mon Jun 06, 2011 8:18 am

Thank you so much for your quick response, Eostic!

I added the flat file into the job. The number of the records in the flat file is what I expect.

Yes, my binding is SOAP and XML.

Please advice!

Thanks
Jane

eostic · Post by **eostic** » Mon Jun 06, 2011 11:32 am

There is more to do per my post above....

What happened when you design (and publish) a new Job that sends the data from the new target flat file, into ISDoutput ? [no oracle stages].

What client tool are you using to test? I suggest Actional Diagnostics or SOAPUI.

Ernie

yijuanl · Post by **yijuanl** » Mon Jun 06, 2011 1:42 pm

Now I have two jobs.
One is oracle stage -> Transformer -> Sequential File.
Another one is Sequential File -> WISDOutput

The number of the records in Sequencial File is 10,997 that is the same as oracle stage output link.

But the number of the records from XML is 709. I still get less data through ISD than in Datastage Server.

Please advice!
Thanks in advance!
Jane

eostic · Post by **eostic** » Mon Jun 06, 2011 4:02 pm

That's a pretty big payload for a SOAP envelope.

You mentioned SOAP and XML..... did you mean "SOAP over HTTP?"

What client tool are you using?

Ernie

eostic · Post by **eostic** » Mon Jun 06, 2011 4:04 pm

reading your first post again....what do you mean by "in the web page" ? are you using a mashup tool and the REST binding?

If you are using the REST binding, then try running the service directly from the address line of your browser....something like:

http://yourserver:9080/wisd-rest/yourap ... rOperation

If you are using SOAP over HTTP, download Actional Diagnostics or SOAPUI from their respective sites. Great tools that will let you see what is being returned.

Ernie

yijuanl · Post by **yijuanl** » Mon Jun 06, 2011 6:17 pm

In ISD, the bindings that I make are SOAP over HTTP and REST.

In Operation, I use XML or JSON. If I define it as XML Or JSON, I will use browser to test my web service and save it as a text file. I also use SOAPUI to test them. None of them returns the number of records that I expect.

Thanks!
Jane

eostic · Post by **eostic** » Tue Jun 07, 2011 5:50 am

Try from other clients. I've never seen a situation where rows are dropped by ISD. I've had client tools blow up because there is no buffering in ISD (ie...it sends everything), but always sends all the data. I can't be sure what's happening in your environment, but I'd start to poke around elsewhere also.

See if the client tools have their own heapsize issues. See if your network has issues for quantities of data.

And see where the break is.....it doesn't sound always consistent from your notes....one time it was 700's, another time in the 1000's..... Create a job that sends back only a single 5 byte column. How many rows then?

Ernie

yijuanl · Post by **yijuanl** » Thu Jun 09, 2011 12:17 pm

Can you tell me what clients are better to test Web Service?
Thanks
Jane

eostic · Post by **eostic** » Thu Jun 09, 2011 2:08 pm

As noted above, I'd suggest experimenting with Actional Diagnostics or perhaps SOAP UI, and also test from other client "machines" in your network, in case there are network issues or other such blocks.....

Also...how much data are you thinking about? The numbers we are talking about here are relatively low, but it raises the question about how much you are eventually planning on......SOAP is not exactly a good transport mechanism for large volumes.....

Ernie