Use trigger column in xml output stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
rchandra1
Participant
Posts: 9
Joined: Tue Jan 24, 2012 11:26 am
Location: Ohio

Use trigger column in xml output stage

Post by rchandra1 »

Hi, I am using webservice client to send data to target system from a source table, so i used xml output stage to aggregate and format the data into single xml record and then invoke service using WS client. Now, i ended up with out of memory issue when the source record count increases, so i tried single record option in the xml output stage, it is working fine. Then, i thought of using the using trigger column, came up with logic to chunk the record into 100 records and creating trigger column without / but i am getting fatal error as below from the xml output stage,

Code: Select all

xmlo_INSERTISSTOUCHES,0: Info: CopyOfws_InsertTouches2.ws_Call_InsertissTouches: 0029`:`[WebService] ---- XML response envelope ----
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 <soapenv:Body>
  <soapenv:Fault>
   <faultcode xmlns:ns1="http://xml.apache.org/axis/">ns1:HTTP</faultcode>
   <faultstring>(400)Bad Request</faultstring>
   <detail>
    <string>return code:  400
</string>
   </detail>
  </soapenv:Fault>
 </soapenv:Body>
</soapenv:Envelope>
and also

Code: Select all

APT_CombinedOperatorController,0: Fatal Error: Fatal: 0026`:`com.ascentialsoftware.wsclient.ServiceInvocationException: Service invocation exception: <ns1:Fault xmlns:ns1="http://schemas.xmlsoap.org/soap/envelope/">
   <faultcode xmlns:ns1="http://xml.apache.org/axis/">ns1:HTTP</faultcode>
   <faultstring>(400)Bad Request</faultstring>
   <detail>
    <string>return code:  400
</string>
   </detail>
  </ns1:Fault>
 at com.ascentialsoftware.wsclient.OperationCall.handleRemoteException(OperationCall.java:369)
 at com.ascentialsoftware.wsclient.OperationCall.invoke(OperationCall.java:274)
 at com.ascentialsoftware.wsclient.WebService.callServiceInternal(WebService.java:1685)
 at com.ascentialsoftware.wsclient.WebService.callService(WebService.java:1591)
 at com.ascentialsoftware.wsclient.WebService.callService(WebService.java:1544)
 at com.ascentialsoftware.wspack.Transformer.process(Transformer.java:104)
 at com.ascentialsoftware.jds.StageFactory.runStage(StageFactory.java:544)
 at com.ascentialsoftware.jds.StageFactory.runTarget(StageFactory.java:348)
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Check it again. Create a copy of you job and REMOVE the WSTransformer....replace it with a Sequential Stage......send ONLY the xmlContent column (your resulting column) down that link [longvarchar, length of something like 99999]. In the Sequential Stage, use delimeter NONE and quote character NONE.

Look at the resulting XML. Also note if you still get the error.

It's unlikely that the xmlOutput Stage would yield a SOAP fault error and an HTTP code.

Something else is probably going on.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

...by the way...can you please share any settings that you might have in your job regarding tracing or other such things? The message you are seeing, especially where it says something like "xml envelope", is typically part of the WSTransformer "tracing" option --- something that generally is only available via Server Jobs using this Stage.

Either way --- to debug this best, take away the WSTransformer entirely and see if you are still getting a 400 error.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
rchandra1
Participant
Posts: 9
Joined: Tue Jan 24, 2012 11:26 am
Location: Ohio

Post by rchandra1 »

Ernie, As u suggested i added sequential file and had the output of xml stage to it, it is working fine and creates the xml file as it needs to be with right name space and tags. Not sure why it fails when it is passed to WS client stage.
But it is always creates 3 segments (3 grouping/3 xml files) being source record as 200 or 300 or 3000, i used the logic to break at every 100 records using a variable and counter and used that variable as trigger column (varchar).
In WS client stage, in the stage tab under Error handling I have radio button checked for Fatal and In the input tab under General I have info radio button checked. have Log response as info
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Ok. Good. So it's not the xmlOutput Stage causing the issue. Just a thought....put a transformer in the middle, between the xmlOutput Stage and the WSTransformer. Maybe some old xpath in the Description is causing a conflict.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
rchandra1
Participant
Posts: 9
Joined: Tue Jan 24, 2012 11:26 am
Location: Ohio

Post by rchandra1 »

I tried with transformer, it is not working. one thing what i noticed is, i had trigger column calculated in transformer to change the trigger value for every 100 records, when i take the output of change value, i see only 3 changes of 222 records, but output from xml output stage is 5 grouping (I checked in flat file too), even i decided to live with 5 grouping it is failing on serivce call.
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

I'd probably break it out to debug it further. Take one of the xml documents and in a new job, put the WHOLE document in a Derivation in an upstream transformer, and then send THAT column to the WSTransformer...just to entirely isolate the debugging process.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

I'd probably break it out to debug it further. Take one of the xml documents and in a new job, put the WHOLE document in a Derivation in an upstream transformer, and then send THAT column to the WSTransformer...just to entirely isolate the debugging process.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply