https for data retrieval?

Dedicated to DataStage and DataStage TX editions featuring IBM<sup>®</sup> Service-Oriented Architectures.

Moderators: chulett, rschirm

lstsaur
Participant
Posts: 1139
Joined: Thu Oct 21, 2004 9:59 pm

Post by lstsaur »

Right, that's typical error message generated from DS, meaningless. What's your target's platform? I have to deal with both mainframe, AS400, and Windows platforms. I found out that for each platform, you have to escape differently. But none of them need to include the port 22 info.
For example, say my target is Windows platform with the cygwin installed, then I code in URI = as follows:
sftp://yourserver//cygdirve/d/yourdataset/your.txt.

One thing can be sure is that your ssh is definetly set up correctly becasue you are able to do the ssh connect manually. Good luck.
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

Yup - we've tried everything - with port. without port. With sftp in front of URI, with ftp in front of URI. And yes, SSH and SFTP work from command line flawlessly. Second tier support confirms it is broken and has forwarded it to engineering for research. Right now I'm trying to connect from RedHat Linux 4 to Windows 2003, or to another Linux platform.

Now - to "unhijack" this thread... Back to the original problem.

We've got the CastIron call to work internally (it can call itself now). But now we are getting out of memory from our side (DataStage).

Code: Select all

sjd_HMS_BigFile_load..wsxfm_SJMFTPGetService: java.lang.OutOfMemoryError
	at java.lang.StringBuffer.ensureCapacityImpl(StringBuffer.java:386)
	at java.lang.StringBuffer.append(StringBuffer.java:234)
	at java.io.StringWriter.write(StringWriter.java:101)
	at org.apache.axis.encoding.SerializationContextImpl.writeString(SerializationContextImpl.java:1060)
	at org.apache.axis.encoding.SerializationContextImpl.writeSafeString(SerializationContextImpl.java:1072)
	at org.apache.axis.encoding.SerializationContextImpl.writeChars(SerializationContextImpl.java:1045)
	at org.apache.axis.message.SAXOutputter.characters(SAXOutputter.java:104)
	at org.apache.axis.message.SAX2EventRecorder.replay(SAX2EventRecorder.java:210)
	at org.apache.axis.message.MessageElement.output(MessageElement.java:743)
	at org.apache.axis.message.MessageElement.getAsString(MessageElement.java:706)
	at org.apache.axis.message.MessageElement.getAsDocument(MessageElement.java:685)
	at org.apache.axis.message.MessageElement.getAsDOM(MessageElement.java:714)
	at com.ascentialsoftware.wsclient.ResponseHandler.getAsDOM(ResponseHandler.java:239)
	at com.ascentialsoftware.wsclient.ResponseHandler.buildBody(ResponseHandler.java:128)
	at com.ascentialsoftware.wsclient.OperationCall.invoke(OperationCall.java:339)
	at com.ascentialsoftware.wsclient.WebService.callServiceInternal(WebService.java:1685)
	at com.ascentialsoftware.wsclient.WebService.callService(WebService.java:1591)
	at com.ascentialsoftware.wsclient.WebService.callService(WebService.java:1544)
	at com.ascentialsoftware.wspack.Transformer.process(Transformer.java:104)
	at com.ascentialsoftware.jds.StageFactory.runStage(StageFactory.java:544)
	at com.ascentialsoftware.jds.StageFactory.runTransformer(StageFactory.java:361)
I've used -Xms768m and -Xmx768m for java virtual machine options in all three locations (Stage properties, input properties, and output properties) for the Web Services Transformer and it doesn't seem to have any effect on the error. Is there anything else I can do to either increase memory available or diagnose what is breaking? Oh - and ulimit -a shows unlimited for all memory settings.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

The WSTransformer uses similar logic to XMLPack. Works the same with XPath, and has issues with documents that are too large.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply