DSXchange

jrothney
Welcome aboard.

Check the doco.

If you have created a delimited sequential file, are you in a position to use a XSLT external or as an after job routine to create your XML?

I haven't done it this way, but I have gone the other with great success.

Step 1. Request move post to Server area (Done)
Step 2. Read the manuals
Step 3. Experiment
Step 4. Consider not using D/Stage XML Pack
Step 5. Then post here if needed

Datastage XML is really designed to handle realtime input (possible why it's in the realtime area? :) ) Run a search on DSX for XML and you'll see some better, more efficient ways. Hint: Saxxon from the apache site. I ran a 45 file extract from 3 XML files in 20 minutes where the XML was 100's MB in...

Arn,
cat all the files together before inputting them.

That way you only have one input file.


Think outside the datastage you work in

GeeK, The problem could be locks on the T/D tables. There are limits to the number you can run at one time and they will just wait (or crash depending on config) for room. I'd break the job into two parts, one the fast export and the other a Multiload. If there is no transofrmation required between ...

Link collectors from active stages should have IPC turned on in the tunables.

If you have lots of processors on your box, you might even get a performance boost.

'000000{','000022E' , '00000936G' look to be negative numbers.

Get the source to send you the data in display format:
9(09).99- or -9(09).99 instead. A lot less hassle to handle and also a lot less hassle to debug the data as it is human readable.

Welcome aboard, always good to see a new face. Try a search here. It's the best way to find answers. That being typed ... Trouble shooting performance is an art. Somethings to consider: Are you using enough/too many nodes? If you are using a distubuted system (ETL and D/B on different servers) do yo...

Troy,
Try a XSLT to do it. A java based one (Saxxon at apache) would do it.

Vince, Have a look at the XSLT's on the apache web site. I think they were supplied in part by IBM. The licence allows comercial use so long as no money is charged further on (it's either a GPL or the apache one, I can't remember). The XML stages are great for a MQ feed, like you said, real time. My...

Senthil, People here may think I;m not a large advocate of the Datastage XML system, and they'd probably be right :). I like it for very little amounts of data, or from data coming in as a feed rather than a static source. My preference in your situation would be a XSLT translator. This allows you t...

One of the problems with the XML we are getting, and anyone who has processed weblog data will know, is the amount of redundant and non-required information. We are trying to get the source system to provide us with the data in a simple flat file. One problem we had was that the originating source c...

I would have to agree about the XML reader in DS Server. Haven't trird TX. When I last had to play with XML files, especially when I had to extract multiple files (record types) out of theone XML file we generated it externally using a script and I think it was the SAXXON XSLT. This was a Java syste...

For reasonable amounts of data (< 2gb) I've always been partial to a sort -u before the job

However, as people probably have noticed by now, I'm not a datastage purist, I think there are other ways of doing things.

Andrew the Heretic

Affan, I'm not sure I follow your question. Are you trying to fastexport data from Teradata or fastload/multiload data to teradata? I've mentioned this before, and probably will again: If you are loading up to teradata, DB2 or Oracle, it is faster, much faster, to create a flat file and then use the...