DSXchange

Hi Gurus,

I have searched the forum before posting the question and found many anwers to some of the issues iam having in reading the xml files. But i have one outstanding issue with my requirement.

Requirement : Have to read around 40 multiple xml files with the same metadata and convert in to fixed flat file.I have done this by putting the
External Source stage --> XML Input --> T/F-->Seq

Iam using the ls #psFilePath#ABCDE*.xml | sort underSource program(ES Stage), XML Source column as ABCDE, Column Content as URL/File Path in XML Input Stage.

The job is running fine if the source xml file sizes are less than 300 kb and aborted with Heap allocation failure pointing to xml input stage with more than 700 kb .But i may end up having some files larger than 800 kb.

can anyone suggest on this??

Thanks in Advance

K or M ?

Anything above 300 Meg is getting borderline. That's for "one" of the xml documents in your subdirectory. If you really mean "k" here, then that's small, and shouldn't cause you a problem....

If truly it's megabytes you are talking about, then you will need to break up the documents before hand.

Ernie

Hi ,

Thanks for your response.

Yes im talking about Kilobytes's only not Megabytes.Thats weird.

Do i have to do any changes in the job design and properties?

Strange for such a small document. Right now I'd suspect something else. Strange characters in the data, perhaps? What else are you doing in the Job...

...and now I see that you are in EE...so it's possible it is a column issue, although I don't usually see "heap" type errors, and generally, column size issues occur when you are sending the entire document in a column...but here it looks like you are sending just the filename into the xmlInput Stage...

I would definitely try a Server Job (use Folder Stage to pick up the document), just to debug it.......Server is often more forgiving with large varchar text and also with odd characters...

Ernie