XML INPUT without Folder?

[trainee9999]

Hi all,

Our requirement is to handle all the incoming XML files and send them to Oracle database after the required processing. We want to achieve this real time or perhaps near real time using RTI input and RTI output stages.

a) Is this possible?

If yes, can someone please help me get a best possible design. I had worked with XML input in the past and used Folder stage to provide the file to the XML input stage...I believe Folder stage consume lot of resources...

b)Is there a way we can avoid folder stage and still use XMLInput. to include in Fully compliant RTI job design.


Thanks a lot in advance.
Ds

[chulett]

RTI jobs are real time, hence the name and implemented as web services. Something 'near real time' would be implemented with a Server or Parallel solution run in short intervals, like every five minutes or so.

The RTI Input stage replaces the Folder stage in an RTI job and is used to feed XML data to the XML Input stage for parsing. You may find it easier to test your RTI job with a Folder stage on the front, reading XML files from disk, before you swap it out for an RTI Input stage and deploy the job as a web service.

Folder stages don't 'consume alot of resources' and will still need to be what you use if you are going to be processing XML files - the RTI Input stage reads XML sent to the web service created by the 'RTI Enabled' job.

If you stick with the Folder stage, one suggestion would be to look into the URL/Filepath method to feed the XML Input stage - one field in the Folder stage, not two, just bring in filenames and then let the XML Input stage directly read the files off disk. This rather than the standard 'two column' approach where the Folder stage also passes in the entire contents of the file being read which is probably where your resources statement comes from.

[trainee9999]

RTI jobs are real time, hence the name and implemented as web services. Something 'near real time' would be implemented with a Server or Parallel solution run in short intervals, like every five minutes or so.

The RTI Input stage replaces the Folder stage in an RTI job and is used to feed XML data to the XML Input stage for parsing. You may find it easier to test your RTI job with a Folder stage on the front, reading XML files from disk, before you swap it out for an RTI Input stage and deploy the job as a web service.

Folder stages don't 'consume alot of resources' and will still need to be what you use if you are going to be processing XML files - the RTI Input stage reads XML sent to the web service created by the 'RTI Enabled' job.

If you stick with the Folder stage, one suggestion would be to look into the URL/Filepath method to feed the XML Input stage - one field in the Folder stage, not two, just bring in filenames and then let the XML Input stage directly read the files off disk. This rather than the standard 'two column' approach where the Folder stage also passes in the entire contents of the file being read which is probably where your resources statement comes from.

Hi All,

Thank you Craig for the excellent explaination differentiating real time and near real time.

The issue I had when setting up RTI input with XML input is assigning a source column in the XML input stage which looks like a mandatory option. I guess I need to create a variable that holds the location and the xml file name within the RTI INput.. PLease correct me here if I am not or can you please provide any other method for this..

Thanks a lot.

[chulett]

The issue I had when setting up RTI input with XML input is assigning a source column in the XML input stage which looks like a mandatory option.

Treat it just like a Folder stage but without the ability to pass a filename - only the contents of the XML sent to it. So (typically) a single LongVarchar field and the XML Input stage points to it as the 'XML Source Column' with a Column Content setting of 'XML Document'.

Don't mix the two up! The RTI Input stage does not read files but rather the XML document sent by whatever calls the web service the 'RTI Enabled' ETL job provides. The Folder stage is what reads files.