file process problem--please help

[anudeep]

My job contains 10 sequential files in a folder. The folder is the source and oracle database is the target. Now i have to read file_1 from the folder stage, process it and then load into oracle database. then i have to read file_2, process and load. Again file_3, process and load and i have to repeat this for all the ten files. Please help me. How can i achieve this.Should i write any subroutines. or can i use any stage to perform this functionality

[ArndW]

The "Folder" stage was created specifically for this type of processing.

[anudeep]

The "Folder" stage was created specifically for this type of processing. ...

I am sorry. i am pretty new to this field.Can u please be in detail. Please give me the logic

Thanks

[chulett]

Only for XML files. The kind of processing the OP wants to do on flat files is what the Start Loop, End Loop and User Variables stages were created for.

[anudeep]

The "Folder" stage was created specifically for this type of processing. ...

I am sorry. i am pretty new to this field.Can u please be in detail. Please give me the logic and as well the imp. point is all the files should be processed sequentially.(file_1 to file_10). (stage should not read the whole data in all files at once. read the first file, process it and then read the second file process it and it repeats. Please help with the logic.

Thanks

[ArndW]

Anudeep - the Folder stage in Server will return the contents of all files matching the pattern as a single column, which then needs to be parsed. As Craig has stated, you can use a Job Sequence and a loop to process all file names in a directory, passing the name to a job as a parameter.

Please, read the documentation on the folder stage and on job sequences before asking for detailed descriptions or assistance.

Craig - I still prefer to use the folder stage, then pass the data through named pipe and redefining the columns to the appropriate metadata. This is quicker than using a sequence (plus I've been using it since before sequences could loop or have user variable stages )

[chulett]

Ok, I could see that approach. What kind of file size limitations have you encountered with the approach, Arnd?

The Sequence job may still be more appropriate depending on the nature of the loads being performed. If a per-file audit needs to be maintained, for example, or each load needs to be handled as a discrete unit of work for whatever reason, using the Folder may... complicate that.