How can we mearge 6 huge flat files

ramdev · Post by **ramdev** » Wed Apr 05, 2006 4:02 pm

HI to everybady,
I have 6 huge flat files. I have to mearge those flat files. all are having same delimiters. each having 25 million records. If we can achive through routine pls give me the routine.

ray.wurlod · Post by **ray.wurlod** » Wed Apr 05, 2006 4:38 pm

This question will not be answered until it has been moved to the correct forum.

kumar_s · Post by **kumar_s** » Thu Apr 06, 2006 1:04 am

Hi Rax,
Edit your both post and mention the correct version and Job type you use along with the OS you use.

ArndW · Post by **ArndW** » Thu Apr 06, 2006 1:37 am

25 Million rows per file is not necessarily considered huge, many sites here deal with larger volumes on an hourly basis. Looking at the files from an ETL perspective, do you even need to merge them? Can you not process them through a DataStage job sequentially?
Depending upon your specifi conditions, you might want to simply use an OS level utility to merge these files if they need to be ordered or you can use DataStage stages to do the same thing. Without more information about the operating system, datastage version/type and if you need a true ordered merge there is little or nothing anyone here can do to assist you.

chulett · Post by **chulett** » Thu Apr 06, 2006 6:59 am

That was going to be my first question - why do you believe they need to be merged?