performance FTP

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
nag0143
Premium Member
Premium Member
Posts: 159
Joined: Fri Nov 14, 2003 1:05 am

performance FTP

Post by nag0143 »

Hi !
I am trying to get Data from the mainframe and splitting the file into 4 files using LC in the dataStage , I am testing with FTP stage and FTP script to get the data from mainframe , but i don't see much difference in the performance,

This doesn't look right to me,
does getting data and splitting the data from a FTP Stage is faster when compared to reading data from a sequential file and splitting ??

Thanks
Nag
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

FTP is pretty; especially if you don't "land" your data enroute. If you can split your file on the host side and run several FTP processes (stages) in parallel you should be getting very good performance. Your speed ought to be limited by your network transfer bandwidth, not your UNIX I/O or CPU capacity. So running 2 FTP processes should be enought to guarantee that you are getting all you can out of the network, the rest is up to how you write your DS job. Start off with an empty job that does no data checking or manipulation, let it write to a sequential file and use those Rows/Second numbers as your optimum speed upon which to benchmark your speed once you add your logic.

-Arnd.
Post Reply