Hi all!
I have been running some tests yesterday using the Named Pipe option (sequential file stage).
While I was expecting the process to be faster, it turns out that using Named Pipe was 60 times slower than sequential file...!!!!
Could I tune some settings to speed up the process? If so, what are they?
We are using DataStage 5.2 on Windows 2000 Server. (2 cpu x 512 MHZ, 1.3G
RAM)
Thanks for any input.
Christian Pelletier
Sr. Engineer
Probusiness
Phone: 1-925-737-7058
Named Pipes on Windows 2000
Hi Chrstian,
I dont know why your process is so slow, I expect it should be faster. I tested the Named Pipe option in the past but only with few records. I think you can verify (on the Sequential file stage properties) if the "flash after every row" option is selected and if read/write timeout values are too large. You can also take a look to the "Shared Memory Disk Caching" technical bulletin, but I dont think youll find any interesting info about Named Pipe.
Regards,
Riccardo
----- Original Message -----
From: "Pelletier, Christian"
To:
Sent: Tuesday, October 15, 2002 5:45 PM
Subject: Named Pipes on Windows 2000
> Hi all!
>
> I have been running some tests yesterday using the Named Pipe option
> (sequential file stage).
>
> While I was expecting the process to be faster, it turns out that
> using Named Pipe was 60 times slower than sequential file...!!!!
>
> Could I tune some settings to speed up the process? If so, what are
> they?
>
> We are using DataStage 5.2 on Windows 2000 Server. (2 cpu x 512 MHZ,
> 1.3G
> RAM)
>
> Thanks for any input.
>
> Christian Pelletier
> Sr. Engineer
> Probusiness
> Phone: 1-925-737-7058
>
>
>
>
>
I dont know why your process is so slow, I expect it should be faster. I tested the Named Pipe option in the past but only with few records. I think you can verify (on the Sequential file stage properties) if the "flash after every row" option is selected and if read/write timeout values are too large. You can also take a look to the "Shared Memory Disk Caching" technical bulletin, but I dont think youll find any interesting info about Named Pipe.
Regards,
Riccardo
----- Original Message -----
From: "Pelletier, Christian"
To:
Sent: Tuesday, October 15, 2002 5:45 PM
Subject: Named Pipes on Windows 2000
> Hi all!
>
> I have been running some tests yesterday using the Named Pipe option
> (sequential file stage).
>
> While I was expecting the process to be faster, it turns out that
> using Named Pipe was 60 times slower than sequential file...!!!!
>
> Could I tune some settings to speed up the process? If so, what are
> they?
>
> We are using DataStage 5.2 on Windows 2000 Server. (2 cpu x 512 MHZ,
> 1.3G
> RAM)
>
> Thanks for any input.
>
> Christian Pelletier
> Sr. Engineer
> Probusiness
> Phone: 1-925-737-7058
>
>
>
>
>
Thanks Ricardo
I just ran another test without the "flash after every row..."
)
It is a little faster (168 rows/sec vs 44 rows/sec), but still far from using a straight sequential file (13000 rows/sec). Each rows contains 750 characters. I will make a test with smaller row.
I also expected the Named Pipe to be faster.
I havent found any parameter I can modify so far.
Christian
-----Original Message-----
From: Riccardo Tani [mailto:riccardo.tani@datamat.it]
Sent: Tuesday, October 15, 2002 10:15 AM
To: datastage-users@oliver.com
Subject: Re: Named Pipes on Windows 2000
Hi Chrstian,
I dont know why your process is so slow, I expect it should be faster. I tested the Named Pipe option in the past but only with few records. I think you can verify (on the Sequential file stage properties) if the "flash after every row" option is selected and if read/write timeout values are too large. You can also take a look to the "Shared Memory Disk Caching" technical bulletin, but I dont think youll find any interesting info about Named Pipe.
Regards,
Riccardo
----- Original Message -----
From: "Pelletier, Christian"
To:
Sent: Tuesday, October 15, 2002 5:45 PM
Subject: Named Pipes on Windows 2000
> Hi all!
>
> I have been running some tests yesterday using the Named Pipe option
> (sequential file stage).
>
> While I was expecting the process to be faster, it turns out that
> using Named Pipe was 60 times slower than sequential file...!!!!
>
> Could I tune some settings to speed up the process? If so, what are
> they?
>
> We are using DataStage 5.2 on Windows 2000 Server. (2 cpu x 512 MHZ,
> 1.3G
> RAM)
>
> Thanks for any input.
>
> Christian Pelletier
> Sr. Engineer
> Probusiness
> Phone: 1-925-737-7058
>
>
>
>
>
I just ran another test without the "flash after every row..."
)
It is a little faster (168 rows/sec vs 44 rows/sec), but still far from using a straight sequential file (13000 rows/sec). Each rows contains 750 characters. I will make a test with smaller row.
I also expected the Named Pipe to be faster.
I havent found any parameter I can modify so far.
Christian
-----Original Message-----
From: Riccardo Tani [mailto:riccardo.tani@datamat.it]
Sent: Tuesday, October 15, 2002 10:15 AM
To: datastage-users@oliver.com
Subject: Re: Named Pipes on Windows 2000
Hi Chrstian,
I dont know why your process is so slow, I expect it should be faster. I tested the Named Pipe option in the past but only with few records. I think you can verify (on the Sequential file stage properties) if the "flash after every row" option is selected and if read/write timeout values are too large. You can also take a look to the "Shared Memory Disk Caching" technical bulletin, but I dont think youll find any interesting info about Named Pipe.
Regards,
Riccardo
----- Original Message -----
From: "Pelletier, Christian"
To:
Sent: Tuesday, October 15, 2002 5:45 PM
Subject: Named Pipes on Windows 2000
> Hi all!
>
> I have been running some tests yesterday using the Named Pipe option
> (sequential file stage).
>
> While I was expecting the process to be faster, it turns out that
> using Named Pipe was 60 times slower than sequential file...!!!!
>
> Could I tune some settings to speed up the process? If so, what are
> they?
>
> We are using DataStage 5.2 on Windows 2000 Server. (2 cpu x 512 MHZ,
> 1.3G
> RAM)
>
> Thanks for any input.
>
> Christian Pelletier
> Sr. Engineer
> Probusiness
> Phone: 1-925-737-7058
>
>
>
>
>