IPC Stage vs Row Buffering:

[prasadnittala]

I noticed recently in one of the projects, the developers have extensively used the IPC stage.

But in I think, this stage may create some problems in production where the system normally waits for the arrival of source files. The default timeout for IPC is 10secs. But if there is little delay in the arrival/consumption of the files the IPC stage causes the job to abort. (becuase of the time stipulation).

For the other cases also, we can "Enable the Row Buffering" and achieve the same performance, I believe.

Now Can anybody pls tell me where we need this IPC Stage exactly and what makes it different from row buffering option?

[DSguru2B]

An IPC stage can be used in between two active stages.
Row-buffering are of two types, in process and inter process.
The following is directly out of ds developer's help

In process. You can improve the performance of most DataStage jobs by turning in-process row buffering on and recompiling the job. This allows connected active stages to pass data via buffers rather than row by row.

Inter process. Use this if you are running server jobs on an SMP parallel system. This enables the job to run using a separate process for each active stage, which will run simultaneously on a separate processor



If no IPC stage is provided between say two transformers, the DSEngine implicitly puts an IPC stage in between.

[kumar_s]

The tiemout setting is configurable. IPC is used between stages where the link buffering is required and happens to only that stage.
Where as the link buffering option can be enabled at job level or at project level. Which will reflect to all the links available in the job.