link collector error

[kollurianu]

Hi All,

I have three output links coming from the three transformers and

iam using link collector to combine these data from above three links

and the output from the link collector is to sequential stage ,

iam getting the following errors when iam running the job ,

NamedAccountNEW..HQ_LkUp.DSLink240: ds_ipcopen() - Error in open(/tmp/dev_proj.NamedAccountNEW..HQ_LkUp-Link_Collector_246.DSLink240) - No such file or directory


NamedAccountNEW..Link_Collector_246: |Error 11 in GCI Link initialisation.|


Attempting to Cleanup after ABORT raised in stage NamedAccountNEW..Link_Collector_246


NamedAccountNEW..opfile.DSLink228: Run stopped after 11969 rows

and the job is Aborted.


And there is red link going from Link Collector to Sequential stage.

Any Ideas,

Thank you ,


















[/code]

Code: Select all

[manteena]

load the each TX output to the seq files and them connect the file outputs to the LC then to the target file.
If u don't want the intermediate files, you can always delete them from after the job subroutine

[kollurianu]

Hi manteena,

Thank you for ur quick response,

But I donot think that is the issue , i have design having all links from transformer to linkcollector that job works fine,

could some please explain , why am i getting those errors.

Thank you all

[kollurianu]

Thank you very much Manteena ,

i tried your method , but instead of sequential stage , i used IPC stage ,

that worked.

But i just need a clarification , on using LC , like on how would i know

whether i need to use sequential stage or IPC stage before or after link collector,

any guidance , greatly appreciated.

Thank you all once again.

[manteena]

Thank you very much Manteena ,

i tried your method , but instead of sequential stage , i used IPC stage ,

that worked.

But i just need a clarification , on using LC , like on how would i know

whether i need to use sequential stage or IPC stage before or after link collector,

any guidance , greatly appreciated.

Thank you all once again.

Great!!! you figured it out.
Looking at your error message "No such file or directory"
I think this file is temp file placed by the DS between two active inks.
By placing IPC DS will not write to the temp files..

[kollurianu]

Thank you so much Manteena for your answer.

Thank you all

[trokosz]

Link Collector and IPC are totally different in what they accomplish....you can't mix one over the other

[aartlett]

Link collectors from active stages should have IPC turned on in the tunables.

If you have lots of processors on your box, you might even get a performance boost.

[ray.wurlod]

Link collectors from active stages should must have IPCrow buffering turned on in the tunables.

IPC is a stage type, that makes row buffering explicit. However, both the Link Partitioner and Link Collector stages also use the same libraries to refer to the data buffers.

[ak77]

Hi everybody,

I had a similar problem
Is it good to include a sequential stage or IPC stage between transformer and Link Collector? I already had the Inter-Process turned on with default settings
Mine worked with the sequential stage in between but the time taken to run this job was much longer

Then I tried just the partioner in a job and collector in the following job
This seems to be much faster


I dont know if this is the best way but it works

Can somebody give thought to this?

Thanks

Kishan