DSXchange

Hi

i have 3 seq file stage and all of them are in the append mode writing to the SAME file.

So , when i see the director for logs

For Seq 1 : it has x records exported
For Seq_2 : it has y records exported
For Seq 3 : It has z records exported

But the count of records in the seq file is less than x+y+z.
What could be the issue ? Can i not keep them in Append mode ?

Any ways to prevent this loss of records ?

mouni wrote:i have 3 seq file stage and all of them are in the append mode writing to the SAME file.

Your problem, in caps, above.

Sequential files, anywhere on any platform or tool, are by their very nature sequential media and that name was not chosen lightly. While they can support multiple reader processes they can only support one writer process. Period.

Write to three different files. Combine them appropriately in another step, which could be as simple as cat'ing them together after job.

Thanks , i had used the same cating in the after job ExecSH option but wanted to know if there are any other options with the combinability modes available or any other properties with the stage itself.

While you may very well be able to merge the three streams (but don't cross them!) in the job before writing out a single link, there are no properties in the Sequential stage that will allow what you are doing otherwise to work.

Hello all.

I am having the same issue with sequential file as mouni.
If I use Dataset will it be ok?

I am thinking its ok but let me know if I need to take anything in to consideration like locks or wait time for locks?

Thanks
nsm.

concurrent writing in append mode to a dataset works.