Round robin

krish889 · Post by **krish889** » Fri Dec 12, 2014 7:44 pm

Hi,
When i used Round robin partition in Remove duplicate stage it is not eliminating properly all duplicate data... can you brief me whats the reason behind that.

Thank you.

chulett · Post by **chulett** » Fri Dec 12, 2014 10:19 pm

Because you're not sending all of the duplicate keys to the same partition. Use hashed partitioning on the key field(s) that drive the duplicate check.

priyadarshikunal · Post by **priyadarshikunal** » Tue Dec 23, 2014 8:21 am

Also I suggest you to read the Parallel job developer guide to understand different partitioning method to make sure you choose the correct partitioning strategy in future.