making jot to run in multiple nodes

bhasannew · Post by **bhasannew** » Wed Dec 09, 2009 1:32 am

Hi,

what are the points to be considered to make a job run in multiple nodes. Other than configuration file changes what are the changes to be done at stage level.
Please provide me the useful links if any.

Thanks,
Bhasan.

bkumar103 · Post by **bkumar103** » Wed Dec 09, 2009 4:40 am

Hi,
You have to create the configuration file for the multiple node.
You might need to check the partitioning in your jobs to get the proper output.

Thanks,

priyadarshikunal · Post by **priyadarshikunal** » Wed Dec 09, 2009 4:52 am

exactly. Just to run a job on multiple node you need configuration file changed to define logical nodes. However to get the correct output from that job you may need to reconsider the partitioning algorithm used for your stages. Most of the time auto partitioning gives the correct result unless you have some specific requirements.

chulett · Post by **chulett** » Wed Dec 09, 2009 7:33 am

Specifically, stages that could be affected by partitioning include the Lookup and Aggregation stages. Sometimes it's about building an efficient job and other times it's about getting the correct results. Both Parallel pdf guides discuss this and it would be well worth your time to read them.