Multi node configuration in standalone installation?

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
laxman.ds
Premium Member
Premium Member
Posts: 66
Joined: Thu Mar 02, 2006 9:00 am

Multi node configuration in standalone installation?

Post by laxman.ds »

Dear Experts,

Is it possible to have multi node config file in standalone and multiple engine tier machines?

To our installation topology, we have installed Services and Engine tier for QualityStage in Virtual Server 1, then installed Engine for DataStage in Virtual Server 2, both Virtual Servers have dedicated file systems, and there is common storage across these servers. SSH configured to login Server 2 from Server 1 without password.

Then, we have connected to the project created in Server 1, and created 2 node config file: node1 points to server 1 locations, while node2 points to server 2 locations, tested the config file, fine except the warning "unable to locate the project location in server 2", actually project is exist in server 1 however it looks the same project in server 2.

Now, the question is is it not possible to use 2 servers as nodes in configuration to start running the job on server 1 and utilize the server 2 resources while having the project at server 1? Is multi node configuration possible when common storage like in clustered installation? Or can we achieve this by using directory link on server 2 to point the project location on server 1? Or any alternative thoughts?

Please advise if any one experienced similar situation?

Thanks in advance.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

The PXEngine directory has to be visible to both machines. It can be cross-mounted, or it can be "distributed" to the second machine.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
electajay
Participant
Posts: 36
Joined: Thu Apr 15, 2010 11:19 am

Re: Multi node configuration in standalone installation?

Post by electajay »

Hi

Yes Its possible. The Project Directories and Binaries /opt/IBM/InformationServer should be shared between the 2 Servers. we are using NAS sharing file system between Head node and computed nodes. NAS is shared between Head nodes and Compute nodes.

As ray said the PXEngine and all binaries should be shared between the servers.

Thanks
Ajay
A Kumar
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

A project has to be cross mounted accross both systems. not just "looks the same" it has to be the same.

As Ray said. The engine has to be cross mounted. The projects have to be cross mounted and the data paths have to be cross mounted. The only thing local can be SCRATCH DISK and TEMPDIR if you wish (speed reasons).
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

I disagree with Paul, although he is correct. But you can also configure your installation such that all your projects are on machine 1, and machine 2 is used only for processing. You do this by mentioning both machines in the parallel execution configuration file, even though the engine is installed only on machine 1. And you have to configure secure password-less remote execution from machine 1 to machine 2.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
laxman.ds
Premium Member
Premium Member
Posts: 66
Joined: Thu Mar 02, 2006 9:00 am

Post by laxman.ds »

Dear Paul,

As Ray said, to our constraint of not using cross mount or common sharing, as he explained on the last post, configured SSH for the dsadm user and configured scp (as default non secured rcp uses by engine), and then used 2 node config file, 2nd node points to the 2nd machine. Only constraint is, we have to make same project path in the 2nd machine so that during run time, engine creates binaries (object file) in the 2nd machine and uses 2nd machine scratch n resource disk space as mentioned in the config file for 2nd node.

Finally achieved. Good learning. Thank you Ray, Paul & electajay for your response.
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

Hmmm... not sure which aspect you disagree with.

So I can't even agree to disagree. :)
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

That the project has to be cross mounted.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply