identifying conductor node in default pool

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
mouthou
Participant
Posts: 208
Joined: Sun Jul 04, 2004 11:57 pm

identifying conductor node in default pool

Post by mouthou »

Hi All,

When there are 4 nodes in a config file and when the job is run, is there a way to find out which node played as the conductor node out of those 4 nodes.

It may be possible if the node has defined the pool as 'conductor' in the config file. But if there is only default pool, what is the way of knowing the chosen node as conductor node name (like node1 or node2 etc). Can it found from the Director logs while the job runs? Appreciate your response.

Thanks
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

Paste your APT config file, change the actual hostnames and paths if you wish.


What are you researching that makes you look for the "conductor" hostname? Are you wondering which host is reading your sequential files?
mouthou
Participant
Posts: 208
Joined: Sun Jul 04, 2004 11:57 pm

Post by mouthou »

I am not rresearching as such. It was a general query as to know which node was picked as the conductor node by the engine when the job runs. If there was a node pool defined with keyword "conductor", then we would know the node easily. But if there is only default pool, how we come to know the conductor node. Is there a place where we can see that node which acted as the conductor
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

By default it's the first-named node in the default node pool.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
mouthou
Participant
Posts: 208
Joined: Sun Jul 04, 2004 11:57 pm

Post by mouthou »

Thanks Ray and Paul.

I am not quite clear on the response with the references of named node and default pool, both of which are mutually exclusive as far as my understanding goes. ie, if there is named node then it cannot be called as default pool. Please correct if it is not the case.

If there is named node I thought the conductor node can be identified easily. Rather my query is with the default definition of everything in the configuration file like [pools " "] etc under each node.

Based on some read materials on this , I think the chosen conductor node can be seen in the job run log. There is some log entry which gives the list of values assigned to env variables. May be the variable APT PM CONDUCTOR NAME gives some idea. I don't have access to Datastage being between jobs! Otherwise this can be verified.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

All nodes must be named. The nodes that are in the default pool are those for which the node pool name "" occurs. This does not prevent the node from being named in other pools as well.

Best practice is to include all nodes in the default node pool.

Code: Select all

pools ""
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
mouthou
Participant
Posts: 208
Joined: Sun Jul 04, 2004 11:57 pm

Post by mouthou »

Thanks Ray. I thought to clarify the question with 2 samples of config files.

config1:

Code: Select all

node1
       {
         fastname dsserver
         pools ""
         ..........
         ...........
       }
node2
       {
         fastname dsserver
         pools "conductor"
         ..........
         ...........
       }
config2:

Code: Select all

node1
       {
         fastname dsserver
         pools ""
         ..........
         ...........
       }
node2
       {
         fastname dsserver
         pools ""
         ..........
         ...........
       }

As we know, in config1 we can easily say node2 is the conductor node. But in config2 sample, we cannot identify that easily. But based on your responses for config2 samples, can I assume that the first node 'node1' will be picked by the engine? And after running the job, where can we witness for such auto picking was my query.

[Note: inserted code blocks / formatting to make it easier to read - Andy]
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

That's correct. Your nodes are named node1 and node2. The first-named node in the default node pool is node1.

DataStage itself does not use node names; these are purely for human consumption. DataStage numbers the nodes (from 0) based on their order in the configuration file.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

mouthou wrote:As we know, in config1 we can easily say node2 is the conductor node.
Apparently, not so easily. :wink:

Ray is saying in both cases that node1 is the conductor node.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Actually I'm not. I was referring specifically to the configuration file that has ONLY a default node pool.

Where there is a "conductor" node pool, that will take precedence.

So:
  • in config1 node2 is the conductor node because it is in the conductor node pool

    in config2 node1 is the conductor node because it is the first-named node in the default node pool and there is no conductor node pool
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Thanks for the clarification... for some reason (mostly since that point hadn't been explicitly mentioned yet) I didn't think it was as simple as calling a pool "conductor" and apparently it is. Been far too long. :oops:
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply