Page 1 of 1

Parallel Engine Config File setup - advise

Posted: Fri Jun 22, 2007 12:18 am
by alan
Greetings,

We're a new site with EE. Need advise on best "options" config file for our current & future workload.

System Info:
Red Hat Linux AS 3.x
2x DELL 2850 servers ( symmetrical )
- DataStage programs & projects on NAS share
- ssh used by conductor to start shell session on other player node

Current single apt file:
{
node "node1"
{
fastname "test01"
pools "" "node1" "test01"
resource disk "/datastage/Datasets" {}
resource disk "/export/datastage/Datasets" {}
resource scratchdisk "/Scratch" {}
}
node "node2"
{
fastname "test02"
pools "" "node2" "test02"
resource disk "/export/datastage/Datasets" {}
resource disk "/datastage/Datasets" {}
resource scratchdisk "/Scratch" {}
}
}

Our Current Loading:
multiple concurrent Jobs - all small, mostly single row processing
- hardly any need for parallel engine ( more like penalty currently )

Future load (projects) will make better use of parallel engine - with huge load and fairly partitioned data to process.

Questions:
1) Use the above APT file for future EE loads (& add more physical nodes as required)
2) Would it be wise to define another APT file to define only one logical node to handle existing load? - effectively leaving player node idle.
3) Is there an optimal limit to number of logical nodes I can define for a 2-CPU server - Manual suggests 1 logical node per 2-CPU because EE launches 2 processes per logical node - is that correct? Does our small jobs mean 1 process could be idle at times?
4) Does our existing APT file above means 4 processes are launched each time a job is started? - regardless if our jobs will only use 1 or two?
5) is there a way to ensure only a single process is launched for our jobs - any special parameters?

any advise, enlightenment will be appreciated

thanks