The open files limit is

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
bobyon
Premium Member
Premium Member
Posts: 200
Joined: Tue Mar 02, 2004 10:25 am
Location: Salisbury, NC

The open files limit is

Post by bobyon »

What setting is it that when changed would cause this warning to start appearing in logs.
node_Compute1: The open files limit is 10240; raising to 1048575.
A number of different changes were made recently to our grid servers and those changes resulted in the above message showing up in logs where prior to the changes the warning was not present.

I thought the setting was the ulimit -n but that has been put back to the original setting and we are still getting the warning.

Thanks,
Bob
Bob
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

I'm not sure off the top of my head but if you modify a job to run "ulimit -a" as a Before Job option, then you could look for which show a current value of 10240. That should at least narrow it down if not answer the question.
-craig

"You can never have too many knives" -- Logan Nine Fingers
bobyon
Premium Member
Premium Member
Posts: 200
Joined: Tue Mar 02, 2004 10:25 am
Location: Salisbury, NC

Post by bobyon »

yup. done that.

We've got the ulimit settings back to where they were before all the changes were applied but something still is causing this message to appear in the logs.

I was quite surprised to still see the msgs in the logs after resetting the ulimit values.
Bob
abhinavagarwal
Participant
Posts: 26
Joined: Thu Jun 19, 2008 12:39 am
Location: Atlanta

Post by abhinavagarwal »

Hi Bob...

While you have tried your hands on the ulimit setting already... I would like to add few more things on the same.

The user-id which is running the job, user-id which has been used to create the node directories, user-id which is used to make communication possible between the grid architecture---- I hope every user-id has ulimit set to unlimited.

Hope this help!
- Thanks and Regards,
Abhinav Agarwal
bobyon
Premium Member
Premium Member
Posts: 200
Joined: Tue Mar 02, 2004 10:25 am
Location: Salisbury, NC

Post by bobyon »

abhinavagarwal wrote:The user-id which is running the job, user-id which has been used to create the node directories, user-id which is used to make communication possible between the grid architecture---- I hope every user-id has ulimit set to unlimited.!
We only changef the ulimit setting on the ids that are used for submitting jobs. How would I determine which other ids are involved?
Bob
abhinavagarwal
Participant
Posts: 26
Joined: Thu Jun 19, 2008 12:39 am
Location: Atlanta

Post by abhinavagarwal »

hi Bob

Plz check -

user-id which has been used to create the node directories - this may be dsadmin id or root or any users on the box.

user-id which is used to make communication possible between the grid architecture - this may be root or may be dsadmin id.

Cheers.
- Thanks and Regards,
Abhinav Agarwal
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Run a job that has ExecSH as its before-job subroutine and execute the id command. The user that runs the job will appear in the job log.

You can (should) also use web console for Information Server to check the Engine credentials.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply