DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
qt_ky



Group memberships:
Premium Members

Joined: 03 Aug 2011
Posts: 2822
Location: USA
Points: 21387

Post Posted: Thu Jan 28, 2016 1:29 pm Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Additional info: Version 11.3.1.2
According to the Ops Console's Workload Management tab, by default there are 3 batch queues for jobs (Low, Medium, and High), and dedicated queues for ISD, IA, and DataClick.

Today we have deployed a new ISD application that is based on a DataStage job, and instead of running, it was Queued in the ISD queue. This seems to have happened because several IA tasks were already running and CPU usage was too high, and there was a very long wait for IA tasks to finish and reduce CPU to below the global threshold of 80%.

Are there any options to give ISD priority over IA, or over anything else for that matter?

_________________
Choose a job you love, and you will never have to work a day in your life. - Confucius
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54407
Location: Sydney, Australia
Points: 295092

Post Posted: Thu Jan 28, 2016 4:01 pm Reply with quote    Back to top    

Basically by tweaking (balancing) the CPU and memory limits for each of the queues. Allow ISD queue to start jobs even though CPU/memory might be 100%, but don't allow this for the other queues. Yo ...

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne (especially seeking good business analysts)
Rate this response:  
Not yet rated
eostic

Premium Poster



Group memberships:
Premium Members

Joined: 17 Oct 2005
Posts: 3790

Points: 30485

Post Posted: Fri Jan 29, 2016 1:03 am Reply with quote    Back to top    

There are a lot of sites who recognize the unique pattern and needs of ISD Jobs vis-a-vis regular batch Jobs, and they deploy a whole separate engine for ISD Jobs.

Ernie

_________________
Ernie Ostic

blogit!
Open IGC is Here!
Rate this response:  
Not yet rated
qt_ky



Group memberships:
Premium Members

Joined: 03 Aug 2011
Posts: 2822
Location: USA
Points: 21387

Post Posted: Fri Jan 29, 2016 7:49 am Reply with quote    Back to top    

Thank you both for sharing! We will look into both options.

_________________
Choose a job you love, and you will never have to work a day in your life. - Confucius
Rate this response:  
Not yet rated
qt_ky



Group memberships:
Premium Members

Joined: 03 Aug 2011
Posts: 2822
Location: USA
Points: 21387

Post Posted: Fri Jan 29, 2016 1:13 pm Reply with quote    Back to top    

By default, the IA queue and the ISD queue are each set to Medium priority and 100 maximum running jobs.

The ISD queue is locked down from making any changes, however the IA queue allowed some changes to made and saved.

_________________
Choose a job you love, and you will never have to work a day in your life. - Confucius
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours