DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
This topic has been marked "Resolved."
Author Message
pbttbis



Group memberships:
Premium Members

Joined: 11 Dec 2014
Posts: 35
Location: South Africa
Points: 515

Post Posted: Wed Mar 16, 2016 10:37 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Hi,

If you look at operations console -> Workload Management -> Queue Management

There are "Maximum Running Jobs" for various queues. The parameter for the ISD queue is set at 100, but it is not configurable (at least in the console) like the others.

Anyone know how to change the value?

Thanks,

Shaun

_________________
PBT TBIS Consultant
PaulVL



Group memberships:
Premium Members

Joined: 17 Dec 2010
Posts: 1258

Points: 8265

Post Posted: Wed Mar 16, 2016 12:58 pm Reply with quote    Back to top    

Google is your friend.

https://www.ibm.com/support/knowledgecenter/SSZJPZ_11.3.0/com.ibm.swg.im.iis.ds.monitor.doc/topics/opcsystempolicies.html

I believe that is what you are looking for.
Rate this response:  
Not yet rated
pbttbis



Group memberships:
Premium Members

Joined: 11 Dec 2014
Posts: 35
Location: South Africa
Points: 515

Post Posted: Thu Mar 17, 2016 4:20 am Reply with quote    Back to top    

Hi Paul,

Yeah that is for system wide, which I have set to 200.

What I need to be able to do is modify the max limit on ISD jobs under Queue Management.

Thanks,

Shaun

_________________
PBT TBIS Consultant
Rate this response:  
Not yet rated
eostic

Premium Poster



Group memberships:
Premium Members

Joined: 17 Oct 2005
Posts: 3794

Points: 30536

Post Posted: Thu Mar 17, 2016 4:53 am Reply with quote    Back to top    

Not sure why you can't edit it, but it's likely because it isn't that meaningful, provided that these are "always on" ISD Jobs (that have an ISDInput)... Carefully consider your ISD Jobs...do tes ...

_________________
Ernie Ostic

blogit!
Open IGC is Here!
Rate this response:  
Not yet rated
pbttbis



Group memberships:
Premium Members

Joined: 11 Dec 2014
Posts: 35
Location: South Africa
Points: 515

Post Posted: Thu Mar 17, 2016 9:34 am Reply with quote    Back to top    

Hi Ernie,

The high water mark we estimated to be at 200-300.

The console is a little confusing that it is under the tab "Queue Management", but the parameter there that I want to change is "Maximum Running Jobs". It is currently set to 100 and when trying to change it get the response "Update or delete is not allowed for non batch queues"

Cheers,

Shaun

_________________
PBT TBIS Consultant
Rate this response:  
Not yet rated
eostic

Premium Poster



Group memberships:
Premium Members

Joined: 17 Oct 2005
Posts: 3794

Points: 30536

Post Posted: Thu Mar 17, 2016 9:56 am Reply with quote    Back to top    

200 to 300? are these individual services? do they all have isd input? ...or are you talking about 200 to 300 concurrent requests? ......thats a really huge number for isd always ...

_________________
Ernie Ostic

blogit!
Open IGC is Here!
Rate this response:  
Not yet rated
pbttbis



Group memberships:
Premium Members

Joined: 11 Dec 2014
Posts: 35
Location: South Africa
Points: 515

Post Posted: Thu Mar 17, 2016 11:12 am Reply with quote    Back to top    

Well for example now we are testing a workload of ~360 request per minute which we are attempting to handle with three applications created in Information Services Director. The applications are configured with 45, 45 and 10 instances as the min and max.

We may be able to reduce the 100 instances down a bit given more load testing but as the ~360 per minute load is only 40% of the expected workload we will need to create more than a 100 instances.

Are you saying ISD is not suited for the above?

_________________
PBT TBIS Consultant
Rate this response:  
Not yet rated
eostic

Premium Poster



Group memberships:
Premium Members

Joined: 17 Oct 2005
Posts: 3794

Points: 30536

Post Posted: Thu Mar 17, 2016 1:33 pm Reply with quote    Back to top    

Just saying that seems like a lot. A single instance can often handle 200+ requests per second, even with a slow protocol like SOAP over HTTP. Of course it could depend on your tranformations, but ...

_________________
Ernie Ostic

blogit!
Open IGC is Here!
Rate this response:  
Not yet rated
pbttbis



Group memberships:
Premium Members

Joined: 11 Dec 2014
Posts: 35
Location: South Africa
Points: 515

Post Posted: Fri Mar 18, 2016 3:10 am Reply with quote    Back to top    

Well the jobs are very light weight. EE but running on a single node.

The reason for creating so many instances is that even though the job is light weight there is a REST call made by a hierarchical stage in the job that takes anything from 5-12sec to respond. As we need to process a single request in under 30 seconds we decided to create so many instances so that we do not have requests queuing up behind that REST call. In this way we are submitting many parallels REST calls allowing us to come in at under 30 seconds reliably.

The machine is reasonable hefty. 8 cores 32GB of RAM. Saying that while the instances are up and handling requests the CPU idles at 5%. The only signficant resource used is memory. In the case of the 100 instances that is ~20GB.

The client is also looking at upgrading the machine to a size of 48 core and 256GB in the future which may be needed once we start beefing up the jobs with more transformations etc.

So the above is a lot of context and interesting to discuss, but the original post was asking how to change the Maximum Running Jobs for ISD. I have also asked the question to IBM via a PMR.

_________________
PBT TBIS Consultant
Rate this response:  
Not yet rated
eostic

Premium Poster



Group memberships:
Premium Members

Joined: 17 Oct 2005
Posts: 3794

Points: 30536

Post Posted: Fri Mar 18, 2016 7:19 am Reply with quote    Back to top    

Ok.. sounds very interesting. Just be careful. Start-up in such situations (with that many osh processes needing to come up) can be very problematic with that few cores, along with that many concur ...

_________________
Ernie Ostic

blogit!
Open IGC is Here!
Rate this response:  
Not yet rated
pbttbis



Group memberships:
Premium Members

Joined: 11 Dec 2014
Posts: 35
Location: South Africa
Points: 515

Post Posted: Fri Mar 18, 2016 8:19 am Reply with quote    Back to top    

yeah I set the workload management to start up only 3 jobs every 30 seconds otherwise things do not go well.

_________________
PBT TBIS Consultant
Rate this response:  
Not yet rated
pbttbis



Group memberships:
Premium Members

Joined: 11 Dec 2014
Posts: 35
Location: South Africa
Points: 515

Post Posted: Mon Apr 04, 2016 2:47 pm Reply with quote    Back to top    

Confirmed with IBM that there is hard limit of 100 ISD jobs that can run concurrently per engine.

To work around this we are implementing a multi engine solution and load balancing request across them.

_________________
PBT TBIS Consultant
Rate this response:  
eostic

Premium Poster



Group memberships:
Premium Members

Joined: 17 Oct 2005
Posts: 3794

Points: 30536

Post Posted: Mon Apr 04, 2016 3:18 pm Reply with quote    Back to top    

Ok... let me know if you need any tips on how to deploy your single Operation across multiple DS Engines with automatic round-robin load balancing.

Ernie

_________________
Ernie Ostic

blogit!
Open IGC is Here!
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours