Author |
Message |
pbttbis

Group memberships: Premium Members
Joined: 11 Dec 2014
Posts: 36
Location: South Africa
Points: 529
|
|
DataStage® Release: 11x |
Job Type: Parallel |
OS: Unix |
|
Hi,
If you look at operations console -> Workload Management -> Queue Management
There are "Maximum Running Jobs" for various queues. The parameter for the ISD queue is set at 100, but it is not configurable (at least in the console) like the others.
Anyone know how to change the value?
Thanks,
Shaun
|
_________________ PBT TBIS Consultant |
|
|
 |
PaulVL

Group memberships: Premium Members
Joined: 17 Dec 2010
Posts: 1314
Points: 8626
|
|
|
 |
pbttbis

Group memberships: Premium Members
Joined: 11 Dec 2014
Posts: 36
Location: South Africa
Points: 529
|
|
|
|
|
|
Hi Paul,
Yeah that is for system wide, which I have set to 200.
What I need to be able to do is modify the max limit on ISD jobs under Queue Management.
Thanks,
Shaun
|
_________________ PBT TBIS Consultant |
|
|
 |
eostic

Group memberships: Premium Members
Joined: 17 Oct 2005
Posts: 3840
Points: 31021
|
|
|
|
|
|
Not sure why you can't edit it, but it's likely because it isn't that meaningful, provided that these are "always on" ISD Jobs (that have an ISDInput)...
Carefully consider your ISD Jobs...do tes ...
|
_________________ Ernie Ostic
blogit!
Open IGC is Here! |
|
|
 |
pbttbis

Group memberships: Premium Members
Joined: 11 Dec 2014
Posts: 36
Location: South Africa
Points: 529
|
|
|
|
|
|
Hi Ernie,
The high water mark we estimated to be at 200-300.
The console is a little confusing that it is under the tab "Queue Management", but the parameter there that I want to change is "Maximum Running Jobs". It is currently set to 100 and when trying to change it get the response "Update or delete is not allowed for non batch queues"
Cheers,
Shaun
|
_________________ PBT TBIS Consultant |
|
|
 |
eostic

Group memberships: Premium Members
Joined: 17 Oct 2005
Posts: 3840
Points: 31021
|
|
|
|
|
|
200 to 300?
are these individual services? do they all have isd input?
...or are you talking about 200 to 300 concurrent requests?
......thats a really huge number for isd always ...
|
_________________ Ernie Ostic
blogit!
Open IGC is Here! |
|
|
 |
pbttbis

Group memberships: Premium Members
Joined: 11 Dec 2014
Posts: 36
Location: South Africa
Points: 529
|
|
|
|
|
|
Well for example now we are testing a workload of ~360 request per minute which we are attempting to handle with three applications created in Information Services Director. The applications are configured with 45, 45 and 10 instances as the min and max.
We may be able to reduce the 100 instances down a bit given more load testing but as the ~360 per minute load is only 40% of the expected workload we will need to create more than a 100 instances.
Are you saying ISD is not suited for the above?
|
_________________ PBT TBIS Consultant |
|
|
 |
eostic

Group memberships: Premium Members
Joined: 17 Oct 2005
Posts: 3840
Points: 31021
|
|
|
|
|
|
Just saying that seems like a lot. A single instance can often handle 200+ requests per second, even with a slow protocol like SOAP over HTTP. Of course it could depend on your tranformations, but ...
|
_________________ Ernie Ostic
blogit!
Open IGC is Here! |
|
|
 |
pbttbis

Group memberships: Premium Members
Joined: 11 Dec 2014
Posts: 36
Location: South Africa
Points: 529
|
|
|
|
|
|
Well the jobs are very light weight. EE but running on a single node.
The reason for creating so many instances is that even though the job is light weight there is a REST call made by a hierarchical stage in the job that takes anything from 5-12sec to respond. As we need to process a single request in under 30 seconds we decided to create so many instances so that we do not have requests queuing up behind that REST call. In this way we are submitting many parallels REST calls allowing us to come in at under 30 seconds reliably.
The machine is reasonable hefty. 8 cores 32GB of RAM. Saying that while the instances are up and handling requests the CPU idles at 5%. The only signficant resource used is memory. In the case of the 100 instances that is ~20GB.
The client is also looking at upgrading the machine to a size of 48 core and 256GB in the future which may be needed once we start beefing up the jobs with more transformations etc.
So the above is a lot of context and interesting to discuss, but the original post was asking how to change the Maximum Running Jobs for ISD. I have also asked the question to IBM via a PMR.
|
_________________ PBT TBIS Consultant |
|
|
 |
eostic

Group memberships: Premium Members
Joined: 17 Oct 2005
Posts: 3840
Points: 31021
|
|
|
|
|
|
Ok.. sounds very interesting. Just be careful. Start-up in such situations (with that many osh processes needing to come up) can be very problematic with that few cores, along with that many concur ...
|
_________________ Ernie Ostic
blogit!
Open IGC is Here! |
|
|
 |
pbttbis

Group memberships: Premium Members
Joined: 11 Dec 2014
Posts: 36
Location: South Africa
Points: 529
|
|
|
|
|
|
yeah I set the workload management to start up only 3 jobs every 30 seconds otherwise things do not go well.
|
_________________ PBT TBIS Consultant |
|
|
 |
pbttbis

Group memberships: Premium Members
Joined: 11 Dec 2014
Posts: 36
Location: South Africa
Points: 529
|
|
|
|
|
|
Confirmed with IBM that there is hard limit of 100 ISD jobs that can run concurrently per engine.
To work around this we are implementing a multi engine solution and load balancing request across them.
|
_________________ PBT TBIS Consultant |
|
|
 |
eostic

Group memberships: Premium Members
Joined: 17 Oct 2005
Posts: 3840
Points: 31021
|
|
|
|
|
|
Ok... let me know if you need any tips on how to deploy your single Operation across multiple DS Engines with automatic round-robin load balancing.
Ernie
|
_________________ Ernie Ostic
blogit!
Open IGC is Here! |
|
|
 |
|