RTI - "No handlers available" error

trusac · Post by **trusac** » Tue Mar 31, 2009 7:42 am

We are getting RTI - "No handlers available error" intermittently in Production while running web service client jobs.
I checked the RTI environment ( RTIAgent and RTIServer) is up and running.
We are running DataStage 7.5.2 (Server edition) on HP-UNIX 11i. The RTI version is 7.5.1 A.
RTI Server runs on WebLogic 8.1.
These web services have been running with no issues for around 2 years.
But now we are getting these errors.
What i have observed that if the web service client job is invoked again sometime after it has failed, then it runs successfully.
But it is not guaranteed and some times we have to bounce the RTI Server and the Agent.
There has also been no increase in the web service calls.
Also the web services run as always "ON" with 2 to 3 instances running at one time.
I am already in touch with the IBM folks and provided them with the logs.
In the mean time i would really appreciate if someone has run into similar issue and can shed some light on it.

Thanks
Sachin

eostic · Post by **eostic** » Tue Mar 31, 2009 7:47 am

Could be many things.... this is a very generic error, but meaning that it can't "reach" the DataStage Job that is deployed for your particular Operation...... "Usually" it is because:

There are no instances running to accept requests
The instances all died or aborted
There is a network problem
The RTI agent isn't running
The pipeline buffer is full, the Operation queue is full, and you had a timeout.

If traffic is the same, it's hard to say, because a common reason is that traffic has increased, and the settings for instances/pipeline/Operationqueue/Timeout are too low.

Look also in the DS log --- did any instances abort? If so, can you determine how or why? I've seen cases where a particular data value resulted in abort of the job because of what it was doing or that it was handling a particular data item incorrectly. A flood of requests, right at the time when instances aborted (and until they are automatically re-started) could result in what looks like sporadic errors.

Ernie

chulett · Post by **chulett** » Tue Mar 31, 2009 7:48 am

Did you search here for that message? I ran into the same issue but for us it was from an increase in the number of web service calls and the defined invocations were overrunning their queues. Haven't seen any other reason for that message, personally. Hopefully IBM will shed some light on it, or Ernie will have some Words of Wisdom I'm sure.

chulett · Post by **chulett** » Tue Mar 31, 2009 7:49 am

(and he did)

trusac · Post by **trusac** » Tue Mar 31, 2009 8:37 am

Thanks for the prompt reply.
Out of the possible causes that have been listed, i think it might be coz.
of network issues or the Pipieline buffer/Queue being full.
I checked that the job instances have been running fine and there are no aborts.
I will check with our networking team for any network issues.
Any idea how do i check the pipeline buffer/Queue parameters ?
Are these parameters on the RTIServer or the RTIAgent side and how to decide on what values to be set ?

Thanks
Sachin

eostic · Post by **eostic** » Tue Mar 31, 2009 10:13 am

The pipeline buffer and instance settings are at the individual Job level, under Operation (lowest level drill down) at the RTI Console........right mouse click on the Job icon and select "settings"......

The Operation values are at the Operation page....

What to set them at? Hard to predict...it's a bit of trial and error and understanding your traffic peaks and doing stress testing....another reason I like SOAPscope.... it gives you the ability to ramp up the number of clients and the number of invocations over time during a test scenario.

Ernie