No handlers are available to process this request

Dedicated to DataStage and DataStage TX editions featuring IBM<sup>®</sup> Service-Oriented Architectures.

Moderators: chulett, rschirm

Post Reply
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

No handlers are available to process this request

Post by chulett »

Having a new problem with some 'RTI Enabled' jobs and not sure which 'side' is having the issue. They are using HTTP over SOAP bindings, Oracle as the repository and WebLogic on the RTI Server side. This is an example of what is returned when one of the jobs is called:
Tue Apr 04 15:28:14 MDT 2006 - Exception while calling PropertyUpdateControl.WsApPropertyUpdate: Tue Apr 04 15:28:14 MDT 2006 - EJB Exception: ; nested exception is: com.bea.control.ServiceControlException: <xml-fragment xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <faultcode>RTIException</faultcode> <faultstring>java.rmi.RemoteException: EJB Exception: ; nested exception is: javax.ejb.EJBException: Exception trying to invoke operation WsApPropertyUpdate: No handlers are available to process this request.</faultstring> <detail/> </xml-fragment>
When it says that "No handlers are available to process this request" is this a problem on the ETL / DataStage Server / RTI Agent side or the WebLogic / RTI Server side? :?

I don't see anything wrong on the DS side - RTI Agent is running, jobs are running, nothing funky is in the logs. I've stopped and restarted them via the RTI Console and the same has allegedly been done on the WebLogic side but it seems to be persisting. Can anyone say for certain what might be the root cause of this error?
-craig

"You can never have too many knives" -- Logan Nine Fingers
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

Hi,
How many instances of the RTI job do you have running?
When this happens can it be you exceeded the amount of RTI instances that you set to be waiting for requests?
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Don't believe that's the case. The volume using it is so low the fact that only one 'invocation' of it is running at any given time really shouldn't be the issue. My BEA folks feel it has something to do with the Connection Pooling and the lack of the ability to 'test' connections automatically with the driver supplied with RTI. Found this in one of their logs:
<Apr 5, 2006 8:33:06 AM MDT> <Error> <JDBC> <BEA-001112> <Test "SELECT 1 FROM DUAL" set up for pool "RTIPool" failed with exception: "java.sql.SQLException: [Ascential][Oracle JDBC Driver]This driver is locked for use with embedded applications.".>
<Apr 5, 2006 8:33:06 AM MDT> <Error> <JDBC> <BEA-001111> <Unable to verify the test "SELECT 1 FROM DUAL" set up for pool "RTIPool". Connections will not be tested.>
Opened a case with Support, we'll see what they say.
-craig

"You can never have too many knives" -- Logan Nine Fingers
russ356
Charter Member
Charter Member
Posts: 38
Joined: Tue Jun 07, 2005 6:58 am

Post by russ356 »

Does anyone know the resolution to this problem? We have a job that has been running for months and today it is giving this error.
APT_CombinedOperatorController(0),0: Fatal Error: Fatal: <ns1:Fault xmlns:ns1="http://schemas.xmlsoap.org/soap/envelope/">
<faultcode>RTIException</faultcode>
<faultstring>java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:
java.rmi.RemoteException: ; nested exception is:
javax.ejb.EJBException: Exception trying to invoke operation ds_apl_style_color.


Timed out waiting for an operation handler to become available for Operation &apos;ds_apl_style_color&apos;.</faultstring>
<detail/>
I have searched here and support but have found nothing. We don't think it's the Datastage job and we have restarted all services that are associated with this but no luck.
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

The NO HANDLER AVAILBLE error is fairly generic, but points to the fact (as indicated in some of the entries of the old thread above) that there is "no instance" of a DS Job available to process the incoming request. This can happen for a multitude of reasons, among them:

The RTI Agent is down.

Something's wrong with the port it is using (be sure 2000 or whatever you chose is "listening" via netstat -a|grep <port>

You've flooded the instances with requests, the pipeline buffer is full, and the queue at the RTI Server is full....

The Job keeps crashing.

I'll think of others...... increase your instances to say, 3, check the Agent (well, you won't be able to increase instances if the agent is down), check the conditions of the error (how many incoming requests, etc.).

Ernie
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

I forgot about this. We were running our jobs as 'always on' jobs, without allowing it to automagically start new invocations if it needed to. For whatever reason 'they' wanted to stick with that always on approach so we bumped the number of simultaneous invocations up to 2 or 3 depending on the service. Solved the problem for us.
-craig

"You can never have too many knives" -- Logan Nine Fingers
russ356
Charter Member
Charter Member
Posts: 38
Joined: Tue Jun 07, 2005 6:58 am

Post by russ356 »

OK, here is what we have done so far but have not resolved the issue.

We have bumped up the simultaneous invocations.
We have restarted all the RTI services and Web Services.
Confirmed the port is listening.

But now we may have the same or a different problem. It appears that the web service is running fine when we look in the log but the calling job is hanging.
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Is there anything else interesting that you can tell us about the job? Is it just a standard "batch style" job with no RTIinput or RTIoutput? Are you passing anything into it at all (Job Parameter values?)? Does the job run fine from the Director (without using RTI to kick it off)?

Do you have any other jobs like it that are working?

Just looking for interesting differences in this one particular job.

Ernie
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You've marked this topic Resolved. What was the resolution? Help others who may encounter this issue in future by posting the solution here.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

*I* marked it as resolved as it was my issue... and my resolution was posted earlier.
-craig

"You can never have too many knives" -- Logan Nine Fingers
russ356
Charter Member
Charter Member
Posts: 38
Joined: Tue Jun 07, 2005 6:58 am

Post by russ356 »

Yes, the original issue was resolved. I just had a follow-up question as we were still having problems. But ours has been resolved as well. Our business unit decided to make changes to the source system without informing us, so our RTI was suppose to return around 200 records was getting around 6,000 and was getting hung.

Here is the post with the resolution for this thread, just to make it easier.
I forgot about this. We were running our jobs as 'always on' jobs, without allowing it to automagically start new invocations if it needed to. For whatever reason 'they' wanted to stick with that always on approach so we bumped the number of simultaneous invocations up to 2 or 3 depending on the service. Solved the problem for us.
Post Reply