Web Service is not running error

russ356 · Post by **russ356** » Wed Dec 12, 2007 4:06 pm

We need some help. We have getting the error below on quite a few of our web services lately. The web services are "always on" with a minimum of 1 to a maximum of 5. The database it is connecting to is not down because we do a db upcheck before we process this job. It doesn't happen all the time and there is no set pattern that we can determine but it does happen at least 2 - 3 times a week. Any help or direction would be greatly appreciated.

Here are the versions of what we are running.
AIX 5.3
RTI : 7.5.2
DataStage : 7.5.1.a
IBM WebSphere Application Server, 6.0.0.1
Build Number: o0445.08
Build Date: 11/10/04

APT_CombinedOperatorController(1),0: Fatal Error: Fatal: <ns1:Fault xmlns:ns1=""http://schemas.xmlsoap.org/soap/envelope/"">
<faultcode>RTIException</faultcode>
<faultstring>java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:
java.rmi.RemoteException: ; nested exception is:
javax.ejb.EJBException: Exception trying to invoke operation ds_get_BC_translations: Job ds_get_BC_translations is not running. Check the RTIAgent error log for errors.</faultstring>

eostic · Post by **eostic** » Wed Dec 12, 2007 4:55 pm

Very hard to tell from just this message. Is there anything unique or interesting about this particular job/Operation? Did it always act sporadically like this?...or did this just recently start happening? Does it improve stability if you make it 2/3 minimum and 5 max?

Ernie

lstsaur · Post by **lstsaur** » Wed Dec 12, 2007 5:31 pm

Check your RTIAgent's log to see more detailed information regarding why your job "ds_get_BC_translations" attached to the operations failed.

g_rkrish · Post by **g_rkrish** » Wed Dec 12, 2007 6:10 pm

russ356 wrote:We need some help. We have getting the error below on quite a few of our web services lately. The web services are "always on" with a minimum of 1 to a maximum of 5. The database it is connecting to is not down because we do a db upcheck before we process this job. It doesn't happen all the time and there is no set pattern that we can determine but it does happen at least 2 - 3 times a week. Any help or direction would be greatly appreciated.

Here are the versions of what we are running.
AIX 5.3
RTI : 7.5.2
DataStage : 7.5.1.a
IBM WebSphere Application Server, 6.0.0.1
Build Number: o0445.08
Build Date: 11/10/04

APT_CombinedOperatorController(1),0: Fatal Error: Fatal: <ns1:Fault xmlns:ns1=""http://schemas.xmlsoap.org/soap/envelope/"">
<faultcode>RTIException</faultcode>
<faultstring>java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:
java.rmi.RemoteException: ; nested exception is:
javax.ejb.EJBException: Exception trying to invoke operation ds_get_BC_translations: Job ds_get_BC_translations is not running. Check the RTIAgent error log for errors.</faultstring>

Does your webservice work from internet Explorer or any other browser...

russ356 · Post by **russ356** » Thu Dec 13, 2007 8:00 am

There is nothing unique or interesting about the job. It's actually very simple and it has been happening on or off for a few months now. I think we tried to bump the minimum up for I can't be sure. I will contact our admin today and have him bump it up. I also wanted to say that this is occuring with all of our "always on" web services. The error I posted is just an example but currently it occurs on all of them.

Below is the error from the RTIAgent job log.

2007-12-10 08:03:54,003 [1197226902062_PRC] ERROR com.ascentialsoftware.rti.agent.handler.datastage.PipeReceiver - [1197226902062_PRC]error during call to init: dspipe_init(1605642): open(/tmp/ade.HR_MGMT_PRD.ds_get_BC_translations.1197226902062.RTI_out) - No such file or directory

And finally, yes the web service works from IE or if we run the job manually.

lstsaur · Post by **lstsaur** » Thu Dec 13, 2007 12:40 pm

Whatever file in the /tmp directory is no longer there for this job. I had almost the same situation as yours. Rerun your job, ds_get_BC_translations, and check all the files.

russ356 · Post by **russ356** » Fri Dec 14, 2007 8:43 am

OK, I bumped up the minimum instances to 3 and it still abended today. In director it is giving me the same error.

APT_CombinedOperatorController(1),0: Fatal Error: Fatal: <ns1:Fault xmlns:ns1=""http://schemas.xmlsoap.org/soap/envelope/"">
<faultcode>RTIException</faultcode>
<faultstring>java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:
java.rmi.RemoteException: ; nested exception is:
javax.ejb.EJBException: Exception trying to invoke operation ds_get_BC_translations: Job ds_get_BC_translations is not running. Check the RTIAgent error log for errors.</faultstring>

And in RTIAgent Log it is giving me the following errors.

2007-12-14 09:34:06,796 [1197226906616_PRC] ERROR com.ascentialsoftware.rti.agent.handler.datastage.PipeReceiver - [1197226906616_PRC]error during call to init: dspipe_init(1605642): open(/tmp/ade.HR_MGMT_PRD.ds_get_BC_translations.1197226906616.RTI_out) - No such file or directory

I looked in the /tmp directory and this file does not exist but director says the job was running. Is there anything I need to look at to find out why this is occuring.

More info to maybe help. This is how we have the job setup in RTI.

Minimum = 3
Maximum = 5
Time to Live = 3600
Load Balancing Delay = 60
Absolute Max Idle Time = 0
Max Pipe Size = 5
Pipe Threshold = 3
Threshold Delay = 1000

eostic · Post by **eostic** » Sat Dec 15, 2007 3:56 pm

This sounds like it might be one for support to take a look at, but in the meantime, is there any connection to the problem's frequency and the number of nodes in your config file? Except in rare circumstances, a single node config is recommneded for RTI jobs that are always on...... since this was working for awhile but has recently been terminating randomly, have there been any config changes? How many nodes are there?

Ernie

russ356 · Post by **russ356** » Fri Jan 11, 2008 9:00 am

Here is an update. I changed the job that I mentioned in the earlier posts from an "Always On" web service to an "On Demand" web service. Thinking that maybe it had something to do with a time out somewhere. However, the same error occurred again calling the On Demand job. It has only happened once, but it still happened. I will be calling tech support on this issue but I was wondering if anyone has come across this issue.

Thanks

eostic · Post by **eostic** » Sat Jan 12, 2008 2:29 pm

This one is difficult because the error at the start of this thread is a resultant error --- it thinks, whether right or wrong, that job Job ds_get_BC_translations isn't running... or isn't available for some reason. The real error may be elsewhere --- in the job itself, or in conditions that can occur in the job that cause it to terminate....the environment, a particular data value, many possibilities. Anything in the DS log itself? You say that it happens when you make it "on demand".... Does that mean you are simply passing in Job Parms instead of using WISD Input? I'll assume that it is still using WISD Output? Maybe it has problems starting the job... ...have you played with the timeout settings in the WISD/RTI stages? These settings are concerned with the time it takes for the Agent to speak with the Job.....large jobs on EE with lots of stages may create issues because the job takes too long to initialize, and this could be dependent on what's happening on the machine at any given time....... are there any interesting pattersn to "when" the job fails?

Ernie

DSXchange

Web Service is not running error

Web Service is not running error

Re: Web Service is not running error