Disk Space getting filled

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
kamesh_sk
Participant
Posts: 43
Joined: Fri May 27, 2005 7:58 am

Disk Space getting filled

Post by kamesh_sk »

HI,

i have a 14G space allocated to /opt/IBM folder but this space gets filled by certain log files . I have found certain files under /opt/IBM/InformationServer/ASBNode/bin occupying around 4G of space.
The file names are
-rw-r--r-- 1 root root 1006275756 Apr 7 13:37 orbtrc.05042010.2211.46.txt
-rw-r--r-- 1 root root 465369 Apr 7 14:11 orbtrc.07042010.1402.21.txt
-rw-r--r-- 1 root root 606381173 Apr 8 14:26 orbtrc.07042010.1413.37.txt
-rw-r--r-- 1 root root 1639260160 Apr 13 14:11 orbtrc.08042010.1426.47.txt
-rw-r--r-- 1 root root 4122763264 Apr 13 14:11 logging-agent.err

Can any one help to understand what are these files are and why such huge size files are created and by which Datasatge operation and what should i do to avoid it.
Due to size issue my jobs are getting aborted.
Can anyone help please

Thanks
Kamesh
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

From the names one can see they are log files. Particularly 'logging-agent.err'. Have you looked into this file to see what errors it is logging?
kamesh_sk
Participant
Posts: 43
Joined: Fri May 27, 2005 7:58 am

Post by kamesh_sk »

The entire Gb space conatains this error.

The file keeps growing

; nested exception is:
javax.ejb.EJBException: nested exception is: java.sql.SQLException: [IBM][Oracle JDBC Driver][Oracle]ORA-01653: unable to extend table XMETA_OWNER.LOGGING_LOGGINGEVENT1466CB5F by 8192 in tablespace INFOS_XMETA_DATA

at com.ascential.acs.logging.server.impl.JDBCLoggingEventDAO.insert(JDBCLoggingEventDAO.java:1031)
at com.ascential.acs.logging.server.impl.JDBCLoggingEventDAO.create(JDBCLoggingEventDAO.java:401)
at com.ascential.acs.logging.server.impl.appender.OperationalRepositoryAppender.append(OperationalRepositoryAppender.java:110)
at com.ascential.acs.logging.server.impl.appender.AppenderController.doAppend(AppenderController.java:73)
at com.ascential.acs.logging.server.impl.appender.OperationalRepositoryAppender.doAppend(OperationalRepositoryAppender.java:84)
at com.ascential.acs.logging.server.impl.config.LoggerImpl.appendLoopOnAppenders(LoggerImpl.java:713)
at com.ascential.acs.logging.server.impl.config.LoggerImpl.callAppenders(LoggerImpl.java:657)
at com.ascential.acs.logging.server.impl.config.LoggerImpl.forcedLog(LoggerImpl.java:615)
at com.ascential.acs.logging.server.impl.config.LoggerImpl.log(LoggerImpl.java:533)
at com.ascential.acs.logging.server.impl.LoggingServiceBean.log(LoggingServiceBean.java:479)
at com.ascential.acs.logging.server.impl.LoggingServiceBean.log(LoggingServiceBean.java:497)
at com.ascential.acs.logging.server.EJSRemoteStatelessLoggingService_5c03a87a.log(Unknown Source)
at com.ascential.acs.logging.server._EJSRemoteStatelessLoggingService_5c03a87a_Tie.log__CORBA_WStringValue__java_util_Collection(_EJSRemoteStatelessLoggingService_5c03a87a_Tie.java:553)
at com.ascential.acs.logging.server._EJSRemoteStatelessLoggingService_5c03a87a_Tie._invoke(_EJSRemoteStatelessLoggingService_5c03a87a_Tie.java:152)
at com.ibm.CORBA.iiop.ServerDelegate.dispatchInvokeHandler(ServerDelegate.java:614)
at com.ibm.CORBA.iiop.ServerDelegate.dispatch(ServerDelegate.java:467)
at com.ibm.rmi.iiop.ORB.process(ORB.java:439)
at com.ibm.CORBA.iiop.ORB.process(ORB.java:1761)
at com.ibm.rmi.iiop.Connection.respondTo(Connection.java:2376)
at com.ibm.rmi.iiop.Connection.doWork(Connection.java:2221)
at com.ibm.rmi.iiop.WorkUnitImpl.doWork(WorkUnitImpl.java:65)
at com.ibm.ejs.oa.pool.PooledThread.run(ThreadPool.java:118)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1475)
Caused by: javax.ejb.EJBException: nested exception is: java.sql.SQLException: [IBM][Oracle JDBC Driver][Oracle]ORA-01653: unable to extend table XMETA_OWNER.LOGGING_LOGGINGEVENT1466CB5F by 8192 in tablespace INFOS_XMETA_DATA
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

kamesh_sk wrote:...unable to extend table XMETA_OWNER.LOGGING_LOGGINGEVENT1466CB5F by 8192 in tablespace INFOS_XMETA_DATA...
So that file seems to be growing because of table extent issues. Fix those and the error log will (probably) stay small. What kind of contents do the other files have?
battaliou
Participant
Posts: 155
Joined: Mon Feb 24, 2003 7:28 am
Location: London
Contact:

Post by battaliou »

Ok, looks like you got a serious problem here. My first instinct is that your "server side tracing" is on. Check it and turn it off using the datastage administrator app. Also check if any of your load jobs are generating warnings, this fills up log space quickly. You will need to tune your jobs if they do. You can lose the orbtrc.05042010.2211.46.txt files without any impact to your system. I'd ditch the logging-agent.err file too.
3NF: Every non-key attribute must provide a fact about the key, the whole key, and nothing but the key. So help me Codd.
kamesh_sk
Participant
Posts: 43
Joined: Fri May 27, 2005 7:58 am

Post by kamesh_sk »

Thanks very much for the timely information.

We have few quality stage Job which is deployed thru WISD to make it available for MDM application.
One of the job is aborting , (reason not known since RTLogging was not enabled).
Since the job is called by WISD , it tries to run continuously with different Invocation ID.
So eventually what happens is the job aborts every second and restarts again with new invocation ID.
This process generates huge amount of log and they are written into table XMETA_OWNER.LOGGING_LOGGINGEVENT1466CB5F by in tablespace INFOS_XMETA_DATA. The Tablespace gets full after some time and then it starts building up error log under /opt/IBM/InformationServer/ASBNode/bin file name logging-agent.err.

This logging-agent.err file contains nothing but the information saying the tablespace is filled. and this file grows bigger in size and occupies the entire disc space.This file has grown to a size of staggering 5GB
This is building up a huge issue.

All i need to know is whether the table XMETA_OWNER.LOGGING_LOGGINGEVENT1466CB5F in tablespace INFOS_XMETA_DATA will be populated only for errors by qualitystage jobs or any log generated by the Datastage project.
Is it possible to disable this logging? what will be the option to disable logging and what will be its effects.

The RTLogging was kept as 0 and hence no log was generated in the director and it was just changed to 1.
Will this have an effect in the way the log information is handled .

How is logging handled with this version , is it not generated as files , ,or does it populates the Repository tables

Kindly help
DSShishya
Premium Member
Premium Member
Posts: 37
Joined: Tue Oct 27, 2009 9:43 pm

Post by DSShishya »

Did you have any success in resolving this issue?

If Yes, please give details for everyone's benifit.
DSShishya
Premium Member
Premium Member
Posts: 37
Joined: Tue Oct 27, 2009 9:43 pm

Post by DSShishya »

Did you have any success in resolving this issue?

If Yes, please give details for everyone's benifit.
Post Reply