Extract fatal log entries.

[rkashyap]

We have a business need to generate an email with Fatal log entries for latest invocation of the job (if the DataStage job has aborted). This job will be running several times day.

We are planning to create a script for this. To extract fatal log entries ... we can use

1.

Code: Select all

dsjob -logsum -type FATAL <ProjectNm> <JobNm>

This extracts 'All' fatal log entries available (i.e it includes information from previous invocations also).
OR
2.

Code: Select all

dsjob -lognewest  <ProjectNm> <JobNm> FATAL

This extracts 'Last' fatal log entry for the latest invocation only.

Please advise, if there is another approach/command to extract all the fatal log entries for latest invocation of the job.

[ray.wurlod]

Yes, but not if you insist upon using command line utilities. Check out the DataStage API (Programmer's Guide) to learn how to code it more exactly suited to your particular requirements, for example filtering on timestamps and establishing the start timestamp of the most recent run.

[stuartjvnorton]

Have you got the Ops Console set up?
If so, you could query DSODB.

[chulett]

Yes, but not if you insist upon using command line utilities.

I'm not aware of anything in the API that does not have a command line equivalent, so it should just be a choice between BASIC coding and shell scripting. Yes?

[rkashyap]

Thanks for your suggestions. Appreciate it.

We do not have DSODB setup yet, so we are:
1. Establishing the start timestamp of the most recent run.
2. Extracting subsequent fatal log entries.

Code: CaptureFatalMsg.ksh

Code: Select all

#!/bin/ksh
#set -x
## Argument Validation
EXPECTED_ARGS=2
if [ $# -ne $EXPECTED_ARGS ];then
  echo "Invalid Number of Arguments: EXPECTED_ARGS=2. Usage: ksh `basename $0` <ProjectName> <JobName/JobName.InvocationId>"
  echo "Example : ksh `basename $0` PROJ1 JOB1.INVC1"
  exit
fi

## Intialize VariableS

# Set Datastage home path and intialize  dsenv
DSHOME=/opt/IBM/InformationServer/Server/DSEngine; export DSHOME
. ${DSHOME}/dsenv
CMDDIR=${DSHOME}/bin; export CMDDIR

# Initialize  script Arguments
ProjectName=$1
Job=$2

# Initialize iteration Variable
i=0
EventIDLimit=50

## Get start event  id of the last job   run
StartEventID=`${CMDDIR}/dsjob  -logsum -type STARTED $ProjectName $Job 2>&1 | nawk 'ORS=(FNR%2)?FS:RS' | awk '/Starting/ {a=$1} END{print a}'`

## Get  fatal ids of the last job failure
LastJobFailureFatalIDList=`${CMDDIR}/dsjob -logsum -type FATAL $ProjectName $Job 2>&1 | awk '/FATAL/ {print $1}'`

## Perform iteration to  capture error messages
 for EventID in ${LastJobFailureFatalIDList}
  do
    if [ "${EventID}" -gt "${StartEventID}" ]  &&  [ "${i}" -lt "${EventIDLimit}" ]
      then
           EventIDErrorMsg=`${CMDDIR}/dsjob  -logdetail $ProjectName $Job   ${EventID} 2>/dev/null`
           ErrMsg="${ErrMsg} ${EventIDErrorMsg}"
           i=`expr $i + 1`
    fi
 done

## Display Last Job Run Error Message
echo $ErrMsg| nawk '{print substr($0,1,4000)}'