Getting Job Logs

[Andal]

I am planning to write a Batch file , to get the job logs from director and write it to individual text file based upon jobnames.

I read the manuls for dsjob and stuck at a point

dsjob -logsum (gives the short details of the entire log)
dsjob -logdetail gives all details of a particular event
dsjob -lognewest will give us the latest event id based upon the Type
(If we specify the type as Started, it is returning me the last event id,(Finishing of a job)
dsjob -jobinfo give me the latest start datetime of a job.

My question is how to get the Max and Min event id for a job , based on latest run.

Within DS we can use the query given by KIm Duke to find the event ids

Code: Select all

SELECT @ID FROM RT_LOG1 WHERE EVAL "@ID" NOT LIKE "//%" AND @ID > CAST(EVAL "FIELD(TRANS('RT_LOG1','//JOB.STARTED.NO',1,'X'), @SVM, 1)"AS DECIMAL) order by @ID,TIMESTAMP 

But while coming to Batch file, how can i get the event ID's?

I know that some posts by David give me exactly what i want. But I am having some access rights for creating files in the Ascential server. For that only , i decided to write a batch file.

[loveojha2]

Hey Anand,
What about this

Code: Select all

SELECT * FROM RT_LOGnn WHERE TIMESTAMP=(SELECT MAX(TIMESTAMP) FROM RT_LOGnn)

This will give all the records of last run.

Where nn is the job no.

[Andal]

Love

I want to know, how to get the eventid's in the batch file.To run ur qry , i should be in Datastage Environment.

[palmeal]

This is what I have done to get the log information for datastage job runs.

(1) RESET Job before running it and then run following commands

dsjob -lognewest $Project $SequencerName.$DSN > Filename
first_event_id=`cat FileName |grep Newest | awk '{print $4}'`
first_event_id=`expr $first_event_id + 1`

(2) At end of job run following command
dsjob -lognewest $Project $SequencerName.$DSN > FileName
second_event_id=`cat FileName |grep Newest | awk '{print $4}'`

This gets the First/Last event id's which enable you to go to the director logs and pull all the info that you require with the command below.

dsjob -logdetail $Project $SequencerName.$DSN $first_event_id $second_event_id > LogFileName.

Note though that if you have multiple instances(invocations) of the same job running then this eventid range may pull out log information
for other instances. I have this and have a perl script that goes through the lof file produced and rips out the required data based on the
.$DSN value.

Hope this is what you were looking for.

[kaps]

Can you please tell me why do you want to reset the job as a first step ?
what If I just run the lognewest command before and after running job and use that to get the latest logs ? As the reset operation may cause the checkpoints to loose track in case of restart...

or is there any way we can just reset the log entry...i.e.similar to dsj.logreset in unix ?

[johnshot]

I'm not going to get fancy here, but here is a quickie we used to
identify pid issues on our box: UNIX
This used 2 routines, one before and one after.
This code is ONLY to show basic approach for others who want to
do something similar, so please don't post complaints about it.
variables are identified with v_

1. First we wrote the Last Pid of the previous run to a file:
*Capture the Last Log ID
CMD=" ":v_DSPath/dsjob:" -lognewest ":v_DS_ProjectName:" ":v_JobName:" |awk -F= '{print $2}'"
Call DSExecute("UNIX",CMD,v_ReturnCd,SysRetCode)

*Clean up the Return-(Remove embedded FM marks)
v_ReturnValue=TRIM(FIELD(CONVERT(@FM,"|",v_ReturnValue),"|",3))

CMD="`echo ":v_ReturnValue:" > ":v_JobLastIDFileName:" `"
Call DSLogInfo("Last ID CMD =":CMD,RoutineName)
Call DSExecute("UNIX",CMD,ReturnCd,SysRetCode)
2. In After Routine, get the last Event ID and write the FULL DETAILS
of the log to a UNIX file:

*get the start id
*Get the Job Start Log ID from the file created at beginning of job
*Capture the Last Log ID
CMD="awk '{print $1}' ":v_JobLastIDFileName:" ;"
Call DSLogInfo("Executing Command":CMD,RoutineName)
Call DSExecute("UNIX",CMD,v_StartReturnCd,SysRetCode)
v_StartReturnValue=CONVERT(@FM,"",v_StartReturnCd)

Get the End ID (not really the LAST, just when Post-job routine is called)
*Capture the Last Log ID
CMD=" ":v_DSPath:" -lognewest ":v_DS_ProjectName:" ":v_JobNameIn:" |awk -F= '{print $2}'"
Call DSLogInfo("Executing Command":CMD,RoutineName)
Call DSExecute("UNIX",CMD,EndReturnCd,SysRetCode)
*Clean up the last entry
EndReturnValue=TRIM(FIELD(CONVERT(@FM,"|",EndReturnValue),"|",3))


*Return the Full Details
*Return FULL Details
CMD=" ":v_DSPath:" -logdetail ":v_DS_ProjectName:" ":v_JobNameIn:" ":StartReturnValue:" ":EndReturnValue:" > ":v_JobLogName:";"
Call DSLogInfo("Executing Command":CMD,RoutineName)
Call DSExecute("UNIX",CMD,EndReturnCd,SysRetCode)