datastage job having high start up time

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Lavanya B
Participant
Posts: 20
Joined: Mon Oct 30, 2006 12:32 am

datastage job having high start up time

Post by Lavanya B »

Hi,
There is a datastage job with Teradata Enterprise stage and Teradata API stages as the source. Data is being loaded to Oracle. All the stages in this job are running in sequential mode.
The start up time of this job is low most of the times. Sometimes the start up time is very high(approx 15 min, once it was approx 45min :( ).
What could be the reason for such high start up time. Why is the job having inconsistent start up times? Is there any environment variable/setting in the DB stage to control the start up time?
Could anyone please help on this?
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

Check in the stage properties for any before-SQL statements. The last time I checked, those SQL executions were counted as startup time.
Choose a job you love, and you will never have to work a day in your life. - Confucius
Lavanya B
Participant
Posts: 20
Joined: Mon Oct 30, 2006 12:32 am

Post by Lavanya B »

There are no before SQL statements that are executed in the source Teradata stages.
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

It's not just before SQL that appears to get captured as Startup time.

From what I've observed, it appears to be a catch-all bucket for any time not specifically measured elsewhere.

Overhead would probably be a better name for it.

Before SQL, after SQL, time spent computing database statistics are all things I've seen captured as "Startup" time.

Mike
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

And, of course, it includes the "real" startup time of waiting for computing resources to become available.

Since you're using Teradata, any chance that you may be waiting for a utility slot to free up?

Mike
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

One last thought...

Since the long startup time is not consistent from run to run, it is probably not directly related to something that your job is doing; but rather, to something that is happening in your environment.

Mike
weiyi_will
Participant
Posts: 10
Joined: Sun Aug 11, 2013 10:46 pm
Location: Dalian

Post by weiyi_will »

I ever also meet the case that it include execution time of after SQL as start up time.
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

My money is on the waiting for Teradata sessions.


Try requesting less sessions and see if your time reduces.

If you have a small result set, request less sessions.

Since all of your stages are in sequential mode (for a parallel job), I am guessing you have a very small amount of data.
ozgurgul
Premium Member
Premium Member
Posts: 9
Joined: Tue Jan 31, 2006 9:07 am

Post by ozgurgul »

Hi there - have a look at this APAR..

JR49593: INFORMATION SERVER 9.1 PARALLEL JOBS HAVE LONG STARTUP TIMES DURING SCHEMA SETUP

Hope it helps..
Ozgur GUL
Assumption is the mother of all mistakes!
Post Reply