Master Sequence is not restartable

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

parag.s.27
Participant
Posts: 221
Joined: Fri Feb 17, 2006 3:38 am
Location: India
Contact:

Master Sequence is not restartable

Post by parag.s.27 »

After browsing through all the posts on restartability, I am still not able to achieve it in my scenario. The condition goes like: -

We are having a scenario where we are having multiple subsequences. Lets say

S1, S2, S3
Now these sequences are having a custom trigger where we are checking job status of each job and if its not equal to 1 then we are terminating the sequence.
Also we are having following parameters set in Administrator: -

1.Add checkpoints so that sequence is restartable on failure.
2.Automatically handle activities that fail.
3.Log warnings.......

Now we are calling these subsequences in a master sequence M1 as: -

Code: Select all

S1---->S2---->S3
. We have attached an Exception Handler with a terminator.
But still if any of the subsequence S1, S2 or S3 fails then the master sequence M1 is not getting aborted, but is showing Finished(See Log), hence the master sequence is not restartable. Also in master sequence I can not use jobstatus logic because there are so many of these subsequences. Can anyone suggest how to achieve this.
Thanks & Regards
Parag Saundattikar
Certified for Infosphere DataStage v8.0
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Your 'Master Sequence' must abort to trigger the restartability. You need to make sure that happens, either using the 'Automatically handle' option or specific triggers and stages.
-craig

"You can never have too many knives" -- Logan Nine Fingers
parag.s.27
Participant
Posts: 221
Joined: Fri Feb 17, 2006 3:38 am
Location: India
Contact:

Post by parag.s.27 »

I do not understand, why the exception handler in master sequence is not working if in this case the subsequence is getting aborted. The exception handler is not able to send a signal to the connected Terminator if the subsequence is having an abort.
Thanks & Regards
Parag Saundattikar
Certified for Infosphere DataStage v8.0
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

That's not how the Exception Handler works. And adding it overrides the 'automatically handle' option. Remove it, use just OK triggers and let us know what happens.
-craig

"You can never have too many knives" -- Logan Nine Fingers
parag.s.27
Participant
Posts: 221
Joined: Fri Feb 17, 2006 3:38 am
Location: India
Contact:

Post by parag.s.27 »

chulett wrote:That's not how the Exception Handler works. And adding it overrides the 'automatically handle' option. Remove it, use just OK triggers and let us know what happens. ...
As per the documentation on 'Automatically Handle' option, It will not work if there are any sort of triggers involved. And if there is an exception handler in the master sequence then the 'Automatically Handle' option will send the error code to that exception handler and it will abort the job.

But I can not use it since I need to have 'Execute OK' trigger in between my subsequences. But still I'l try removing the exception handler and will let u know.
Thanks & Regards
Parag Saundattikar
Certified for Infosphere DataStage v8.0
parag.s.27
Participant
Posts: 221
Joined: Fri Feb 17, 2006 3:38 am
Location: India
Contact:

Post by parag.s.27 »

Hi Chulett,

As you suggested, I tried running the master sequence after removing the exception handler and kept the triggers intact. But still the master sequence was showing 'Finished(See Log)'. And it was not getting aborted.

I think the only way is now to connect each subsequence activity to a sequencer(Operational Mode = 'Any'). And each link will be Custom Condition on job status. offcourse it will be creating complexity as we are having almost 140 subsequences. But eventually it will ensure restartability.. :roll:
Thanks & Regards
Parag Saundattikar
Certified for Infosphere DataStage v8.0
balajisr
Charter Member
Charter Member
Posts: 785
Joined: Thu Jul 28, 2005 8:58 am

Post by balajisr »

Craig did not ask you to remove exception handler. He had asked you to remove custom triggers and use OK triggers instead.
parag.s.27
Participant
Posts: 221
Joined: Fri Feb 17, 2006 3:38 am
Location: India
Contact:

Post by parag.s.27 »

balajisr wrote:Craig did not ask you to remove exception handler. He had asked you to remove custom triggers and use OK triggers instead.
May be you have not read my full reply properly.
I've done the same thing...Later I said the alternative to it is to use the custom trigger.

May be if you have some good solution then You can share
Thanks & Regards
Parag Saundattikar
Certified for Infosphere DataStage v8.0
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Actually, Craig did specifically ask him to remove the Exception Handler. :wink:
parag.s.27 wrote:As you suggested, I tried running the master sequence after removing the exception handler and kept the triggers intact. But still the master sequence was showing 'Finished(See Log)'. And it was not getting aborted.
What exact options do you have enabled in the Job Properties of the Master Sequence job? Are you using Job Activity stages to run the subsequences?

With the correct options, when the "subsequence" aborts, if the master Job Activity stage just has an "OK" trigger to the next stage (and no exception handler) then it should abort as well.
-craig

"You can never have too many knives" -- Logan Nine Fingers
balajisr
Charter Member
Charter Member
Posts: 785
Joined: Thu Jul 28, 2005 8:58 am

Post by balajisr »

Actually, Craig did specifically ask him to remove the Exception Handler.
Oops! i had misinterperted your statement. Sorry for that.

Parag,

Also post your custom trigger code.
parag.s.27
Participant
Posts: 221
Joined: Fri Feb 17, 2006 3:38 am
Location: India
Contact:

Post by parag.s.27 »

balajisr wrote:
Actually, Craig did specifically ask him to remove the Exception Handler.
Oops! i had misinterperted your statement. Sorry for that.

Parag,

Also post your custom trigger code.
Hey Not a problem...

Basically for e.g, My job design after using custom trigger, is restartable.

There are 3 subsequences S1, S2, S3 in my Master Sequence M1. The design is: -

Code: Select all

       S1--Exec OK------>S2---Exec OK---->S3
        |                |                |
 S1.JobStatus<>1    S2.JobStatus<>1    S3.JobStatus<>1
        |                |                |
       Common Sequencer used with mode = Any
                         |
                   Terminator
Thanks & Regards
Parag Saundattikar
Certified for Infosphere DataStage v8.0
balajisr
Charter Member
Charter Member
Posts: 785
Joined: Thu Jul 28, 2005 8:58 am

Post by balajisr »

Looks like you are handling failure explicitly due to which the master job sequence is not aborted.

Verify whether this design works.

Code: Select all


S1 ---OK Trigger -----> S2 ------OK Trigger ----->S3 
And Set "Automatically handle activites that fail" to true by selecting the check box in master sequence job properties.

Now your master job sequence should abort.
parag.s.27
Participant
Posts: 221
Joined: Fri Feb 17, 2006 3:38 am
Location: India
Contact:

Post by parag.s.27 »

balajisr wrote:Looks like you are handling failure explicitly due to which the master job sequence is not aborted.

Verify whether this design works.

Code: Select all


S1 ---OK Trigger -----> S2 ------OK Trigger ----->S3 
And Set "Automatically handle activites that fail" to true by selecting the check box in master sequence job properties.

Now your master job sequence should abort.
No actually, The design you showed was the original one with "Automatically Handle" option set. I also tried with Exception Handler. So finally the design I showed is the one in which I am getting successful restartability process. i.e. It is working fine..But in practical in my project we have around 140 subsequences, so it will be very difficult to implement this logic.
Thanks & Regards
Parag Saundattikar
Certified for Infosphere DataStage v8.0
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

You need to open a case with your Support provider. The need for all that triggering and explicit abort handling went out the window when the 'Automatically handle' options came on the scene. The simple 'OK trigger only' flow should do exactly what you need - I do this many places in our 7.5.x release, perhaps there is an issue introduced with this in the 8.x release?

ps. In the design as balajisr posted, you'll need a final Sequencer to allow proper restart handling of S3:

Code: Select all

S1 ---OK Trigger -----> S2 ------OK Trigger -----> S3 ------OK Trigger -----> Sequencer
pps. You never answered my question as to which exact options of the four available on the Sequence job's Properties page that you had enabled when this didn't work.
-craig

"You can never have too many knives" -- Logan Nine Fingers
gateleys
Premium Member
Premium Member
Posts: 992
Joined: Mon Aug 08, 2005 5:08 pm
Location: USA

Post by gateleys »

Since S1, S2 and S3 themselves are also sequences, and you have been successful in getting them to abort (if any of its job failed), you would apply the same logic to get the Master sequence (say M) to abort in case any of S1, S2 or S3 failed.

So, can you tell us about your job design for either S1, S2 or S3 that causes it to abort when any of its job fails?
gateleys
Post Reply