DSXchange

Hi guys,
I have devised what I thought was a method of running an infinite loop in a sequence job. Basically a DO WHILE. I will explain the process in a moment but I went for this method as opposed to using the Loop Activity stages because I wanted the loop to exit on a condition, which those activities did not appear they would do.

My job layout is as follows:

Sorry for the weird chemical formula looking diagram, my employer does not allow screenshots of anything internal to uploaded. Basically the job is designed to do the following:
Wait for a trigger file, then go to the first sequencer, SEQ1, set to ANY. Flow then continues to CMD1 which checks the status of a job in a different project (dsjob command). This output is sent over the the first Nested Condition, NC1, which runs an index() on the output to see if the job is running or not. If NOT then sends the flow to a sleep command, SLP1, which waits for a time and sends flow back to SEQ1 to repeat until the job being checked is in a running status. At which point flow then hits the second sequencer, SEQ2, set to ANY, and drops down to it's command, CMD2. This checks the same job status again and flows to the second Nested Condition, NC2. Here the job status is checked to see if the job completed, OK or with warnings, or is still running, or is not in any of those statuses. If it's still running SLP2, sleep, will be triggered and flow should go back around to check the status again until the job is completed or failed (no longer running nor completed). If completed, NC2 will trigger the link to run a job. If not completed and not running: Then an email will be sent. My issue is that when I tested this I got both a FAILED link to run AND the Still Running link. The log then showed the following message (not error):
RawStageTrigger..JobControl (@Coordinator): Note: Sequencer 'SEQ2' was entered, but never exited

Any ideas why?

Upon further review, sometimes typing things out, reading it, then reading it again in a few minutes helps.... I see that my Or conditions in the code example I gave, should be And conditions....

Ha! No-one has ever had that problem before.

Ok, so tested this again and confirmed it still doesn't work. It appears the job shuts down when the flow goes back to a sequencer that was not exited. Anyone know a functional way to run an infinite loop in a sequence job? DO/WHILE?

"Note: Sequencer 'SEQ2' was entered, but never exited"

that means you have it set to "All" and not all input links were triggered. Maybe it needs to be "Any"? Sorry, haven't taken the time to fully absorb your design / requirements but thought that might help. And on the "infinite loop" question I really don't think that's possible.

The two sequences used are definitely set to ANY, datastage just doesn't seem to like running a loop back to the same sequence. Essentially it bombs when the process flow follows a path as follows:

Sequence Stage, Exec Command, Nested Condition, Exec Command, back to Sequence Stage. This is laid out in a circle.

Ok, I went around the rules a bit here and got the image uploaded here:
https://ibb.co/iOzRHf

Hmmm... from what I recall they don't like "circles" either but there's a specific fatal error for that, I thought. But maybe that's the root of your issue.

Okay, just looked at your canvas. The "circles" I meant were when the end loops back to the beginning, I have vague memories of trying to do something like that back in the day and getting busted for it. But still, there's no way even those smaller sequencer circles will work like that. Perhaps as a Start/End Loop construct? Easy enough to fly around in one if conditions aren't met and then exit past the End Loop to move forward when they are. From what I recall.

Seems like I ran into that circular error once myself and just moved on without trying to debug it.

We use begin/end loops with parameters for the amount of sleep time and number of retries. While it's not an infinite loop, you could set the number of retries to a very high number if that would help.

We set these up at the highest level of our sequence jobs and place that job on schedule. We did that in order to auto-retry failed jobs, because most of the time our jobs fail in the middle of the night is because of a temporary network glitch. A simple automatic retry (when jobs are set to the action of reset then run) is able to work around the shoddy network in most cases.

Ok, using the loop stages was my initial plan but I figured I would try doing an infinite. Not a huge deal, set the number of steps to 9999 with 5 second wait times that still 13 hours a loop could run. MORE than enough time. I do have an issue however..... the first loop setup fine and should exit to another loop. However, the second loop will not allow me to return link on the End Loop stage back to the Start Loop stage for that loop. Please see image here:
https://ibb.co/n3qbk0

If I remove the "Running" link from StartLoop_Activity_155 I can then link the End Loop to the start but it's a standard link, not the dotted line link (sorry the name escapes me)

That's odd and sounds like a bug in Designer. The same job design works fine on my Designer v11.3.1.2. What's the exact version you are on? We ran into something similar one time on a dot-zero release and had to get a patch from Support.

Hmmm wonder if that is it... I was able to trick it to giving me a link from the end loop back to the start loop by removing the end loop stage, adding a new one with no incoming links to it. Linking that to the start loop, which correctly showed the dotted line link. Then re-linking things to the end loop. So the designer shows all the links are there correctly. However, when I compile it still says the start loop has no end loop that links back to it.... We're on 11.5.0.1. I may open a case with IBM on it....

IBM has confirmed the job compiles on their end using 11.5.0.2, working on getting patches installed.

Marking this as Resolved since IBM confirmed an issue with rollup version in our environment. No idea when it'll get fixed here...