"No output from command " from routine stage

[ray.wurlod]

I was under the impression that what you really wanted was a success/otherwise result from the routine.

On that basis I have been recommending returning any other information by other means, so that Ans contains only the success/otherwise value.

[RodBarnes]

I was under the impression that what you really wanted was a success/otherwise result from the routine.

On that basis I have been recommending returning any other information by other means, so that Ans contains only the success/otherwise value.

Oh, then I understand your recommendation and generally agree with it. However, in this case, the problem is that the routine works fine when run via the [Test] button but fails when in a stage. Tagging the output to the result in Ans, allows me to easily see what is coming through from the command during [Test] verses what I see when running from a stage. I don't know of any other way to see the output from the command while running via the [Test] button.

Whether Ans contains only the result of a combined value from which the result must be parsed -- FIELD(return_value,@TM,1) -- really doesn't affect the operation.

Once I resolve the discrepancy, certainly, I will only have Ans return the result -- which is my desired outcome. Until then, I'm just trying to get it working.

[RodBarnes]

It may be moot but can you post the full log entry from executing this from (I assume) a Routine Activity stage? And just for grins, what exact O/S and Service Pack are you running?

This is Windows 2003 Server SP2, running DS 7.5

Unfortunately, the log doesn't show much. I've included the pertinent bits below. (The entire log is 204 lines long so I didn't think you'd want me posting it here but I still can if you really want me to.) I've added a couple calls to DSLogInfo to output the "cmd:" and the "out:". Obviously because of the error, it isn't getting to the point of showing any output -- which implies to me that it isn't able to even invoke the command correctly.

You can see in this log I even tried giving the fullpathname to the FC command but that made no difference.

What could be different in context between running the routine from the [Test] button (where it works successfully) vs. running it from a Routine Activity?

Code: Select all

   Item #: 4
   Event ID: 3
   Message: CompareFiles2..JobControl (@ControlVars): Set control variables...

   Item #: 5
   Event ID: 4
   Message: CompareFiles2..JobControl (CompareFiles): cmd:C:\Windows\system32\FC /B "C:\Data\Ascential\DataStage\Staging\Orbit_rod\Test\Temp\SourceIssues_Rapid.csv" "C:\Data\Ascential\DataStage\Staging\Orbit_rod\Test\Temp\xSourceIssues_Rapid.csv"

   Item #: 6
   Event ID: 5
   Message: CompareFiles2..JobControl (CompareFiles): out:

   Item #: 7
   Event ID: 6
   Message: CompareFiles2..JobControl (@CompareFiles): Routine DSU.CompareFiles did not finish OK, return code = '-1'

   Item #: 8
   Event ID: 7
   Message: CompareFiles2..JobControl (@CompareFiles): Controller problem: Unhandled failure (-1) encountered calling routine DSU.CompareFiles

   Item #: 9
   Event ID: 8
   Message: CompareFiles2..JobControl (@Coordinator): Summary of sequence run
14:43:15: Sequence started
14:43:15: CompareFiles (ROUTINE DSU.CompareFiles) started
14:43:16: CompareFiles finished, reply=-1
14:43:16: Exception raised: @CompareFiles, Unhandled failure (-1) encountered calling routine DSU.CompareFiles
14:43:16: Sequence failed

   Item #: 10
   Event ID: 9
   Message: CompareFiles2..JobControl (fatal error from @Coordinator): Sequence job will abort due to previous unrecoverable errors

[ray.wurlod]

OK, without explicit triggers a job sequence will interpret a Routine activity as having failed if its return value is anything other than zero.

Add an explicit Failure trigger so that "failures" are handled, even though the trigger goes to exactly the same place as the Success trigger (through a Sequencer, of course).

[RodBarnes]

OK, without explicit triggers a job sequence will interpret a Routine activity as having failed if its return value is anything other than zero.

Add an explicit Failure trigger so that "failures" are handled, even though the trigger goes to exactly the same place as the Success trigger (through a Sequencer, of course).

That will only prevent the sequence from aborting because of the routine failure. It doesn't alter the fact that the routine is failing as can be seen by the -1 return value regardless of the state of the files. Yet I can run the routine via the [Test] button and it returns the correct values -- a 0 if the files are the same, 1 if they are different, and 2 if one or more is missing. Again, the routine works fine when run via the [Test] button. The same command using the exact same parameters works fine from the command-line. I can test this because I am writing the cmd to the log and can cut-paste it to the command-line and it works, producing the correct output.

If I had a place to do so, I could post images of these results and it would demonstrate these results.

Yet, to eliminate the abort of the sequence as a factor, I went ahead and added the failure trigger and have included the pertinent log entries below. Note that it still returns a -1 failure from the routine and produces no output:

Code: Select all

   Item #: 4
   Event ID: 3
   Message: CompareFiles2..JobControl (@ControlVars): Set control variables...

   Item #: 5
   Event ID: 4
   Message: CompareFiles2..JobControl (CompareFiles): cmd:C:\Windows\system32\FC /B "C:\Data\Ascential\DataStage\Staging\Orbit_rod\Test\Temp\SourceIssues_Rapid.csv" "C:\Data\Ascential\DataStage\Staging\Orbit_rod\Test\Temp\xSourceIssues_Rapid.csv"

   Item #: 6
   Event ID: 5
   Message: CompareFiles2..JobControl (CompareFiles): out:

   Item #: 7
   Event ID: 6
   Message: CompareFiles2..JobControl (@CompareFiles): Routine DSU.CompareFiles did not finish OK, return code = '-1'

   Item #: 8
   Event ID: 7
   Message: CompareFiles2..JobControl (@check): Check result...

   Item #: 9
   Event ID: 8
   Message: CompareFiles2..JobControl (@OTHER): Record result

   Item #: 10
   Event ID: 9
   Message: CompareFiles2..JobControl (@FNAME1): Record Filename1

   Item #: 11
   Event ID: 10
   Message: CompareFiles2..JobControl (@FNAME2): Record Filename2

   Item #: 12
   Event ID: 11
   Message: CompareFiles2..JobControl (@Coordinator): Summary of sequence run
15:38:55: Sequence started
15:38:55: CompareFiles (ROUTINE DSU.CompareFiles) started
15:38:55: CompareFiles finished, reply=-1
15:38:55: OTHER (ROUTINE DSU.WriteValue) started
15:38:55: OTHER finished, reply=0
15:38:55: FNAME1 (ROUTINE DSU.WriteValue) started
15:38:55: FNAME1 finished, reply=0
15:38:55: FNAME2 (ROUTINE DSU.WriteValue) started
15:38:55: FNAME2 finished, reply=0
15:38:55: Sequence finished OK

   Item #: 13
   Event ID: 12
   Message: Finished Job CompareFiles2.

End of report.

[chulett]

If I had a place to do so, I could post images of these results and it would demonstrate these results.

You could upload them to any file sharing site, places like ImageShack and MediaFire (amongst others) are still free AFAIK. Then you can include a link to the images in your post and the sites will generally provide the tags for you.

[ray.wurlod]

Can you post the code for CompareFiles if it's changed from since you posted it, and include some DSLogInfo() calls to log what actual pathnames the routine receives? Presumably the -1 codes are caused by OpenSeq failing.

If it seems fruitful can you please add all the pertinent clauses to each OpenSeq statement?

Code: Select all

File1OpenedOK = @False

OpenSeq FileName1 To fvar1
On Error
   Msg = "System error opening file ": FileName1 : ", error code = " : Status()
End
Locked
   Msg = "File ": FileName1 : " locked by user number " : Status()
End
Then
   File1OpenedOK = @True
   Msg = "File " : FileName1 : "opened OK."
End
Else
   If Status() = 0
   Then
      Msg = "File " : FileName1 : " does not exist."
   End
   Else
      Msg = "Unable to open file " : FileName1 : ", code = " : Status()
   End
   Call DSLogInfo(Msg, "Debugging")
End

[RodBarnes]

If I had a place to do so, I could post images of these results and it would demonstrate these results.

You could upload them to any file sharing site, places like ImageShack and MediaFire (amongst others) are still free AFAIK. Then you can include a link to the images in your post and the sites will generally provide the tags for you.

Unfortunately, our firewalls restrict is from accessing any such sites.

[chulett]

Post them from home.

[RodBarnes]

This is the current code from the CompareFiles routine. (NOTE: I am no longer using the OPENSEQ to test for existence since the DSExecute() successfully returns a 2 when a file is missing -- at least, when I run via the [Test] button. Also, I added an explicit path to the cmd in case that was a factor -- but it made no difference.)

Code: Select all

*-- Set the return code as OK; do not stop the process
result = 0

cmd = 'C:\Windows\system32\FC /B "': Filename1 :'" "': Filename2 :'"'
Call DSLogInfo("File1:": Filename1, 'CompareFiles') 
Call DSLogInfo("File2:": Filename2, 'CompareFiles') 
Call DSLogInfo("cmd:": cmd, 'CompareFiles')
Call DSExecute("DOS", cmd, cmdout, result)
Call DSLogInfo("out:": cmdout, 'CompareFiles')

Ans = result

This is the output from running the routine via [Test] with the files being different, one missing, and with them identical. Note that the routine correctly returns a result of 1, 2, and 0:

Code: Select all

TEST #1
*******
 
Arg1 = C:\Temp\Test\file1.csv
Arg2 = C:\Temp\Test\file2.csv
 
Test completed.
 
DSLogInfo called from : CompareFiles
Message to be logged is...
> File1:C:\Temp\Test\file1.csv
DSLogInfo called from : CompareFiles
Message to be logged is...
> File2:C:\Temp\Test\file2.csv
DSLogInfo called from : CompareFiles
Message to be logged is...
> cmd:C:\Windows\system32\FC /B "C:\Temp\Test\file1.csv" "C:\Temp\Test\file2.csv"
DSLogInfo called from : CompareFiles
Message to be logged is...
> out:Comparing files C:\TEMP\TEST\file1.csv and C:\TEMP\TEST\FILE2.CSV
> FC: C:\TEMP\TEST\FILE2.CSV longer than C:\TEMP\TEST\file1.csv
> 
> 
> 

Result = 1



TEST #1
*******
 
Arg1 = C:\Temp\Test\file1.csv
Arg2 = C:\Temp\Test\file2.csv
 
Test completed.
 
DSLogInfo called from : CompareFiles
Message to be logged is...
> File1:C:\Temp\Test\file1.csv
DSLogInfo called from : CompareFiles
Message to be logged is...
> File2:C:\Temp\Test\file2.csv
DSLogInfo called from : CompareFiles
Message to be logged is...
> cmd:C:\Windows\system32\FC /B "C:\Temp\Test\file1.csv" "C:\Temp\Test\file2.csv"
DSLogInfo called from : CompareFiles
Message to be logged is...
> out:Comparing files C:\TEMP\TEST\file1.csv and C:\TEMP\TEST\FILE2.CSV
> FC: cannot open C:\TEMP\TEST\FILE2.CSV - No such file or folder
> 
> 

Result = 2



TEST #1
*******
 
Arg1 = C:\Temp\Test\file1.csv
Arg2 = C:\Temp\Test\file2.csv
 
Test completed.
 
DSLogInfo called from : CompareFiles
Message to be logged is...
> File1:C:\Temp\Test\file1.csv
DSLogInfo called from : CompareFiles
Message to be logged is...
> File2:C:\Temp\Test\file2.csv
DSLogInfo called from : CompareFiles
Message to be logged is...
> cmd:C:\Windows\system32\FC /B "C:\Temp\Test\file1.csv" "C:\Temp\Test\file2.csv"
DSLogInfo called from : CompareFiles
Message to be logged is...
> out:Comparing files C:\TEMP\TEST\file1.csv and C:\TEMP\TEST\FILE2.CSV
> FC: no differences encountered
> 
> 

Result = 0

These are the pertinent log entries from a run of the test program using the routine in a Routine Activity (NOTE: It doesn't whether the files are the same, different, or one/both are missing. When run from the Routine Activity, the routine always fails to execute and always returns -1):

Code: Select all

   Item #: 4
   Event ID: 3
   Message: CompareFiles2..JobControl (@ControlVars): Set control variables...

   Item #: 5
   Event ID: 4
   Message: CompareFiles2..JobControl (CompareFiles): File1:C:\Temp\Test\file1.csv

   Item #: 6
   Event ID: 5
   Message: CompareFiles2..JobControl (CompareFiles): File2:C:\Temp\Test\file2.csv

   Item #: 7
   Event ID: 6
   Message: CompareFiles2..JobControl (CompareFiles): cmd:C:\Windows\system32\FC /B "C:\Temp\Test\file1.csv" "C:\Temp\Test\file2.csv"

   Item #: 8
   Event ID: 7
   Message: CompareFiles2..JobControl (CompareFiles): out:

   Item #: 9
   Event ID: 8
   Message: CompareFiles2..JobControl (@CompareFiles): Routine DSU.CompareFiles did not finish OK, return code = '-1'

   Item #: 10
   Event ID: 9
   Message: CompareFiles2..JobControl (@check): Check result...

   Item #: 11
   Event ID: 10
   Message: CompareFiles2..JobControl (@OTHER): Record result

   Item #: 12
   Event ID: 11
   Message: CompareFiles2..JobControl (@FNAME1): Record Filename1

   Item #: 13
   Event ID: 12
   Message: CompareFiles2..JobControl (@FNAME2): Record Filename2

   Item #: 14
   Event ID: 13
   Message: CompareFiles2..JobControl (@Coordinator): Summary of sequence run
08:49:05: Sequence started
08:49:05: CompareFiles (ROUTINE DSU.CompareFiles) started
08:49:05: CompareFiles finished, reply=-1
08:49:05: OTHER (ROUTINE DSU.WriteValue) started
08:49:05: OTHER finished, reply=0
08:49:05: FNAME1 (ROUTINE DSU.WriteValue) started
08:49:05: FNAME1 finished, reply=0
08:49:05: FNAME2 (ROUTINE DSU.WriteValue) started
08:49:05: FNAME2 finished, reply=0
08:49:05: Sequence finished OK

   Item #: 15
   Event ID: 14
   Message: Finished Job CompareFiles2.

Here's the output of running the exact same command (as copied from the log entry and pasted to a command-line and run from the same project folder) when the files are different, missing, and the same. The command succeeds with no errors.

Code: Select all

C:\Data\Ascential\DataStage\Projects\Orbit_rod>C:\Windows\system32\FC /B "C:\Tem
p\Test\file1.csv" "C:\Temp\Test\file2.csv"
Comparing files C:\TEMP\TEST\file1.csv and C:\TEMP\TEST\FILE2.CSV
FC: C:\TEMP\TEST\FILE2.CSV longer than C:\TEMP\TEST\file1.csv


C:\Data\Ascential\DataStage\Projects\Orbit_rod>C:\Windows\system32\FC /B "C:\Tem
p\Test\file1.csv" "C:\Temp\Test\file2.csv"
Comparing files C:\TEMP\TEST\file1.csv and C:\TEMP\TEST\FILE2.CSV
FC: cannot open C:\TEMP\TEST\FILE2.CSV - No such file or folder


C:\Data\Ascential\DataStage\Projects\Orbit_rod>C:\Windows\system32\FC /B "C:\Tem
p\Test\file1.csv" "C:\Temp\Test\file2.csv"
Comparing files C:\TEMP\TEST\file1.csv and C:\TEMP\TEST\FILE2.CSV
FC: no differences encountered

I just don't get it...

[RodBarnes]

Since this road is simply not getting me there, time to try a new route. I wrote a DATA-BASIC routine that returns the same values (0=same, 1=different, 2=missing). It simply opens the files, reads the content into two variables and checks whether they are equal. So, I'll tag this as a workaround.

Thanks for all your time, guys.

Code: Select all

FNAME1 = Filename1
FNAME2 = Filename2
*FNAME1 = ''
*FNAME2 = ''
*
*GET(ARG.)FNAME1
*GET(ARG.)FNAME2
*
RESULT = 0
CONTENT1 = ''
CONTENT2 = ''
OPENSEQ FNAME1 TO FV1 THEN
   OPENSEQ FNAME2 TO FV2 THEN
      *--- Read in the first file
      EOF = 0
      LOOP
         READSEQ LN1 FROM FV1 THEN
            IF CONTENT1 NE '' THEN CONTENT1 := @TM
            CONTENT1 := LN1
         END ELSE
            EOF = -1
         END
      WHILE EOF EQ 0 REPEAT
      *--- Read in the second file
      EOF = 0
      LOOP
         READSEQ LN2 FROM FV2 THEN
            IF CONTENT2 NE '' THEN CONTENT2 := @TM
            CONTENT2 := LN2
         END ELSE
            EOF = -1
         END
      WHILE EOF EQ 0 REPEAT
      *--- Compare the content
      IF CONTENT1 EQ CONTENT2 THEN
         RESULT = 0
      END ELSE
         RESULT = 1
      END
      CLOSESEQ FV2
   END ELSE
      RESULT = 2
   END
   CLOSESEQ FV1
END ELSE
   RESULT = 2
END
*
Ans = RESULT

[ray.wurlod]

I'm still curious. Would you be so kind as to post the previous iteration of CompareFiles routine?

[RodBarnes]

I'm still curious. Would you be so kind as to post the previous iteration of CompareFiles routine?

Believe me, so am I!

The current version was included in my prior post.
The version using DSExecute() can be found at the top of the post prior to that (the really long one).
The original version (which used DSU.ExecDOS() and relied upon OPENSEQ to check existence of the files) can be found in the original post in this thread.

[ray.wurlod]

Probably be next week before I get a chance to play with it. This week's work is on AIX, and in my copious free time I'm taking a class in the 8.1 Foundation Tools.