How to read a file and rewrite it?

gwen · Post by **gwen** » Fri Aug 19, 2005 10:12 am

HELLO ALL~

I'M TRYING TO WRITE TO ROUTINE AS BELOW:

PathName="E:\ETL\weightAO.txt"

OpenSeq PathName To FileVar Else
Call DSLogWarn("Cannot open ":PathName, MyRoutine)
End
Loop
ReadSeq FileLine From FileVar 
On Error
  Call DSLogWarn("Error from ":PathName:" status=":Status(),
                  "MyRoutine")
End

Then
    
BRANCH=FIELD(FileLine,",",1)
MAX=FIELD(FileLine,",",3)
CNT=FIELD(FileLine,",",4)
AO=FIELD(FileLine,",",2)

    If BRANCH=BRANCH_NBR AND MAX>CNT  then
       NEWFileLine=Fieldstore(FileLine,",",4,1,CNT+1);
       Exec=NEWFileLine;
       Exit;
    End Else
       Exec="999";
    End
End Else
  Exit         ;* at end-of-file
End
Repeat
CloseSeq FileVar

Ans=Exec

I FOUND THE FILELINE BY THE BRANCH_NBR AND USE Fieldstore FUNCTION TO GENERATE THE NEWLINE.
CAN I REPLACE THE OLD FILELINE BY THE NEWLINE AND SAVE TO THE FILE IMMEDIATELY?

chulett · Post by **chulett** » Fri Aug 19, 2005 11:53 am

No. You need to write to a new (or at least different) name, then decide what you want to do in the way of post processing. Rename new back to old, perhaps. Save off original file somewhere first, perhaps. Your call.

kumar_s · Post by **kumar_s** » Fri Aug 19, 2005 11:04 pm

Yes same thread cannot read and write in to a file.
probably u can try opening another thread parallely to write in to....
but i am not sure about its stability....

regards
kumar

ray.wurlod · Post by **ray.wurlod** » Sat Aug 20, 2005 1:31 am

You can open the file's directory with OpenPath which treats the directory as a database table. You can then use ReadU to read the record in its entirety, looping through it with Remove.
Using this algorithm, you CAN update individual lines. Make sure to release the update lock if you don't Write the file back.
The following omits error handling, but only for clarity. You should not omit error handling.

Code: Select all

FileDir = "E:\ETL"
FileName = "weightA0.txt"

OpenPath FileDir To FileVar
Then

   Read File From FileVar, FileName
   Then

      LineNumber = 0

      Loop
      Remove FileLine From File Setting MoreLines

         * logic to extract pieces most efficient in field order
         BRANCH=FIELD(FileLine,",",1) 
         AO=FIELD(FileLine,",",2) 
         MAX=FIELD(FileLine,",",3) 
         CNT=FIELD(FileLine,",",4) 

         LineNumber += 1

         If BRANCH=BRANCH_NBR AND MAX>CNT  
         Then 
             FileLine = Fieldstore(FileLine,",",4,1,CNT+1)
             Exec = FileLine
             File<LineNumber> = FileLine
             Exit 
         End 
         Else 
             Exec="999"
         End                    ; * end of IF statement

      While MoreLines
      Repeat

      Write File To FileVar, FileName
      Else
         Call DSLogWarn('Unable to write to  file', 'Routine')
      End

   End   ; * end of READ statement

End      ; * end of OPEN statement

Note that trailing semi-colons are not required in DataStage BASIC.

chulett · Post by **chulett** » Sat Aug 20, 2005 7:18 am

So, just to be clear...

Because you are using a routine that reads the entire file into memory, this allows you to write everything back to the same filename when you are done - something that is not possible when using Sequential File stages.

Ray, you mention 'ReadU' but don't use it in the posted code. ?

Kumar - same thread, different thread, no matter - still no writing to a sequential file that is actively being read.

gwen · Post by **gwen** » Sun Aug 21, 2005 9:26 pm

Thanks Ray~

I'm trying your method as the routine but seems failed each time I tested

Code: Select all

FileDir = "E:\ETL" 
FileName = "weightA0.txt" 

OpenPath FileDir To FileVar 
Then 

   Read File From FileVar, FileName 
   Then 

      LineNumber = 0 

      Loop 
      Remove FileLine From File Setting MoreLines 

         * logic to extract pieces most efficient in field order 
         BRANCH=FIELD(FileLine,",",1) 
         AO=FIELD(FileLine,",",2) 
         MAX=FIELD(FileLine,",",3) 
         CNT=FIELD(FileLine,",",4) 

         LineNumber += 1 

         If BRANCH=BRANCH_NBR AND MAX>CNT  
         Then 
             FileLine = Fieldstore(FileLine,",",4,1,CNT+1) 
             Exec = FileLine 
             File<LineNumber> = FileLine 
             Exit 
         End 
         Else 
             Exec="999" 
         End                    ; * end of IF statement 

      While MoreLines 
      Repeat 

      Write File To FileVar, FileName 
      Else 
         Call DSLogWarn('Unable to write to  file', 'Routine') 
      End 

   End   ; * end of READ statement 

End      ; * end of OPEN statement

*return the fileline which is found to be rewirted
Ans=Exec

I put the return value to see if any line was replaced but always go the result below

Code: Select all

TEST #1
*******
 
Arg1 = 675
 
Test completed.
 

Result = Program "TSTWeightAO.B": Line 5, Variable "ANS" previously undefined.  Empty string used.

Maybe the file wasn't read into the memory,How do I testified if the file did load into the memory?

ray.wurlod · Post by **ray.wurlod** » Mon Aug 22, 2005 5:12 pm

That message can indicate that you've used the variable name ANS rather than the name Ans. Variable names are case sensitive.

It can also mean that there is a path through your code in which the Ans variable is not set at all - maybe you failed to open the "table" (directory) or to read the "record" (file).

To avoid the latter you need to initialize the Ans variable.

And, Craig did spot an omission, I intended to use ReadU not Read - this sets an update lock on the "record" so you don't lose updates made from multiple DataStage processes to this file.

Code: Select all

FileDir = "E:\ETL" 
FileName = "weightA0.txt" 
Ans = @NULL  ; * initialize return variable

OpenPath FileDir To FileVar 
Then 

   * Because the ReadU statement lacks a LOCKED clause, it will wait
   * indefinitely for any lock to be released.  You could, alternately, 
   * build in a sleep/retry mechanism if the lock is detected.
   ReadU File From FileVar, FileName 
   Then 

      LineNumber = 0 

      Loop 
      Remove FileLine From File Setting MoreLines 

         * logic to extract pieces most efficient in field order 
         BRANCH=FIELD(FileLine,",",1) 
         AO=FIELD(FileLine,",",2) 
         MAX=FIELD(FileLine,",",3) 
         CNT=FIELD(FileLine,",",4) 

         LineNumber += 1 

         If BRANCH=BRANCH_NBR AND MAX>CNT  
         Then 
             FileLine = Fieldstore(FileLine,",",4,1,CNT+1) 
             Exec = FileLine 
             File<LineNumber> = FileLine 
             Exit 
         End 
         Else 
             Exec="999" 
         End                    ; * end of IF statement 

      While MoreLines 
      Repeat 

      Write File To FileVar, FileName 
      Else 
         Call DSLogWarn('Unable to write to  file', 'Routine') 
      End 

   End   
   Else

      * File does not exist, so release the update lock
      Release FileVar, FileName

   End   ; * end of READ statement 

End      ; * end of OPEN statement 

* Return the fileline which is found to be rewritten 
Ans=Exec

gwen · Post by **gwen** » Mon Aug 22, 2005 9:26 pm

RAY~THANK YOU AGAIN!

I'VE BEEN TRYING TO TEST THE ROUITINE CODE. AND I STILL GOT THE SAME RESULT.

Code: Select all

TEST #1
*******
 
Arg1 = 676
 
Test completed.
 

Result = Program "TSTWeightAO.B": Line 5, Variable "ANS" previously undefined.  Empty string used.

I THINK THE REASON IT'S ALWAYS FAILED IS "failed to open the "table" (directory) or to read the "record" (file)" WHICH RAY MENTIONED BEFORE.
HOWEVER,PREVIOUSELY,I USED "OPENSEQ" FUNCTION TO OPEN THE SAME FILE,IT'S BEEN SUCCESSFULLY OPENED.I DON'T KNOW WHY~BECAUSE OF THE OS PLATFORM OR THE TEXT FILE FORMAT?[/quote]

ray.wurlod · Post by **ray.wurlod** » Tue Aug 23, 2005 12:09 am

Try this (it has a LOCKED clause on the READU statement and an ELSE clause on the OPENPATH statement) and let us know how it tests. Don't forget to double click on the result cell so you can see what is logged.

Code: Select all

FileDir = "E:\ETL" 
FileName = "weightA0.txt" 
Ans = @NULL  ; * initialize return variable 

OpenPath FileDir To FileVar 
On Error

   Call DSLogWarn('System error opening ' : Quote(FileDir), 'Routine')

End
Then 

   * Because the ReadU statement lacks a LOCKED clause, it will wait 
   * indefinitely for any lock to be released.  You could, alternately, 
   * build in a sleep/retry mechanism if the lock is detected. 
   ReadU File From FileVar, FileName 
   On Error
 
      Call DSLogWarn('System error reading file ' : Quote(FileName) : ', status = ' : Status(), 'Routine')

   End
   Locked

      Call DSLogWarn('File update-locked by process number ' : Status(), 'Routine')

   End
   Then 

      LineNumber = 0 

      Loop 
      Remove FileLine From File Setting MoreLines 

         * logic to extract pieces most efficient in field order 
         BRANCH=FIELD(FileLine,",",1) 
         AO=FIELD(FileLine,",",2) 
         MAX=FIELD(FileLine,",",3) 
         CNT=FIELD(FileLine,",",4) 

         LineNumber += 1 

         If BRANCH=BRANCH_NBR AND MAX>CNT  
         Then 
             FileLine = Fieldstore(FileLine,",",4,1,CNT+1) 
             Exec = FileLine 
             File<LineNumber> = FileLine 
             Exit 
         End 
         Else 
             Exec="999" 
         End                    ; * end of IF statement 

      While MoreLines 
      Repeat 

      Write File To FileVar, FileName 
      Else 
         Call DSLogWarn('Unable to write to  file', 'Routine') 
      End 

   End    
   Else 

      * File does not exist, so release the update lock 
      Release FileVar, FileName 

   End   ; * end of READ statement 

End
Else

   Call DSLogWarn('Unable to open directory ' : Quote(FileDir), 'Routine')

End      ; * end of OPEN statement 

* Return the fileline which is found to be rewritten 
Ans=Exec