Can CFF in PX on DS 7.5.2 do redefines?

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
bcarlson
Premium Member
Premium Member
Posts: 772
Joined: Fri Oct 01, 2004 3:06 pm
Location: Minnesota

Can CFF in PX on DS 7.5.2 do redefines?

Post by bcarlson »

I have used DataStage for several years now, but this is actually my first foray into the Complex File Format stage - up until now, we have simply used sequential file stages and work arounds. I am trying to see if there is a more efficient way to handle multiple file formats in a single file.


So, with that said, I am doing a simple test. I have the following file layout with a header record and detail records (the CFD below is actually generated from the CFF stage):

Code: Select all

******************************************************************
      * COBOL DECLARATION FOR TABLE COMPLEX-FLAT-FILE-36
      ******************************************************************
       01  COMPLEX-FLAT-FILE-36.
        02  MCRE-HEADER.
         05  TD-RECORD-ID PIC X(3).
         05  TD-APPL-CODE PIC X(2).
         05  TD-BANK-NUMBER PIC X(6).
         05  TD-FILLER1 PIC X(5).
         05  TD-DATA-TYPE PIC X(8).
         05  TD-FILLER2 PIC X(5).
         05  TD-ENTRY-DATE PIC X(6).
         05  TD-ACTION-CODE PIC X(1).
         05  TD-EXTRACT-DATE PIC X(6).
         05  TD-SOURCE PIC X(4).
         05  TD-PACKET-ID PIC X(8).
         05  TD-FILLER3 PIC X(18).
         05  TD-CREATE-APPL PIC X(2).
         05  TD-FILLER4 PIC X(4).
         05  TD-FORMAT PIC X(1).
         05  TD-FILLER5 PIC X(26).
        02  MCRE-DATA
          REDEFINES MCRE-HEADER.
         05  MCRE-SITE-CODE PIC X(1).
         05  MCRE-BANK-GROUP PIC X(3).
         05  MCRE-CYCLE PIC X(1).
         05  MCRE-CYCLE-DATE PIC X(8).
         05  MCRE-ENDPOINT PIC X(8).
         05  MCRE-STRING-NAME PIC X(24).
         05  MCRE-CASH-LETTER-TIMESTAMP PIC X(8).
         05  MCRE-ROUTING-TRANSIT PIC X(8).
         05  MCRE-AMT-X PIC X(10).
         05  MCRE-SEQ-FILL PIC X(2).
         05  MCRE-BANK-NUMBER PIC X(6).
         05  MCRE-SEQ12 PIC X(12).
         05  MCRE-RECORD-SEQ PIC X(9).
         05  MCRE-CRD-DBT-FLAG PIC X(1).
         05  MCRE-ONUS-TRNST-FLAG PIC X(1).
         05  MCRE-SORT-PATTERN PIC X(3).
      ******************************************************************
      * THE NUMBER OF COLUMNS DESCRIBED BY THIS DECLARATION IS 34
      ******************************************************************
I have heard rumors that the CFF stage can do redefines. In fact, I have defined this layout already. I would like the header record to go to one dataset and the detail records to go to another. How do I do this?


Again, I am on DataStage PX 7.5.2.

Brad.
bcarlson
Premium Member
Premium Member
Posts: 772
Joined: Fri Oct 01, 2004 3:06 pm
Location: Minnesota

Post by bcarlson »

Okay, I never got this example working back then and let it sit until now. Still can't get it working... at least not the way I want it to.

The job is simple. The CFF uses the file layout from above and has 3 outputs: reject, header and data (write directly to datasets).

The file imports and outputs to the datasets just fine. I get 0 records in the rejects (expected), but get all records in BOTH the header and data datasets.

How does the CFF know which record to send to which link? I would have expected all but 1 record to go the data link, and 1 record to go to the header link.

Any suggestions?

Brad.
bcarlson
Premium Member
Premium Member
Posts: 772
Joined: Fri Oct 01, 2004 3:06 pm
Location: Minnesota

Post by bcarlson »

One other note. When I look at the schemas for the data and header datasets, they do have the correct layouts. The header just has all records in it...

Brad.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

I'm not a site where I can test this, but recall that you need a record-id column that is used, in a manner similar to a switch stage, to identify a column position that differentiates the different record types. I know I've gotten it working, but I did struggle with it.
bcarlson
Premium Member
Premium Member
Posts: 772
Joined: Fri Oct 01, 2004 3:06 pm
Location: Minnesota

Post by bcarlson »

Thanks. I'll look into it. Man, I bet that means I have to look at documentation, doesn't it? Are men allowed to read directions? :)

Brad.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Sure, many of us occasionally read the manuals but make sure we are not watched or disturbed in the process. I will usually only admit under duress that I even own a manual, and only the utmost coercion will force me to admit having actually read from it :D
Aruna Gutti
Premium Member
Premium Member
Posts: 145
Joined: Fri Sep 21, 2007 9:35 am
Location: Boston

Post by Aruna Gutti »

If I recall correctly, we used @INROWNUM=0 constraint in the Header Output link. But not able to recall whether it is on a Server Job or parallel. I don't have access to 7.5.2 right now.

Also on the Header file input link selected 'first row column names' option.
bcarlson
Premium Member
Premium Member
Posts: 772
Joined: Fri Oct 01, 2004 3:06 pm
Location: Minnesota

Post by bcarlson »

Will the column layout in CFF allow us to define default values for a column or a null field value? If not, how have others handled this situation?

Brad.
flashgordon
Premium Member
Premium Member
Posts: 99
Joined: Tue Aug 17, 2004 7:50 am
Location: Boulder, Colorado

Post by flashgordon »

Hi,

arndt is a stud so I'll defer to him. But I will say in our organization that if you have completely different record formats (cff stage can definitely handle small redefines) in the input (an a in column 1 means header record - format x, a b in column 1 means detail - format y) we split them out and process each separately. cff stage is a technical marvel, but one transformer the other side of cff stage handling two completely different input record format seems very confusing. Think about how you would do this with ascii rows, each row would have a completely different format.

... Flash
Flash Gordon
Hyperborean Software Solution
flashgordon
Premium Member
Premium Member
Posts: 99
Joined: Tue Aug 17, 2004 7:50 am
Location: Boulder, Colorado

Post by flashgordon »

Hi,

arndt is a stud so I'll defer to him. But I will say in our organization that if you have completely different record formats (cff stage can definitely handle small redefines) in the input (an a in column 1 means header record - format x, a b in column 1 means detail - format y) we split them out and process each separately. cff stage is a technical marvel, but one transformer the other side of cff stage handling two completely different input record format seems very confusing. Think about how you would do this with ascii rows, each row would have a completely different format.

... Flash
Flash Gordon
Hyperborean Software Solution
Post Reply