Complex file stage vs Sequential file stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
iamds
Participant
Posts: 3
Joined: Tue Jan 13, 2015 12:56 am
Location: Hyderabad

Complex file stage vs Sequential file stage

Post by iamds »

Hi Guys,

I have couple of questions where need your advice.

Q1) My requirement is to read a mainframe file(ebcidic) having text data. The file is fixed width. I should read it through Complex file stage or Sequential file stage? It's simple file with volume not big enough. In term of performance wise which one is preferable? Overall which stage to use to read?

Q22) Can I read delimited file(pipe) through complex file stage?

Thanks
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

1. Always try to use the simplest stage possible for a given task, that means in your case the normal sequential file stage is preferable for this file.
2. -not sure- at the moment, since I have only used the stage for COBOL non-delimited files, but if you have the stage in front of you, you should be able to find the section with the delimiters to see what options you have.
Checked the Stage - and the answer is that there is no way to specify a delimiter, variable length types get mapped to static lengths.
iamds
Participant
Posts: 3
Joined: Tue Jan 13, 2015 12:56 am
Location: Hyderabad

Post by iamds »

Thanks. So can we say complex file stage take more more
Resources than sequential file stage?
vibwipro
Participant
Posts: 40
Joined: Thu Jan 24, 2013 7:34 am

Re: Complex file stage vs Sequential file stage

Post by vibwipro »

Mainframe file(EBCIDIC) should be read with Complex Flat files, because this stage can handle decoding of Compressed and non- compressed fields based on Copy Book provided to CFF stage. Out put will be in ASCII format over which trannsformations rules can be implemented.

Yes you can recd (Pile) delimited files through CFF stage, by giving delimiter under "Record Option" Tab on CFF stage.
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

Both answers are correct, almost. There is no reason to prefer CFF over Sequential except for one thing: multiple record types.

Edit: Sorry, there are other considerations. CFF lets you filter data rather than using a Filter stage, and CFF and Sequential have different compatibilities with source channels. For example, my previous design used FTP to stream data directly to the next stage instead of landing a file to the server, but CFF cannot be the next stage.

CFF's biggest advantage is reading multiple record types from one source.

Using a sequential file usually means needing a transformer to parse the record into the fields of the copybook.

In the mainframe legacy world, delimited records are rare. Storage is designed for efficiency, and adding one byte for every field takes up unnecessary space when you have fixed-width fields and records. There's no accounting for lazy developers, of course.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
Post Reply