Complex Flat File Stage server vs parallel

jlitherl · Post by **jlitherl** » Fri Jul 28, 2006 10:17 am

I'm writing a parallel job to read a complex flat file (defined by COBOL Copybook) and interpret the data according to an ID field, signifying which of the REDEFINES sections applies.
I can do this in a server job fairly easily and intuitively with multiple output links, defining the select columns and selection criteria.
When I try to do the equivalent task in a parallel CFF stage, the user interface is different, it doesn't let me define multiple output links, select columns etc.
It has been suggested I wrap the server job in a parallel container, which seems to work, but looks a bit like a bodge to me.
Does anyone know a prettier (i.e. the correct) way to do this, please?
Thanks

ray.wurlod · Post by **ray.wurlod** » Fri Jul 28, 2006 2:18 pm

Welcome aboard. :D

You are correct to assert that the interface is different, remember that the parallel version must take into account the need to split the rows into multiple parallel streams.

Is there anything in particular about which you want to ask concerning the parallel CFF stage?

jlitherl · Post by **jlitherl** » Mon Jul 31, 2006 1:52 am

Thanks for your reply, Ray.

I'm just trying to do what seems to be the obvious thing (well, it is very easy to do with the server CFF stage), that is, to read a record in the CFF stage. Then, according to the value of the record id (record type), to branch off to process the record appropriately to which of the REDEFINE clauses apply.
In the CFF server stage, on the Selection Criteria tab of the Output tab, you can choose which of the output links is chosen depending on the value of the record ID.
Whereas, with the parallel CFF stage, I seem to be constrained to a single output link.
To do this in a parallel job should I be using a combination of two jobs? A CFF followed by a switch, perhaps?
I'll be grateful for any further advice. Thanks.

kumar_s · Post by **kumar_s** » Mon Jul 31, 2006 4:12 am

You are right. CFF in PX cannot change its metadata dynamically based on data, as what redefines is for. So either you need to seperatly process the data for different conditions othere than expected from soruce, before reading into CFF or use other stages to filter it out. Read it as varchar (if possible) and deal withing job.

jlitherl · Post by **jlitherl** » Tue Aug 01, 2006 10:38 am

Thanks for the help.

Apparently it is possible to read a CFF with redefines in parallel with a little jiggery-pokery, a switch and a Column Import stage.

The description is on :
http://www.dsxchange.com/viewtopic.php? ... +flat+file

I'm getting nearer but still not quite there...

Regards
JL