Filter through Complex Flat file stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
anudeepdasari001
Participant
Posts: 58
Joined: Mon Dec 14, 2015 3:16 pm
Location: Arizona

Filter through Complex Flat file stage

Post by anudeepdasari001 »

I have source as Complex flat file, File layout is binary,

I see there is a filter options in file options tab on complex flat file stage,
How can we use that,

I have a scenarios where I need to filter a column values which reflects this functionality

i.e

substr(Column1,-3,3)= 360

How can this be accompolished?
anudpETL
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

That filter option is where you would put a UNIX command to operate on your file (not going to be much use on a binary file).

You could use a transformer constraint after the CFF stage.

The constraint tab on the output tab might be an option if you convert the substring you want to do into a defined column... perhaps with a REDEFINES.

Mike
anudeepdasari001
Participant
Posts: 58
Joined: Mon Dec 14, 2015 3:16 pm
Location: Arizona

Post by anudeepdasari001 »

I am pretty much familiar with the constraint on transformer, Just wanted to know the filter importance on CFF stage, Thanks Mike


As I am retreving the binary file through CFF stage, when i am trying to do a substring in transformer, its behaving different, beacuse after each feild value there is a .(dot), So that's bothering me.
anudpETL
rameshrr3
Premium Member
Premium Member
Posts: 609
Joined: Mon May 10, 2004 3:32 am
Location: BRENTWOOD, TN

Post by rameshrr3 »

You can go to the Output Master tab of CFF stage ,and you will see a sub tab called constraints - it offers some level of capability of filtering data -, but I don't think any string functions can be used. You can just filter on a column or bunch of columns based on equality or inequality criteria. NO Fancy logic would be supported - only Equals and GreaterThan/LessThan(or equal to) .
Post Reply