Pattern - String Extraction

koti9 · Post by **koti9** » Thu Jul 30, 2015 6:59 pm

Hello all,

We need to extract year and quarter from a string. Please find our source string below.

Source String may contain below values.

ABCD1998.3EFH.GHAI.678ABCD.EF
1998.3ABCD.EFH.AJG.SS.123SSS...SSS...
ABC.DEF.EF345GH.124SJLAS..1998.3

We need to extract the 1998 as the year from above 3 strings and also quarter which follows the year with a dot...

I have gone through the forum, explored with index function,field function,convert function..but none of them worked..

Thanks & Regards
Koti

rkashyap · Post by **rkashyap** » Thu Jul 30, 2015 8:29 pm

Use following Unix command in External Filter stage.

Code: Select all

sed -n  's/.*\([0-9][0-9][0-9][0-9]\.[1-4]\).*/\1/p'

chulett · Post by **chulett** » Thu Jul 30, 2015 8:41 pm

So... you need to find the first occurrence of four numeric digits and then the single digit that follows immediately behind that after a 'dot'? Two separate fields or one? And is that always the case or if the four don't have a dot right there after them do you keep looking deeper in the string? Give up? Want to make sure we understand the full glory of your requirements.

Why don't you show us what you tried and what didn't work about it. Show us your thought process, how you approached this.

Oh, and what is your source? Don't want to make any assumptions there.

koti9 · Post by **koti9** » Fri Jul 31, 2015 1:04 pm

Thanks Kashyap, you are awesome..I was able to do this with Stage Variables though...

StageVar1 - Converted all '.' and numbers to tilda(~) using convert function.
StageVar2 - Used index function to get the position of 6 consecutive tildas in StageVar1 Variable. Used that position pull the substring from actual field.

I made sure that tildas~ were not there as part of the actual data.

Thanks Craig, your question helped me raise a question with source.

Thanks & Regards
Koti

rkashyap · Post by **rkashyap** » Sat Aug 01, 2015 7:05 am

You are welcome.

Just FYI for anyone using External Filter stage. In previous versions, External Filter stage has an issue in processing of special characters. See technote. If this issue still occurs, then push the arguments to a file either as described in technote or by using "-f" option of sed.

BalajiL · Post by **BalajiL** » Mon Aug 03, 2015 12:17 am

Hi Koti,

Need your help here.

How did you convert all the numbers to a specific value. When i use convert function it replaces only one character. Could you please share the code which you have used for your convert function.

chulett · Post by **chulett** » Mon Aug 03, 2015 7:06 am

Sounds like you need to re-read the documentation on the Convert function.

If you need more help that that, start your own post and let us know what problem you are trying to solve. Thanks.

rkashyap · Post by **rkashyap** » Thu Aug 06, 2015 3:07 pm

Just for future reference ... Basic Transformer's MATCHFIELD function could also have been used for pattern matching and string extraction.