Stages that support regular expressions (regex)

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
timsmith_s
Participant
Posts: 54
Joined: Sun Nov 13, 2005 9:25 pm

Stages that support regular expressions (regex)

Post by timsmith_s »

I have read posts regarding regular expression in filters, but would anyone please help with understand if the Modify or Transformer stages support them too? Can they be used for parsing values, or only to exclude records?

Thx - T
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

The answer to your first question is NO, which makes your second question moot.

A Filter stage uses something like regular expressions in its WHERE clause, but does not support the full gamut of regular expression functionality.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

External Filter stage can be leverage more for this purpose.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
jhmckeever
Premium Member
Premium Member
Posts: 301
Joined: Thu Jul 14, 2005 10:27 am
Location: Melbourne, Australia
Contact:

Post by jhmckeever »

It is possible to use regular expressions in parallel transformers, but only after you've done all the hard work yourself...

You can define a Parallel Transformer by creating a C function using the <regex.h> library. We're using this at the moment and it works well. It provides full regular expression support in parallel transformers and performance is pretty good too. You can get extended expression support using the REG_EXTENDED flag.

Let me know if you require further details. If you want a place to start, lookup the 'regcomp' and 'regexec' functions in the library.

J.
<b>John McKeever</b>
Data Migrators
<b><a href="https://www.mettleci.com">MettleCI</a> - DevOps for DataStage</b>
<a href="http://www.datamigrators.com/"><img src="https://www.datamigrators.com/assets/im ... l.png"></a>
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

jhmckeever, you mean you have created a BuildOps?
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
jhmckeever
Premium Member
Premium Member
Posts: 301
Joined: Thu Jul 14, 2005 10:27 am
Location: Melbourne, Australia
Contact:

Post by jhmckeever »

DOH! Sorry - That post was typed Pre-Coffee (hence my excuse for a brain wasn't awake).

Nope, I created a Parallel Routine (Not 'parallel transformer'), referencing a function I defined in a custom C library.

By way of payment for my daft error, here's the code ...

Code: Select all

#include <sys/types.h>
#include <regex.h>

int matchRegexp(char *inString, char *inPattern)
{
    int i;
    regex_t re;
    char buf[256];

    i=regcomp(&re, inPattern, REG_EXTENDED|REG_NOSUB);
    if (i != 0) {
        (void)regerror(i,&re,buf,sizeof buf);
        /* printf("%s\n",buf); */
        return(0);			    /* report error */
    }
    i = regexec(&re, inString, (size_t) 0, NULL, 0);
    regfree(&re);
    if (i != 0) {
        (void)regerror(i,&re,buf,sizeof buf);
        /* printf("%s\n",buf); */
        return(0);			    /* report error */
    }
    return(1);
}
Rgds,
John.
<b>John McKeever</b>
Data Migrators
<b><a href="https://www.mettleci.com">MettleCI</a> - DevOps for DataStage</b>
<a href="http://www.datamigrators.com/"><img src="https://www.datamigrators.com/assets/im ... l.png"></a>
timsmith_s
Participant
Posts: 54
Joined: Sun Nov 13, 2005 9:25 pm

Post by timsmith_s »

Thank you.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Search for "Editing Multiple Derivations" under transformer section in the Parallel Job Developer's Guide. There is something mentioned for Regular expression. I haven't gone through it fully.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Post Reply