External Filter - Run in one project, not the other

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
IASAQ
Premium Member
Premium Member
Posts: 31
Joined: Wed May 04, 2016 11:07 am
Location: Montréal

External Filter - Run in one project, not the other

Post by IASAQ »

Testing the stage with this. Exactly the same job:

SeqFile_0 ---> EF ---> SeqFile_1

The external filter contains a "sed" instruction. It executes fine in Project A, but does absolutely nothing in Project B, which was based on Project A.

In the job log from Project A, I see that external filter stage was executed, but in the log from Project B, there is not mention of the stage, like it was bypassed. There is no error running both jobs.

Is there a setting I might have forgotten when creating Project B?

Edit: Well, another detail I just found. Here's the sed command I use:

sed 's/#PS_ORA_JDE_SRC.SCH_CTL#/PRODCTL/g'

In Project B, it looks like the "#" character is causing the problem. Works fine in Project A. If I do this in Project B:

sed 's/PS_ORA_JDE_SRC.SCH_CTL/PRODCTL/g', the string will be substituted correctly, but with the "#" characters left intact, which I don't want.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Those # mark the beginning and end of a Job Parameter, so first guess would be it exists in Project A but not in Project B, which cause the hash/pound signs to be left there.
-craig

"You can never have too many knives" -- Logan Nine Fingers
IASAQ
Premium Member
Premium Member
Posts: 31
Joined: Wed May 04, 2016 11:07 am
Location: Montréal

Post by IASAQ »

Chulett, the parameter set exists in both projects as well, so that's not the issue. Could have been a good clue though so thanks.

I have another way of going about it with different stages, but the external filter would have made it simpler.

All this because the "read select statement from file" option doesn't work with parameters.
JRodriguez
Premium Member
Premium Member
Posts: 425
Joined: Sat Nov 19, 2005 9:26 am
Location: New York City
Contact:

Post by JRodriguez »

Did you compile the code in Project B after importing the parameterSet??
Julio Rodriguez
ETL Developer by choice

"Sure we have lots of reasons for being rude - But no excuses
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

IASAQ wrote:All this because the "read select statement from file" option doesn't work with parameters.
True but not insurmountable. We had the same need years ago so developed a small routine to take the original sql file statement (with parameters) and substitute in the current runtime values to a new version of the file. The stage always used the modified version.

Just an FYI. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply