DSXchange

Forget data combining stages (Lookup, Join, Merge) - they do not allow you to identify the rows that were not processed.

Look at one of the change detection stage types (Difference, Compare or, perhaps, Change Capture).

Is a solution required for any possible time zone in the world? For example Indian Standard Time will require you to change minutes as well as hours.

Some conversions - even within the USA - will require the day to be changed and then, as a possible flow-on, the month and maybe even year.

Use the UNIX diff command either from an Execute Command activity in a sequence or from a before-job subroutine. Use the exit status of diff to determine whether to proceed with the other job. Yes, I know you're on Windows, but you have MKS Toolkit installed so that you can run the parallel engine.

Actually the *best* structure for staging data between parallel jobs is the Data Set, since it preserves partitioning, internal data types and sorting. Neither files nor database tables do that.

So you're looking to match whenever Name OR Account is that same?

Have you tried a two-pass Match, one blocking on Account, the other blocking on NYSIIS of Name and matching on Name (and anything else you might need) as independent passes?

These are not documented by the vendor.

I suspect, but have not had the opportunity to verify, that they are related to two of the other options in the Job Run Options dialog, probably whether or not to disable project and local message handlers.

Definitely British (and definitely not Australian). Peter Cook and Dudley Moore - though the latter ended up being Californian if my memory serves me correctly.

See also Derek and Clive.

I don't understand your algorithm.

What is it that gets NEW YORK and CALIFORNIA grouped with TRENTON ?

Does "group" here mean the same thing as a "block" in QualityStage match - a set of potential duplicates - with your Group being the same as QualityStage match's SetID ?

If you can have Dud data, can you have Pete data?

I had your problem the wrong way around.

What you actually need is a Lookup stage with four separate lookups, one for each of ProfitCenter, FunctionalArea, DocType and LogType (always in conjunction with Account), and some logic downstream to isolate which actually had the value supplied from source.

Project can be determined by mapping the agent process PID to the project name in the project&!DS.ADMIN!& entry in list_readu output. Determining the jobs that a particular is running is fraught with difficulty. For example, was the job started by a connected client's agent process, by a dsj...

Set parameters in the job that loads the status table. Use calls to DataStage function DSGetJobInfo() in a routine, and use the return value from the routine (or user-defined system variables) to populated values into the individual parameters. FUNCTION GetJobStats(JobName) $INCLUDE DSINCLUDE JOBCON...

The sqlplus command output includes a final linefeed, which DataStage converts to a field mark (@FM). You could simply have removed that.

Assemble rows using stage variables to test whether "*" or an explicit value has been given, and preserve the explicit values into the output rows. Follow the Transformer stage with a Remove Duplicates stage to preserve only the last of each group (based on Account).

asorrell wrote:Restrictions:
1) Can't write a routine to accomplish it.
2) Must be done in parallel.
3) Can be done with multiple jobs, but since it is a mod to an existing job, it would be easier if it was self-contained.

Ideas?

Resist stupid requirements!

I'd use a server job calling a server routine.