DSXchange

Can you post the detail of one of these invalid handle messages, in particular the stage that is throwing it? You may need to disable operator combination to get a sensible report.

Do you really need the sh -c part? Given that the External Target stage consumes rows via stdin, so your command probably does need to process stdin (maybe via the - token). The "cannot preserve partitioning" is easily solved in the upstream stage; you can find that solution on DSXchange.

A horribly inefficient method would be to perform a "not equals" self-join in the source and return the first row from that. You could not use a Join or Lookup stage for that; it would have to be done in source.

There is a technique for changing stage variables only when you need to; namely to assign the stage variable to itself when no change is required. If condition_met Then calcualted_value Else svMyVariable --> svMyVariable It is necessary, because the condition might not be met in row #1, that the sta...

All keys in XMETA, XMETASR and DSODB are defined as VARGRAPHIC. (At least when DB2 is used.)

Was the file created on a Windows system, or copied/pasted from a Windows system? In that case the extraneous character is probably a carriage return - you can use Convert() function to eliminate it or, if you're reading from a text file, specify DOS-style as the Record Delimiter String property.

There is a separate RT_STATUSnnn table for each job. The information is not summarised anywhere. The nnn is the unique job number allocated by DataStage when the job is created and which is recorded as JOBNO in the DS_JOBS table (whose key is NAME, the job name). The status of each active or hybrid ...

Another link in the job design, perhaps from a Copy stage, to write into the new location.

A stage variable that is reset each time the grouping value changes. Change detection can be done through stage variables or, if you are on a recent enough version, through functions.

brupun wrote:I may have to reference around 70 lookup tables in one lookup stage.

No you won't.
There's no reason not to use multiple Lookup stages. You can make the downstream lookups conditional on the success of the upstream ones if that is your wont.

My suggestion would be to try it.

Seconds since midnight requires storage of 0..86400 (17 bits minimum)and microseconds requires storage of 0..99999 (18 bits minimum). I guess you can construct that into five bytes easily - why two formats I'm not sure.

Could anyone throw inputs? It's a given among the premium posters at least that, if anyone has anything constructive to provide, they will. Silence means that we have nothing to contribute or that we have not yet gotten around to reading the thread. The latter is unlikely if 24 hours have gone by (...

You've marked this as Resolved before I got to read it. Presumably you are no longer in need of help. But please do post how the problem was resolved, in case any future searcher encounters the same issue.

You could, of course, have DataStage write to the new location and simply delete the original on successful completion.