DSXchange

You could probably find the technique by searching the Forum. The trick is that stage variables are evaluated in the order in which they are defined, so that, if SV1 access the value of SV3, it is accessing the value from the previous row. SV1 derivation InLink.C1 <> SV3 And @INROWNUM <> 1 SV2 deriv...

The important thing to remember is that DataStage is just another client, as far as the database server is concerned. What this means in practice is that you have to configure any necessary client software and connectivity just the same as if you were configuring any other database client. There are...

To get full information, determine the most recent "job started" event using DSGetNewestLogId function then use DSGetLogEntry for that event number and each higher event number. You'll need a loop incrementing by 1, and detect failure of DSGetLogEntry (or use DSGetNewestLogId to determine the highes...

Read each line as 12 bytes with no terminators, using a sequential file stage. In a Transformer stage stage variable (let's call it stagevar1), apply OCONV with "MB0C". For example Oconv(InLink.TheLine, "MB0C") Instead of 96 bits you now have 96 characters, each 1 or 0. In your fou...

Slightly more serious answer... LEN(string) returns the number of characters in a string. LEN("") returns 0. LEN(@NULL) also returns 0, curiously. BYTELEN(string) returns the number of bytes in a string. In non-ASCII environments this may not be the same as the number of characters. There are other ...

There's a separate manual (sort.pdf) for the Sort stage. Among other things this reveals that the sorting mechanism is based on the UNIX sort command. The model for the Sort stage is the UNIX sort command, as used in a shell pipeline. Input data rows to be sorted arrive as lines of ASCII characters ...

You have global control over the hashed file cache sizes via the Administrator client's project Tunables tab. The configuration parameter DISKCACHE can also be used.
There is a separate manual on caching hashed files (dsdskche.pdf); reading this should answer all your questions.

Perhaps we're in search of an answer to the perennial "how long is a piece of string" question here!

Can these warnings be shut off somehow? The DataStage philosophy is to log all warnings. Best practice is to heed them! You can use a filter in director to suppress display of warnings, but it's important that you are aware of them. In your particular case, four parameters are not assigned values. ...

What I suggested was to add a Constraint to the output link, so that only the four digit strings would get through. You've used the expression as the output column derivation, rather than as a column constraint. To get in to the constraints grid, double click on the grey "Constraints" area at the to...

0. You also need to set "normalized on Page" when viewing. If you're only seeing one value your update rule may have been set to "insert new rows only" or you may not have normalized on when loading. Remember that the row's identity is based on the key column, which is not multi-valued. 1. All value...

Depends on, among other things: what hardware resources you're throwing at it (e.g. CPU count, CPU speed, memory) what stage type(s) you're using (e.g. ORAOCI, ORABULK, ODBC, SeqFile) how you're handling transactions how large the rollback segment is Oracle parallelism How long is a piece of string?

If this still doesn't make sense or it doesn't seem like an approach you want to take, how about a variation of the Ray Wurlod method in your other thread? Break this up into two jobs. The first one creates the initial dataset, does the lookups and writes the data you need to do the grouping on out...

If you really think about it, reference lookups per se are left outer joins.

You only make them inner joins if you constrain the output to those rows for which the lookup succeeded!

Create another job that generates the row in a before-job subroutine using ExecSH echo ",,,,,," > dummyfile which then processes that row from dummyfile into your target table.
Use a job sequence or job control to run your main job ahead of this simple job.