DSXchange

When a PX job starts, the first process is the Conductor, which starts a Player process for each node in the configuration file. (It WAS called "Orchestrate" after all!). Each player may then start one or more osh processes to do the actual work, the number also depending on your partitioning scheme...

I am going to assume that DSLink1 is the main (stream) input to the Transformer stage, and the DSLink3 is an output link. I will call the stage variable IsCEO rather than V_One. Derive the stage variable using the following expression: (DSLink1.EmpJob < "VS_PRES") Derive DSLink3.EmpJob using the fol...

The Transformer stage performs the lookup. There is a reference key expression "MasterTable.Column1" for the reference key "Lookup.Column1". The reference input link returns Column1 and Column2 from the lookup table. (Here I've used MasterTable and Lookup as the names of the primary and reference in...

Try using a UV stage rather than a hashed file stage at least for the partial key lookups. Data Source Name is "localuv". A hashed file stage insists on the entire key (I should have mentioned this; it's been discussed on this forum in the past). And, yes, clearing the pre-load to memory check box d...

If your text file lacks a final end-of-line character you can add one (for example use ExecSH to do echo "" >> filename) in a before-job subroutine.

Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518

Just because no-one mentioned it, you CAN pre-process your text file with the UNIX sort command. It's not nearly as fast as CoSort or syncsort but it is faster than the Sort stage in server jobs.

Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518

1) True. I think we need an enhancement request on this one, now that Version Control is about. 2) Have no-one in the Developers group on the production system, and allow Operators to view full log contents. 3) That makes sense, since to do anything in DataStage you must be able to access the data f...

There's no need to encapsulate your jobs in containers, unless you're planning to implement them within a Parallel Extender environment. A job sequence is just another kind of DataStage job, so it is perfectly legal and feasible for one job sequence to invoke another. In fact, at the most recent pro...

There's not really enough detail in your description of what you want to do. When you say "log error" for example, do you mean log an entry into the job log, or write a row into a file? If the latter, as Craig suggests, all you need is a Transformer stage with two outputs. One output has a constrain...

Simplistically, you could have a control job that sleeps until the desired time (SLEEP hh:mm) then calls DSRunJob. I take it that's not what you have in mind. Again it's not clear why you want to reinvent the wheel. What's wrong with the facilities provided with DataStage? There is a subroutine call...

Yes, it can be done, but there are many things of which you need to be careful. That kind of specialist knowledge tends to be available in exchange for consultancy fees. I am curious to see your answer to Craig's question, which I would have asked if he hadn't. Ray Wurlod Education and Consulting Se...

This kind of thing has been addressed on a number of occasions both here and on the Oliver list. Check the archives for detail. Basically, you need to have a licensed ODBC driver, you need to have configured the DSN correctly (in ODBC.INI), you need to have an entry in the uvodbc.config file in the ...

You need three separate lookups, one based on three key columns, one based on two key columns and one based on one key column. Or perhaps six. The technique is to lie to DataStage about what the key columns are. If you specify that there are three key columns, you will need three reference key expre...

I grew up writing my own Job Control code (from version 1), so tend to prefer that. It gives me total flexibility with what I want to do, for example a "busy wait". I will use job sequences where my clients prefer that approach; sometimes directly for production, sometimes merely to prototype (then ...

It doesn't matter where you're doing it. If two processes want to access a server of some kind (whether a hashed file, a routine, a server process or whatever) to obtain the next value in the same sequence, that server mechanism must be single-threaded. Most usually this is accomplished via locks of...