DSXchange

I'm guessing they only have one server available, or they only have one license, and they want to test the new version before the completely upgrade the production environment. The problem here is that they will be installing the new version on the production machine alongside the old version and th...

There is a FAQ on implementing a counter in a parallel transformer using the @PARTITIONNUM and @PARTITIONCOUNT variables. Consider this design: sort by KEY1 and KEY2. In a lookup stage retrieve the KEY1, KEY2 and MAX(KEY3) from the database to return MAX_KEY3. This should be a very efficient lookup ...

Does ProfileStage use the parallel framework? I know the in next release the Sorcerer product will be parallel but I doubt there is a 7.x Windows version which is parallel other then DataStage.

You can install DataStage 7.5x2 on Windows servers and it will run in parallel mode. You also need Visual Studio compilers on the server to compile any buildops or jobs with transformers in them. Focus just on your largest jobs, you will find many of your smaller jobs may actually be slower as paral...

Have a look at the FAQ on DataStage reporting options. Probably the easiest way is to load up the DataStage Reporting Assistant and write your own query to retrieve all jobs using that column.

It's a very poor ETL design that requires permanent hash files. Usually hash files get built during each run, or can be rebuilt as a recovery operation by utility jobs. How many of your hash files do you need to move? How many are built by each batch schedule? Hash files are not a database engine wi...

I've been on a project with exactly the same config, DB2 v8 and AIX. We also had problems with the load option, but we redesigned most of our jobs to be delta sized loads and we set everything to upserts. I didn't know at the time whether the load failures were caused by a DB2 config problem or a Da...

Have a look at the Advanced Parallel Job Developers Guide, it has some sections on header files. Assume you can drop your header files in the same location as DataStage header files in the apt_util folder and start using them, or create your own header folder. DataStage comes with a range of header ...

Been thinking of doing something similar in a parallel job, a shared container that receives a propogated input link with almost no columns defined and can derive the key and value fields for error reporting. May be possible via a custom parallel stage. Best approach for a server job is to use a col...

I gave it a serious tilt, we had a lot of dispersed users without much knowledge, I wanted templates to work so they could create new jobs without having to copy and rename existing jobs. In the end the process for updating these templates was too time consuming, there were a few steps involved in l...

Go to the Vegas conference and heckle during the opening session.

The only time I've seen it was due to a misunderstanding with the license keys. They thought they only had to use the Enterprise license keys, and since they only used parallel jobs they never missed server jobs. From memory they could still use sequence jobs, it was only the server jobs that were m...

At a guess I would say that when you don't include your own WHERE clause then DataStage builds it for you using the key columns. If you do include a WHERE clause you effectively overwrite whatever the sparse join was going to generate. So your custom WHERE clause that included both the filter and th...

Some sites don't have server jobs installed. They don't think they need them and they do everything with parallel jobs. I would be inclined to put the code into a routine rather then a job. This gives you better control over the input and output arguments and lets you use the Test button for testing.

It takes between one hour and three weeks to get a remote DB2 connection working on a new DataStage server, depending no how friendly your DB2 administrators are. I found it helps to give full schema rights to all required DB2 databases to the dsadm user for running the setup and grant scripts, even...