DSXchange

I wouldn't be surprised if there was a single tool that had a conversion engine and had various adaptors for Informatica, DataStage server jobs and Ab Initio. In theory they could build a converter for any ETL tool if the demand was there.

kcbland wrote:Proctologists

I would call that a necessary discomfort rather than a necessary evil! They do have the patients best interest at heart (unlike the parking inspector).

Sounds like a good candidate for a custom built stage. Something that could navigate through the list of columns looking for nulls in a loop with an output link and a reject link. It is code intensive but you can do it in a transformer stage. Most functions will reject a row if that row has nulls in...

I have the definitive "necessary evil": parking inspectors.

There are certain minor useability things that I think make or break a forum. For the old developernet the registration and login restrictions broke it and restricted it to 1/100th the volume of dsxchange. Getting a login was hard, once you had a login it did not automatically log you in and this wa...

Hey heah, I had forgotten about that blog entry! It even has a link to my wiki entry on retrieving SQL codes for DB2 stages! The other article of interest is Data Quality Firewall 2: inside an ETL job which has some bitmaps of how to pass your reject flow into a shared container.

Ab Initio was the first of the parallel ETL tools and on scalable performance it is hard to beat. The main difference is that DataStage parallel is now more widely used, has easier access to training and documentation and online help and better integration with related data integration products. Inf...

Yes, the developerworks forums are terrible. The interface is really, really bad. The search is poor and doesn't seem to work at all well and has no advanced search option. These are mirrored forums from real newsgroups so I assume there are newsgroup tools out there that we could use instead?

There is a trick to it. Turn on colum propogation out of the DB2 stage reject link and type in the two column names manually. They will appear! For a peek stage it automatically shows all propogated columns such as the existing rejected columns and the two new database columns. Changing this to a se...

The transformer stage has no reason to repartition data, it has no requirements for grouping key fields for example. The easiest design is to leave the transformer as default - same partitioning so it retains whatever the input link has set partitioning to be. If it is the first partitioning stage t...

On Tuesday I blogged about Developernet being unavailable and Developerworks not being ready yet. I actually got a blog comment from a US IBM employee when the forums were ready! It appeared at that point as though they had put it in the DB2 forums (which would have resulted in a very critical blog)...

There are two ways to get and apply your change capture. You start with a Before set of data and an After set of data. If you use the change capture stage alone it gives you new/changed/deleted/unchanged records. There is a flag to turn each one of these on or off and an extra field is written out t...

If I had DataStage running I would run a quick test to see. I am assuming that transforms are run first followed by constraints. You could test it by creating a transformer with a reject link, a constraint that blocked all rows and a transform function that triggered a rejection. If the constraint r...

The transformer is a good allrounder: offering constraints, transforms, stage variables etc. The problem is that it performs all those transforms on all rows before the constraints are applied. This means if your input to the lookup has 100mill records, your range lookup expands it to 300mill record...

That is spot on, you need a lookup stage followed by a filter stage. Resist the urge to put in a transformer with a constraint as it will be slower in this instance. Better to have a lookup - filter - transformer rather than a lookup - transformer. DataStage 8 (Hawk) adds range lookup functionality ...