DSXchange

I also agree that vmware is not for a production ETL server. It is fantastic for maintaining portable training and evaluation environments. On a Windows server your RAM and CPU power gets divided between the host and the vmware session so you never really get 100% of the resources on the box for you...

To quickly see the rejection error message add a peek stage to your job with a reject link out of your EE database stage. Within the DB stage set the "Reject" optional property to true, you should be able to find it somewhere in the properties list. Re-run your job. The sql code and sql error code s...

This is quite common when you put a transformer after a join or lookup stage. You end up running a transformation function against a reference field that can be null. You need to put null handling around any transformer field that might contain a null and requires some type of transformation change ...

Make sure you fire it off to the Hawk support team, getting feedback on installation problems is part of the beta testing process and will help make a better production release. Hopefully they are already monitoring the forums for Hawk posts.

Try this: read your reference table and send it to a copy stage, have three output links to a lookup stage. Each link has a different key field: cnum, pnum and qnum. In the lookup stage link your input stream to all three lookups. In your output mapping take the lookup fields from one of the three l...

The parallel sequential file stage has a file mask option for defining file names, this lets you process files in a folder that match the mask, defining * as the mask gives you every file in the folder.

A sequence job can call another sequence job via the job activity stage, so in a way all sequence jobs are shareable containers.

A very good point. If you look at Google Trends for the last two years the top 10 cities doing a Google search for the term "DataStage" since 2004 are: 1. Pune, India 2. Bangalore, India 3. Hyderabad, India 4. Chennai, India 5. Mumbai, India 6. Singapore, Singapore 7. New Delhi, India 8. Minneapolis...

I have put most of the DataStage Certification links I could find over on a squidoo lens: http://www.squidoo.com/datastage

You can also do a Google search for "Datastage certification" where you will find the first two results are my certification blogs, the next two are my certification wiki pages followed by IBM links.

I have noticed that SSIS bloggers tend to outnumber DataStage and Informatica bloggers by about 10 to 1 and they are often coming from an established network of SQL Server code cutters. They have some very detailed discussions on things like SCDs, balancing between ETL and ELT and surrogate key gene...

It's about time. There is almost no WebSphere II content anywhere in Developerworks and this big forum dump will give it some a big kickstart. Now if only they started putting WebSphere II articles into Developerworks...

The installation requirements and learning curve of MetaStage has put up a formidable barrier for a lot of ETL projects. IBM-Ascential are addressing this in the next release (version 8 or Hawk) where a lot of the MetaStage functions have been moved onto the DataStage Designer and given a better GUI...

Parallel jobs are faster on large volumes of data via all the parallel partitioning capabilities, they are also faster with sorting and aggregation functions even when compared in a non-parallel mode. The C++ stages of parallel jobs seem to be more efficient. However parallel jobs have a slower star...

You need a copybook that defines the structure of the file, you then use the CFF importer to read this copybook and turn it into a complex flat file table definition. The data can then be read by the CFF stage and the complex structure flattened into a relational structure. Unpacking of fields will ...

As I said in Is the DataStage parallel transformer evil? the benefits of the transformer stage far outweigh any time overhead. I would always design with transformers and switch them later if extra performance is required. There was some bad press about transformers a couple ascentialworlds ago and ...