DSXchange

I have a DB2 version for anyone who is interested. We needed good process metadata for administration and troubleshooting so we chose it ahead of MetaStage. We were looking for a interim solution to take us through to the end of the year when DataStage Hawk delivers better metadata reporting against...

Ascential/IBM will hate this idea. It would be easy for a competitor to take the worst results for each platform and compare them to the results they have produced in a favourable lab environment and publish them or present them to potential customers. Meanwhile IBM/Ascential cannot use the favourab...

Most PX comparison stages (eg. lookups, joins, change data capture) work by bringing all the data from the primary stream and all the data from the reference stream onto the ETL server where the comparison action is performed. In the case of a lookup the reference data is loaded into memory for even...

With DataStage 7.5.1 the command line job report with the /R report switch turned on provides a bitmap and a HTML document with links between the bitmaps and the attributes section of the HTML document via some clever rectangle reference coordinates properties on the bitmap image. <img src=" imagena...

Using DB2 7.2.9 (shortly going to version 8.x) and AIX version 5.2 maintenance level 3 and version 7.5.1A of DataStage Enterprise Edition.

I am finding that error codes or error messages for the DB2 Enterprise stage is the equivelent to the loch ness monster. A couple people claim to have seen such a beast but I have yet to come across it. I have a job with a DB2 target stage, Write Method is set to Upsert, Output Rejects is set to Tru...

We have found sparse lookups to be very flaky. In one instance the DROP option drops all rows regardless of whether the lookup was successful or not. When we set it to DROP all rows get dropped. When we set it to continue rows get outputted with the lookup fields fully populated. In another instance...

I didn't think partitioning of the DB2 source would be required since a sparse lookup goes directly against the table. We are willing to pay the price for a sparse lookup since the lookup tables are typically over a million rows and the primary data has less then a thousand rows and the DB2 database...

I'm not so sure. I did testing on a Windows machine processing 100,000 primary rows against 100,000 lookup rows with a 1 to 1 match. Two key fields of char 255 and two non key fields also of char 255. I deliberately chose fat key fields. The dataset as a lookup took 2-3 minutes. The fileset as a loo...

Yes, we use the message handler to filter out known warnings that do not lead to loss of rows or data corruption. You get a few of these in parallel jobs. In some stages you get a warning for an event that is not a problem at all. See the message handler FAQ. Warning messages were being completely i...

DataStage is not a replicating tool, if you want real time replication between DB2 and Oracle you need to search the market for a better solution or have a look at DB2 replication tools. The best you could do through DataStage is get a DB2 log scraper, there are a few on the market and I think Ascen...

Is a lookup fileset any more efficient then a normal dataset?

I would class it as usable with most of the problems in the new 7.5 and 7.5.1 features. Most of our problems are on the client designer side and not on the server side. There is the 100% CPU utilisation you sometimes get in Designer when you have a hanging link open (which I think is linked to the n...

You have several options when moving from a server job to a parallel job: - use a BASIC Transformer in your parallel job or a server transformer in a parallel shared container. It can call routines but it is a lot slower then most parallel stages. - move your routine code into a parallel transformer...

Having problems with the lookup stage. Let's say I have 10 input rows and I am looking up a surrogate key in a DB2 table with 1 million rows. When I use the lookup stage in Normal mode it loads all 1 million rows down my reference link (even though I only need 10 of them). When I use sparse lookup m...