DSXchange

I absolutely 100% agree with you, Barbara. I also caught a challenge out there for someone to reconnect legacy DataStage jobs to the MetaData/MetaStage framework. That's something that myself and others have talked about doing since we've already reverse engineered everything else it seems like. As ...

Here's my 2 cents worth: ETL is hard because: (1) The data warehouse team lacks an experienced and "blooded" data warehouse architect. One intimately knowledgable in all aspects, from software development lifecycle, to modular programming techniques, to data warehouse architectures and phi...

It can't hurt unless users are designing jobs, but I think it tells you that users are in. It's pretty innocuous, though not a fix all. If you ever delete a job and you get a lot of messages saying unable to delete RT_LOGxxxx or the like, then this helps clean that up. It's not critical to cleanup i...

One of the downloads I have on the board does a normalization of most of the DataStage repository and outputs the data into different files. By reverse-engineering the job design database, I was able to create "tables" of passive stage usage. For example, a common task is to find all of th...

I've eliminated what I believe are all of the variables. After several replies on the Oliver maillist pointed me to the archives on the www.tools4datastage.com companion website, I think the problem is related to release 5.2 and earlier. Just to give an idea, my problem is with a 1400 job project. T...

I actually have all of the jobs categorized, and it makes no difference. Any job running activity outside the immediate folder still causes the horrendous refresh hang time. Apparently the focus is not on the jobs in the immediate category folder, so a job starting or finishing somewhere else trigge...

I was curious as to anyone with a lot of jobs in a single project. Here's some of my observations: 1. 500+ jobs in a project causes a long refresh time in the DataStage Director. During this refresh, your Director client is completely locked up. Any edit windows open are hung until the refresh compl...

You're absolutely correct. Quality Manager, Integrity, and MetaRecon are not really integrated with DataStage, more like bundled. As DataStage Server processes each row, the overhead of doing any external call for any reason will stagnate the data stream until that call returns. Using multiple job i...

I initially designed 2 approaches to prove a point. When you introduce anything to a single threaded process, you are limited to single threaded execution. So, doing an external call to an API to interrogate the cleanliness of a row means that you have this interface overhead and the single threaded...

A normalized warehouse would follow rules of normalization. A dimensional model follows rules for the star schema. They are not necessarily mutually exclusive. Recommend you read The Data Warehouse Lifecycle Toollkit by Ralph Kimball. You also need to visit www.BillImon.com and www.RalphKimball.com....

Michael, from my perspective, I do not use the packaging wizard. I prefer a good old fashioned written technical specification to cover documenting a migration set. I find the packaging paradigm in DataStage to be more of a "shiny object" or sales "bullet item" than anything else...

There's another reason that any kind of reporting model needs to be denormalized and that is performance. A denormalized, aggregated, shrunken, or collapsed structure "pre-joins" the data. If you have 30 variations of the same report navigating the same tables over and over, conducting the...