DSXchange

How many of the following documentation techniques and processes are being fulfilled in your current engagement? <ul> <li>ETL naming conventions</li> <li>ETL source to target documents</li> <li>ETL source to target map table</li> <li>ETL load outline</li> <li>Use of system tables to help design ETL</li> <li>Use of Version Control</li> <li>Documents for each job </li> <li>Row counts of ETL</li> <li>Jobs by table report </ul> <p>If you’d like some help with some of these above processes download KgdGenHtml.zip and EtlStats.zip, both of which are available for free on <a href="http://www.duke-consulting.com">Duke-Consulting.com</a> and ADN. KgdGenHtml.zip will create documentation for each job in a project. Of further help is DSaveAsBmp.bat which will create a bit map image file for each DataStage job in a project. Also EtlStats.zip will gather row counts by link and should be run after every job sequence. Lastly don’t forget Kim Duke's product DwNav. It’s almost free so buy a copy! DwNav will give you html documents for Jobs by table report.</p> <p>For more information on the above products contact <a href="mailto:kim_g_duke@hotmail.com">Kim Duke</a>.
</p>

All of the products from Kim Duke are nice, especially for autogenerating documentation is a format relevent to developers and and the business side. I highly recommend those not familiar with the offering to check out the Duke Consulting website

We recently down loaded the Duke's doco suite. Now we can view our jobs without loading data stage.

I had to do a few tweaks to the bat files and html as the BMP's are too "steenkin'" big. Ran them through a batch BMP > JPEG, sed'd all the html s/bmp/jpg/ and now what was 700MB doco is 200MB and loads quicker.

We use a UNIX DS engine so had to extract the list on the UNIX box and bypass the bat files getting of the list, no biggy but a trap for people who aren't familiar with bat files (none around here anyway, what am I typing :) ).

The only thing I would like to see in the html doco is the derivations in the transformer stages, but beggars can't be choosers.

Kim, thank you for a great suite of utilities, they have made a 1 month project to try and document our 200+ programs into a 4 hour sit back and let the computer do all the work afternoon. Even the boss was impressed.

Thanks. And the price is right too. They are free. Tony from ASCL wrote the JobReport routine. So I cannot change where the derivations print. I wrote a wrapper so it would run it for all the jobs and I wrote the routine documentation. Also an index to all the jobs and routines. It was fun. Learned a lot.

I hate that save as bitmap stuff too. Way faster than SnagIt though. Takes too long to pull up the jobs. You should post the save as jpg and back to bmp routine. It used to do gif files from SnagIt. They are very small. Look better too but speed is important when you have hundreds of jobs.

Thanks Tony. A very cool routine JobReport.

BMP-> JPEG used Easy Image Converter from http://www.homeplansoftware.com.
I've got the Unix/win32 suite of programs so I could:
ls *.bmp > a | sed s/.bmp// > a
then
gawk -f a.awk a > aa.bat
where a.awk is:
{
print "\"C:\\Program Files\\Home Plan Software\\Easy Image Convertor\\imgconv.exe\" "$0".bmp "$0".jpg F-"
}

then ran aa.bat (this took a while)

While that was going:
ls *.html > b
gawk -f b.awk b > bb.bat
where b.awk is
{
print "sed s/bmp/jpg/g "$0" > C:\\data\\Doco\\WDR\\LOLOxDSL\\"$0
}

and ran bb.bat

Because we use a unix D/S server I had to modify the DataStageBackup.bat to not pull the list of jobs from the project and run it manually on the unix box and bring the file down, just a coupe of comments.

The HTML was of cource deposited on the unix machine, had to make a couple of minor changes to the output path, no biggy and then ftp'd the html down to the windoes machine for the above maniputaltion,

A few hours but the boss' are pleased (kudos for me and indirectly to Kim and Tony). Like I mentioned earlier, the only addition I would like is the transformation derivations etc.