DataStage Job Standards Checking Tool

rwierdsm · Post by **rwierdsm** » Tue Dec 03, 2013 2:58 pm

Folks,

On a bit of a trolling mission here...

I'm documenting ETL best practices and standards at my current client. One of the managers had experience with a tool that would check 3GL code for naming conventions, code layout, commenting, etc. and wondered if such a thing exists in our world. While I have never heard of such a tool for DataStage, I thought I would ask here to see what our community's experience in this area is.

Rob

bart12872 · Post by **bart12872** » Tue Dec 03, 2013 3:08 pm

such process exist, but it was proprietary code. You can export dsx and analyze it, you can import it in a database or whatever. There is full of possibilities. Just do it or find it.

ray.wurlod · Post by **ray.wurlod** » Wed Dec 04, 2013 3:40 am

I have a routine that checks for existence of default names on stages and links, but not much more than that. We have many more standards but I have not given any thought to automatically checking; we normally advocate a process of peer review, because that also involves documentation and unit testing.

rwierdsm · Post by **rwierdsm** » Wed Dec 04, 2013 7:06 am

My approach has always been to create templates of each data flow pattern. I ensure that the pattern adheres to all the standards. Each detail design specifies a starting pattern and adherence to the pattern / standards is enforced at code review time.

IBM is suggesting much the same approach.

Some of the development team here has used various tools that do a lot more for Java development. I can't see how something like that could work for DataStage though.

Rob

eostic · Post by **eostic** » Wed Dec 04, 2013 8:39 am

I like the idea....it certainly is do-able, but I haven't heard of anyone building such a tool.......one could comb thru [probably by examining a .dsx] link names, job names, job parameter names and lengths, etc. etc. etc.

As reflected above, most sites enforce best practices through review, and no one's had time (or hasn't publicized the success of such a thing) to create a "helper" tool along the way. A nice as such a tool would be, there would still need to be human/community review to ensure compliance.

Ernie

ray.wurlod · Post by **ray.wurlod** » Thu Dec 05, 2013 3:51 am

You don't need a .dsx, at least until IBM eschews use of the local repository for design-time information. It's all there in DS_JOBOBJECTS (and can be verified against RT_CONFIGnnn if necessary).

rwierdsm · Post by **rwierdsm** » Thu Dec 05, 2013 12:32 pm

The conclusion to this thread seems to be that the necessary bits to accomplish this are available in DataStage and it would be a great idea if someone were to build something, however, at this time is no tool that accomplishes this.

Rob

FranklinE · Post by **FranklinE** » Fri Dec 06, 2013 12:28 pm

A good impact analysis application should be able to accomplish this, though inconveniently requiring the user to manually set up search routines.

We use Smart Technologies TS XL. It readily parses designer code from an xml-formatted export.