DataStage Analysis spreadsheet

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

PhilHibbs
Premium Member
Premium Member
Posts: 1044
Joined: Wed Sep 29, 2004 3:30 am
Location: Nottingham, UK
Contact:

Post by PhilHibbs »

I think I've found a significant error in the File List tab. The Stage name will not always be correct - in fact I guess it will only be correct 50% of the time. :oops: I'm working on it now.
*Update:* False alarm... I think...
*Update:* No, but it is less widespread than I thought.
Phil Hibbs | Capgemini
Technical Consultant
PhilHibbs
Premium Member
Premium Member
Posts: 1044
Joined: Wed Sep 29, 2004 3:30 am
Location: Nottingham, UK
Contact:

Post by PhilHibbs »

OK, I've found and fixed the file stage problem - it occurs if there is another stage in between the file stage and the input/output link. Files now hook back to the stage via the ID which should be 100% reliable.

Version 1.4 also picks up NLS details. Hashed file paths are recorded against the Stage. Some hashed files may have had incorrect paths prior to this version.
Download
Phil Hibbs | Capgemini
Technical Consultant
PhilHibbs
Premium Member
Premium Member
Posts: 1044
Joined: Wed Sep 29, 2004 3:30 am
Location: Nottingham, UK
Contact:

Post by PhilHibbs »

Version 1.5 now consolidates the modified date and time into one column, and there's a new button that does an update of the job sheet with the compiled time. Jobs with a modified timestamp later than the compiled timestamp are highlighted in red. This is gathered from the DSEXECJOB sections of the dsx file, which I am now splitting out into a separate file as part of my sort process. In future I might integrate the two steps so you can process a single dsx that contains both the DSJOB and DSEXECJOB sections, but it works better for me this way as I am already sorting and splitting the DSJOB sections prior to analysis. The sort and split scripts are also included in the zip file.
Download
Phil Hibbs | Capgemini
Technical Consultant
PhilHibbs
Premium Member
Premium Member
Posts: 1044
Joined: Wed Sep 29, 2004 3:30 am
Location: Nottingham, UK
Contact:

Post by PhilHibbs »

Version 1.6 now decodes all your passwords, just to teach you that you shouldn't assume that they are safe just because they look encrypted.
Download
Phil Hibbs | Capgemini
Technical Consultant
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You naughty man!

I'm sticking with version 1.5.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
PhilHibbs
Premium Member
Premium Member
Posts: 1044
Joined: Wed Sep 29, 2004 3:30 am
Location: Nottingham, UK
Contact:

Post by PhilHibbs »

ray.wurlod wrote:You naughty man!

I'm sticking with version 1.5.
1.5 stores encrypted parameter defaults and SAP stage password strings, which are trivial to decode. I didn't even need to work out how the values are encoded, I just used AutoIt to create a job that had 96 parameters that were all 100-repeated-character strings of every ASCII value, ripped the encoded versions out, and the VBA uses them to do a positional search of each 3-character group of an encoded string.

But if it makes you happy I'll add it as an option that is turned off by default so it doesn't store any encoded values in the first place.
Phil Hibbs | Capgemini
Technical Consultant
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Someone else some time ago posted about how simple the encryption was and even the details on how to break it. They were very proud of themselves. Like it was meant to be hacker-proof or something. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
PhilHibbs
Premium Member
Premium Member
Posts: 1044
Joined: Wed Sep 29, 2004 3:30 am
Location: Nottingham, UK
Contact:

Post by PhilHibbs »

Version 1.6a - now has a Yes/No box for passwords, to control pick up of encrypted parameter defaults, and SAPPWD and PASSWORD values.
Download
Phil Hibbs | Capgemini
Technical Consultant
PhilHibbs
Premium Member
Premium Member
Posts: 1044
Joined: Wed Sep 29, 2004 3:30 am
Location: Nottingham, UK
Contact:

Post by PhilHibbs »

Version 1.7 separates the Control Sheet from the Job List tab, and adds a new button to run the dscmdexport.exe utility. The path to the Ascential software is currently hard-coded at the top of the DataStageAnalysis module. You only need to specify the Server and Project, it will generate a path for the dsx file if you do not specify one, and if you just specify a directory then it will put the dsx file there.
Download
Want to use DropBox? Ask me for a referral and we both get extra space!
Phil Hibbs | Capgemini
Technical Consultant
serices
Participant
Posts: 12
Joined: Tue Jan 03, 2006 12:31 pm
Location: Chile

Post by serices »

very good contribution, thank you very much!
PhilHibbs
Premium Member
Premium Member
Posts: 1044
Joined: Wed Sep 29, 2004 3:30 am
Location: Nottingham, UK
Contact:

Post by PhilHibbs »

Version 1.8d, significant changes to user interface and a few new tricks.

Front page now has config items that were previously hard-coded.

Any errors encountered will be logged on the front page. Columns can now be deleted without causing a crash, but you will get warnings that the cell tags can't be found.

Job sizes in bytes are now recorded so you can see what your biggest jobs are.

FTP server is now recorded in Stage notes, and Sequence Job parameter usage counts are now gathered.

Oh, and don't get too excited over the new tabs for Executables, Shared Containers, Stage Types, Table Defs, Transforms, and Routines, I haven't actually written the code to populate them yet!

p.s. Is anyone actually using this?

Want to use DropBox? Ask me for a referral and we both get extra space!
Phil Hibbs | Capgemini
Technical Consultant
rcanaran
Premium Member
Premium Member
Posts: 64
Joined: Wed Jun 14, 2006 3:51 pm
Location: CANADA

Post by rcanaran »

Not yet. But it sounds interesting. Will have a look at it tonight.
PhilHibbs
Premium Member
Premium Member
Posts: 1044
Joined: Wed Sep 29, 2004 3:30 am
Location: Nottingham, UK
Contact:

Post by PhilHibbs »

Version 1.8g.

Improved handling of deleted spreadsheet columns.
Detects job parameters in User Variable Stages.
Remove requirement for access to VBA Object Model.

Want to use DropBox? Ask me for a referral and we both get extra space!
Phil Hibbs | Capgemini
Technical Consultant
PhilHibbs
Premium Member
Premium Member
Posts: 1044
Joined: Wed Sep 29, 2004 3:30 am
Location: Nottingham, UK
Contact:

Post by PhilHibbs »

Version 1.9b.

Job log retrieval! Retrieves job logs and stores them in new tabs. Requires Perl and the included "tnet.cmd" Perl script, which executes telnet commands on a Unix DataStage server.

Also, documentation! In colour!

Want to use DropBox? Ask me for a referral and we both get extra space!
Phil Hibbs | Capgemini
Technical Consultant
PhilHibbs
Premium Member
Premium Member
Posts: 1044
Joined: Wed Sep 29, 2004 3:30 am
Location: Nottingham, UK
Contact:

Post by PhilHibbs »

Version 1.9d.

Director functionality extended - now can run jobs (in -wait mode), retrieve logs, and reset jobs. Still needs Perl and the included tnet.cmd script. If anyone knows a way of doing Telnet natively in VBA, I'd love to do it that way.

Want to use DropBox? Ask me for a referral and we both get extra space!
Phil Hibbs | Capgemini
Technical Consultant
Post Reply