Search found 23 matches

by rhys.jones@target.com
Tue Oct 03, 2006 11:43 am
Forum: Information Analyzer (formerly ProfileStage)
Topic: Using DB2Connect instead of Data Direct
Replies: 5
Views: 4308

If you're familiar with the dsenv file for DataStage, all you need to do is copy the environment variables pertaining to your DB2 connect installation location. You're correct, they go into the .ctl file for the analysis server. Make sure you restart the analysis server in order to make the changes take effect. You will also need to add entries into your .odbc.ini file that point to the correct DB2 Connect library file. Then you can reference that DSN name in the .odbc.ini file and all should work (provided your .odbc.ini database name corresponds to a catalogued alias in your DB2 Connect installation). Good luck!
by rhys.jones@target.com
Thu Feb 02, 2006 5:35 pm
Forum: Information Analyzer (formerly ProfileStage)
Topic: Complex format files Profiling - addendum
Replies: 3
Views: 3015

I see what you're saying now, Roy. You really do need to profile each sub-schema of the flat file independently. Which would mean spinning those related records into separate sequential files. That should still be possible in DataStage (CFF Stage to Transformer to multiple Sequential Files).

dscnn,
In ProfileStage you'd have separate logical databases setup to profile each file. If calling from DataStage, then, you'd have multiple ProfileStage packages you'd have to call from command line. Or create a new shell script that calls all your other runpackage.sh scripts. The end result you'd have to piece together manually to get a holistic profile of an entire complex file. But it's certainly possible.

I wonder - could you profile the distribution values table in your ProfileStage repository to get a picture of your entire complex file??
by rhys.jones@target.com
Wed Feb 01, 2006 6:27 pm
Forum: Information Analyzer (formerly ProfileStage)
Topic: Complex format files Profiling - addendum
Replies: 3
Views: 3015

Re: Complex format files Profiling - addendum

dscnn wrote:I have a complex file that needs to be profiled. There is a post on this earlier and it was suggested to break up similar rows into seperate bunches and then profile them.
But what is the use of buying the tool if it has to be done manually?

Is there any other better way to profile complex format data? All help would be greatly appreciated.


I don't think that it's possible in its current version, although I haven't tried yet. I recently had a conversation with one of their Support Engineers, and they won't support any middleware other than the DataDirect ODBC drivers. Which give you limited text file interaction (.csv is probably about it). You'd have to "flatten" your data. Try this with a CFF stage in Datastage writing to a sequential file, then kick off your ProfileStage package (see my response to your other post).

Information Analyzer and Hawk may improve upon this as the hype is the products will share common "connectors" to interact with source data. So this may indicate being able to extract in parallel, deal with CFF's, XML, etc.
by rhys.jones@target.com
Wed Feb 01, 2006 6:16 pm
Forum: Information Analyzer (formerly ProfileStage)
Topic: Calling ProfileStage jobs from DataStage
Replies: 5
Views: 3121

Actually, you can call ProfileStage packages at the command line. You posted the same question on Developer Net, so check there for my response.
by rhys.jones@target.com
Tue Dec 27, 2005 12:39 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: SIGSEGV, SIGBUS
Replies: 6
Views: 7069

Sorry, good ol' USA support engineers. :wink:

Yes we did receive a patch, and it resolved the SIGBUS/SIGSEGV errors that were coming from this bug. We now have been able to reenable time-based monitoring. Check with your support engineer for the patch under ecase e74035. We received a patch for both AIX and Sun.
by rhys.jones@target.com
Tue Nov 22, 2005 6:16 pm
Forum: Enhancement Wish List
Topic: ProfileStage Log File Cleanup
Replies: 0
Views: 1047

ProfileStage Log File Cleanup

One annoying thing about ProfileStage is the fact that it doesn't cleanup after itself. For every action you perform (analyzing, importing metadata, etc), it generates several flat files to keep track of that job (*.log, *.ini, *.err, *cleanup.log, etc). Not to mention that the Analysis Server and Message Switch both continue appending to their own log files. All the while you're consuming disk space like crazy and will eventually use up all space in those directories. It would be really nice if the tool would take care of older files automatically (and that the purge process was configurable so you could decide to keep the last x number of days/weeks/months of log files).
by rhys.jones@target.com
Tue Nov 22, 2005 5:39 pm
Forum: Information Analyzer (formerly ProfileStage)
Topic: Using DB2Connect instead of Data Direct
Replies: 5
Views: 4308

Just in case others were curious, the official word I got from Ascential was they wouldn't support using drivers other than DataDirect for ProfileStage. This isn't to say it's not possible, it's just not supported. I'm still waiting to hear back from them in regards to whether this will change with Information Analyzer, but I have to assume it will.
by rhys.jones@target.com
Mon Nov 21, 2005 10:21 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Unable to start Director
Replies: 14
Views: 4441

Is it working on any other desktops? How about on a different project? That may indicate whether the problem is just on your computer, or with a specific project, or possibly with the Unix server.
by rhys.jones@target.com
Mon Nov 21, 2005 9:42 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Unable to start Director
Replies: 14
Views: 4441

Since this is Windows, you can use the tried and true method of reboot!

Seriously, did you happen to install the Multi-Client Manager? If so, open that up, select the version that you have installed and change to that version (even if it's already selected). This goes through and registers all your libraries again, so sometimes self-corrects problematic installs. If that doesn't work, you may just have to reinstall.
by rhys.jones@target.com
Mon Nov 21, 2005 9:09 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Accessing mainframe DB2 databases via DB2 Connect from Linux
Replies: 6
Views: 3098

sparse lookup

Yes, the sparse lookup is possible via the enterprise stage, but only when going against a non-mainframe UDB/DB2. You can only connect to mainframe using the API stage (unless on USS), which doesn't support sparse lookup.

The trick on an AIX or Sun platform is that you have to make sure that all the DB2 nodes that are configured in your config file are included in your DB2NODES.CFG file. For some reason the operators of the sparse lookup validate the nodes in that file, but don't when doing a normal lookup.
by rhys.jones@target.com
Thu Nov 17, 2005 5:55 pm
Forum: Information Analyzer (formerly ProfileStage)
Topic: Who will use ProfileStage?
Replies: 7
Views: 3801

Re: Who will use ProfileStage?

olgc wrote:Hi there,

Who will use ProfileStage: Data Modeller or ETL developer? At which stage or stages of your project, do you use ProfileStage: project analyzing stage, data model, or ETL coding?

Thanks,


Yes and yes. Your data modeller and person developing your ETL designs will find it of most use. If you generate ETL jobs from the tool, then your ETL developer might find it useful to go back and see what decisions were made to generate the target DDL and jobs, but beyond that if they are profiling source data during construction, they've missed the boat. Really, you're going to leverage the most ROI by making data profiling a practice as early in your project's lifecycle as possible. By identifying your source data defects early, you can decide to address the problems in the source (retraining, business owners paying to fix the source, etc), or build a more intelligent design around correcting the bugs in your ETL jobs. This will save you more money in the long run, maybe even prevent you from making costly mistakes.
by rhys.jones@target.com
Tue Nov 15, 2005 7:59 pm
Forum: Information Analyzer (formerly ProfileStage)
Topic: Error loading Metadata.
Replies: 9
Views: 6834

Y'know, trainee, I'd be interested to know if you had configured your DSN using the wired protocol driver, or just the standard Oracle driver. You shouldn't have had problems with TNSNAMES.ORA entries if using the wired protocol driver (or so says Data Direct!). That's supposed to bypass client software and make its own connection.
by rhys.jones@target.com
Tue Nov 15, 2005 7:56 pm
Forum: Information Analyzer (formerly ProfileStage)
Topic: Error loading Metadata.
Replies: 9
Views: 6834

Ray was correct in that ProfileStage error handling is unlike any of IBM/Ascential's other tools. That error is a generic error that means you've opened a socket to your database, but don't receive a timely response. I've gotten it trying to connect to DB2 and Oracle sources. The other common generic error is the "Error 88 - Contact Support" which shows up just about everywhere, and doesn't mean jack!

Most of the really useful information is going to be in the log file that gets generated under the Analysis Server engine (could be many types, depending on the action you were doing: selecttables, genodbc, analysisadmin, etc). That error indicates that you need to open the corresponding log file in order to glean what the actual cause of the problem was. In there you'd definitely find your typical TNS error (or in the case of DB2 some CLI error). The log file location is going to be something like:

/apps/Ascential/ProfileStage/AnalysisServer/LogFiles

A word of advice for those of you with Unix-based Analysis Servers - get UltraEdit! It's really useful for looking in the log file directory for the most current logfile to open (open via FTP, sort on modified date). There are a LOT of files in that directory, and unless you've written your own cleanup script, searching for all files with a wildcard (ls -l selectTables*) can be a nightmare!
by rhys.jones@target.com
Tue Nov 15, 2005 7:44 pm
Forum: Information Analyzer (formerly ProfileStage)
Topic: Data profiling with PX license
Replies: 2
Views: 1985

The PX engine is the same Orchestrate framework that Datastage uses. We don't presently have it licensed, but had an evaluation copy a while back, so I've configured it and used it briefly. It's pretty slick and easy to setup. However, you're not platformed properly. The PXEngine presently ONLY works on a Unix platform for ProfileStage. So you'd have to move your Analysis Server over to Unix to get it to work properly.
by rhys.jones@target.com
Tue Nov 15, 2005 7:41 pm
Forum: Information Analyzer (formerly ProfileStage)
Topic: Multiple users
Replies: 1
Views: 1617

Re: Multiple users

I needed to understand how is it possible for multiple users (different users) to connect to the same profiling repository.

If you're using Windows, use active directory to create a security group. Lump all users into that security group, and then add that security group to your SQL Server and presto! Just add users.... :wink:

You could do the same in Unix - setup a security group on the server, add users, tie to the database (UDB or Oracle), etc.

Can any please tell me if it is possible to package the profiling jobs to run in a different system or to do a batch run on the same system.

Have you looked into creating packages? You can setup an analysis, and rather than running it from the GUI, create a package to run later. Then once the package is created, use a scheduling tool to trigger it. You'd have to experiment a little with the proper command-line verbiage (tail the log file as you kick off a package via the GUI to see it).