Quality Stage and DS

Infosphere's Quality Product

Moderators: chulett, rschirm

Post Reply
gomez
Premium Member
Premium Member
Posts: 83
Joined: Tue Jun 22, 2004 6:11 am

Quality Stage and DS

Post by gomez »

I have a few basic doubts with using QualityStage
1. Can I call a QualityStage job from a DataStage Server Job ?
2. I have a set of files on which frequency analysis is to be performed. I can use the Investigate stage from QualityStage for this. Now the analysis is performed on different set of columns for different files. Can we create just one QualityStage job to achieve this by parameterising the set of fields. Can any scripting done with Quality Stage can help us in this. ?


Thanks in advance
[/list]
T42
Participant
Posts: 499
Joined: Thu Nov 11, 2004 6:45 pm

Re: Quality Stage and DS

Post by T42 »

gomez wrote:I have a few basic doubts with using QualityStage
1. Can I call a QualityStage job from a DataStage Server Job ?
Yes, you can. You will need to install QSforDS plugin for both client and server in order for this to work. See the eService web site or contact your Ascential rep for the CD.
2. I have a set of files on which frequency analysis is to be performed. I can use the Investigate stage from QualityStage for this. Now the analysis is performed on different set of columns for different files. Can we create just one QualityStage job to achieve this by parameterising the set of fields. Can any scripting done with Quality Stage can help us in this. ?
#1 will solve #2 as long as the fields are exactly the same.
PilotBaha
Premium Member
Premium Member
Posts: 202
Joined: Mon Jan 12, 2004 8:05 pm

Re: Quality Stage and DS

Post by PilotBaha »

gomez wrote:I have a few basic doubts with using QualityStage
1. Can I call a QualityStage job from a DataStage Server Job ?
2. I have a set of files on which frequency analysis is to be performed. I can use the Investigate stage from QualityStage for this. Now the analysis is performed on different set of columns for different files. Can we create just one QualityStage job to achieve this by parameterising the set of fields. Can any scripting done with Quality Stage can help us in this. ?


Thanks in advance
[/list]
You've got your answer for the #1 in the previous post. I am not sure if I agree with the #2.

First of all, I need to understand what kind of parameterization you are reffering to. If you are trying to call a QS job from DS and pass on some parameters so that QS will perform differently, the answer is no, you cannot do that..

If you are doing one type of investigation, then moving data into QS in a single format will solve some of the flexibility you are looking for. Then, there is the issue of sending different type of data and requiring word/character investigation on them. If your investigation requirement varies from data to data, "one size fits all" type of investigation won't be a good approach..

I may sound confusing, but if you can give me a bit more concrete example I know I can answer your question better :wink:
dls
Premium Member
Premium Member
Posts: 96
Joined: Tue Sep 09, 2003 5:15 pm

Post by dls »

Installing and configuring the QualityStage Plugin should probably be straight-forward, but I've been working at it for a couple of days now.

E.g., I had a bit of trouble just starting the QSRT Manager. Don't assume that port 6010 has been registered/reserved.

The QualityStageMetadata Plugin works as I expected.

However, when I open the QualityStage stage in my DataStage job, I'm not seeing the jobs and projects in their respective drop-down boxes. I was forewarned about this, but the 'Open Repository' command from the QualityStage File menu doesn't solve that.

Since another developer is responsible for creating the QS jobs that I plan to access in my DS job, I didn't assume that I would need the QS client on my desktop machine. However, it was only after I installed the QS client on MY desktop, established a run profile, imported the IMF file from the OTHER desktop, and opened the repository, that I was able to see the QS projects and jobs.

Learning as I go.........
PilotBaha
Premium Member
Premium Member
Posts: 202
Joined: Mon Jan 12, 2004 8:05 pm

Post by PilotBaha »

DLS,
you should have asked. :)
DS and QS are completely different animals, different architectures, different mentalitied. QS is not a true client/server software to boot. :roll:
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

QualityStage with the Real Time Manager comes pretty close to "true" client/server, however.
Currently setting up just such an animal.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
PilotBaha
Premium Member
Premium Member
Posts: 202
Joined: Mon Jan 12, 2004 8:05 pm

Post by PilotBaha »

That's good to know Ray. I have never worked with RTI before , so I am completely clueless on that "animal" :)
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

To be pedantic for a moment (who, me?), it's not really RTI. Quality Stage has its own "real time manager" (runs as qsrtmngr) which only DataStage uses. It's that to which I referred. (Incidentally, it works well straight out of the box if one follows the instructions accurately).

If you want to expose DataStage jobs, or QualityStage jobs, or DataStage jobs that invoke QualityStage, as web services, that is when you need the Real Time Integration (RTI) component.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
PilotBaha
Premium Member
Premium Member
Posts: 202
Joined: Mon Jan 12, 2004 8:05 pm

Post by PilotBaha »

Ray,
I don't think I would argue with you, but I don't believe using QS from DS comes as close to a "true" client/server as DS is. I think QS has to come long way to be as flexible as DS..

In your initial post , I thought you were refferring to the RTI ..

I am not questioning your knowledge, or your pedantness (if there is a word like that exist) , but this is the second time this week I was replied with "not so pedantic, but down right rude" comments by you.

viewtopic.php?t=89920

:?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You misread it. It was I who was being pedantic. :D

Sorry if I caused any offence; certainly none was intended.

ETL is a world in which taking pains to be accurate is rewarded by getting things to work correctly first time. Doubtless some of this rubs off on one's personality.

And the other post to which you referred was an attempt at humour. Oh, well. Never was that good at telling jokes.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
PilotBaha
Premium Member
Premium Member
Posts: 202
Joined: Mon Jan 12, 2004 8:05 pm

Post by PilotBaha »

I am sorry if I was oversensitive.. Last few days have been pretty rough..
Wish I had some emoticons with two hands cheering with beer glasses..

Once again, my apologies..
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

I've heard it said that the DataStage plugin for the Trillium data quality project is better integrated with DataStage then the QualityStage plugin.

My experiences with version 7.0 is that they were still panel beating QS into a client/server product. QS still had client based repositories, it still produced different behaviour when running a job from the command line on the server then when it was run from the QS client, it had lots of metadata integration problems with DS if you kept changing the definition of data files and it the server deployment functions were unreliable.

By the end of 2005 QS will be merged into the DataStage pallette and they share a single repository and it should be one of the best data integration products on the market.

Despite the integration problems I would recommend running QS from within a DS job instead of running QS on its own, much more control and output options and better monitoring services.
PilotBaha
Premium Member
Premium Member
Posts: 202
Joined: Mon Jan 12, 2004 8:05 pm

Post by PilotBaha »

I didn't know Trillium had a plug-in for DS. Is this new?
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

Welcome to the confusing world of vendor partnerships. Neither the Ascential or Trillium (Harte-Hanks) web sites mention the existance of any partnership between the companies or a plug-in, however I have a link to the press release!!!!
http://www.findarticles.com/p/articles/ ... i_63648895
Informix Corp. recently formed a partnership with Trillium Software, a division of Harte-Hanks Inc., to integrate the Trillium Software System with Informix's Ardent DataStage suite.

Under the terms of the agreement, Informix (www.informix.com) customers will be able to tap into Trillium's suite using a DataStage plug-in that will be made available in late June (2000). It is expected to sell for about $15,000.
I have been told recently by a Trillium sales person that the plugin is available, I don't know whether it is an option for version 7.x of the suite.

At the time Ascential had a similar agreement with First Logic. I'd say the relationship with both vendors turned very sour when Ascential acquired a competing tool. On the partner site Trillium lists Informatica and Ab Initio but not Ascential. Trillium also got the cold shoulder from Informatica who last year announced an OEM arrangement with First Logic for data quality, relegating Trillium to a second string option. Ab Initio still list Trillium as the data quality option and the Ab Initio/Trillium combination competes directly with DataStage/QualityStage for massively scalable parallel processing capability.

The Trillium/Ab Initio and Informatica/First Logic products will be challenged by Ascential Hawk that puts DS and QS into a single designer on a single repository.
PilotBaha
Premium Member
Premium Member
Posts: 202
Joined: Mon Jan 12, 2004 8:05 pm

Post by PilotBaha »

It is interesting. Actually earlier this year when I posed the question about writing plugins I was thinking about this :) At the time I was not "made aware" by neither parties that there is such a plug in :)

Let's wait for Ascential Hawk and let's hope that they won't change the name every 5 months , this time following the bird names :lol: :lol:
Post Reply