test the data

Infosphere's Quality Product

Moderators: chulett, rschirm

Post Reply
raghunind
Participant
Posts: 2
Joined: Mon Jan 31, 2005 9:08 pm

test the data

Post by raghunind »

Hi,

I need to perform the following check on the input data which is a field which contains the description of an item. I need to check if every word of the text starts with an uppercase letter or not and report an error if it doesn't start with an uppercase letter(in this case write to an output file the records that do not satisfy this condition)

Any ideas how I can accomplish this ?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

This isn't really a job for QualityStage; it's probably better done with a DataStage server job.

Read the file using a Sequential File stage. In a Transformer use an output constraint expression to test the string against its converted value.

Code: Select all

string = Oconv(string, "MCT")
If this test is satisfied, the string meets your criteria. On a second output link either mark the link as handling rejects, or use the logical inverse of the above expression (that is, use "<>" rather than "="). This will handle input rows in which the string does not satisfy your criteria.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
PilotBaha
Premium Member
Premium Member
Posts: 202
Joined: Mon Jan 12, 2004 8:05 pm

Post by PilotBaha »

If someone really knew QS they would have already known that they couldn't do this. I am basically tired of people claiming QS when they know squat about it.. I was once told "I don't know QS but I have friends that do" by a "consultant" .. So i have a PhD in Chemical Engineering then, since I have a friend who is.
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

The first time I worked on a project that combined DataStage with QualityStage there was a lot of confusion over what tasks should be done in each product since they both have ETL and text transformation functionality. Since this is a problem of handling and validating free text it is reasonable to look for a solution in QualityStage first.

I'm sure QualityStage could carry out Ray's solution, change the field to title case and save it to a new field, compare the old field to the new field and reject those that are different. DataStage would do this in a single step in a transformer, QualityStage would do this over several steps, therefore it is likely that DataStage will be faster and more efficient.
JamasE
Participant
Posts: 32
Joined: Sun Aug 31, 2003 5:52 pm

Post by JamasE »

vmcburney wrote:I'm sure QualityStage could carry out Ray's solution, change the field to title case and save it to a new field, compare the old field to the new field and reject those that are different.
Could do it in QualityStage using a Select Stage. First define a field that is the first character of the original field, then Select and accept if =A, =B, =C, etc...

This does mean you'd need to put in 26 = accept statements, so would be quicker to implement in DS, but can be done in one Stage in QS.

Cheers,
Jamas
PilotBaha
Premium Member
Premium Member
Posts: 202
Joined: Mon Jan 12, 2004 8:05 pm

Post by PilotBaha »

Jamas,
you can do what you suggest, if (and that's a big one) if the first character of the field was the case. The original requirement is for the field where every letter in the description field has to be checked for upper case.
Your solution deals only with the first character of the entire field.

If I am understanding the requirement corectly of course. I hope I didn't come around sounding like an *ss; but there are a lot of impersonators in this business, and some IT companies, go for the el chepo ones..
PilotBaha
Premium Member
Premium Member
Posts: 202
Joined: Mon Jan 12, 2004 8:05 pm

Post by PilotBaha »

Nope,
it was posted in QS forum, because the initial poster was told by his client "let's use QS for this" :) Both are clueless though..
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Nope, also.

It's just thst there's no place to put in a QualityStage version number. :roll:

But QualityStage version 7 can run on Windows perfectly adequately (given enough memory especially).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
JamasE
Participant
Posts: 32
Joined: Sun Aug 31, 2003 5:52 pm

Post by JamasE »

PilotBaha wrote:Jamas,
The original requirement is for the field where every letter in the description field has to be checked for upper case.
Your solution deals only with the first character of the entire field.
Yes, quite right, quite right...

Erm, can't think of a simple way to do that in QS then. Maybe something in the Unijoin stage, but I'd have to investigate harder.

(Myself, I'd do it in SAS because we use that far more than QS anyway.)

Cheers,
Jamas
PilotBaha
Premium Member
Premium Member
Posts: 202
Joined: Mon Jan 12, 2004 8:05 pm

Post by PilotBaha »

Jamas,
it can be "sort of done" in QS. you can absolutely pars the field into words through some pattern access stuff, and then you can take a look at the first character of each parsed field through a SLC stage.
Besides being highly inefficient in many aspects, you cannot really guage how many words are in one description field.

I just wish the people who are recruiting so called "QS experts" knew at least this much about it

</vent> :)
Post Reply