DSXchange

This posters problem is the all-in-one job that new users always write. They never understand that their transformation pipeline runs at the speed of the slowest consuming process. In this case, it's obviously the receiving database putting back-pressure all the way up the pipeline. I thought this ...

remove duplicate will remove duplicate records and select unique records only; it will not remove all the records which are duplicate... You lost me on that one. What's the difference between what it will do and what it won't do again? Never mind, for some reason the distinction was lost on me the ...

keshav0307 wrote:remove duplicate will remove duplicate records and select unique records only; it will not remove all the records which are duplicate...

You lost me on that one. What's the difference between what it will do and what it won't do again?

What about the Remove Duplicates stage?

Not that I don't trust you or anything, but I'd have to try this for myself before I believed it.

No, it always needs to be a character type - decimal may 'accept' leading zeroes but it sure as heck won't go out into the world with them still in place.

What is your target, a sequential file? Your output field really needs to be a character field of some sort to retain the leading zeroes, hence my question.

What's not working about it? Is your output field defined as a Char/Varchar field or something else?

Is the kid of Norse stock?

(figured why not continue to litter a perfectly good VC thread with silliness)

Just so we're all on the same page, if you do this: select count(*) from ( SELECT A.*, B.*, C.* FROM TABLE_A, TABLE_B, TABLE_C WHERE A.XYZ=B.XYZ AND B.MNO=C.MNO AND B.EFFDT BETWEEN C.EFFDT AND C.ENDDT ) I just verified it doesn't change the original explain plan nor the cost or cardi...

Yah, there's nothing like back-pressure all the way up your pipeline to ruin a perfectly good day. ps. Ken, I don't mean replace what you are selecting with a COUNT(*), I mean wrap the whole original query in a count select of the result set. Don't believe that changes the explain plan of the origin...

True, figured cow might be the most common ruminant... and you keep goats where? As the Ever Travelling Man, I can't imagine they would travel well with you... or recall hearing them in your hotel room in Texas.

Perhaps they're just well trained...

Still, no amount of phantoms is ok. The suggestion is to try and make them go away and the added bonus may be that this other problem goes away as well!

In other words, what you are going to find is your 'performance problem' isn't on the source side, it's on the target side. And dollars to doughnuts it's your update link. Are the fields you are using as your Keys - the ones used on the where clause - indexed?