DSXchange

Since the date is your key, lookup hash file 1 (will give you either a record not found or a value), and hash file 2 (will give you a value that might be zero) and then the derivation does a

srinagesh, you will need to talk to your DBA here and have him/her monitor the database while you are loading. They will certainly find something (since there is always something to find) that can improve you jobs's performance. The indices are going to be a great place to start; if you could split ...

I am not sure that the previous row will hold what you are looking for - since hashed files come in no particular order. Can you not do 2 lookups and performthe IF 0 logic in the transform stage?

Craig,

10% finished so far and no duplicates. Have you got a fast machine?

I wrote a small program that is chewing up CPU which checks to see if a 10 character string produces any CRC32 duplicates. If not, then you can use the CRC32 method. It will take a couple of hours to run to completion, though.

I wrote a small program that is chewing up CPU which checks to see if a 10 character string produces any CRC32 duplicates. If not, then you can use the CRC32 method. It will take a couple of hours to run, though.

DataStage will forget about the previous line; but you can store the value in a stage variable in a transform stage and use that value in the next line. This is very commonly used for many different type of applications.

Ken, my original response to the original response would have been identical. Reminds of the Pink Panther episode where Clouseau is in the lobby of an old-fashioned Swiss hotel with the elderly hotelier and a little growling dog. Clouseau asks "Does your dog bite" and the receptionist responds "no"....

You can always modify parameter values outside of the job. But the "before job" is executed after the job has been invoked but before it does anything, so it has gotten parameter values and at that point in time they can no longer be modified. With older versions of DataStage one would pass in "dumm...

Craig, Sorry for posting again. If you only need to compress this text without duplicates and don't need to retrieve the original string from the compressed one then try this approach: If you need to put a maximum of 200 characters into 50 characters of space you would need to reduce volume to 25% o...

olgc,

this functionality has been removed and won't work at your version 7. Job parameters are no longer modifiable.

You could always call your subroutine in a sequencer and use that result as the input parameter for your job.

Sorry, one more comment - if a large proportion of the string is whitespace then assigning a short binary code to space and 6-bit codes to all characters used then you would be almost there. Depending upon your level of desperation I might be able to write this for you in my off hours.

Craig,

if the new string is limited to displayable characters then there is no room to compress without incurring some sort of loss of data.

Craig, if the ASCII values are always going to be between 32 and 126 that gives you 94 discrete values that you need to represent, in an available 256 "space" - letting you easily compress this into under 100 characters. By not limiting yourself to byte boundaries this will get smaller. 96 is repres...

Craig, I should have added some more text to the post. The Huffmann encoding would create an unbalanced binary tree of 0's and 1's, using the relative frequency of character octets in order of frequency to decide on which branch to place a character. What ends up happening is that frequently used oc...