Clear a multiple input hash file

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
MounikaP
Premium Member
Premium Member
Posts: 6
Joined: Mon Nov 06, 2017 3:48 am

Clear a multiple input hash file

Post by MounikaP »

I have a job having the below sequence of stages.

DB2 Connector Stage -> Transformer (10 output links basis various constraints) -> Hash File -> Sequential File

As I see, there is "Clear File before writing" property set for only 3 of the input links. Till now, the job is running as is.

But, we are migrating all our jobs from DS 7.5 to 11.3. When working on data validation, post migration, we are seeing issues in the output sequential file generated. For multiple runs of the same job with same input parameters, 10 outputs of transformer is same, but, the sequential file output is different every time.

Does it do anything with the Clear Hash file property to be set for other 7 input links?

Please help me.
Thanks and Regards,
Mounika P
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Do the hashed files all have unique names or do they all specify the exact same name? Just because you have a single hashed file stage doesn't necessarily mean you are writing to a single hashed file, it could up any number of them from one to ten. Please clarify.

Not knowing the requirements you are trying to fulfill, I'm not sure why you wouldn't have the same property set in all of them. However, it's simple enough to try setting them and see if that corrects your issue.
-craig

"You can never have too many knives" -- Logan Nine Fingers
MounikaP
Premium Member
Premium Member
Posts: 6
Joined: Mon Nov 06, 2017 3:48 am

Post by MounikaP »

Thanks for the reply!!

Actually, the hash file name is same across all input links. Not sure why developers prior to me has done it this way.

As per your suggestion, I have tested the job by changing the property to clear across all input links.

But, the thing which i am worried now is, once i have set the property to clear in all input links, the output which i got previously is not the same as output which i am getting now.

But, to confirm, the output which i am getting now is the correct one. I should somehow update this to my business.
Thanks and Regards,
Mounika P
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Well, it seems in this case that different is good. I'm going to mark your post as "Resolved" since it seems to be and you haven't done so yet.

Unfortunately, we see issues like this post all the time when people upgrade, especially long leaps ahead like you have done. Things that used to "work fine" no longer do and the vast majority of the time that translates to it wasn't really working fine. A bug was being accidentally exploited or a loophole was helping and in the latest release these had been squashed / closed / fixed. Sometimes they can be very subtle, other times more obvious when say the job aborts where it didn't before. Keep an eye out for other issues, hopefully you are planning a plethora of regression testing! :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
MounikaP
Premium Member
Premium Member
Posts: 6
Joined: Mon Nov 06, 2017 3:48 am

Post by MounikaP »

Thanks Chulett... I forgot... I shall get used to the same.
Thanks and Regards,
Mounika P
Post Reply