DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
This topic has been marked "Resolved."
Author Message
vskr72



Group memberships:
Premium Members

Joined: 28 Apr 2004
Posts: 128

Points: 1411

Post Posted: Wed Dec 21, 2011 5:16 pm Reply with quote    Back to top    

DataStage® Release: 8x
Job Type: Parallel
OS: Unix
I have a job that loads data from a SEQ file to Ora table. While loading I need to check the following conditions for Date and Number columns:

1. If its a valid date, then insert the value else set it to NULL
2. If its a valid number, then insert the value else set it to NULL.

Implementing them is not an issue. I have used 'IF..THEN..ELSE' statements. Works fine. I am using a Transformer for this. But the only issue is there are too many Date or Number columns. Writing these statements for each of them is very painful and it takes a lot of time. There are many jobs also. Is there any other way to handle this outside Transformer or using any Stage variables(???). Thank you for your inputs.
shamshad



Group memberships:
Premium Members

Joined: 25 Aug 2004
Posts: 147
Location: Detroit,MI
Points: 1383

Post Posted: Wed Dec 21, 2011 6:56 pm Reply with quote    Back to top    

Can there be 2 routines, one for DATE CHECK and one for NUMBER CHECK. Just pass the column value to the routine and get either a value or NULL as OUTPUT. Then insert into table.

_________________
Datawarehouse Consultant
Rate this response:  
Not yet rated
vskr72



Group memberships:
Premium Members

Joined: 28 Apr 2004
Posts: 128

Points: 1411

Post Posted: Wed Dec 21, 2011 8:07 pm Reply with quote    Back to top    

But, I still need to call thsi routine in the derivtion in the Tfm stage right? Hope I am not missign anything here.
Rate this response:  
Not yet rated
pandeesh



Group memberships:
Premium Members, Heartland Usergroup

Joined: 24 Oct 2010
Posts: 1399
Location: CHENNAI, TAMIL NADU
Points: 9855

Post Posted: Wed Dec 21, 2011 8:11 pm Reply with quote    Back to top    

Why not IsValid()?

_________________
pandeeswaran
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 43085
Location: Denver, CO
Points: 222463

Post Posted: Wed Dec 21, 2011 8:50 pm Reply with quote    Back to top    

I would assume that is what they are doing. And I assume the goal here would be to reduce all that if-then-else activity down to something more manageable. As noted, a parallel routine could cut that ...

_________________
-craig

"You can never have too many knives" -- Logan Nine Fingers
Rate this response:  
Not yet rated
jwiles



Group memberships:
Premium Members

Joined: 14 Nov 2004
Posts: 1274

Points: 10406

Post Posted: Wed Dec 21, 2011 9:21 pm Reply with quote    Back to top    

Writing this logic within a transformer is going to take some measure of repetition, whether that is the if-then-else logic you are currently using or the parallel routines mentioned in the thread. This is the nature of the transformer interface and logic...no dynamic access to the columns passing in or out of the stage.

The parallel routines will likely be your best solution unless you want to tackle something more dynamic such as a Java Transformer or custom operator (C++). They will be somewhat more difficult to implement than a single-function parallel routine.

Regards,

_________________
- james wiles


All generalizations are false, including this one - Mark Twain.
Rate this response:  
Not yet rated
vskr72



Group memberships:
Premium Members

Joined: 28 Apr 2004
Posts: 128

Points: 1411

Post Posted: Wed Dec 21, 2011 11:51 pm Reply with quote    Back to top    

Thank you for all your inputs. So, I guess I will have to manage this cumbersome process. I will at least try to develop a routine to reduce the lengthy if-then-else statements
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours