DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
peterveenis
Participant



Joined: 16 Dec 2010
Posts: 1

Points: 32

Post Posted: Tue Dec 09, 2014 5:07 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Windows
Hi all,

I'm having troubles with my Filter Stage. Here is the situation

Sequence:
StartLoop -----> Job ---->
^ |
| |
| V
--------------------EndLoop

Job:
Sequential File ---> Filter --->

The file contains more records than the target can handle at a time. That's why I need to split up the sequential file in different pieces.

In an early job of the sequence I make up the number of records of the sequential file and divide it by the number what the target can handle at a time (10 in this example).
I loop the job the number of times it needs and within the filter I do this:
(ROWID > ((#loopcounter#*10)-10)) and (ROWID < ((#loopcounter#*10)+1))

The #loopcounter# is StartLoop.$counter.

The error messages I get are:
- Filter: Error occurred during initializeFromArgs().
- Filter: Parse error: Expected field or literal after comparison operator,got: "("
Parse error: Expected boolean operator (and, or) got: "("
Parse error: Expected 'is' statement ('is true', 'is false') or boolean operator (and, or) got: "-10".

Would anyone mind helping me? Thanks in advance! Smile

Peter Veenis
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54541
Location: Sydney, Australia
Points: 295742

Post Posted: Tue Dec 09, 2014 7:29 pm Reply with quote    Back to top    

Welcome aboard.

I'm surprised to hear of any target that can only handle a limited number of rows.

The filter expression you use in a Filter stage is actually quite limited (check the Parallel Job Developer's Guide for details). In the meantime replace your Filter stage with a Transformer stage, which can handle far more complex expressions.

ROWID is not a native part of DataStage, I assume it's in your data or generated by the Sequential File stage.

_________________
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Rate this response:  
Not yet rated
priyadarshikunal



Group memberships:
Premium Members

Joined: 01 Mar 2007
Posts: 1735
Location: Troy, MI
Points: 9319

Post Posted: Tue Dec 23, 2014 8:15 am Reply with quote    Back to top    

I have seen it while interacting with SAP systems with limited resources.

Check if the split command works there in command prompt. It does work for me with MKS toolkit installed. here -l is the option to define the number of records. This you can use after the file is created. but for 11.3 you should check as they have tried to reduce the dependency on mks toolkit.

Since you are on 11.3, did you check the properties of sequential file stage which should be capable to generating multiple files?

_________________
Priyadarshi Kunal

Genius may have its limitations, but stupidity is not thus handicapped. Wink
Rate this response:  
Not yet rated
rschirm

Premium Poster



Group memberships:
Premium Members, Inner Circle

Joined: 13 Dec 2002
Posts: 27

Points: 203

Post Posted: Tue Dec 23, 2014 11:24 pm Reply with quote    Back to top    

The () characters are what is throwing the error. Something a complex as you are doing belong using a transformer or a build op stage.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours