DSXchange

It is a very good point, Mike :
Using Stage variables, you have to be very very carefull with the last line (or the first depending on the case)...

Usually, it try to avoid them for this kind of things...

kcbland, it is not a mod(), but a div... I you truncate the resulting number, you will have the right group (truncate and not round)... 1 / 5 = 0.20 int(0.20) = 0 2 / 5 = 0.40 int(0.40) = 0 3 / 5 = 0.60 int(0.40) = 0 4 / 5 = 0.80 int(0.40) = 0 5 / 5 = 1.00 int(0.40) = 1 6 / 5 = 1.20 int(0.40) = 1 .....

In your example, it is a sum and not an avg... Anyway, this might do the trick : In a transformer before the aggregator add a column to your stream containing the value Int (@OUTROWNUM / 5) Like this, your data now looks like : a,5,0 a,3,0 r,6,0 y,2,0 i,5,0 q,3,1 o,4,1 t,3,1 If you then sum (or avg)...

I think it is more like :

Have you checked for trailing blank lines ?

Moreover, when you have a lot of activity with the log file (like one warning per line) the performance greatly suffer... It also remind me of this time, when I get used to see in director : Finished (see log) and missed a really important warning... From this day, all my jobs are finishing without ...

You don't have data cleaning facility in Datastage, for this, take a look at QualityStage... But, in Datastage you can do "Data Quality", ie. analysis of your data in order to produce reports (unicity, count, etc. ...) and apply business rules (in transformer, based on routines or using re...

I wasn't talking about designer manipulation but about "pattern extraction". In order to reverse engineer the XML format, you need some example. From them, you can extract usefull information like : how are the stage defined, how are the link defined and so on... Once you found out the str...

Another possibility is to reverse engineer the xml file generated.
Then, based on some template job (for example) you can create an xml file that suit you needs and import it using the manager...
This way, you can create job without using designer, but it is really tricky.

150 row/sec is quite slow...
Have you checked the director to see if there are some warnings (too many columns or things like that...)
I had performance problem due to warning issued at each line, once the warning was removed, the performance took off...

If you would like to receive these notifications, but do not have an RSS aggregator, you can use <a href="http://www.ceruleanstudios.com/products/">Trillian Pro</a> ($25 download), <a href="http://www.feedreader.com/">FeedReader</a> (Free download) or any other RSS 0.9x compatib...

ketfos wrote: ls -t > temp
tail -1 temp

You can have the last line using pipes instead of using a temporary file : Alexandre.