DSXchange

@James, Thats what I am saying, if I use Auto partitioning method in Join, Merge etc stages then datastage by itself will use Hash partition. Thats why we dont have to explicitly use Hash partitioning method. The same is documented too, pasting here a part of the document The data sets input to the ...

@Sura, Thats what I said earlier, and I am still saying the same :( It is not mandatory to use explicit partitioning methods and there's no harm too in doing that. it all depends upon the developer and the requirement. ray.wurlod wrote: It's mandatory for quite a few stages, including Join For above...

This topic has been discussed so much times and as xinhuang66 said it is still unclear to me too. I'll take my project for an example, we have developed more than 1000 jobs here which contains atleast joins or aggregators etc. For these stages, we have never used explicit partitioning methods. We ha...

@Craig,
You also posted two times..

Actually Ray, this is a problem with the website.
While posting comments, if the net speed is slow then the comment is posted twice(or thrice)

@Craig, I also found the same but I think the last logic is for conversion only, not for addition. found the list of conversions logic from the same document but there was nothing for addition. @mobashshar I will try that, but what if any of the columns has null value. Then may be total amount value...

We faced the same issue in our project.
Either you can convert the incoming columns into Bigint and then apply the MIN function.
Or you can use a typical If..then.. Else Statement and check using greater/less then function.

Yeah Craig, My job completes in around 1.5 hrs, and I need to make it finish with in 30 min.
Can you give me the logic that can be used in Modify stage ? I have checked but didnt get the solution.
And my target is a DB2 table.

No Craig, the job is running fine.
Just wanted to know if there's anything which I may be missing.
Additionally I wanted tune the job for performance.
(As I said my source will have 150+ million records)
Also is addition of amount columns possible in transformer only ?
Or Modify stage can also do it ?

Thanks Ray,
But I cant use a rank function since my source is a Seq File.
Is there any way where I can eliminate a transformer ?
Or is addition of columns possible in some other stage ??

Hi All, I am preparing a job for a new requirement in our Project. I have to find out the Top 20 Customers for each Branch. Now my source is a Seq File(~150 million rows) and it contains 4 Amount columns, Customer No and Branch No. I have to add these Amount columns then find out the customers with ...

Congrats Ray !!!
When's the party ???

@RK72 If the job is still getting aborted then I guess you have only two options. We also faced this issue in our Project, so we 1) Increased the transaction log space. 2) Reduced the commit size. using ODBC enterprice stage Why not DB2 connector stage ?? :? also tried with Insert array size=500000 ...

Then I guess you have lower the commit size(like 500000)
By the way how are you deleting the records ??

Yup, you are right.
Check the below post as well:

viewtopic.php?t=145383