DSXchange

Continued here by the way.

Well yah... duh.

The argument centered on the fact that we have already made sure the input-string is well-formed and valid. That certainly was never part of my "argument" as I was making a generic statement. :wink: I'm also about to split this into your own thread as the original conversation came off th...

I'd suggest taking DataStage out of the picture for now. How would you do this in general, what approach would you take? Once you figure that out then it would be easy to implement those steps in the tool, meaning it executes and monitors them. Unless you're thinking you want a "pure DataStage&...

Sure... but you don't "read row -1". What you can do it hold on to the previous row and then use data from it while processing the current row. End result is the same, just wanted to make that clarification. Stage variables in a Transformer can do that for you. New versions have more optio...

Apparently. :wink: I'm just going to have to guess, not having any way to test this but what if you use the full path to the package file? Does that make any difference? I agree it shouldn't but I'd be curious if it changes the error at all. Worst case contact your official support provider, if you ...

Still - and this is just my opinion - with something as strongly typed as a DataStage PX job is under the covers, I consider it a Best Practice to not let it do any implicit conversions. Especially with a flat file source. Heck, Informatica is about half-way between a Server & PX job from a type...

I agree with Prasanna's solution. For whatever that is worth.

Can you create a work table in the source to hold the PK values you've extracted? Or are they willing to add a timestamp to the source? We're running out of options here.

This "unique key", is it a surrogate or any kind of field that increments in such a manner you can use it for your high water mark? Without that or a timestamp all you can do is pull everything and compare.

Couldn't pull it out of my hat earlier today, thanks for that.

Makes sense. I removed the rather large list of column names as all it was doing was pushing the edge of the post way too far to the right. I don't recalling there being anything that looked improper with it, perhaps just column name issues?

And yet all it complained about was the array size and then logged a fatal error. So the array size isn't the source of the fatality, something else is going on here? Perhaps something not included in the post?

There's an option in the stage to add an ouput link for the filename, one of the properties from what I recall. Don't have any documentation here so can't quote chapter and verse I'm afraid. You may also need to add a specific $APT environment variable to help it but try it first without it and see ...

crystal_pup wrote:Decrease the array size value to somewhere like 150 or 200.It will solve the problem.

150 or 200 won't... do the math and you'll see that the maximum array size without increasing the buffer would be 149.0951276102088 which is why I said less than 150.