DSXchange

Krazykoolrohit wrote:It just blows up when run with 12 other jobs parallely through sequence.

All trying to write to the same sequential file? Tch! Tch!

It will be slow, but it will get there. Just make sure you're not starting a 27 hour task on a daily basis. Or a 90 minute task on an hourly basis, if you get my meaning.

Which of the methods did you try? The second (create an extra column) will fail with the message you got, because there is no actual delimiter character between the last column and the \r character. What is the data type of the last field in your data record? Did you try setting the final delimiter ...

How are the data partitioned on both inputs to the Lookup stage?

Set APT_SUMP_SCORE to True. In the score (an entry in the job log) composite operators are identifed as Composite Operator.

I think it was in 1897 that the Indiana state legislature legislated that the value of PI should be 3, to make life easier for geometry teachers. Maybe this has finally reached Virginia? I find it curious that you are only getting four digits after the decimal place even though you specified a scale...

That's entirely an issue for those running that site. If they want to retain control, that's their choice.

You don't need execute permission for dsenv; it's read-only and serves mainly to set some environment variables in your private memory area, so there are no permissions issues with that.

Sometimes it is enough to re-boot the client (the "Microsoft solution").

Dump the score and inspect it. Identify the composite operator, and the operators it contains. This will help to narrow the focus of your diagnostic activities. Note that a composite operator is not the same thing as a combined operator. For example, the Lookup stage generates a composite operator; ...

Data type of the incoming column is date (if I read the OP correctly). Can you apply a Field() function to a date column? That is, is the conversion to string implicit? You can find the answer to that question by trying it, or by consulting the Parallel Job Developer's Guide

... except for the ones that use LIBPATH

But the initial problem is with PATH, not with any of these.

The best way is to generate the physical model and, perhaps, the DDL for the database, and hand that to the DBA for review. The DBA may have input concerning, for example, the best placement on disk for table space, index space, and so on. Might even observe efficiencies that can be obtained (such a...

Why did you do the chmod +x command?

What's the key of the reference table for the lookup? What partitioning algorithm are you using for the reference input link? I suspect you may be missing some calid lookups if you're not hash partitioning on calid on both inputs to the Lookup stage (or Entire partitioning on calid on the reference ...

... or set the Final Delimiter property to \R

... or create a dummy Char(1) column and set it's "drop on import" property to True

There's always more than one way to do something in DataStage.