DSXchange

I disagree with Ray - the previous post isn't "almost impossible" to read - it is unreadable.

Sumeet, please do a reset in the director and see if you get a log entry with the title "from previous run" and if that contains additional information.

The error message is very clear, you are using an incorrect schema for your input or output. Did you import the schema file from a text file?

Can you put in a condition into your transform stage output derivation for this value which tests for a null or notfound?

Could you please post your error to this thread?

Har, 1 is a pretty inefficient transaction size. Do you know why your job was hanging when you used 0? Did you get a job error or see something on the database size? I trust you stopped your job from the Director and not from UNIX; otherwise you might still have sessions connected to the database th...

Ultramundane - thanks for doing the research work that the original poster didn't :D You were more energetic than I, since the only real learning out of a question such that that can come by actually doing the exercise. Thanks!

Har, you stated that your job fails, and kwwilliams suggested you reduced your transaction size; then you stated that it is running with a transaction size of 1 but not with a transaction size of 0 (which is no commits until the last row in the job has been process). Did kwwilliams advice help you? ...

lstsaur, the way the question was phrased didn't really let anyone give you an answer. SAS is complex. It is a big application residing on some SQL database; just like SAP and PeopleSoft. They discourage direct access to the tables (similar to the DataStage repository ) DataStage is about accessing ...

The error message you get when running the job will be much clearer than something during validation. Try running it (it's only writing to a sequential so you won't be messing up any data) and see what you get.

Deleting and creating small hashed file is much faster than deleting a big one; this is for speed test purposes to see why you job is acting oddly. Always use buffering unless you are forced to turn it off. It is going to be at least an order of magnitude faster. But your performance stats will show...

Try runnign the job once with a limit of 1 row (this will force DataStage to delete and then re-create an empty hashed file). Then run the full data off this empty hashed file and the speed should be roughly linear across the load if your source is a sequential file. Also, are you using buffering to...

How long does your total load take? The row counts shown by the monitor are snapshots and really use too small samples to be reliable. You could put a sequential link out of your transform with a constraint of "MOD(@OUTROWNUM,10000)=0" and output the current time to see if the actual job has linear ...

Techtips might still be free, but doesn't help you much with regards to answering DataStage questions. Some of that forum's answers either originate in this forum or point here. IBM/Ascential's developernet forums remains completely free as well, as is the new Yahoo! one mentioned elsewhere. DSXChan...

What is your source? If you are reading from a database and doing a complex or time-consuming SELECT or ORDER BY then this would be the type of performance I'd expect to see.

A couple of things: ... 2. The disks are apparently set up using striping. One then would not expect read/write IO wait. Striping won't magically remove waits on I/O, just perhaps help spread the load across spindles and controllers. Most EMCs are configured with parity disks as well - is that the c...