DSXchange

As Craig has already pointed out, using FTP as a source means that the remote FTP server has to read the file, and FTP is limited to flat files. You can FTP the Excel file (in binary mode, as mentioned earlier) to your DataStage Server and then use ODBC connectivity to read the data from the Excel f...

Well - that really sucks; a memory-leak-by-design!

Phil, I see what you mean and yes, the malloc()d space for the return pointer value may be the culprit - but on the other hand DataStage may be free()ing up that allocated space. Since the reference in a free() call is a void pointer it could be done within DataStage. Although in c++ don't return va...

Sorry, I thought you were reading the remote excel file.

If you replace your ODBC connection with a sequential file stage writing just the column "areaname" without quotes then is there a line in that file which contains quotes which may not be balanced?

You need to FTP the Excel file in binary mode, have you done that?

In writing custom build-ops and operators I've found that the only place with potential for memory leak is when using the builtin pointers to input record contents. These are not allocated on a word boundary as would be the case with malloc() and the like and thus these record buffers are difficult ...

BI-RMA wrote:...I said I can't think of a situation where going from one to two nodes will duplicate data...

The only one I can think of was already mentioned, an ill-placed "entire" partitioning will do that.

*edit* If you look at one of the first entries in your log, you will see the runtime settings of the environment variables. It might be a good idea to compare those values with the ones in the dsenv file; particularly if a line break is in there as Craig hinted at being possible. *edit* What did you...

I disagree with BI-RMA. When going from a 1-node configuration (and design) to a multi-node configuration there are several conditions where the results between the two environments may be different. The outcome depends upon factors such as the hashing algorithms chosen and the settings of "APT...

Is the error occurring on the read or the write?

You've already hit upon the location in your post. The project directory contains a subdirectory call "&COMO&" which contains these (and other) files. They are flat files and can be viewed from UNIX.

Normally those files are deleted, sometimes when jobs abort some files might remain in the scratch disks. DataStage doesn't have any builtin mechanism to delete these files; I usually go and manually delete the files in Scratch once a week or once a month, depending upon the number and the sizes.

...8.7 does not support variable length of records. Only fixed length of record... No, that is incorrect. If you do not specify a length then the record is deemed to be of variable length. If your column specifications include variable length fields (i.e. VarChar() ), then the file is deemed to be ...

Your DataStage timestamp column must also have the Extended attribute of "milliseconds" in order for those values to be passed through or used in DataStage.

Both input links must have identical hash columns and ordering as well as being sorted on the join key. To keep things simple, hash both input links to the join on "Col1" and then sort both on Col1, Col2, Col3, Col5 and Col6. Alternatively run the job with a 1-node configuration and see if...