DSXchange

I'm reading in the whole record as one string and breaking it up into many fields later on. It is entirely possible that one field(record) will be longer than 100000 bytes. I've found an evironment variable that is supposed to allow for a bigger limit APT_MAX_DELIMITED_READ_SIZE I'll let you know ho...

Is anyone aware of a setting that will bump up the 100000 bytes to a higher amount?

I need to import records that may have several 32k fields, possibly going over the 100000 byte limit.

Yikes

Ray, What I had in mind was to write a message from a transformer, outputing some control date values to the log - and maintaining my parallel processing. As such, I didn't want to use a Basic Routine. By the time the after job subroutine happens, the data is lost. I might have used UtilityMessageTo...

Is there a way of replicating the UtilityMessageToLog functionality found in the Server edition?

Seems like a common thing to want to do and yet I can't find any facility to do this

Rob W.

Any other thoughts out there?

Rob W

Mahesh,

We came accross this error some time ago.

See this thread. viewtopic.php?t=96485&highlight=is+writable

Rob W

Thanks for the reply Ray.

I think I'll leave this one open for awhile, see if anyone else has some input.

Rob W

Given a multiple node installation, will a job that reads a dataset, performs some manipulation to columns and then writes to another dataset maintain the row order, assuming there is no sort being applied on the partitioning tab (say, if I choose Auto)? I would think not, however, I have seen nothi...

Ultramundane,

That's done the trick!

DataSet is just a little larger than the orginal sequential file, throughput is up by an order of magnitude and more.

Thanks!

Rob W

DSXChangers, I'm reading in a sequential file, 14,500 rows, about 3 meg. Defined as a single column varchar(75,000) (some of the record types could be this wide). When reading this file into a dataset, this thing bloats to almost 2 GB. Needless to say this bloated dataset is the opposite of fast. Is...

Thanks Ray,

I suspected as much but thought there might some people out there smarter than me that had a better solution.

Rob

List the available dataset available in the required pattern either by ExecuteCommand activity. Pass it as parameter to the job one after other. You can also store it in a file and read and pass it as parameter in a loop by using Loop activity as well. Hi Kumar, I actually thought of creating a mul...

Well it wouldn't be a Data Set (which uses internal, binary, forms for numeric data), but it might be a File Set. You can craft a File Set control file containing the pathnames of the actual files (which require identical record structure, and probably no "first line is column names") and...

We parameterize almost all the values in our MLoad stages. Check the values in your job parameters, there will be a mistake (typo) somewhere in there. Actually, if you right click on the job in Director and pick Details, it will tell you all the values for your job parameters in the last run. Check ...

Hey DSXChangers, I need to read a bunch of files and really like the idea of the 'Read Method = File Pattern' in the sequential stage, however, I also like the efficiency of DataSets. Is there a way (on Windows!) to emulate the file pattern read when reading DataSets? I read through the manual, did ...