internal storage

mcs_suman · Post by **mcs_suman** » Wed Feb 27, 2008 11:19 pm

Does data stage has internal storage place to store data.

Maveric · Post by **Maveric** » Wed Feb 27, 2008 11:28 pm

On a temporary basis, yes. Datasets. That will be stored in DataStage internal format, in the resource disc. They are the most efficient way in terms of space, read/write efficiency to store data. You also have hashed files, file sets and sequential files.

ray.wurlod · Post by **ray.wurlod** » Wed Feb 27, 2008 11:36 pm

In parallel jobs you don't really have hashed files readily available.

mcs_suman · Post by **mcs_suman** » Thu Feb 28, 2008 12:35 am

could you explain me about data stage internal format.

ray.wurlod · Post by **ray.wurlod** » Thu Feb 28, 2008 12:42 am

As much as possible, in parallel jobs, we strive not to use internal storage (that is, not to touch data down to disk), as this slows things down. The structure of data in memory is given by the record schema that you can see in the generated OSH and the score. The same structure is used in Data Sets. Numeric values are stored in binary formats, while variable-length strings have a prefix showing how many characters there are in the string. If NLS is enabled internal storage is Unicode, if NLS is not enabled internal storage is "eight-bit ASCII".