Using Schema Files with decimals for Sequential Files

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
iShore ETL
Premium Member
Premium Member
Posts: 2
Joined: Mon Oct 07, 2013 3:33 pm
Contact:

Using Schema Files with decimals for Sequential Files

Post by iShore ETL »

Greetings,
I have a series of CSV's I import into a database via Datastage. I am attempting to do this using RCP and schema files.

I generate the schema files from the CSVs using an accompanied master table list that comes with the CSVs.

I am down to one problem. When I find that a numeral is the last column in a particular table, it is the last entry in a schema file. My problem is null handling. The CSV is comma-delimited, double quoted for strings, and no data for null.

The master list identifies some of these number columns as number(), which is indicative of an oracle description of the output. To that end, I am trying this:

<column>:nullable decimal[38,9] { default=0, text };
in this example, the scale and precision are defaulted, to 38,9....unless specified elsewhere, such as decimal[10,2].

A null entry results in this error:
When validating import/export function: APT_GFIX_Decimal::validateParameters: the decimal "text" format is variable length, and no external length is specified;
you should possibly specify an appropriate "width" property; external format: {text, padchar=32, nofix_zero, precision=38, scale=9, round=trunc_zero, ascii}. [decimal/impexp.C:939]

so I tried:
<column>:nullable decimal[38,9] { default=0, text, width=47 };
in this example, the scale and precision are defaulted, to 38,9. The width is the sum of the two values (38 + 9 = 47...unless specified elsewhere, such as decimal[10,2].

and I got:
ODBC_Connector_3,0: Input buffer overrun at field "<column>", at offset: ### [impexp/group_comp.C:6006]

Lastly, I tried exactly what it said, and did this:
<column>:nullable decimal[38,9] { default=0, text, padchar=32, nofix_zero, precision=, scale=, round=trunc_zero, ascii, width=47 };
in this example, the scale and precision are defaulted, to 38,9. The width is the sum of the two values (38 + 9 = 47...unless specified elsewhere, such as decimal[10,2].

For this third time, I received this error: Input buffer overrun at field "<column>", at offset: ### [impexp/group_comp.C:6006]


Has anyone ran into this? this only happens if decimal is the last column in the table.

my record settings are: {intact, final_delim=none, record_delim='\n', charset='UTF8', delim=','}


Thank you very much.
ray.wurlod
Participant
Posts: 54595
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Try width = 40. That is, 38 + sign + decimal placeholder.

The 9 decimal digits are included in the total of 38 significant digits.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
iShore ETL
Premium Member
Premium Member
Posts: 2
Joined: Mon Oct 07, 2013 3:33 pm
Contact:

Post by iShore ETL »

Thank you so much...
I tried your recommendation, but the response still gave me a buffer overrun error.

Any other ideas?
Thanks again.
Post Reply