Unicode with RCP?

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
jweir
Participant
Posts: 134
Joined: Wed Aug 04, 2010 12:31 pm

Unicode with RCP?

Post by jweir »

Hi all,

Is there a way to use Unicode with RCP?

Thanks in advance.
Jweir

--- If strength were all, tiger would not fear scorpion.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Have you tried... specifying "ustring" in the schema file? I'm sure there's more to it than that, but that's the first thing that comes to mind.
-craig

"You can never have too many knives" -- Logan Nine Fingers
jweir
Participant
Posts: 134
Joined: Wed Aug 04, 2010 12:31 pm

Post by jweir »

I do not use schema files to specify the table definition. My job is a very generic job. It takes data from Sybase to Oracle - a one to one move. The Select statement is generated by a routine by gathering the column names from the Sybase system table. This is done so we can load many tables using only a few jobs, rather than a parallel job for every single table.
Jweir

--- If strength were all, tiger would not fear scorpion.
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

RCP isn't going to do any type conversion. If your source is a ustring then it will propagate as a ustring. If it is a string then it will propagate as a string.

If you need to convert between ustring and string, you can use a modify stage with a parameterized specification.

$OSH_PRINT_SCHEMAS will show you what you are working with.

Mike
jweir
Participant
Posts: 134
Joined: Wed Aug 04, 2010 12:31 pm

Post by jweir »

Ok. So if my job is:

Code: Select all

Sybase stage --> Oracle stage
There is no way to apply Unicode unless I add a Modify stage? And that is with RCP?
Jweir

--- If strength were all, tiger would not fear scorpion.
UCDI
Premium Member
Premium Member
Posts: 383
Joined: Mon Mar 21, 2016 2:00 pm

Post by UCDI »

What exactly is the error?

Assuming its string and not uc in the source, string and not uc in the flow, and at most uc on the target side .... but datastage should be able to write string source to uc target column without any extra code (might warn, but no fail?)

Can you explain exactly where it is failing and what you are trying to do in that region of code?
jweir
Participant
Posts: 134
Joined: Wed Aug 04, 2010 12:31 pm

Post by jweir »

It is failing because on of the target columns is VARCHAR(20) and one record is 21 length (it is not 21 in the source). When the data is extracted out of Sybase, it add extra bytes to the data for special characters (it hold telephone data). So the job fails because it cannot insert a record that is 21 length into a 20 length field.

I created a test job and extracted the problem record, and when I applied Unicode, it was not 21 length anymore. It makes it more difficult since I use this "generic" job to load many tables, and pass in the table name and SELECT and INSERT statements from a user-defined IIS routine.
Jweir

--- If strength were all, tiger would not fear scorpion.
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

You could experiment with the NLS Code page at the job and stage levels. See if UTF-16 will help.

Mike
JRodriguez
Premium Member
Premium Member
Posts: 425
Joined: Sat Nov 19, 2005 9:26 am
Location: New York City
Contact:

Post by JRodriguez »

The Oracle client needs to know the NLS of the target database, just set DS job to go by the NLS of the target database, here is how is done:
-Get the NLS of Oracle database
- Add a NLS_LANG to user defined environmental variables in DS Admin Client. Leave the value empty
- Add the $NLS_LANG as a parameter in the job, ser the default value to the value obtained in step 1
Julio Rodriguez
ETL Developer by choice

"Sure we have lots of reasons for being rude - But no excuses
jweir
Participant
Posts: 134
Joined: Wed Aug 04, 2010 12:31 pm

Post by jweir »

UTF16 did not work. And the problem is before the Oracle client. It is when the data is extracted into IIS, the extra chars are added.
Jweir

--- If strength were all, tiger would not fear scorpion.
cdp
Premium Member
Premium Member
Posts: 113
Joined: Tue Dec 15, 2009 9:28 pm
Location: New Zealand

Post by cdp »

jweir wrote:It is failing because on of the target columns is VARCHAR(20) .
VARCHAR(20 byte) or VARCHAR(20 char) ?
https://stackoverflow.com/questions/814 ... -datatypes

What happens if alter your target column to VARCHAR(20 char) ?

If you need to data type convert with RCP on, I can only think of the Modify stage.
http://it.toolbox.com/blogs/infosphere/ ... ster-20368

have fun
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

It's got to be BYTE semantics or it would be working now, I'd wager.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply