connection issues between hive and datastage 11.3.1

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
nimcurry
Participant
Posts: 7
Joined: Mon Nov 02, 2015 10:42 am
Location: toronto

connection issues between hive and datastage 11.3.1

Post by nimcurry »

I am trying to connect to a hive database on a edge node (residing on another linux server secured with Keberos protocol). Following is my dsn connection parameters in .odbc.ini file:

Code: Select all

[EDL_Hive]
Driver=/opt/IBM/InformationServer11-3/Server/branded_odbc/lib/VMhive00.so
Description=DataDirect Apache Hive Wire Protocol
ArraySize=16384
AuthenticationMethod=1
Database=ISL_DB
DefaultLongDataBuffLen=1024
EnableDescribeParam=0
GSSClient=native
HostName=10.1.21.45
LoginTimeout=30
LogonID=ndeshpande
MaxVarcharSize=8192
Password=
PortNumber=10000
ProxyUser=
RemoveColumnQualifiers=0
ServicePrincipalName=
StringDescribeType=12
TransactionMode=0
UseNativeCatalogFunctions=0
UseCurrentSchema=0
WireProtocolVersion=thrift
and following is the error i got:

SQLSTATE = 08001
NATIVE ERROR = 0
MSG = [IBM(DataDirect OEM)][ODBC Apache Hive Wire Protocol driver]Client unable to establish connection

I am just wondering if it is even possible at all to connect to a hive database using a user name/password?
TNZL_BI
Premium Member
Premium Member
Posts: 24
Joined: Mon Aug 20, 2012 5:15 am
Location: NZ

Post by TNZL_BI »

Yes it is and we have just managed a connection to the Hive database using the ODBC stage and Hive Connector
DelSenato
Premium Member
Premium Member
Posts: 1
Joined: Fri Mar 31, 2006 5:06 am
Location: Glasgow

Post by DelSenato »

TNZL_BI wrote:Yes it is and we have just managed a connection to the Hive database using the ODBC stage and Hive Connector
Hi, I realise that this is 11.3.1 specific but my question relates to v9.1.2. so hope that this is ok posting my question here.

Were you using version 7.1.6 of the drivers?

I'm having difficulty connecting to Hive using odbc in an AIX environment.

I've updated uvodbc.config and .odbc.ini

We are not using Kerberos so GSSClient is native.
Server and port information is good as is the HTTPPath as we can connect using odbc on windows.

My concern is around the .pem file as I created that manually on the server by doing the following

1.Generated a .cer certificate.
2.Created the jks from the .cer.
3.Created a .pk12 from the .jks
4.Created a .pem from the pk12

Frankly, configuring certificates is new to me and I'm not getting much help at all from the various internal teams who I would expect to direct me in this.

My ODBC entry.

[HiveSSL]
Driver=/opt/IBM/InfoSphere/InformationServer/Server/branded_odbc/lib/VMhive00.so
Description=DataDirect Apache Hive Wire Protocol
ArraySize=16384
AuthenticationMethod=0
Database=default
DefaultLongDataBuffLen=1024
EnableDescribeParam=0
GSSClient=native
HostName=***********
LoginTimeout=30
EncryptionMethod=1
LogonID=
MaxVarcharSize=8192
Password=
PortNumber=8444
ProxyUser=
RemoveColumnQualifiers=0
ServicePrincipalName=
StringDescribeType=12
TransactionMode=0
UseNativeCatalogFunctions=0
UseCurrentSchema=0
TransportMode=1
HTTPPath=gateway/default/hive
WireProtocolVersion=2
TrustStore=/tmp/dshive.pem
TrustStorePassword=dshive
ValidateServerCertificate=0

I'm getting


SQLSTATE = 08001
NATIVE ERROR = 0
MSG = [IBM(DataDirect OEM)][ODBC Apache Hive Wire Protocol driver]Client unable to establish connection.


message currently but I'm now trying various variations on the authentication and validation and getting nowhere.

When connecting from windows successfully, that is done using a preprod Root certificate (.cer).

I've tried copying this upto AIX and then following the steps outlined above i.e.

2.Created the jks from the .cer.
3.Created a .pk12 from the .jks
4.Created a .pem from the pk12

to no avail.

I've now engaged the unix admins to ask if they can install version 7.1.6 of the drivers as I only realised today that those were available but not installed.

Note, if I change the WireProtocolVersion=thrift my error message is


SQLSTATE = S1000
NATIVE ERROR = 0
MSG = [IBM(DataDirect OEM)][ODBC Apache Hive Wire Protocol driver]General error.Driver was unable to connect to the Server. Please check your Hive Server is running and the Host/Port are configured correctly.


I can telnet to the server on port 8444 successfully.
I cannot set trace on unfortunately as /opt/IBM/InfoSphere/InformationServer/Server/branded_odbc/lib/odbctrac.so is not present.

Any help or pointers from anyone pointing out stupid errors gratefully accepted!
Post Reply