Datastage 8.7 compatibility with Hive 0.14 (Hortonworks)

skrai11 · Post by **skrai11** » Tue May 31, 2016 12:17 am

Hi Team,

One of our clients requirement is to implement an EDW project using DataStage 8.7 and Hive 0.14 (Hortonwork).
As per information available in forums and web, DataStage 8X does not support Hadoop infra so hive as well.

However, we are able to connect with Hive 0.14 with DataStage using Progress DataDirect 7.1 Apache Hive Wire Protocol driver.

Could you please help understand if there would be any problem in going with DS8.7 and Hive 0.14.

Thanks in advance!!!

Regards,
Santosh

ray.wurlod · Post by **ray.wurlod** » Wed Jun 01, 2016 4:34 pm

Welcome aboard.

The only "problems" I might envisage are speed (lack of) and limited functionality of the ODBC driver. You can do anything the ODBC driver supports.

skrai11 · Post by **skrai11** » Wed Jun 01, 2016 11:56 pm

Thanks Ray, hope we would able to manage.

Teej · Post by **Teej** » Thu Jun 02, 2016 11:10 am

Why 8.7? Why not upgrade to 11.5.0.1, which have a lot of additions for the hadoop environment, including the ability to run PX jobs using Hadoop's YARN?

There is also a number of additional support for Hive in 11.5:

http://www.ibm.com/support/knowledgecen ... ables.html

All of this goodness really started in 9.1 with BDFS (Big Data File Stage), but we really prefers you use the File Connector stage (11.3.1+), as that is actively worked on.

So why not 11.5.0.1?

-T.J.

qt_ky · Post by **qt_ky** » Thu Jun 02, 2016 1:46 pm

Version 8.7 was released in 2011 with some Big Data support included. However, it's over 4 years old now and it has a published End of Support date of 30-Apr-2017 (less than a year away).

http://www-01.ibm.com/software/support/ ... dex_i.html

Teej · Post by **Teej** » Mon Jun 06, 2016 12:31 pm

Actually no. BDFS was in Beta form for 8.7, and any customers using 8.7 with BDFS must upgrade to 9.1 in order to get full IBM support.

skrai11 · Post by **skrai11** » Tue Jun 07, 2016 12:34 am

Thanks T.J, Its just that client does not want to migrate to latest version but wanted to implement Big data.

Teej · Post by **Teej** » Tue Jun 07, 2016 11:12 am

Okay, do remind them that if they wish to have IBM provide an easy-to-use option for accessing Big Data, they should consider upgrading anyway. Plenty of cool stuff added for 9.1, 11.3 and 11.5, after all!