How can we encrypt data?

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Post Reply
silvachjuan
Participant
Posts: 4
Joined: Tue Aug 28, 2018 11:50 am

How can we encrypt data?

Post by silvachjuan »

Hello, we need some help, because we have a new project in datastage and we have to ingest in Hadoop (Hortonworks), but we need encrypt our clients data using (aes 128 to encrypt and dencrypt data), is it possible in datastage?

How can we do it?

Regards
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Welcome.

Just to clarify the scope of this requirement, are you needing to encrypt the data all the way through the process - i.e. in flight and at rest? Or just at rest?
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

The encrypt.sh tool in both ASBNode and ASBServer will encrypt using AES128 (by default), though it is quite slow. It's intended for manual encryption of passwords and the like.

You would be faster writing a custom routine that calls one of the standard encryption utilities out there in the world.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
silvachjuan
Participant
Posts: 4
Joined: Tue Aug 28, 2018 11:50 am

How can we encrypt data?

Post by silvachjuan »

We need extract data from data bases and text files, encrypt only columms about our clients personal information, after that ingest to hadoop. We need use (aes 128 to encrypt and dencrypt data). We can't put data without encrypt in hadoop

thank you
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Looks like you need to encrypt as the data leave the data source, and decrypt after the data leave Hadoop. Is that correct?

If that's the case, you'll not be doing the encryption in DataStage; it will need to be done earlier than that.

Determine whether your database has encryption capabilities available, and use those. Otherwise create scripts/procedures that can encrypt the results of a SELECT query and use those.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
silvachjuan
Participant
Posts: 4
Joined: Tue Aug 28, 2018 11:50 am

Post by silvachjuan »

Hello, thanks

We need encrypt data with datastage and after that ingest to hadoop
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Then you have a security gap between the source and DataStage. Please think this through again.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply