ETL of Twitter,LinkedIn,GoogleAnalytics data using Datastage

mydsworld · Post by **mydsworld** » Thu Jun 09, 2016 4:56 am

Please let me know how can I access the following data source for ETL work, using Datastage.

1. Twitter
2. Google Analytics
3. LinkedIn

Thanks.

eostic · Post by **eostic** » Thu Jun 09, 2016 5:50 am

Absolutely......but ask first, "how" do you get to all of those data sources? [ie...what technology is supported by those vendors or outlets].... For example, Twitter supports a JSON based REST web service. DataStage can issue such calls....so there's your answer. The others are probably the same, but maybe they have other mechanisms (downloads, other APIs, etc.).

Ernie

chulett · Post by **chulett** » Thu Jun 09, 2016 6:36 am

Right... don't worry about the "from DataStage" part quite yet. Find out what they support, what external access methods / API they have and go from there.

Teej · Post by **Teej** » Thu Jun 09, 2016 12:40 pm

With knowledge of C/C++, and the ability to build a custom stage, you can access just about anything. The real question is whether or not this will provide you with information that can be useful in a structured data format, or if a different solution is ideal due to unstructured data.

ray.wurlod · Post by **ray.wurlod** » Fri Jun 10, 2016 12:38 am

With knowledge of Java, and the ability to work with the Java stages, you can access just about anything. The real question is whether or not this will provide you with information that can be useful in a structured data format, or if a different solution is ideal due to unstructured data.

Note also that some environments have security requirements.