Picking metadata at run time

mfshah · Post by **mfshah** » Mon Feb 15, 2010 11:11 am

I have to load data from oracle(source) to teradata(target). The jobs should be dynamic/flexible and pick the metadata information at run time i.e. If a change has made to the data model it shouldn't effect the DataStage job design. Say Col A was previously varchar 5 and now the lenght has increased to varchar10 Or an additional column has been added to the oracle table and we would like to pass it along to the target table. Is there any way to pick the metadata at runtime and make the datastage code flexible?
We are using Datastage 7.5.2 PX.

chulett · Post by **chulett** » Mon Feb 15, 2010 1:47 pm

As long as there are zero transformations involved in the process, yes - it's called RCP or Runtime Column Propogation. Search here or in your pdf manuals for all of the gory details.

ray.wurlod · Post by **ray.wurlod** » Mon Feb 15, 2010 1:47 pm

Welcome aboard.

mfshah wrote:The jobs should be dynamic/flexible and pick the metadata information at run time

Why?

mfshah · Post by **mfshah** » Tue Feb 16, 2010 8:57 am

ray.wurlod wrote:Welcome aboard.
mfshah wrote:The jobs should be dynamic/flexible and pick the metadata information at run time
Why?

If a DDL is made to the database, it should not effect job i.e. we shouldn't be changing datastage job

chulett · Post by **chulett** » Tue Feb 16, 2010 9:02 am

Good luck with that.

ray.wurlod · Post by **ray.wurlod** » Tue Feb 16, 2010 1:14 pm

Seriously, how often are table definitions changed in a real production environment.

The usual (professional) approach is that such changes are planned, and advised to everyone involved, including ETL developers so that they can assess the impact. A proper software development lifecycle should apply to ETL programming just as much as to any other form of programming.

What happens (for example) in the Business Intelligence tool that touches this amoeba table? Is that expected to be just as dynamic?