DSXchange

Welcome aboard.

Research "runtime column propagation" (RCP) here, in the manuals or on IBM Knowledge Center
(http://www-01.ibm.com/support/knowledge ... ge_IS.html)

Does the folder in which the library/object exists appear in the PATH when this job is executed? (Check the entry in the job log that contains environment variable values.)

You might experiment with a Connector with multiple input links, which should give you you ability to create a transaction across all three tables where a particular key value is shared.

But what you are doing will work too.

I agree with Craig. Your symptom strongly suggests that whatever method you are using to request job runs is re-using invocation ID.

Copy stage can drop columns - you probably don't need Modify stages for that.
Various options exist for elimination of duplicates or filtering based on dates; investigate the possibilities - some of which have been posted here - and run with whatever works best for you.

Welcome aboard.

Perhaps you could start with enabling ODBC tracing, to see what actually gets passed "over the wire".

Seems to me (based on your example) that the more appropriate delimiter to use in your Field() function is " " rather than ":".

What you have described should work. What specific error message was thrown?

The UNIX command executed via ExecSH can be anything you can do on the UNIX command line. Therefore, for example, you could construct a pipeline based around a for loop picking up all the files in the directory.

Have you tried it?

I can't envisage how you would manage the three separate grouping (sorting) requirements.

That comes under the heading of consultancy. Which comes at a cost. The way we work here is that you come up with your own design and try it out, and we can then help you work out why it doesn't work, or why it's inefficient, or whatever else you may need help with. But we're NOT here to provide a d...

The "orch..." modules you mentioned have nothing to do with Oracle - they have to do with Orchestrate (the parallel execution engine of DataStage). Are you really sure there are no other entries in the job log, and yet the Oracle table gets created? Possibly your logging system has become ...

Are there other files in the folder that match the pattern *.csv ?

DataStage Operations Console, which has the added benefit of preserving history in a database so you can monitor trends.

Can you check that the Hadoop server is listening (successfully) on port number 9000? This is the next step on from a ping test, which only shows network connectivity. Use netstat and, perhaps, telnet commands to test the port.