DSXchange

Welcome aboard. :D

As I understand it, you don't use the liborch library, the stage (operator) does. What the documentation means is that the library must be findable through your shared library search list (LD_LIBRARY_PATH).

Welcome aboard. :D

Via "user defined SQL" you can execute any SQL that is legal in the database, provided that the correct number of columns is bound to the query.

Read the description of DSGetStageInfo() carefully. If my memory serves it is pertinent only to active stage types.

UPDATE is always much slower than INSERT. Split the stream into insert only and update only.

Not if you have a deadlock. If you have a deadlock nothing at all happens.

So pivot 30 columns then filter out those whose length is 0 or which contain null.

The reference data (do you mean 4,00,00,000 bytes or 4,00,00,000 rows?) has to be totally loaded into memory using a Lookup stage. It is probably here that you're running out of memory. Consider using a Join stage.

The first contains the entire list. The $Counter activity variable contains the single value for the current iteration through the loop.

Please perform the checks I asked you to perform.

$PROJDEF picks up its values from the environment variables that have been set in the Administrator client, and which are stored on the server in a file called DSParams in the project directory. I suspect you've missed migrating these environment variable definitions.

How do you calculate the array size that you use?

DataStage does not use Outlook; it uses simple mail transfer protocol (SMTP).

Welcome aboard. :D Assuming you mean "in" a routine, then you should be able to refer directly to the environment variable, for example by capturing its value via echo command. Call DSExecute("UNIX", "echo " : $APT_CONFIG_FILE, Result, ExitStatus) Result = Convert(@FM...

Use the expression builder to construct your expression (you don't need the "#" characters).
Check, using another Execute Command activity to echo the previously calculated return value, that all four names are being picked up. Check also that there is a terminator following the fourth.

Zero-length string is valid as a key value. NULL is not.