DSXchange

Primarily DataStage's own Performance Analyzer.

You might also gather useful information from the Resource Estimation tool.

Both are available from within Designer. Performance analysis does require some preparation, all of which is described in the manuals.

That is totally the opposite to the requirement you descrived originally. Please confirm what your requirement actually is.

What you are seeking to achieve is called a "vertical pivot". Search DSXchange for examples.

You need version 8.5 to accomplish a vertical pivot using a PXPivot stage.

Why did the job abort?

Your current job design can not load the remaining one million rows without modification.

That modification is to lookup each of the four million input rows against the target and proceed only if the lookup fails (or some equivalent strategy).

No, because there are just too many other variables. Disk topology (mix of local and remote) and disk device speed, to name just one category.

I'm wondering if it's imprecision in decimal storage, though your precision and scale look OK for the values you're using. As a test, can you try converting them to string for the purpose of comparison? Amending Suzanne's suggestion slightly, can you use a reject link and set your Condition Not Met ...

1) Correct.
2) Array size has nothing to do with commit - it's simply the number of rows transmitted at a time. Commit rowcount should be a whole multiple of array size for best performance.
3) Correct.

Parallel Sequential File stage seems to have trouble with these. Prefer a server Sequential File stage, perhaps in a server Shared Container in your parallel job.

ajithaselvan wrote:Can anybody help me out?

Groucho Marx wrote:I'd love to help you out - which way did you come in?

Define "performance".

You've run all three - did you bother to take relevant measures of whatever it is you consider performance to be, and did you use a statistically meaningful volume of data when doing so?

Does your Transformer stage call any routine? Is this a server Transformer stage, or a BASIC Transformer stage called from a parallel job? Please post proof that you can find the file RT_BP368/JOB.1161039537.DT.1455447146.TRANS1 in the project directory and show us the permissions on it by using an ...

Can you guarantee that someone isn't viewing the file with, say, Notepad? This is not the behaviour of after-job subroutine that I understand; it is not invoked until all stages have closed. Could you try introducing a short sleep into your DOS command, in case it's just a timing issue with operatin...

It's not possible with any dsjob option to clear a job log.

You could create a server job or a routine called from a sequence to purge the log, and run that from dsjob.

(I'm pretty certain there's code somewhere on DSXchange for that routine.)

It is definitely not OK simply to move things, because locations (pathnames) are recorded in the common repository. It probably is possible to move them then record all those items of information, but I'd worry that I'd missed one and would therefore prefer to export, uninstall, reinstall and import.

When do you propose to dump the score and inspect the source code as I requested?