DSXchange

Hi DS gurus,

I am new to DataStage.I have created some parallel jobs.Now these Jobs are running properly in my Development environment.They were also running properly in my Quality environment.But nowadays some of the Jobs while running simply hangs up.These jobs are not very complex having an aggregator,joiner and 2 to 3 transformers.They run properly some times and hangs up the other times.Could you tell me what is the reason behind this and how can I correct this issue.

Thanks a lot
Rony

Rony,

a "hang" is a tough condition to analyze. When the job "hangs" does it use any CPU or I/O or other resources? Do the players still exist on the system or have they stopped? Does the job "hang" right at the beginning or after some period of processing rows? Does the the amount of data being processed through the Aggregator matter and/or does the job "hang" in the same place with the same data? Will the job "hang" with a different configuration file (i.e. different numbers of nodes)?

You posted in the Server section but declared the job as parallel. Which one is it? I assumed parallel for some of the questions listed above, but the other items apply to server as well.

Hi,

I have created all parallel Jobs.We access DataStage from the Client machine through Citrix connection.so I am not sure how to check the resources used over there.I am sorry but i dont kno what you mean by "Players"...The job hangs after some time with usually the last messages in the logs as
"agg_Qty,3: Hash table has grown to 131072 entries."
OR
"A sequential operator cannot preserve the partitioning
of the parallel data set on input port 0."

It doesnt hang at the same place all the time.
We only use a single configuration file and dont change that.I hope this gives a little more insite into the problem

Regards
Rony

My initial thought is that you might be "overloading" your Windows server.

I would recommend using a DataStage PX configuration file that only defines 1 node for testing, and also set APT_DUMP_SCORE to true to get some additional information. Also, how many nodes do you define in your current file and what does your Windows server configuration look like (# of CPUs, physical memory)?

The two messages are not necessarily issues.

Consider using Sort as the aggregation method rather than Hash Table.

Hi,

Sorry for putting this in the sever Forum...wasn't aware of that when I posted.I have now used sorter instead of Hash Tables as suggested by Ray Wurlod and it seems to be working.Thanks for your responses

Regards
Rony

Do you understand why?