DBQUERY blocked by Input card database retrieve

Formally known as "Mercator Inside Integrator 6.7", DataStage TX enables high-volume, complex transactions without the need for additional coding.

Moderators: chulett, rschirm

Post Reply
VPech
Participant
Posts: 1
Joined: Fri Oct 24, 2008 8:41 am

DBQUERY blocked by Input card database retrieve

Post by VPech »

We have a Datastage TX 8.0 system with a map that's creating multiple connections to the database (Microsoft SQL Server 2000) and blocking itself, causing it to hang. This blocking occurs when there are more than 300 or so rows to be retrieved from the table. It also ONLY occurs on our production database server with 8 CPUs and we can't replicate it on our QA database server with less processing power, even with many more (4000+) rows.

Details:
The map has an input card that uses a database as the source. It calls a stored procedure which selects rows from a table for processing.
The rows pulled from the database are passed to a functional map, which calls one of two child maps on each row. The child maps process the row creating a flat file. When the child map finishes, the parent functional map calls a stored procedure to update the status of the row in the database. The stored procedure is invoked via DBQUERY, with -CSTMT flag for commit and using an mdq file. This update statement gets blocked by the select that's for some reason still holding on to the table (from the retrieve of the input card). We can see the blocking on the database level (one datastagetx SPID blocking another).

Settings:
The input card GET is currently set to FetchAs Burst and FetchUnit 1. We tried doing FetchAs Integral and FetchUnit 'S', but this made only the first row get processed (only one flat file created) out of all the ones that were retrieved, but all of them were updated by the functional map as if they succeeded. The transaction scope is set to 'Map'. We also tried it with 'Card' scope, but nothing changed.

In the .ini, we have MaxThreads set to 1. We tried setting IdleDB to 30 to force it to close the idle SELECT connection. We tried setting HLimDB to 1 to force it to only open one connection to the database - this caused it to hang (but not block) as it seemed to be trying to open more connections instead of reusing the 1 connection it already had. We also tried setting HLimTimeout to 30.

How can we ensure the retrieve closes its connection properly? Are there any other avenues we should be investigating?

Any help would be greatly appreciated.
Post Reply