I am new to websphere MQ connector stage . I am trying to understand how does the MQ connector stage can prevent sending the same messages to the queue in the event the job failed after processing few messages and then i re-run the job again. is there a setting i can use in the connector stage or how can i design my job to prevent sending same messages again after the job failure/re-run.
I am picking the required records from a table in DB2, creating a required XML(sequentially) for each record and sending them on to the cluster queue. I am using the default config file while running the job and with default properties on the MQ stage.
First thing to ask yourself:
Is there any harm to the downstream consumers if they reprocess a message that they've already processed?
If there is no harm, then don't worry about it.
If there is a potential for harm, then Prasoon's solution is practical as long as it's not a matter of life and death. There is some risk in updating a database table and writing to a queue in separate transactions which is what you'll have when using a post update job.
If it is a matter of life and death, then you need to update the processed database row and write to the MQ queue in a single transaction. You'll need to utilize DTS (distributed transaction stage) to make that happen.
Discussed with the downstream consumers about re-sending the messages again and concluded to implement the logic on their side(similar to the logic as per prasson_ibm) to not re-process the previously sent messages. So, there was no change to the datastage job on my side and the functionality is addressed.
. I am trying to understand how does the MQ connector stage can prevent sending the same messages to the queue in the event the job failed after processing few messages and then i re-run the job again. is there a setting i can use in the connector stage or how can i design my job to prevent sending same messages again after the job failure/re-run.