Delay Processing of records

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
ankursaxena.2003
Participant
Posts: 96
Joined: Mon May 14, 2012 1:30 pm

Delay Processing of records

Post by ankursaxena.2003 »

Hi,

I have a requirement where I am reading MQ. And after I read message, I want the job to hold the message for 30 secs before dumping into oracle table.

MQ Read -------------> Transformer -------------> Oracle table
Thanks,
Ankur Saxena
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

Really? That's a requirement? The reason most applications use messaging systems is to get the data securely delivered in near real-time, not to add delays to the delivery.

The only semi-reliable way I could think of that might work would require the records to each be timestamped as they are read from the MQ system. Then you'd have to build a transformer routine that would be called for each record that would take that timestamp as an argument. The routine would determine what the current time is and wait for the required number of seconds before returning. In essence, adding a "sleep" before passing the record downstream.

Warning - jobs that have any sort of "sleep" or "wait on a semaphore" internally are notorious for being difficult to stop cleanly if something goes wrong.

Out of curiosity - could you state why it is supposed to wait 30 seconds?
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
ankursaxena.2003
Participant
Posts: 96
Joined: Mon May 14, 2012 1:30 pm

Post by ankursaxena.2003 »

Hi Andy,

Actually I found a logic on dsxchange, but I have to still test it.

viewtopic.php?p=425333

And the reason for putting delay is that application sends data to 2 different systems (Java and DataStage). And I depend on getting some data from Java and datastage is processing data faster than java :D .
So I need to put a delay in job of 30 secs.
Thanks,
Ankur Saxena
Post Reply