HornetQ: consuming distributed queue sequentially

Question

For a current project, I'm trying to set up the following scenario with JBoss 7.1 and HornetQ (JMS), which I think is a fairly common use case: There are three applications servers. A number of MDBs should each process a broken-down fragment of a lengthy calculation process, the tasks should be distributed among the three servers. When one fragment is finished and a corresponding result is ready, the result should be sent to a distributed queue, from where it is consumed an the total result is assembled. In order to avoid race conditions during total result assembly, the "result" queue must be processed sequentially, although it may be distributed among several servers. No message in the result queue may be processed while another message ist still in progress. An administrative constraint is that the consumers (MDB or session beans) consuming messages from the result queue can be deployed on all of the cluster nodes, i.e. the EAR deployed on the cluster nodes are identical. In that case, the same consumer code will be deployed on each of the nodes. Is there still a way to synchronize access the the queue?

Answer 1

I don't fully understand your use case, but It sounds you need message grouping.

http://docs.jboss.org/hornetq/2.4.0.beta1/docs/user-manual/html/message-grouping.html

if you post edit your question with something simpler I can understand without digging on your testcase I may be able to add more information to this answer.

You also talked about lengthy process. what sounds that you may have clients buffering. take a look on treating slow consumers by setting consumer-window-size=0 on a serverLocator.

this example here on hornetq exemplifies how that could be achieved:

http://docs.jboss.org/hornetq/2.4.0.beta1/docs/user-manual/html/examples.html#examples.no-consumer-buffering