What open source message queuing software provides durability with strict ordering?

Question

What we need is RabbitMQ that actually works as a queue and doesn't do this. Messages should stay at the head of a queue untill client dequeues them explicitly.

It seems like a very straightforward scenario, but for some reason I can't find any broker to support it.. A broker should run on Windows OS.

Answer 1

Apache Qpid is probably your best option. Of all of the message queues, this one has a number of interesting things going for it, including strict ordering.

Answer 2

If it is only one message that is the problem, why not write it to a file (and flush the file) before you process the message. After acking the message, delete the file.

And if you are concerned about the message broker crashing, first step is to upgrade it to RabbitMQ 2.4.1 running on Erlang R14B02. Second step is to cluster it so that you have multiple servers acting as the MQ broker. And only then, change your app to track the messages that have been processed, either by timestamp or by saving message IDs. Then, if RabbitMQ requeues a message, you will already have it and will process it and remember it. When it comes around a second time you will ignore it.

You may need to set prefetch to 0 for this to work right.

And there is another alternative too. You could consider writing your own RabbitMQ plugin to provide the exact behaviour that you need. Erlang may look complex at first sight, but it really isn't that hard to learn for an experienced programmer who has already learned a few languages. In particular, if you have anyone with functional programming experience in languages like Haskell or CAML, they will quickly pick up enough Erlang to do the job.

Because of Erlang's internal model of message-passing processes, RabbitMQ plugins can essentially do anything that they want. There is no specific limited plugin API that they need to conform to.

In other words, if RabbitMQ only does 99% of what you need, consider yourself lucky that with a small amount of work, you can leverage that 99% and achieve everything that you need. But in order to do this you have to get away from the idea that RabbitMQ is yet another package that you install with your system's package installation tools. In cases like yours RabbitMQ should be considered to be a mission critical tool, and you should install Erlang and RabbitMQ from source, and configure them to your needs without letting your OS limit you.

Answer 3

RabbitMQ also supports strict ordering as of release 2.7.0 and so should be an option for your scenario again.