Message Queue: Can we trigger events when a message enters the queue?

Question

At work we're discussing whether or not to implement a message queue for our PHP application. Currently we're looking at Apache's ActiveMQ. One thing we're not entirely clear on is whether or not it's possible to trigger a process based on a message arriving in a queue.

The literature we've found so far seems to indicate that messages queues are a pull-based mechanism: the process runs regularly (either as a daemon or a cron), and pulls its incoming messages from the queue. Is it possible to turn this into a push mechanism? That is, is there a way to have the Message Queue actually initiate an HTTP request (or a process) when a message arrives? One option we have found is the Publish/Subscribe model, but this requires running our PHP app in an infinite loop to maintain an open (TCP) connection to the ActiveMQ instance, which feels like a bit of a kludge.

Any input would be much appreciated.

Answer 1

An obvious solution is to let the publisher initiate the HTTP request right after storing the message, but this begs the question, why then are you using a message queue?

Having a set of consumers listening on a queue and doing their job as messages come is not a kludge, it is good, scalable design. (Though I agree that running a PHP process in an infinite loop has its cons.)

Why have you chosen a message queue as opposed to, say, a database which stores the messages? The "producer" could store the message as a row in a table and then trigger the "consumer" with the message's PK.

Answer 2

Consider creating a Camel route that pulls messages from the queue (JMS component) and forwards to an HTTP endpoint (HTTP component). You may even decide to host the Camel route on the ActiveMQ broker process -- creating a sort of intelligent routing broker. ActiveMQ makes this easy by bundling the Camel Core libraries in the ActiveMQ distribution.

Here is a recent relevant / related post:

HTTP Post From ActiveMQ using Camel

Answer 3

Your design concept of an app to monitor the queue is the industry way of triggering an external application to run. Have a look at IBM Websphere "Trigger Monitor Applications". In their implementation you can have multiple trigger monitors to start apps that process messages in a queue, so it scales very well as the trigger monitor runs on the server the apps are installed on (and if you design for scalability to you should be able to have several app servers if needed).