Broadcast message in rabbitmq and wait for all replies

Question

I'm using a fan out with a reply to address in a header. The tricky part is that I want to wait for the replies of all workers but I don't know how many I will recieve (I don't know how many workers exist when I send the broadcast).

Is there a way to know how many workers are currently connected?

Thanks in advance!

Answer 1

Brief answer: no, there isn't.

A general solution is to let the workers send sort of "work in progress" heartbeat messages (not RabbitMQ heartbeats, that go only to the broker, I mean at application level) to the reply-to exchange.

So you let the workers send their first heartbeat as soon as they get the message and start working. Then once every X seconds, until they haven't completed their task.

You never have the guarantee to know how many they are. For example it's possible that a worker have been reached by the first message, have started working but the replies can't reach the broker. Or a working worker fails and never report the result back (but monitoring the heartbeats the consumer(s) of the replies can be aware of it).

IMHO, it's a good practice to let a monitor application (your collector/reducer task) be aware of the state of the work, especially when it's a long-lasting one, instead of just "waiting and hoping".