Is it possible to dispatch no more than N number of messages from the queue at a given time with distributed consumers?

Question

I have a distributed system that reads messages from RabbitMQ. In my situation now I need to process no more than N msgs/s.

For example: Imagine service A that sends text messages (SMS). This service can only handle 100 msgs/s. My system has multiple consumers that reads data from RabbitMQ. Each message needs to be processed first and than be send to service A. Processing time is always different.

So the question:

• Is it possible to configure queue to dispatch no more than 100 msgs/s to multiple consumers?

Answer 1

You can use the prefetch_size parameter of the qos method to limit throughput to your consumers.

channel.basic_qos(100);

See also: http://www.rabbitmq.com/blog/2012/05/11/some-queuing-theory-throughput-latency-and-bandwidth/