How to load test Publisher/Subscribing queue?

Question

I am trying to load test a queue that works on a pub sub pattern and can't figure out how response time is to be calculate and the TPS(transactions per second).

Problem statement :

The pub sub pattern requires us to develop different script for the publishing user and the subscribing user.

Every message that is sent to the queue by the publisher will be intercepted by each subscriber to that queue.

The user who publishes and the other user who consumes the response might/might not be different.

How should the latency be calculated now?

Case-1 Calculate only the time for publishing the messages to the queue.

Case-2 Calculate the time for publishing the message + time taken by subscriber to retrieve it.

Case-3 Calculate only the time for subscriber to extract the messages from the queue.

The subscriber is working on a polling mechanism, i.e, it checks the queue for any messages after every interval. The time taken to receive the message by a subscriber from a publisher depends on the following factors -

When the last polling took place?

Load on the queue because of the publisher.

This gives a hint that there might be a correlation between publisher and subscriber users due to dependence of load on queue due to both publisher and subscriber.

There also a chance of difference in the rate of speed of publishing and subscribing of the users. For example - If in a queue, 20 messages are queues per second but a subscriber can only extract 10 messages per second. We have to factor in the time due to extra messages queued in before a particular message is a read from the queue.


Solution till now

While preparing the script for publisher and subscriber, keep the rate of pushing and extracting messages, similar/with least possible induced difference by keeping polling and publishing interval same. Though still the difference in the number of publishers/subscribers can load the queue and difference will still be induced.

What is still not clear is that for what factor the load testing is to be done -

How many transactions a publisher can do in a sec (TPS)? How many transactions a subscriber can do in a sec (TPS)? How much load can the queue can handle by making transactions? The TPS of queue depends on both publisher and subscriber making transactions in it.

To study the Publisher | Subscriber | Queue Load separately (keeping the polling rate and publishing rate same in the script) should I - Calculate response time to publish a message separately. ( To test the publishing rate) Calculate subscriber's extraction time separately. ( To test the subscribing rate) Calculate time to reach from publisher to subscriber separately. (To check the load on queue separately)

Is there something I am doing wrong, how can I test it for multiple users?

Answer 1

The whole integrated system acts with the speed of its slowest member. So when you conduct the load you will see that certain component works slower than others hence the messages targeting this component will be queuing up.

Standard metrics like response time, latency and transactions per second are not very applicable for queues so you need to look for a queue monitoring solution which might be different depending on your MQ solution vendor:

• Microsoft Message Queue (MSMQ) Monitoring
• A web-based monitoring tool for IBM WebSphere MQ
• How can I monitor ActiveMQ
• etc.

You can also consider using Prometheus or Grafana which will report messages processing rate and queue sizes.

With regards to generating the load itself, if you don't have anything better in mind you can try out Apache JMeter which supports simulating publishers, subscribers and point-to-point scenarios.