Azure worker role + number of message queue consumer

Question

I am trying to understand the best practice when hosting a message queue consumer in a azure worker role. I have many different type of message consumers that subscribe to different azure service bus subscriptions (or queue if you like to call it). I am wondering if I should instantiate multiple threads for each consumer in one Worker Role or should I deploy to multiple Worker Role for each consumer.

Answer 1

This is really dependent on your app and workload. If you have tasks that are I/O-blocked, then you should be running several threads; otherwise, you'll have a virtual machine instance which isn't being used efficiently. If it's primarily CPU-based, you may find that you can run efficiently with a lower number of threads.

You should only scale out your worker instances if you can't handle the capacity in a single instance (or if you need high-availability, in which you'd need at least two instances). Just remember that a worker role instance is a full VM, so adding one VM per queue consumer scales in cost, and you still might not see great throughput in an I/O-bound app (or one that blocks on other things such as a Web Service call).

You'll need to do a bit of experimenting to see how many threads to work with on the worker side.