How to ensure only once instance of an Azure web-job is running at any time

Question

I've got a continuously running WebJob on my auto-scale Azure website.

My WebJob is a simple console application with a while(true) loop that subscribes to certain messages on the Azure service bus and processes them. I don't want to process the same message twice, so when the web site is scaled an another WebJob is started I need for it to detect that another instance is already running and just sit there doing nothing until it's either killed (by scaling down again) or the other instance is killed. In the last scenario the second WebJob should detect that's the other is no longer take over.

Any takers?

Answer 1

You should create a queue (either using the Service Bus or storage queues) and pull the jobs off (create and manage a lease to the message) and process them from there. If that lease is managed properly, the job should only get processed once although you should make sure it's idempotent just in case as there are fringe cases where it will be processed more than once.