Is it possible to get a message from an azure storage queue twice?

Question

I know that if a worker fails to process a message off of the queue that it will become visible again and you have to code against this (idempotent). But is it possible that a worker can dequeue a message twice? Based on my logging, I seem to be seeing this behavior and I'm not sure why. I'm even deleting the message in between going go get the next message and it seems like I got it again.

Answer 1

You shouldn't be able to dequeue it twice. And if I recall things properly, even deleting it twice shouldn't be possible because the pop receipt should change after the second dequeue and lock.

As SilverNinja suggests, I'd look to see if perhaps the message was inadvertantly queued twice.

Answer 2

Yes, you can dequeue same message twice. This can happen for two reasons:

1. Worker A dequeues Message B and invisibility timeout expires. Message B becomes visible again and Worker C dequeues Message B, invalidating Worker A's pop receipt. Worker A finishes work, goes to delete Message B and error is thrown. This is most common.
2. In certain conditions (very frequent queue polling) you can get the same message twice on a GetMessage. This is a type of race condition that while rare does occur. Worker A and B are polling very quickly and hit the queue simultaneously and both get same message. This used to be much more common (SDK 1.0 time frame) under high polling scenarios, but it has become much more rare now in later storage updates (can't recall seeing this recently).

That being said - if you only have 1 worker popping messages, then you are queueing message twice. 1 and 2 only happen when you have more than 1 worker.

Answer 3

Do you have more than one worker role?

It is possible (especially with processes that take a while) that the timeout on the queue item visibility could end before your role has finished processing whatever it is doing. In this case another identical role could pick up the same message (which is effectively what you need to allow for - you do not want it to be a problem if the same message is processed multiple times).

At this point the first role will finish and dequeue the message and then the other role that picked it up after the timeout will end and attempt to dequeue the message. Off the top of my head I don't recall what exactly happens when a role attempts to dequeue an already dequeued message.