Azure Queue storage not sending queue message

Question

I am using a azure queue storage to send emails. The emails are stored in the queue storage and the queue sends 20 emails at a time.

//Checks for messages inn the queue
foreach (CloudQueueMessage msgin sendEmailQueue.GetMessages(20, TimeSpan.FromSeconds(50)))
   {
        ProcessQueueMessage(msg);

   }

The problem I am having is that when an email is added to the queue with incorrect SMTP details(i.e wrong password), the message stays in the queue as it fails to send and prevents other messages in the queue from sending.

private void ProcessQueueMessage(CloudQueueMessage msg)
{
  try
  {
    //We try to send an email
    SendEmail(emailRowInMessageTable, htmlMessageBodyRef, textMessageBodyRef);

  } catch (SmtpException e)
  {
    string err = e.Message;

    //When an error occurs we check to see if the message failed to send certain no. of      
      times
    if (msg.DequeueCount > 10)
    {
      //We delete the message from queue
      sendEmailQueue.DeleteMessage(msg);

      return;
    } else
    {
      //delete from top of queue
      sendEmailQueue.DeleteMessage(msg);

      //insert into end of queue
      sendEmailQueue.AddMessage(msg);

      return;
    }
  }
 }

The solution I tried was to delete the message from the queue if there was an error and add it back to the end of the queue resulting in the correct emails being send out. But deleting and adding the message back into the queue resets its dequeue property which is not ideal since I am using the dequeue property to make sure a message is not in the queue forever.

What would the best solution be in this situation?

Answer 1

The solution was to use different queue's for different SMTP servers and run it all from one worker role using multiple threads.

Used the framework here: http://www.31a2ba2a-b718-11dc-8314-0800200c9a66.com/2010/12/running-multiple-threads-on-windows.html to run multiple threads inside a single worker role.

Answer 2

You don't really have to delete the message and add it. Once the message's visibility timeout period (50 seconds in your case) has expired it will automatically appear back in the queue. This way, your DequeueCount logic would also work as the same message is dequeued and enqueued again.

Please note that Windows Azure Queues are best-effort FIFO ... so it is not always necessary and that messages will be picked on the basis of the time they were added. You could do a few things:

• Reduce the number of retries (currently you have 10, you can reduce it to 5) or
• Check for actual exception. If the process failed because of incorrect credentials, it will fail the next time and next time after that. There's no point retrying that message. You could just move that message into a poison queue so that you can inspect the message later.