Getting a Message by MessageId on the Azure Service Bus Dead Letter queue

Question

I'm writing functionality for monitoring and fixing messages in the dead letter queue of a topic subscription on Azure.

I can get a list of messages using _subscriptionClient.PeekBatch(10) for display; however we are stuck when we want to actually remove one of the messages.

The workflow would be:

1. Get a list of all messages
2. Send a specific message (say the 5th on in the collection) back to the original queue for reprocessing
3. Mark the associated dead letter message as .Complete().

The issue is that whilst we have the list of messages, we cannot call .Complete() on a specific one without using .subscriptionClient.Receive() first.

As you cannot .Receive() a message by MessageId, does this mean we have to loop through the messages one by one like below?

public BrokeredMessage GetMessageById(string messageIdentifier)
{
    BrokeredMessage matchingMessage = null;
    var messageNotFound = true;

    var messagesToAbandon = new List<BrokeredMessage>();

    while (messageNotFound)
    {
        var message = _subscriptionClient.Receive();

        if (message == null)
        {
            throw new Exception("Could not find the message on the queue");
        }

        if (message.MessageId == messageIdentifier)
        {
            messageNotFound = false;
            matchingMessage = message;
        }
        else
        {
            messagesToAbandon.Add(message);
        }
    }

    // Unlock all messages that do not match the matching one received.
    foreach (var message in messagesToAbandon)
    {
        message.Abandon();
    }

    return matchingMessage;
}

My issue with this approach is:

• It's not scalable; what if the message is the 100th in the collection? We'd have to loop through 99 and mark each as abandoned
• We risk losing the lock on the matchingMessage if there are too many items to abandon
• It's a long running process

I've toyed with the idea of marking each message as abandoned if not matched within the loop; however this creates the risk of us looping through the same items infinitely (as .Abandon() puts them back on the queue).

Has anyone found an efficient way around this? Maybe using .Defer() with .Receive(sequenceNumber)?

Answer 1

You may use sessions here, set the session id for each message. Later on when you browse the DeadLetter, pick the message and its sessionID, open a session using the sessionID and receive on the message.

Update: The DeadLetter queue, being a special queue, doesnt allow sessions and one cannot receive a message from DeadLetter queue using sequenceNumber as well, those are the two options i found if you want to cherry pick a message.

So here is what one shall do in the cases described in question , Take all the messages from DeadLetter, Replay certain messages to original queue for reprocessing, move the rest of the messages to another queue (ErrorQueue) and clear the dead letter queue. I think this will be more appropriate as DeadLetter queue is more of an internal queue for ServicBus to handle errors/expiry. If application needs to deal with the error , moving those to app specific queue will give more flexibility and not to over load the system queue (DeadLetter Queue)

Answer 2

You can delete a specific message from the dead letter queue quite simply if you have the message SequenceNumber:

public static void DeleteDeadLetterMessage(string connectionString, string queueName, long sequenceNumber)
{
    var deadLetterQueue = QueueClient.CreateFromConnectionString(connectionString, QueueClient.FormatDeadLetterPath(queueName));
    var msg = deadLetterQueue.Peek(sequenceNumber);
    if (msg.SequenceNumber == sequenceNumber)
    {
        msg = deadLetterQueue.Receive();
        msg.Complete();
    }
}

The only thing you have to be careful about is that Peek may return another message if there isn't any message with the specified SequenceNumber.

Therefore you'll need to check the SequenceNumber, so that you don't Complete the wrong message.

Answer 3

We do the same as Dhana Krishnamsamy.

We created a DeadLetterQueueCompanionQueue Then for each subscriptions we registered a client for the deadletter queue That client re-submits a copy of the original message to the new queue

Our queue has a message expiration and also allows for us to monitor in a single place.

We then complete the message in the original deadletter queue.