MSMQ consuming large amounts of memory when processing messages with NServiceBus

Question

I have two windows services which use NserviceBus. One writes messages to the queue and the other reads from it and do some processing. All the queues are transactional and the NserviceBus endpoints are configured as below.

.IsTransactional(true)
.IsolationLevel(IsolationLevel.ReadCommitted)
.MsmqTransport()
.RunTimeoutManager()
.UseInMemoryTimeoutPersister()
.MsmqSubscriptionStorage()
.DisableRavenInstall()
.JsonSerializer()

The issue is when a large amount of messages (170,000+) are queued, MSMQ service (mqsvc.exe) chews up quite a bit of memory (1.5 - 2.0 GB) and that memory doesn't get released for at least 5 - 6 hours. The average message size is around 5 - 10 KB. And it seems like the more messages you queue the more memory it uses. The NServiceBus based Windows Services memory consumption are in perfectly acceptable limits (50 - 100 MB) and do not increase no matter how many messages they process.

Any ideas on why MSMQ would use this much memory and takes quite long to release it? Thanks heaps.

Answer 1

This is perfectly normal. MSMQ uses storage in 4MB blocks of memory which map to the files in the Storage folder. 170,000 messages at 5-10kb each is 0.85-1.7GB so no surprise you're seeing so much virtual memory being allocated. To reduce the overhead of deleting and creating files as messages are removed or arrive, the storage files are kept for 6 hours. After this period, the empty files are deleted. You can configure this, as discussed in my blog post:

Forcing MSMQ to clean up its storage files

Answer 2

On the off-chance it will help anyone - this post on google groups by msmq legend John Breakwell documents how to actually clean down all the messages in storage completely, which is sometimes desirable/necessary

https://groups.google.com/d/msg/microsoft.public.msmq.performance/jByfXUwXFw8/i1hVP1WJpJgJ

I had 8GB of files but no messages in any queues, and the msmq service would take around 2 hours just to enter the started state. Purging any queue would take 10s of minutes and cause massive memory spikes, which did not then get released for days, if ever.

If you're ever in this situation, rather than re-install message queuing you can just follow these steps:

1. Stop Message Queuing service
2. Go to the MSMQ storage location (usually C:\Windows\System32\msmq\storage)
3. Delete ONLY the P*.MQ, J*.MQ, R*.MQ, and L*.MQ files