C# .NET workload processing a'la job queue processing examples ideas?

Question

As part of my constant learning curve into what you can do to make apps scale better, I am currently trying to get a direction to go with queuing, i.e. job queuing or workload processing whichever phrase you like.

In the distant past I used IBM MQ/Series - it worked for a financial app but quite heavy if I remember.

I know of MSMQ, and I have also heard of quite a few others.

But first, here is my context

I have a C#/.NET back-end web app which serves data etc to a Javascript (mostly jQuery etc) front-end via AJAX calls etc. I have a situation where a certain action involves uploading some files, setting up a few record entries in the database, emailing some users etc. So of course I don't want to make this process "online"/"real-time" due to the possible time delay and I am sure the overheads on the webserver/database etc.

So given the type of "messages" that I need to queue and process, what would be (I shouldn't just say easy here I guess!) a good start point? should I run with MSMQ and/or the SQL 2008 service broker stuff, or something like ZeroMQ - or should I simply create my own lightweight workload queue service?

I realise again without seeing the full picture it is hard to make full recommendations, however any start points gratefully received!

David

Answer 1

Don't try to make your own, please! There are so many things to take into account that you will spend more time on it than the rest of your project most probably.

I'd say go for MSMQ, it's very easy to use with WCF, the queues are transactional, have a retry mechanism, etc, and you benefit from the MSMQ UI to see the messages, move them and so on.