How to marshal work onto single thread c#

Question

I have an app that needs to process some packets from a UDP broadcast, and write the result to a SQL database. During intervals the rate of incoming packets gets quite high, and I find that the underlying buffer overflows, even at 8MB, because of the slow database writing.

To solve this I have come up with two options, cache some of the DB writing, and/or marshal the db writes onto another sequentially operating thread. (order of db writes must be preserved).

My question is how to marshal the work most effectively, with built-in structures/features of c#?

Should I just have a synchronized queue, put the delegate work items onto it and have a single thread service it in an event loop?

Answer 1

The easiest way is to create a separate thread for the UDP reading.

The thread just receives the packages and puts them into a BlockingCollection. (Don't know how you do the packet reading so this will be pseudocode)

BlockingCollection<Packet> _packets = new BlockingCollection<Packet>();
...

while (true) {
    var packet = udp.GetNextPacket();
    _packets.Add(packet);
}

This will ensure you don't miss any packages because the reading thread is busy.

Then you create a worker thread that does the same thing as your current code but it reads from the blockingcollection instead.

while (true) {
   var nextPacket = _packets.Take();
   ...
}

If your packages are independent (which they should be since you use UDP) you can fire up several worker threads.

Answer 2

If you are using .Net 4, Tasks and TaskSchedulers can help you out here. On MSDN you'll find the LimitedConcurrencyLevelTaskScheduler. Create a single instance of that, with a concurrency level of 1, and each time you have work to do, create a Task and schedule it with that scheduler.