What socket-based model should I use for many simultaneous connections?

Question

I need to know what socket-based model is the best for servers with many simultaneous connections, exactly as in an MMORPG server.

I've read about c10k, but what should I use, if for example my server would allow only 5000 clients at one time? Blocking sockets, threaded (one thread per client)? Non-blocking sockets, non-threaded? a loop handling all the clients in only a few threads? IOCP?

What are the advantages and disadvantages, in short? I've already googled but I didn't find a concrete 'versus' article about all these models.

Or is there a simpler way to handle the c10k problem?

Platform: windows XP

Answer 1

Personally I'd suggest that you use an IOCP based design, but then I would say that as I have an IOCP based framework that I've used for years to develop high performance servers for clients.

I recently put together some of my reasons for prefering IOCP to the other models available on windows and they're available here on my blog here.

Often 'complexity' is cited as a reason to avoid IOCP, but, to be honest, if you're doing anything other than 'thread per connection' the code will be reasonably complex anyway. I have a free set of C++ code with example servers that you could take a look at, it does all the IOCP work for you and the server examples are a good starting point for your own code. You can get it from here.

Alternatively I also have a free (for non commercial use) pluggable server platform which might interest you. With WASP you simply write a dll with some specific entry points and I do all of the networking stuff for you... See here for the download and here for the tutorial.

Whichever model you decide on THE single most important thing, IMHO, is to start doing your performance and load testing from DAY 0. This way you KNOW how many concurrent connections you can support on your target hardware and you can quickly spot the inappropriate design decisions that cause this to drop. It's easy to squander the concurrency of your server. See here for more info.

Since Jay mentioned ENet... I've found that our IOCP based ENet server works well with the standard Enet client code. But remember, ENet is designed to be a peer to peer protocol and so there are some challenges when you start trying to use it on a server. With the standard ENet code you essentially have a single threaded network pump and that just can't compare to the scalability you get from IOCP. I ended up rewriting the ENet protocol handling code from scratch to work with async I/O and IOCP and that's a non trivial amount of work. I'm currently doing a new version of this code which uses the latest ENet protocol as a base and hopefully this will be available early in 2011 with complete perf comparisons to the standard ENet code used in large number of concurrent connection server situations.

Answer 2

The decision as to which way to interact with the server depends heavily on how the client will also be sending data, what type of data is sent, expected rate of transfer and traffic pattern.

For example, if you are writing a game that will allow people to come together in a virtual auditorium and play music, then you are basically doing a great deal of streaming.

If you have a multi-player pacman then you can send small packets, when the direction changes, so it won't be constant, but is bursty.

Then you can decide if you must guarantee packet order and how important every packet sent is delivered, as then you will need to look at TPC or UDP.(http://www.skullbox.net/tcpudp.php)

If you want to support 5000 clients then having that number of threads can be a performance issue, depending on how much memory, but more importantly, how many cores you have and how cpu-intensive the processing is, but, odds are it will be a performance killer.

Then you can look at select() (http://www.lowtek.com/sockets/select.html), which would allow a smaller number of threads to handle the load, but, if you are streaming then that could be a problem as a dedicated thread makes more sense.

So, how to decide which is best depends on what type of game you are writing, as both can be useful, depending on the scenario.

Answer 3

Did you read this article?

I did similar research and found the eNet library. I believe it uses Select() instead of any of the newer methods but there are many users who've indicated they've had very good results with it. It wouldn't be difficult to change if you absolutely need that many simultaneous connections.

Answer 4

You should use IOCPs and socket I/O (any simple I/O) using ReadFile and WriteFile.

Your code should be focused on waiting for I/O completions, loading the context of the completion, processing and finally produce new I/Os on whose completion you can wait.

Your service code (I think in the context of using C) should be multithreaded but with no more threads than you have cores.

I/O-completion ports cost next to nothing and that is also the case with just a few threads. So if you can do your bits quickly enough there is really no limit as to how many I/O-completions an NT-engine can handle. Your internet connection will probably be saturated first.