Designing “Scalable Multi Threaded Socket Push Server” in .Net

Question

We are at the beginning of designing Scalable Multi Threaded Socket Push Server in .Net which will initially handle 10K clients per node. Since designing this can be very complex, it would really help me if you all experts can provide any guidence, resources, best practices etc.

Since I dont know much about socket programming, it would really good if someone can suggest me an opensource project to get start with... I googled it, but couldn't find any good solution.

This will also be helpful for other people who would like to design such server.

[For your information we are planning to use .Net 4.0 for this server]

I have been reading lots of information regarding Async Socket and its Pinning related problem. Is that problem still exists in .Net 4.0?

Thanks in advanced...

Answer 1

10k concurrent connections using .Net on reasonable hardware shouldn't be that much of a problem but, as always, it will depend on what you're actually doing in your server and how isolated each connection is from each other. After all, saying 'handle 10k clients per node' is pretty meaningless when you don't define your hardware requirements for each node, the amount of data that will be flowing in and out on each connection and how much CPU each connection is likely to need to do its work. I talk about these vague, hand wavey scalability questions here: http://www.serverframework.com/asynchronousevents/2010/12/one-million-tcp-connections.html

However:

First you should be looking at the asynchronous sockets API and pretty much ignoring the threading issue. .Net's async sockets will take care of the threading for you and use I/O completion ports (a very scalable kernel object) to manage your threading for you.

Also, bear in mind that if you are using async sends then these can take a long while to complete if TCP flow control kicks in, see here.

and...

Secondly you should test your scalability from day 0 and keep testing it all the way through your development - I write about why here: http://www.serverframework.com/asynchronousevents/2010/10/how-to-support-10000-or-more-concurrent-tcp-connections---part-2---perf-tests-from-day-0.html

Answer 2

Some people might say that 'scalable' and 'multi-threaded' are mutually contradictory. You need to explore the design space before making decisions like that. If you need scalability it may be possible to use just one thread, but it depends on what you are serving up.

Answer 3

With scalability being one of your high priorities and I am also going to assume stability I have found that Erlang is a good fit. It is fault tolerant and highly scalable. If you have the time to investigate different options I would suggest at least looking into that.