How does a (full featured) long polling server work abstractly

Question

Since you're using an event loop as opposed to threads, how does the actual server look?

I know it uses an event loop, but how do you separate out the requests? And how do you prevent your server from running extremely slowly (since it, I assume, can only push one thing at a time since it's threadless?)

Some sort of pseudo-code would be great.

Forgive my ignorance; of course, if there's somewhere that explains it in a non-basic "this is good enough until you get 1000 visitors way", I'd be glad to know of it.

Answer 1

The implementation details of a long poll server would vary so much from platform to platform that your assumptions might not be correct.

I implemented a COMET server for our website using .NET. I leveraged HttpListener to do all the boring http stuff and Microsoft CCR to deal with all the async IO. It uses a pool of threads to service requests as and when they come in. It's not a thread per client, but it's not single threaded either generally requiring a few tens of threads to stay fluid as user numbers rise. This approach means that we scale easily across multiple CPU cores. CCRs async enumerator pattern really helped keep the asynchronous logic nice and tidy, and I can read the code fairly easily a year later.

This approach has proved extremely scalable. I've tested up to 20000 clients, whereupon we became bound by network IO. It handles all our clients (who are "permanently" connected, reconnecting every 30s) ticking along at 1-2% server load. It's definitely worth reconsidering your assumption that you must either choose an event loop architecture as opposed to multiple threads. The middle ground works very nicely for me, and the .NET asynchronous programming model for dealing with IO bound tasks really takes you away from needing to micro-manage threads. Effectively, when there's IO data to process, a thread is borrowed from the pool to do that processing, and subsequently returned to the pool ready to service another request. All the complicated IOCP stuff is abstracted away.