Having more than one client per thread in c10k

Question

Is there any optimal reason why one thread per client is normally used in the c10k problem?

Can we put 5, 10, or 25 per thread? Would there be any negative changes to performance?

Having one thread per client seems to be very taxing on the system and if we had 25 per thread we could have 400 threads for 10,000 instead of 10,000 for 10,000.

Answer 1

There are two completely different models for dealing with 10K+ connections:

• Multi-Threaded with one thread per connection
• Single-Threaded with non-blocking operations and asynchronous I/O (possibly with several independent processes per machine to fully utilise the available cores)

Both models can be used to work with more than 100K connections, and both models have their benefits in certain areas. In direct comparison, they differ as follows:

Multi-Threaded:

• high memory footprint caused by the stacks
• need for synchronization operations to avoid data races

Single-Threaded:

• everything has to be non-blocking (bad support by operating systems, e.g. for file-system operations on Linux)
• everything has to be asynchronous (bad support by programming languages, e.g. for control structures and error handling)

It is possible to combine these two models. In this case, one thread per core is typically used (instead of one thread for N connections). There are some good use cases for this model. However, there is also a huge problem: It combines the drawbacks of both models. That means, you have to spent the efforts to everything thread-safe, non-blocking and asynchronous. Usually the effort is twice as high as for one of the pure models.

That's the reason, why the pure models are usually preferred over the mixed model for most applications. The exceptions are edge components, which have to deal with millions of requests per second, e.g. load-balancers and proxies.