Why the need for Async IO when reading sockets for non HTTP server

Question

Im designing a c++ client application that listens to multiple ports for streams of short messages. After reading up on ACE, POCO, boost::asio and all the Proactor like design patterns, I am about to start with boost::asio.

One thing I notice it a constant theme of using Asynchronous socket IO, yet have not read a good problem description that async io solves. Are all these design patterns based on the assumption of HTTP web server design?

Since web servers are the most common application for complex latency sensitive concurrent socket programming, im starting to wonder if most of these patterns/idioms are geared for this one application.

My application will listen to a handful of sockets for short and frequent messages. A separate thread will need to combine all messages for processing. One thing that I am looking at design patterns for, is separating the connection management from the data processing. I wish to have the connections to try reconnect after a disconnect and have the processing thread continue as if nothing happened. What design pattern is recommended here?

I dont see how async io will improve performance in my case.

Answer 1

You're on the right track. It's smart to ask "why" especially with all the hype around asynchronous and event driven. There are applications other than the web. Consider message queues and financial transactions like high frequency trading. Basically any time that waiting costs money or loses an opportunity to serve a customer is a candidate for async. The web is a big example because the network is so much faster than the database. As always, ask "does this make sense" for your app. Async adds a lot of complexity if you're not benefiting from it.

Your short, rapid messages may actually benefit a lot from async if the mean time between messages is comparable to the time required to process each message, especially if that processing includes persistence. But you don't have to rush into async. Instrument your blocking code and see whether you really have a bottleneck.

I hope this helps.

Answer 2

Using the blocking calls pattern would entail:

1. Listening on a socket vector of size N
2. When a message arrives, you wake up with a start in time K, find and start processing the message, employing a time T (it does not matter if the processing is offloaded to another thread: in this case T becomes your offloading time)
3. You finish examining the vector and GOTO 1

So you might say that if M messages arrive, and another message arrives during the K+M*T dispatching time, the M+1-th message will find itself waiting K+M*T time. Is this acceptable for your expected values of K (constant), M (function of traffic) and T (function of resources and system load)?

Asynchronous processing, well, actually does not exist. There will always be a "synchronous" IO loop somewhere, only it will be so well integrated in the kernel (or even hardware) that it will run 10-100x faster than your own, and therefore will be likely to scale better with large values of M. The delay is still in the form K1+M*T1, but this time K1 and T1 are much lower. Or maybe K1 is a bit higher and T1 is significantly lower: the architecture "scales better" for large values of M.

If your values of M are usually low, then the advantages of asynchronicity are proportionally smaller. In the absurd case when you only have one message in the application lifetime, synchronous or asynchronous makes next to no difference.

Take into account another factor: if the number of messages becomes really large, asynchronicity has its advantages; but if the messages themselves are independent (the changes caused by message A do not influence processing of message B), then you can remain synchronous and scale horizontally, preparing a number Z of "message concentrators" each receiving a fraction M/Z of the total traffic.

If the processing requires performing other calls to other services (cache, persistence, information retrieval, authentication...), increasing the T factor, then you'll be better off turning to multithreaded or even asynchronous. With multithreading you shave T to a fraction of its value (the dispatching time only). Asynchronous in a sense does the same, but shaving even more, and taking care of more programming boilerplate for you.