What are the underlying patterns in systems allowing asynchronous operations?

Question

Say I'm working with a system that allows async, nonblocking operations. If I queue up a set of those operations and specify their result buffer references:

nonblocking_write( message, write_buffer );
nonblocking_read( source, read_buffer );
nonblocking_read( other_source, other_read_buffer );

wait();

// do stuff with results now in the buffers

or queue up the same set of operations, but specify result callbacks to do stuff with results directly

nonblocking_write( message, write_handler );
nonblocking_read( source, read_handler );
nonblocking_read( other_source, other_read_handler );

// doing stuff happens in the result handlers, not here

My question is about the library or environment exposing the nonblocking_*() operations. What software structures and patterns allow this kind of thing to happen?

These are my thoughts so far:

• In a multithreaded environment, spawn a new thread within the nonblocking operation, then run an equivalent blocking operation within that thread.

• In a single threaded environment, define an event construct and tasks that handle 'events', then have nonblocking operations push event/task pairs into a queue for an event loop to run later. (I think this is how JavaScript does it.)

Answer 1

Since no one else has answered and I've now had a full day to remember all my old comp sci courses, I'll try writing a proper answer this time.

To the best of my knowledge, all asynchronous behavior in computers is at some level implemented by putting things in a queue and coming back to that queue later to process the things when it's more convenient. When you create a thread, you're pushing some thread-identifying information onto a queue that live inside the OS kernel. When you fire an event or send a message--in every language and framework I'm familiar with--you're pushing an event or message object into an event queue or a mailbox or whatever one wishes to call it.

Of course, the devil is in the (implementation) details, and there are a lot of details.

• With Javascript, what I just said is the whole story: the browser has a simple queue of events that it resolves one at a time in order. In particular, it assumes each event handler is a single function call that will simply run to completion without incident.

• With OS threads, you don't have functions to call, but tasks to execute for a set time frame. Thus you have a hardware timer interrupt that periodically forces the CPU to go execute the OS kernel's thread scheduling routine.

• Network sockets are a bit special because the "queue" is not really a queue, in that it's not FIFO or LIFO or any other definite ordering. Instead, the OS has to constantly poll your networking hardware to see if it has any new data, and when it does, it has to figure out who that data belongs to, because it has no way of knowing whose requests will finish first. The "queue" is probably more like a map from port numbers to execution contexts that need to be resumed when data matching that port number finally appears.

It should go without saying that these implementations have very different priorities with regards to stability, predictability, optimizeability, and interoperability.