Where do asynchronous operations wait when they are not being dealt with?

Question

If I start a group of asynchronous operations in C# (Tasks), each downloading some content off the web and then I process the first available Task result using Task.WhenAny(), where do the other async operations "wait" while the main thread processes the results?

I can only assume that there's some sort of internal queue at the OS level which stores state and also the results of the downloads occurring in the background.

My question is, where is this queue located and are there any dangers of this queue overflowing with the results of the async operations which have not yet been handled?

Answer 1

TPL Task Schedulers keep track of those tasks started via Task.Run, Task.Factory.StartNew, Task.ContinueWith, Task.Run or Task.RunSynchronously.

For promise-style tasks (those created with TaskCompletionSource), the references are kept by I/O completion callbacks or event handlers. Stephen Cleary has a great blog post relevant to this category of tasks.

For compiler-generated state machine tasks (those returned by an async method with await statements inside), the task is kept alive for as long as any "inner" task (or a custom awaiter) it awaits is "in-flight". In this case, the continuation callback is kept by the task awaiter (e.g., TaskAwaiter). This compiler-generated callback holds a strong indirect reference to the ambient ("outer") task. When the "inner" task completes, the callback will be scheduled via either SynchronizationContext.Post or TaskScheduler.Current task scheduler (if there was no synchronization context captured at the point of await).

In case with custom awaiters, one may need to put a strong hold onto the await continuation callback passed to INotifyCompletion.OnCompleted, to keep the ambient task from being garbage-collected, while "in-flight".

My question is, where is this queue located and are there any dangers of this queue overflowing with the results of the async operations which have not yet been handled?

If tasks are queued faster than they get completed, you may eventually run out of memory, in the long run. This is a common problem The Queuing Theory deals with, it's not something specific to the TPL tasks.