How to limit Parallel.foreach for asynchronous operations?

Question

I'm writing a tool that sends queries to an azure table, the amount of queries depends on the user. I want to send queries in parallel but only up to a given number ( i don't want to send all 100 queries at once). Is there any built in mechanism i can use to sent say up to 20 queries in parallel each time ?

I know there is Parallel.Foreach which can be limited using ParallelOptions.MaxDegreeOfParallelism but for asynchronous operation like mine this will just send all the queries really fast and my tool will handle all 100 callbacks at once.

Answer 1

You should use SemaphoreSlim. It's especially nice in the case of async operations because it has WaitAsync that returns a task you can await on. The first 20 will go right through, and the rest will asynchronously wait for an operation to end so they can start.

SemaphoreSlim _semaphore = new SemaphoreSlim(20);

async Task DoSomethingAsync()
{
    await _semaphore.WaitAsync();
    try
    {
        // possibly async operations limited to 20
    }
    finally
    {
        _semaphore.Release();
    }
}

Usage:

for(int i=0; i < 100; i++)
{
    await DoSomethingAsync();
}