How to throttle number of Tasks created?

Question

I'm using ODP.NET, which doesn't provide any asych methods like the SQL driver does or other Oracle drivers.

I have lots of slow queries, sometimes I need to call several of them on a single MVC controller call. So I'm trying to wrap them in Task calls. So I'm thinking of using this pattern:

(This is somewhat contrived example I wouldn't call the same query 10 times for real, it would be some heterogeneous workload)

List<Task<DbDataReader>> list = new List<Task<DbDataReader>>(10);
for (int i = 0; i < 10; i++)
{
    ERROR_LOG_PKG_DR package = new ERROR_LOG_PKG_DR();
    //Could be a bunch of different queries.
    //Returns a task that has the longrunningtask property set, so it's on it's own thread
    list.Add(package.QUERY_ALL_ASYNC());
}

Task.WaitAll(list.ToArray());

And here is the task:

public Task<DbDataReader> QUERY_ALL_ASYNC()
{
    CancellationToken ct = new CancellationToken();
    return Task.Factory.StartNew(_ =>
                                     {               
                                         System.Diagnostics.Debug.WriteLine("InTask: Thread: " +
                                                                            Thread.CurrentThread.ManagedThreadId);
                                         System.Diagnostics.Debug.WriteLine("InTask: Is Background: " +
                                                                            Thread.CurrentThread.IsBackground);
                                         System.Diagnostics.Debug.WriteLine("InTask: Is ThreadPool: " +
                                                          Thread.CurrentThread.IsThreadPoolThread);
                                         return QUERY_ALL<DbDataReader>();
                                     }, null, ct,TaskCreationOptions.LongRunning, TaskScheduler.Default);
}

This fires up 10 threads. What if I wanted some thread pool like behavior-- ie. only about 4 concurrent threads at a time-- re-use threads, etc, but I want it outside of the ASP.NET thread pool (which is being used to service requests)

How do I do that?

Answer 1

If you are happy to sacrifice your web app scalability, you could use SemaphoreSlim to throttle the number of parallel tasks:

const int MAX_PARALLEL_TASKS = 4;

DbDataReader GetData(CancellationToken token)
{
    DbDataReader reader = ... // execute the synchronous DB API
    return reader;
}

// this can be called form an async controller method
async Task ProcessAsync()
{
    // list of synchronous methods or lambdas to offload to thread pool
    var funcs = new Func<CancellationToken, DbDataReader>[] { 
        GetData, GetData2, ...  };

    Task<DbDataReader>[] tasks;

    using (var semaphore = new SemaphoreSlim(MAX_PARALLEL_TASKS))
    {
        tasks = funcs.Select(async(func) => 
        {
            await semaphore.WaitAsync();
            try
            {
                return await Task.Run(() => func(token));
            }
            finally
            {
                semaphore.Release();
            }
        }).ToArray();

        await Task.WhenAll(tasks);
    }

    // process the results, e.g: tasks[0].Result
}

Answer 2

I think this is what you need - (msdn) How to: Create a Task Scheduler That Limits Concurrency.

Shortly put idea is to create custom TaskScheduler that limits number of concurrently runninga tasks.

Answer 3

Why not just use ThreadPool?

https://msdn.microsoft.com/en-us/library/kbf0f1ct.aspx

This provides system-aware throttling and has been around for a very long time