large number of threads is wait state in threadpool causing performance problems

Question

My application connects to a large number of clients over http, downloads data from those clients and processes data as these results are received. Each request is sent in a separate thread so that the main thread does not remain occupied.

We have started encountering performance issues and it seems like these are mostly related to large number of threads in the ThreadPool that are just waiting for getting data back from those requests. I know with .NET 4.5 we have async and await for the same type of problem but we are still using .NET 3.5.

Any thoughts on what's the best way of sending these requests in a different thread but not to keep that thread alive while all its doing is to keep waiting for request to come back?

Answer 1

You can use async operations in .NET 3.5, it's just not as convenient as in .NET 4.5. Most IO methods have a BeginX/EndX method pair that is the async equivalent of the X method. This is called the Asynchronous Programming Model (APM).

For instance, instead of Stream.Read, you could use Stream.BeginRead and Stream.EndRead.

Actually, many async IO methods in .NET 4.5 are just wrappers around the Begin/End methods.

Answer 2

If you cannot use .NET 4.x and async/await, you still can achieve a sort of similar behavior using IEnumerator and yield. It allows to use pseudo-synchronous linear code flow with Begin/End-style callbacks, including statements like using, try/finally, while/for/foreach etc. You cannot use try/catch, though.

There are a few implementations of the asynchronous enumerator driver out there, e.g. Jeffrey Richter's AsyncEnumerator.

I used something like below in the past:

class AsyncIO
{
    void ReadFileAsync(string fileName)
    {
        AsyncOperationExt.Start(
            start => ReadFileAsyncHelper(fileName, start),
            result => Console.WriteLine("Result: " + result),
            error => Console.WriteLine("Error: " + error));
    }

    static IEnumerator<object> ReadFileAsyncHelper(string fileName, Action nextStep)
    {
        using (var stream = new FileStream(
            fileName, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize: 1024, useAsync: true))
        {
            IAsyncResult asyncResult = null;
            AsyncCallback asyncCallback = ar => { asyncResult = ar; nextStep(); };
            var buff = new byte[1024];
            while (true)
            {
                stream.BeginRead(buff, 0, buff.Length, asyncCallback, null);
                yield return Type.Missing;
                int readBytes = stream.EndRead(asyncResult);
                if (readBytes == 0)
                    break;
                // process the buff
            }
        }
        yield return true;
    }
}

// ...

// implement AsyncOperationExt.Start
public static class AsyncOperationExt
{
    public static void Start<TResult>(
        Func<Action, IEnumerator<TResult>> start,
        Action<TResult> oncomplete,
        Action<Exception> onerror)
    {
        IEnumerator<TResult> enumerator = null;

        Action nextStep = () =>
        {
            try
            {
                var current = enumerator.Current;
                if (!enumerator.MoveNext())
                    oncomplete(current);
            }
            catch (Exception ex)
            {
                onerror(ex);
            }
            enumerator.Dispose();
        };

        try
        {
            enumerator = start(nextStep);
        }
        catch (Exception ex)
        {
            onerror(ex);
            enumerator.Dispose();
        }
    }
}