Efficient forwarding HTTP requests with an IHttpAsyncHandler

Question

I'm developing a HTTP front controller, based on the pattern of Martin Fowler (link). In my case the controller has the following responsibilities: - Unmarshall encapsulated data - Authorize the request - Logging - Relay / forward the request to another server

The following to possible solutions came to mind: - (Synchronous) IHttpHandler, forwarding the request with the WebClient or the HttpWebRequest class - (Asynchronous) IHttpListener (non-IIS solution) - (Asynchronous) IHttpAsyncHandler

The ideal situation being that the FC can handle a lots of concurrent requests (>10000 TPS) without burning the CPU.

To test the solutions I've created a small framework that has 3 clients making requests, a front controller that sits in the middle and 2 servers that respond to the requests passed by the FC. The framework benchmarks 3 scenario's, firstly it tests with fast responses with small payloads, secondly: fast responses with big payloads (> 100KB) and lastly its tests with slow responses (>3 seconds) and small payloads.

The transactions per seconds (TPS) drops to an ultimate low (<25 TPS) with the last test with the synchronous HTTP handlers. My guess would be this is due the handler blocks the thread when is waiting for a response. To overcome this problem I've started to implement an asynchronous handler (see code below). The problem is, it simple doesn't work.

The last (untested) solution that came to mind is to use the HttpListener class. Guessing that this is more lightweight solution with a fine grain control for concurrency. I've seen a example implementation using the RX-framework by José F. Romaniello (link).

My question would be this, why doesn't handler code work?, is this the most efficient way to do so? or should I be in favor of the HttpListener solution.

Asynchronous HTTP-handler code:

public class ForwardRequestHandler : IHttpAsyncHandler
{
    public IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData)
    {
        var uri = GetForwardUriFor(context.Request.Url.PathAndQuery);

        var proxy = HttpWebRequest.Create(uri) as HttpWebRequest;
        return proxy.BeginGetResponse(new AsyncCallback(EndProcessRequest), new ForwardedRequestContext(context, proxy));
    }

    public void EndProcessRequest(IAsyncResult result)
    {
        var proxy = result.AsyncState as ForwardedRequestContext;
        proxy.TransferResponse(result);
    }

    public bool IsReusable
    {
        get { return true; }
    }

    public void ProcessRequest(HttpContext context)
    {
        throw new NotSupportedException();
    }

    private Uri GetForwardUriFor(string path)
    {
        var loadbalancer = new RoundRobinLoadBalancer();
        var endpoint = loadbalancer.GetRandomEndPoint();

        return new Uri(
            string.Format("http://{0}{1}", endpoint, path)
        );
    }
}

public class ForwardedRequestContext
{
    private readonly HttpContext context;
    private readonly HttpWebRequest forwarder;

    public ForwardedRequestContext(HttpContext context, HttpWebRequest forwarder)
    {
        this.context = context;
        this.forwarder = forwarder;
    }

    public void TransferResponse(IAsyncResult ar)
    {
        var response = GetResponse();

        var result = forwarder.EndGetResponse(ar);
        response.StatusCode = 200;
        response.ContentType = result.ContentType;
        response.AddHeader("Content-Length", result.ContentLength.ToString());

        result.GetResponseStream().CopyTo(response.OutputStream);
        response.Flush();

        result.Close();
    }

    private HttpResponse GetResponse()
    {
        return context.Response;
    }
}

Answer 1

One possible solution to this might be to use ARR to do the forwarding for you.

There is an example of doing this on Azure here:

https://github.com/richorama/AzureARR

Answer 2

in return proxy.BeginGetResponse(new AsyncCallback(EndProcessRequest), ... you create a new callback, and ASP.net doesn't get informed about the end of the request therefore it can't stop processing thre request (this it at least what I have observed by many concurrent requests). So use the callback passed in as argument instead.

return proxy.BeginGetResponse(cb, ...

Cheers