Maybe you can invoke the ProcessRequest method on the IHttpHandler object directly and pass it your own http context with your own response object. I'm not sure but I think the BinaryWrite method of the http response uses the output stream of the TextWriter object that is passed into it's constructor. So if you have a memory stream set in the TextWriter then you could use it as your output.
So:
- Create MemoryStream
- Create StreamWriter and pass memory stream into the constructor
- Create a HttpResponse and pass it the created StreamWriter
- Create an HttpContext with the created HttpResponse and a created or current http request
- Invoke IHttpHandler.ProcessRequest