Simulate Incorrect Content-Length Headers for HTTP in C#

Question

We are building a comprehensive integration test framework in C# for our application which exists on top of HTTP using IIS7 to host our applications.

As part of our integration tests we want to test incoming requests which will result in EndOfStreamExceptions ("Unable to read beyond end of stream") that occur when a client sends up a HTTP header indicating a larger body size than it actually transmits as part of the body. We want to test our error recovery code for this condition so we need to simulate these sorts of requests.

I am looking for a .NET Fx-based socket library or custom HttpWebRequest replacement that specifically allows developers to simulate such conditions to add to our integration test suite. Does anyone know of any such libraries? A scriptable solution would work as well.

Answer 1

Set the ContentLength property prior to calling GetRequestStream (or BeginGetRequestStream) and then write fewer bytes to that stream. ContentLength will throw if you try to set it after you've gotten the request stream. If you don't set ContentLength, HttpWebRequest will buffer the headers until the stream is closed so that it can set ContentLength appropriately (alternately, you can use SendChunked but that won't work for you here). If you want maximum control over this, create a malormed request or two by hand and then open a socket to port 80 on the server and write the request to the TCP stream and then read the response and check the connection to see if it was closed.

HOWEVER: I don't think that this test is a good idea. Here's the problem:

Client sends a request to the server. It claims the content will be 100 bytes. It then sends 90 bytes and then just stops sending, leaving the connection open. The server reads 90 bytes, then waits for the rest since the client said that 100 bytes would be sent. Now, the Client sends a second request. What will the server do with the first 10 bytes of the new request?

The answer is that the server will assume that those bytes were part of the previous request and treat them as such, then start reading the "new" request 10 bytes after its start, which will obviously result in malformed headers. The server won't like that, so it'll send a 4xx error and then it will close the connection. The reason it closes the connection is because it now has no way to know what the data being sent to it means and no way to recover. Also, the closing of the connection will not be graceful, it'll be sudden and the HttpWebRequest on the other end that is submitting a second request (or a third or fourth if they're queued up) will throw a WebException saying that the underlying connection was closed and leave you guessing as to why.

This same behavior is what causes the connection to be closed with the Expect 100-continue header and the server returns a 100-continue followed by a 4xx such as when Auth is required. Even though it's rejected the request, it still MUST assume that the next bytes are part of the same request since it has committed to receiving those bytes by sending the 100-continue. If it won't service that request or if the client wants to cancel the request and submit a new one (presumably with auth credentials) then it has to close the connection and open a new one.

Finally, testing for an EndOfStreamException from a network stream using TCP doesn't make any sense to me at all. TCP doesn't mark the "end" of a stream, it just keeps sending data as it is written to the socket. There is no "EOF" for it and no way to detect if the data was all transmitted unless you know how much data to expect. TCP doesn't really have an "end" to its stream unless the connection is closed, in which case you'll get a WebException or a SocketException depending on where in the stack you're operating. Any transmission errors in the protocol are dealt with in winsock. If no more data is sent, one end of the connection will eventually send a keepalive to the other to ensure that the connection is still actually open and one host didn't just drop it. If the keepalive times out, the connection will be closed and you may get a WebException the next time you try to read from it. Given how all of this works, I just don't see how you're going to get any value out of this test. You're much better off just sending malformed requests and ensuring that the errors are processed accordingly and the appropriate 4xx message is being sent back to the client and the connection is being closed properly.

Answer 2

I don't know of any such library, but it seems to me it would be very simple on the receiving-side, if that is where you are coding.

Simply retrieve the entire WebRequest, truncate as much or as little as you want, and then forward it on. You might need to make a new copy of the WebRequest in the process, rather than truncate in-place, but it should still be straight-forward.

Answer 3

Can't you just set the HttpWebRequest.ContentLength property on the clients to a smaller or larger value than the actual size of the data?

Answer 4

If you want to simulate any things like this, where you are violating the HTTP protocol, then you will have to write your own code to do it. HttpWebRequest does not allow you to do things like this.