TcpClient's NetworkStream reads incomplete data unless Thread.Sleep(1) is called

Question

I have a method that reads some data through a TcpClient's NetworkStream, and it has been giving me some errors.

During investigation, I discovered that it actually worked fine... but only if I stepped through the code with Visual Studio 2012's debugger, using a breakpoint.

Here's my code:

public static byte[] DownloadStream(string hostname, int port, 
    byte[] requestBytes, int bufferSize = 4096)
{
    byte[] responseBytes = null;

    var client = new System.Net.Sockets.TcpClient(hostname, port);

    if (client.Connected)
    {
        using (var stream = client.GetStream())
        {
            stream.Write(requestBytes, 0, requestBytes.Length);
            stream.Flush();

            if (stream.CanRead)
            {
                var responseStream = new System.IO.MemoryStream();

                byte[] buffer = new byte[bufferSize];
                int bytesRead = 0;

                do
                {
                    bytesRead = stream.Read(buffer, 0, buffer.Length);

                    responseStream.Write(buffer, 0, bytesRead);
                }
                while (stream.DataAvailable);

                responseBytes = responseStream.ToArray();
            }
        }
    }

    client.Close();

    return responseBytes;
}

This is pretty frustrating, since there's no real error. It apparently just needs the debugger to hold its hand while it reads the NetworkStream.

Does anybody know why this is happening? How can I fix it?

---

Edit:

For some reason making this change eliminates the problem:

                do
                {
                    bytesRead = stream.Read(buffer, 0, buffer.Length);

                    responseStream.Write(buffer, 0, bytesRead);

                     System.Threading.Thread.Sleep(1); //added this line
                }
                while (stream.DataAvailable);

Any insight on this?

Answer 1

The NetworkStream.DataAvailable property is not a reliable way to detect the end of the response; if the response is split into multiple TCP packets and a packet has not yet been delivered at the moment you check DataAvailable, then the property will return false, terminating your loop prematurely.

Apparently, your network connection is fast enough that Thread.Sleep(1) provides sufficient time for each successive packet to arrive. On a slower connection, though, it might not be adequate.

For reliable communication, the client and the server need to agree on a way to signal the end of the response. For example:

• The server can send the length of the response as a prefix before the actual response. The client reads the length, and then it reads from the stream until it has received that number of bytes.
• The server can send a special terminator as a suffix after the actual response. The client reads from the stream until it encounters the terminator. (Obviously, the terminator cannot appear anywhere in the response itself.)
• The server can close the connection after it sends the entire response. The client reads from the stream until the connection is closed.