Best way to indicate that all data has been received over the network?

Question

I've got a client / server application that works pretty well, but it's missing one crucial piece of behavior to make it a bit more solid.

Right now, it's far from "strong" in terms of network capabilities. I'm trying to get it there, and research has lead me to believe that I need some sort of protocol in place to ensure that no data is lost during network transmissions.

I've heard of a few methods. One that I think will work best for our situations is to use a terminator, something like an <EOF> tag. My issue is that I'm not sure of the best way to implement this.

Here's a couple code snippets that I'll be modifying to include a terminator after figuring out the best solution.

Client:

TcpClient client = new TcpClient();

client.Connect(hostname, portNo);

using (var stream = client.GetStream())
{
    //send request

    stream.Write(data, 0, data.Length);
    stream.Flush();

    //read server response

    if (stream.CanRead)
    {
        byte[] buffer = new byte[1024];
        string response = "";
        int bytesRead = 0;

        do
        {
            bytesRead = stream.Read(buffer, 0, buffer.Length);

            response += Encoding.ASCII.GetString(buffer, 0, bytesRead);

        } //trying to replace 'DataAvailable', it doesn't work well
        while (stream.DataAvailable); 
    }
}

Note that I'm trying to replace the stream.DataAvailable method of checking for more data in the stream. It's been causing problems.

Server:

var listener = new TcpListener(IPAddress.Any, portNo);

listener.Start();

var client = listener.AcceptTcpClient();

using (var stream = client.GetStream())
{
    var ms = new System.IO.MemoryStream();

    byte[] buffer = new byte[4096];
    int bytesRead = 0;

    do
    {
        bytesRead = stream.Read(buffer, 0, buffer.Length);

        ms.Write(buffer, 0, bytesRead);

    } //also trying to replace this 'stream.DataAvailable'
    while (stream.DataAvailable);

    ms.Position = 0;

    string requestString = Encoding.UTF8.GetString(ms.ToArray());

    ms.Position = 0;

    /*
        process request and create 'response'
    */

    byte[] responseBytes = Encoding.UTF8.GetBytes(response);

    stream.Write(responseBytes, 0, responseBytes.Length);
    stream.Flush();
}

So, given these two code examples, how can I modify these to both include and check for some sort of data terminator that indicates it's safe to stop reading data?

Answer 1

We ended up using an EOS (end of stream) indicator on both ends of our project. I won't post the full code example, but here's a small snippet of how it works:

stream.Write(data, 0, data.Length);
stream.WriteByte(Convert.ToByte(ConsoleKey.Escape));
stream.Flush();

On the receiving end of this stream, the loop reads data byte-by-byte. Once it receives the ConsoleKey.Escape byte, it terminates the loop. It works!

Answer 2

You can rely on TCP transmitting all the data before the FIN. The problem with your code is that available() is not a valid test for end of stream. It is a test for data being available now.

So you are terminating your reading loop prematurely, and thus missing data at the receiver, and getting resets at the sender.

You should just block in the Read() method until you receive the EOS indication from the API, whatever that is in C#.

You don't need your own additional EOS indicator.