Networkstream.ReadAsync behaviour. Checksums not matching

Question

I'm encountering a pretty difficult problem in an asynchronous RSS feed aggregator application I am creating.

The system is built from a communicating client and server. The server collects RSS feed items from different sources and then distributes them to clients based on their subscriptions.

The issue that I am having is - My specification states that I must implement a pre-defined byte based protocol to communicate between the client and server. Part of this protocol is that when the server sends a payload to the client, that payload must have a checksum created and sent with it, then when it is received by the client the checksum is equated based on the received payload. We then compare the two checksums and if they do not match then we have an issue.

The thing that is baffling me is that, sometimes the checksums will match perfectly and other times they will not, using the same algorithm for all sending and receiving operations.

Here is the method that checks incoming ->

        private bool CheckChecksum(uint receivedChecksum, IEnumerable<byte> bytes) {
        uint ourChecksum = 0;
        foreach (var b in bytes)
            ourChecksum += b;
        if (ourChecksum != receivedChecksum) {
            Debug.WriteLine("received {0}, calculated {1}", receivedChecksum, ourChecksum);
            _writeOutToFile = true;
        }
        return receivedChecksum == ourChecksum;
    }

and when calculating it before sending ->

        uint checksum = payloadBytes.Aggregate<byte, uint>(0,
            (currentChecksum, currentByte) => currentChecksum + currentByte);

Since the behaviour seems to occur on updates that have very large checksums (generally 2 million+) then the only thing I can think that would be causing it is these large byte[] sizes.

As you can see above, I write out to a file the contents of the payload when the checksums do not match. What I found was that the byte[] just ends early (despite the fact that the reading/writing lengths match on both the client and server). The end of the byte[] is just filled with empty spaces.

I am using NetworkStream.ReadAsync and WriteAsync to do all of my I/O operations. I thought that it may be a fault with the way I am using or understanding these.

I realise that this is a very difficult and vague problem (because I'm not sure what is going wrong myself) but if you need any further information I will provide it.

Here is some extra information:

• The checksums are of type uint, and their endianness is encoded correctly at both ends of the system.

• Payloads are strings which are encoded into ASCII bytes and decoded on the client-side.

• All messages are sent with a checksum and a payload length. The payload length represents how many bytes to read off the stream. Since the I read payload length amount of bytes from the networkstream, when the checksums do not match the payload becomes white space after a varying length (but is correct before this point).

• Sometimes (rarely) the checksums will match even when they are large.

• I am running both the client and server locally on one machine.

• I have tried having a Thread.Sleep(5000) before the read and this makes no difference.

Here is a sample of sent and received data that fails.

Sent from server - http://pastebin.com/jvbCbQmJ

Received by client - http://pastebin.com/eNkWymwi

Answer 1

Your receiving code tries to read before all the bytes have been posted.

When you try to send huge chunks of data on only one package, your readAsync will detect there's something to read, try to read for full lenght, which may not be available, and fill the part that was not posted yet with 0s

You can either divide your message on the server, read by parts on the client, or try to read what's available until you have received everything or some time has passed