recv receiving not whole data sometime

Question

I have following issue: here is the chunk of code:

void get_all_buf(int sock, std::string & inStr) {
    int n = 1;
    char c;
    char temp[1024*1024]; 

    bzero(temp, sizeof(temp));

    n = recv(sock, temp, sizeof(temp), 0);

    inStr = temp;
};

but sometimes recv returning not whole data (data length always less then sizeof(temp)), only its part. Write side always sends me whole data (I got it with sniffer). What matter? Thx.

P.S. I know, good manner suggests me to check n (if (n < 0) perror ("error while receiving data")), but it doesn't matter now - it's not reason of my problem.

P.S.2 I've forgot - it's blocking socket.

Answer 1

The TCP standard allows for fragmentation of data packets. In practice this doesn't happen with small data packets of a few hundred bytes or so, but a megabyte of data is almost certain to get fragmented.

Secondly, when you say the sniffer says all the data gets sent, in one packet or in many?

Good network programming practice requires you to not assume that messages arrive in singular chunks. Two sequential messages can arrive as one packet (in theory but almost never in practice) and even if they arrive in multiple packets can be read as a single read. One message can get fragmented into multiple packets and they might not all arrive at once which is probably what you are seeing.

Your program should buffer all its reads and have a mechanism to determine when a whole message has arrived, either via a delimiter (e.g. HTTP headers which are delimited with CRLFCRLF) or by a byte count (e.g. HTTP bodies where the length is specified in the header) or by closing the connection to indicate the end of the data (e.g. HTTP bodies when the content length isn't specified in the header). There may be other mechanisms too.

Answer 2

A much better way is to use following:

void get_all_buf(int sock, std::string & output) {
    char buffer[1024];

    int n;
    while((errno = 0, (n = recv(sock, buffer, sizeof(buffer), 0))>0) || 
          errno == EINTR)
    {
        if(n>0)
            output.append(buffer, n);
    } 

    if(n < 0){
        /* handle error - for example throw an exception*/
    }
};

Also note that the buffer allocated on the stack is much smaller. Having 1M buffer on stack may cause stack overflow.

Additional note: You probably don't want to read until the socket is closed, so you may need to add another termination condition to the while loop.

Answer 3

TCP works as a layer on top of other layers: IP and Ethernet. IP allows data fragmentation, and Ethernet allows some data to get lost over the wire. That leads to data loss, and it's reflected on your calls to recv.

When you call recv, the underlaying operating system will try to read as much data as it can up to the size you specified, but might return the call having read less bytes, even one single byte.

You need to create some protocol of your own to keep reading data up to finishing your data piece.

For example, you can use "\n" as a delimiter. This code can be improved, but I hope will get you the idea:

void get_all_buf(int sock, std::string & inStr) {
    int n = 1, total = 0, found = 0;
    char c;
    char temp[1024*1024]; 

    // Keep reading up to a '\n'

    while (!found) {
        n = recv(sock, &temp[total], sizeof(temp) - total - 1, 0);
        if (n == -1) {
            /* Error, check 'errno' for more details */
            break;
        }
        total += n;
        temp[total] = '\0';
        found = (strchr(temp, '\n') != 0);
    }

    inStr = temp;
}