sendto() returns values greater than MTU on SOCK_DGRAM (UDP) socket

Question

I'm currently writing a UDP client/server (on GNU/Linux). I'm using sendto() for sending messages on a SOCK_DGRAM socket that has not been bound to a port.

The send(2) manpage states that:

On success, these calls return the number of characters sent. On error, -1 is returned, and errno is set appropriately.

However, sendto always returns its length parameter, indicating success. With messages greater than 65507 (0xFFE3) bytes, it returns a Message too long error.

For messages greater than the MTU of 1500 bytes, the server always receives (via recvfrom()) messages of exactly 1500 bytes, simply cutting the message, without further notice.

(Why) is this behaviour intended, and is there a way of getting notified that something went wrong?

The only workaround I can currently think of would be to simply assume a MTU of 1500 bytes and never send bigger packets.

This is the relevant method:

int udp_send(uint32_t dst, uint16_t port, char *msg, unsigned len) {
    int sock = socket(AF_INET, SOCK_DGRAM, 0);
    if(sock < 0) {
            perror("Could not open socket");
            return -1;
    }

    struct sockaddr_in addr;
    addr.sin_family = AF_INET;
    addr.sin_port = htons(port);
    addr.sin_addr.s_addr = htonl(dst);

    int count = sendto(sock, msg, len, 0,
            (struct sockaddr *) &addr, sizeof(addr));
    printf("bytes sent: %d\n", count);

    if(count < 0) {
            perror("Could not send message");
            return -3;
    }

    close(sock);
    return count;
}

A call where the described undesired behaviour occurs would be udp_send(0x7F000001, 1337, bigbuf, 1501); which would return 1501 while sending only 1500 bytes.

Answer 1

It turns out that, as suggested by nos in his comment, my sending code was actually perfectly fine; my receive code had a simple bug: At some point, I defined its buffer to be just 1500 Bytes in size, and forgot about it later.

Furthermore, I was led by the misconception that GNU/Linux can send UDP packets only to the interface's MTU, which is not true. Even though the standard only guarantess 576 Bytes, at least in my case it seems possible to send (and receive) UDP packets up to 65507 bytes, after which the Message too long error is returned by the sendto() call. The UDP packets are automatically fragmented by the Kernel's IP layer, and re-assembled by the receiver - I did not consider that UDP could be capable of such advanced techniques.

Answer 2

UDP is not connection oriented, you can never say that two packets belong to each other, you can use getsockopt() to determine MTU and stay under it, preparing packets by hand with proper offsets.

udp packet fragmentation for raw sockets has some explanation to it.

On the other hand you can try and play with MTU discovery. Basically while sending UDP packets larger than MTU, IP fragmentation can occur if it's supported on your platform but not all platforms support it. Some network equipment inbetween will discard fragmented packets, some will pass them through, you have no guarantee of reassembly upon delivery.

Here is a bit of background info: http://michael.toren.net/mirrors/sock-faq/unix-socket-faq-5.html

Answer 3

From the Linux udp(7) man page:

By default, Linux UDP does path MTU (Maximum Transmission Unit) discovery. This means the kernel will keep track of the MTU to a specific target IP address and return EMSGSIZE when a UDP packet write exceeds it. When this happens, the application should decrease the packet size. Path MTU discovery can be also turned off using the IP_MTU_DISCOVER socket option or the /proc/sys/net/ipv4/ip_no_pmtu_disc file; see ip(7) for details. When turned off, UDP will fragment outgoing UDP packets that exceed the interface MTU. However, disabling it is not recommended for performance and reliability reasons.

So this implies that any sendto with a size larger than the MTU (~1500 bytes for you?) should return EMSGSIZE. If that's not what you are seeing, I'm not at all sure what is happenning.

Answer 4

(Why) is this behaviour intended, and is there a way of getting notified that something went wrong?

--> UDP is like that(connectionless, unreliable, datagram protocol), you need to check for every error when transmetting data using UDP

The only workaround I can currently think of would be to simply assume a MTU of 1500 bytes

--> In our days 99% of traffic is sent via Ethernet, and on Ethernet, MTU is 1500 bytes

A call where the described undesired behaviour occurs would be udp_send(0x7F000001, 1337, bigbuf, 1501); which would return 1501 while sending only 1500 bytes.

--> By default, Linux UDP does path MTU discovery. This means the kernel will keep track of the MTU to a specific target IP address and return EMSGSIZE when a UDP packet write exceeds it.

This is the default behavior inside the krenel, you can modify this and got framgmented packets (for performanes it's a bad idea). More on that here: http://www.kernel.org/doc/man-pages/online/pages/man7/udp.7.html