Multiple sendto() using UDP socket

Question

I have a network software which uses UDP to communicate with other instances of the same program. For different reasons, I must use UDP here.

I recently had problems sending huge ammounts of data over UDP and had to implement a fragmentation system to split my messages into small data chunks. So far, it worked well but I now encounter an issue when I have to send a lot of data chunks.

I have the following algorithm:

1. Split message into small data chunks (around 1500 bytes)
2. Iterate over the data chunks list and for each, send it using sendto()

However, when I send a lot of data chunks, the receiver only gets the first 6 messages. Sometimes it misses the sixth and receives the seventh. It depends.

Anyway, sendto() always indicates success. This always happen when I test my software over a loopback interface (127.0.0.1) but never over my LAN network.

If I add something like std::cout << "test" << std::endl; between the sendto() then every frame is received.

I am aware that UDP allows packet loss and that my frames might be loss for a lot of reasons and I suppose it has to do with the rate I am sending the data chunks at.

What would be the right approach here ?

• Implementing some acknowledgement mechanism (just like TCP) seems overkill.
• Adding some arbitrary waiting time between the sendto() is ugly and will probably decrease performance.
• Increasing (if possible) the receiver UDP internal buffer ? I don't even know if this is possible.
• Something else ?

I really need your advices here.

Thank very much.

Additional information as requested

The reason I must use UDP is because I have several constraints:

1. TCP doesn't work well with NAT traversal (at least without a specific configuration)
2. Some messages can be lost. Some others can't.
3. The message delivery order doesn't matter.

Answer 1

Implementing an acknowledgement mechanism sounds like exactly what you need to do. That way you can ensure that no more than N packets are "in-flight" at once, and you can retransmit packets that have gone unacknowledged for too long.

Answer 2

If you are losing packets over the loopback interface after sending only 6 or 7 packets, then it sounds like maybe your receive buffer is too small. You can increase the size with setsockopt using the SO_RCVBUF option. However, if you are sending 1500 bytes, then if this is indeed the issue, it means the receive buffer is only about 9K (or more likely 8K, however that seems a rather small default). I believe on Windows the default receive buffer is 16K.

Even assuming increasing the receive buffer helps, you still have to address the issues others have mentioned. A couple other things to consider is maybe to try to dynamically determine the max packet size to avoid fragmentation. Also, it may make sense to make the packet size and number of packets sent between acks configurable manually.

Answer 3

Calling UDP unreliable is a simplification which tries to put TCP as a panacea of all network ills. In the same vein defining TCP as reliable is again erroneous. While it is true TCP has mechanisms to attempt to ensure data is transferred, many of the faults that cause the failure of a UDP packet to arrive will also cause TCP to fail.

For example a hardware network fault will have the same affect on UDP and TCP packets. If the fault persists then TCP will not get through as surely as UDP. In fact in this case TCP has the disadvantage in that it will attempt to complete a lost cause for longer. Now if you are sending data over the internet, the TCP has some advantages because the route the packet is sent cannot be predefined. However for sending data over a LAN, UDP is perfectly adequate. If your packets are not getting to the destination, then it indicates a hardware fault that needs to be corrected. TCP will not help here.

Also when choosing your protocol you must also understand your data. If your data is transient, for example a reading from a sensor, it makes far more sense to use UDP over TCP. If a packet is lost in this situation then it is of little consequence since another packet will be along shortly. TCP on the other hand will back off and retry. By the time the data arrives it will already be out of date.

The truth is that TCP was designed for streaming data. In this situation it is important that all data packets arrive reliable and in order. UDP is for packet data and is for this type of data UDP is perfectly acceptable, as reliable, has less overhead and quicker to detect and recover from network faults.

Answer 4

You should implement acknowledge, and retransmission. Require e.g. acks for every N packets, and keep N packets in your retransmission buffer.

(maybe you can get some ideas from rudp, or implement Il over UDP )

UDP is unreliable, and also provides no flow control. The short story is, you will lose packets every now and then - especially if you send data fast - the kernel or any routers/switches in between will drop packets if there's not enough space - and there's little magic you can employ to not make that occur.

Answer 5

TCP exists to solve exactly this kind of problem. Why isn't TCP an option? You are going to have to solve all the same problems and ultimately end up with the same solution, only without the benefit of decades of research, development and debugging of the TCP stack.

If you really must use UDP, the first question to ask is, what are you willing to give up with respect to TCP's guarantees? Are you happy to receive packets out of order? Is it OK to lose some percentage of packets? Can you handle the arrival of duplicate packets? The answers to these questions will hopefully lead to a design.

Without knowing your specifics, it is impossible to answer your question with a simple, "Do this and you'll be fine," except, of course, "Do TCP and you'll be fine".

Answer 6

UDP transmits datagrams, and is unreliable.

TCP transmits data streams, and is reliable.

What you want appears to be datagram based, but reliable. So you need to build something onto either UDP or TCP to give you that. I am sure there are complete protocol specs building onto UDP that provide just that. You only have to find and implement them.

Answer 7

Anyway, sendto() always indicates success. This always happen when I test my software over a loopback interface (127.0.0.1) but never over my LAN network.

If I add something like std::cout << "test" << std::endl; between the sendto() then every frame is received.

That sounds like your receiver buffer is too small.

some advice:

1. Increase the receiver buffer. setsockopt SO_RCVBUF
2. Let the application decide whether retransmission the lose package.
3. Ensure the payload fit the MTU, which means payload + HEADER <= 1500, in order to avoid ip fragmentation.