Packets looping forever in libnetfilter_queue

Question

I want to implement a network delay model for TCP/UDP traffic as described in Linux libnetfilter_queue delayed packet problem. I have followed the suggestion of Andy there, copying entire packet to my program and placing it an a priority queue. As time passes, packets in priority queue are removed and dispatched using RAW sockets.

The problem I am facing is this: Initial capture of packets by libnetfilter_queue is being done by matching the ports (sudo iptables -A OUTPUT -p udp --dport 8000 -j NFQUEUE --queue-num 0). When these packets are reinjected by RAW sockets, they are picked up once again by libnetfilter_queue (since the port remains the same) and hence continue to loop forever.

I am really confused and cannot think of a way out. Please help me.

Answer 1

Use skb->mark. It's a marking which only exists within the IP stack of your host. It does not affect anything in the network packet itself.

You can filter it using iptables using the '--mark' filter. Use it to return from your delay chain so that your re-inserted packets are not delayed again.

iptables -A DELAY -m mark --mark 0xE -j RETURN
iptables -A DELAY -j DELAY

You can configure the raw socket to apply a mark, using setsockopt(fd, SOL_SOCKET, SO_MARK, ...). You only need to do this once after opening the socket. The mark value will be automatically applied to each packet you send through the socket.

Answer 2

This probably isn't the best way to do it, but here is one possible solution. You could use the DSCP field in the IP header to differentiate new packets and packets you are re-injecting. Change your iptables rule to only enqueue packets with a DSCP of 0 (see http://www.frozentux.net/iptables-tutorial/iptables-tutorial.html#DSCPMATCH). This assumes when your OS sends a packet, it will set the DSCP to 0. Now all new packets generated by the OS will be sent to your program because they still match the iptables rule. When you are creating a new packet in your program using a RAW socket, set the DSCP value to a non-zero value. When your new packet is re-injected, it will no longer match the iptables rule and will go out over the network.

If you don't want packets going out over the network with DSCP values set, you could add another iptables rule to re-write the dscp values to 0.