Unable to set speed limit using tc tbf

Question

I am using tc to do QOS on an asterisk server. I want to prioritize voice and sip traffic but also cap all the rest to a fixed limit.

Here is my script:

#!/bin/bash
IFACE=eth1
UPSPEED=1.5mbit
tc qdisc del dev $IFACE root
tc qdisc add dev $IFACE root handle 1: prio priomap 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 0
tc qdisc add dev $IFACE parent 1:1 handle 10: sfq perturb 10
tc qdisc add dev $IFACE parent 1:2 handle 20: sfq perturb 10
tc qdisc add dev $IFACE parent 1:3 handle 30: tbf rate $UPSPEED burst 4kb mtu 1500   latency 100ms
tc filter add dev $IFACE protocol ip parent 1: prio 1 u32 match ip tos 0xb8 0xff flowid 1:1
tc filter add dev $IFACE protocol ip parent 1: prio 1 u32 match ip tos 0x60 0xff flowid 1:2

RTP and SIP traffic is well managed, being sent to the first and the second band. All other traffic is also well managed, being sent to the third band. However, for some reasons, If I download from the server, it is always at 10-16k/sec instead of the 185-190k (1.5mbit) specified in my script.

Worst, it seems that no matter how I change the tbf variables, the speed stays the same.

Using qdisc -s ls, I managed to find out that packets were dropped :

qdisc prio 1: bands 3 priomap  2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 0
Sent 12610974 bytes 45683 pkt (dropped 1147, overlimits 0 requeues 0) 
rate 0bit 0pps backlog 0b 0p requeues 0 
qdisc sfq 10: parent 1:1 limit 126p quantum 1514b perturb 10sec 
Sent 7802180 bytes 36590 pkt (dropped 0, overlimits 0 requeues 0) 
rate 0bit 0pps backlog 0b 0p requeues 0 
qdisc sfq 20: parent 1:2 limit 126p quantum 1514b perturb 10sec 
Sent 181620 bytes 283 pkt (dropped 0, overlimits 0 requeues 0) 
rate 0bit 0pps backlog 0b 0p requeues 0 
qdisc tbf 30: parent 1:3 rate 1500Kbit burst 4Kb lat 100.0ms 
Sent 4627174 bytes 8810 pkt (dropped 1147, overlimits 0 requeues 0) 
rate 0bit 0pps backlog 0b 0p requeues 0

But I don't know why. Again, changed the tbf variables doesn't change anything, packets keep being dropped.

Please note that the mtu of eth1 is 1500.

Anyone?

Answer 1

Add mtu 100000 to your tc qdisc creation command. See this post for more info.

Quoting the post, in case it disappears:

Basically if your interface has TSO/GSO enabled (check using ethtool -k ethX), or you're using the loopback interface - then you'll probably hit a problem. It turns out that the loopback interface has GSO/TSO enabled as default, plus since it is a software interface its default mtu is 16384 (as compared to 1500 for normal Ethernet interface). This matters as the tbf queue checks the size of the incoming 'packets' - which in the case of GSO/TSO are much larger than a normal on-the-wire packet - instead they're up to 9 x iface's mtu. So for normal interfaces it's about 12K, but for loopback it is about 100k!

This appears to apply to Xen interfaces, too:

# ethtool -k eth0
Features for eth0:
rx-checksumming: on [fixed]
tx-checksumming: on
    tx-checksum-ipv4: on
    tx-checksum-unneeded: off [fixed]
    tx-checksum-ip-generic: off [fixed]
    tx-checksum-ipv6: off [fixed]
    tx-checksum-fcoe-crc: off [fixed]
    tx-checksum-sctp: off [fixed]
scatter-gather: on
    tx-scatter-gather: on
    tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on     # THIS IS TSO
    tx-tcp-segmentation: on
    tx-tcp-ecn-segmentation: off [fixed]
    tx-tcp6-segmentation: off [fixed]
udp-fragmentation-offload: off [fixed] 
generic-segmentation-offload: on     # THIS IS GSO
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: off [fixed]
tx-vlan-offload: off [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: off [fixed]
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: on [fixed]
tx-fcoe-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: on
loopback: off [fixed]

Answer 2

I had the same issue. Turning off segmentation offloading will solve the problem (and you won't need to pass mtu to tc anymore).

sudo ethtool -K eth1 tso off
sudo ethtool -K eth1 gso off
sudo ethtool -K eth1 gro off
sudo ethtool --offload  eth1  rx off  tx off