[Bloat] Inaccurate rates with HTB/HFSC+fq_codel on router due to VLAN?

[Eric Dumazet]

On Thu, 2017-03-16 at 22:14 +0000, xnor wrote:
> Hello,
> 
> I have a tl-wdr3600 router with openwrt (linux 3.18, config at [1]) and 
> want to shape ingress traffic.
> 
> The WAN port, interface eth0.2, is connected to a cable modem.
> I use the following script:
> 
> tc qdisc add dev eth0.2 handle ffff: ingress
> ifconfig ifb0 up txqueuelen 1000
> tc qdisc add dev ifb0 root        handle 1: htb default 10
> tc class add dev ifb0 parent 1: classid 1:1 htb rate 18000kbit
> tc class add dev ifb0 parent 1:1 classid 1:10 htb rate 18000kbit ceil 
> 18000kbit
> tc filter add dev eth0.2 parent ffff: protocol all prio 1 u32 match u32 
> 0 0 action mirred egress redirect dev ifb0
> tc qdisc add dev ifb0 parent 1:10 handle 100: fq_codel limit 1000 
> quantum 1514 noecn
> 
> Now the problem is that it is very inaccurate, and this inaccuracy 
> increases with number of TCP download connections.
> With HTB (it's worse with HFSC) and 20 connections (rates calculated 
> with /sys/class/net/eth0.2/statistics/rx_bytes over 5 seconds each):
> 19012.610 kbit
> 19266.877 kbit
> 19303.923 kbit

Sure, but you shape on ifb0, so the rate you observe on the ingress
interface is before any rate limit...

This could be 77.77 Mbit, and the tc scripts have no say about that.


> 
> With a single connection and HTB, I only get about 17.2 Mbit. Curiously, 
> /sys/class/net/ifb0/statistics show about 18.1 +/- 0.1 Mbit
> 
> So I did a tcpdump of eth0.2 and ifb0:
> eth0.2's largest frames are 1514 bytes ethernet type IP.
> ifb0's are 1518 bytes (including the VLAN tag I guess) with some unknown 
> type.
> 
> 
> What do I need to change to make shaping with ifb accurate? Ideally with 
> both HTB and HFSC.
> 

Have you checked your syslog ?

It seems you would need 'quantum 8000' or something like that.

tc class add dev ifb0 parent 1: classid 1:1 htb rate 18000kbit \
     quantum 8000
tc class add dev ifb0 parent 1:1 classid 1:10 htb rate 18000kbit \
     ceil 18000kbit quantum 8000