[Cake] cake infinite loop(?) with hfsc on one-armed router

[Pete Heist]

Note that this doesn’t happen when prio is used in place of hfsc and cake is used in the leafs to do the rate limiting, i.e.:

tc qdisc add dev eth0 root handle 1: prio bands 2 priomap 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
tc qdisc add dev eth0 parent 1:1 handle 10: cake besteffort bandwidth 100mbit ethernet
tc qdisc add dev eth0 parent 1:2 handle 11: cake besteffort bandwidth 100mbit ethernet ether-vlan
tc filter add dev eth0 protocol all parent 1:0 prio 1 basic match "meta(vlan mask 0xfff eq 0xce4)" flowid 1:2
tc filter add dev eth0 protocol all parent 1:0 prio 2 u32 match u32 0 0 flowid 1:1

But it does happen when drr is used instead of prio:

tc qdisc add dev eth0 root handle 1: drr
tc class add dev eth0 parent 1: classid 1:1
tc class add dev eth0 parent 1: classid 1:2
tc qdisc add dev eth0 parent 1:1 handle 10: cake besteffort bandwidth 100mbit
tc qdisc add dev eth0 parent 1:2 handle 11: cake besteffort bandwidth 100mbit ether-vlan
tc filter add dev eth0 protocol all parent 1:0 prio 1 basic match "meta(vlan mask 0xfff eq 0xce4)" flowid 1:2
tc filter add dev eth0 protocol all parent 1:0 prio 2 u32 match u32 0 0 flowid 1:1

drr might ultimately be what I want to use for this, so I can use cake to do the rate limiting instead of htb. prio works well but leads to starvation when the rate limit is above what the CPU can handle.

Meanwhile, using htb classes with rate limits way above the actual, then rate limiting in the cake leafs, works as well, but this seems like a hack:

tc qdisc add dev eth0 root handle 1: htb default 10
tc class add dev eth0 parent 1: classid 1:1 htb rate 10gbit
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 5gbit
tc class add dev eth0 parent 1:1 classid 1:11 htb rate 5gbit
tc filter add dev eth0 protocol ip parent 1:0 prio 1 basic match "meta(vlan mask 0xfff eq 0xce4)" flowid 1:11
tc qdisc add dev eth0 parent 1:10 handle 20: cake besteffort bandwidth 100mbit ethernet
tc qdisc add dev eth0 parent 1:11 handle 21: cake besteffort bandwidth 100mbit ethernet ether-vlan

> On Dec 28, 2018, at 12:30 AM, Pete Heist <pete at heistp.net> wrote:
> 
> I’m seeing what I think it an infinite loop when cake is used in a one-armed router configuration with hfsc as the rate limiter. Three APUs are connected to the same switch and the “middle” APU (apu1a) routes between the default VLAN and a tagged VLAN.
> 
> apu2a   <— default VLAN —>   apu1a   <— VLAN 3300 —>   apu2b
> 
> After qos is set up, ping from apu2a to apu2b still works fine. When iperf3 is run from apu2a to apu2b it works fine, but when it goes in reverse (apu2b to apu2a), all traffic stops flowing from apu1a on the default VLAN. Traffic still flows from apu1a on VLAN 3300 however, with very high RTT (mean 500ms), leading me to believe that the cake instance on the default VLAN is in an infinite loop.
> 
> It does not happen with hfsc+fq_codel, or with htb+cake in the same configuration.
> 
> Here are the commands that set up qos, and it only locks up when cake is used as the instance at handle 20, not at handle 21:
> 
> -----
> tc qdisc add dev eth0 root handle 1: hfsc default 10
> tc class add dev eth0 parent 1: classid 1:1 hfsc sc rate 200mbit ul rate 200mbit
> tc class add dev eth0 parent 1:1 classid 1:10 hfsc sc rate 100mbit ul rate 100mbit
> tc class add dev eth0 parent 1:1 classid 1:11 hfsc sc rate 100mbit ul rate 100mbit
> tc filter add dev eth0 protocol ip parent 1:0 prio 1 \
> 	basic match "meta(vlan mask 0xfff eq 0xce4)" flowid 1:11
> tc qdisc add dev eth0 parent 1:10 handle 20: fq_codel # using cake here locks up !!!
> tc qdisc add dev eth0 parent 1:11 handle 21: cake
> ——
> 
> I’m using sch_cake and tc-adv from the current HEAD, on kernel 3.16.7 (yeah, I know).
> 
> root at apu1a:~/qos# uname -a
> Linux apu1a 3.16.7-ckt9-voyage #1 SMP Thu Apr 23 11:10:44 HKT 2015 i686 GNU/Linux
> 
> Any ideas just from just this? Otherwise, I can only think to hook up the serial cable and start with the printk’s…
>