[Cake] cake infinite loop(?) with hfsc on one-armed router

[Pete Heist]

> On Jan 8, 2019, at 11:33 PM, Dave Taht <dave.taht at gmail.com> wrote:
> 
> On Tue, Jan 8, 2019 at 2:01 PM Pete Heist <pete at heistp.net <mailto:pete at heistp.net>> wrote:
>> 
>> I should have done that: https://www.heistp.net/downloads/htb_split_gso_patched/ <https://www.heistp.net/downloads/htb_split_gso_patched/>
>> 
>> Note that I changed the names in the plots to match the convention of my first email, but it should be clear which is which and I left all plots in. The text output is there too as I sometimes like to open several up in different browser tabs and switch between tabs to compare values.
>> 
>> It looks like about 100 usec to me. Throughput also looks consistently about 0.3 Mbit higher (~1.3%) in the split results.
> 
> My guess is with ecn on would have the highest latency and the same
> throughput. ?

I see a slight increase in icmp/udp rtt (1.04ms-1.02ms = 20us) and a slight increase in throughput (180.12Mbit - 179.50Mbit = 620kbit).

https://www.heistp.net/downloads/htb_cakep2_ecnon/
https://www.heistp.net/downloads/htb_cakep2_ecnon2/

These results are not five sigma. :)

> Since we started this effort in an era when seconds of added latency
> was common, a mere 100us improvement seems insignificant, except that
> that's a 10% improvement over the present-day baseline, and *that's
> worth it*. ;) This is also a function of the number of flows, kernel

Who knows how many timeouts are avoided in the future, just because of this 100us. :)

> I have noticed that BQL's values can get quite large with cake doing
> the shaping, btw, much larger than they do with htb.

Wonder why that is.

So far it’s harder to use cake’s shaper in the current setup I’m working on, because it appears “better" to have eth0 and eth0.3300 under one link sharing hierarchy for eth0:

tc qdisc add dev eth0 root handle 1: htb default 1
tc class add dev eth0 parent 1: classid 1:1 htb rate $RATE
tc class add dev eth0 parent 1: classid 1:2 htb rate $RATE
tc qdisc add dev eth0 parent 1:1 cake besteffort
tc qdisc add dev eth0 parent 1:2 cake besteffort
tc filter add dev eth0 parent 1:0 prio 1 protocol all \
	basic match not "meta(vlan mask 0xfff gt 0x0)" flowid 1:1
tc filter add dev eth0 parent 1:0 prio 2 protocol all \
	basic match "meta(vlan mask 0xfff eq `printf \"0x%x\" $VLAN_TAG`)" flowid 1:2

but I can’t do that with cake’s shaper. It’s better in the sense that when you’re out of CPU, latency doesn’t increase suddenly. I _can_ use cake’s shaper by adding cake to the VLAN interface and filtering out vlan traffic from the main interface:

tc qdisc add dev eth0 root handle 1: prio bands 2 priomap 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
tc qdisc add dev eth0 parent 1:1 handle 10: cake besteffort bandwidth $RATE
tc filter add dev eth0 parent 1:0 prio 1 protocol all \
	basic match not "meta(vlan mask 0xfff gt 0x0)" flowid 10:1
tc qdisc add dev eth0.3300 root cake besteffort bandwidth $RATE

but when you’re out of CPU, you get starvation and inter-flow latency increases. So so far, hfsc or htb is working better in this case. It might be something to think about for an “ISP Cake”…

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20190109/e2dab87f/attachment-0001.html>