[Cake] Recomended HW to run cake and fq_codel?

[Pete Heist]

> Message: 1
> Date: Tue, 2 May 2017 10:34:45 +0000
> From: <erik.taraldsen at telenor.com>
> To: <me at lochnair.net>
> Cc: <cake at lists.bufferbloat.net>
> Subject: Re: [Cake] Recomended HW to run cake and fq_codel?
> Message-ID: <1493721285271.28909 at telenor.com>
> Content-Type: text/plain; charset="iso-8859-1"
> 
> I'm actually most interested in how this works on low bandwidth accesses.  Typicaly, what can we as an ISP do to make ADSL and VDSL less sucky for our customers.  So an Edge Router PoE-5  1) or the X sfp 2) would be a good platform for this?  (Don't need the PoE or sfp, but it's the easiest accessible version here in Norway).
> 
> Nils, very good of you to keep such packages precompiled! That will save me a lot of time.

> ------------------------------
> 
> Message: 2
> Date: Tue, 02 May 2017 14:11:20 +0200
> From: Nils Andreas Svee <me at lochnair.net>
> To: erik.taraldsen at telenor.com
> Cc: cake at lists.bufferbloat.net
> Subject: Re: [Cake] Recomended HW to run cake and fq_codel?
> Message-ID:
> 	<1493727080.1510042.962956680.40220FCB at webmail.messagingengine.com>
> Content-Type: text/plain; charset="utf-8"
> 
> Kinda surprising that the plain ER-X isn't readily available. I know
> Dustin used to have them, but they're out of stock. Both of them will do
> just fine, but I'd probably pick the ER-X-SFP for the beefier CPU, if
> only to get some extra headroom. Mind the ER-X only have 256 MB RAM and
> 256 MB flash, if that matters to you.
> 
> DSL tends to suck pretty (read: very) bad without proper shaping, I
> know. On that note, are you planning to run an AQM on both ends of the
> bottleneck, or shape ingress traffic via a IFB device? CAKE helps a lot
> when running on ingress, but it can't come close to running on both
> ends.
> 
> Best Regards
> Nils

Just sharing some experience/thoughts from a few angles:

- As for low bandwidth, in my experience AQM works great on low bandwidth ADSL. A few years ago I used fq_codel at a campground to shape a 0.5 / 5 Mbps ADSL connection. With up to 130 people in the camp, it was a disaster before fq_codel, where one person saturating either the up or downstream could easily cause 600+ ms of induced latency. fq_codel could keep that to 40-50 ms under load, enough to make it usable for web browsing, at least, and Cake does better.

- I did the ER-X testing referred to in the UBNT forums (pgage). I’ve since learned much more about AQM by testing point-to-point WiFi setups, so I should really repeat those ER-X tests some time to make sure my results were accurate, but afaik ~120 Mbps using Cake is possible with the ER-X. I’m now using Cake in production on the ER-X at rates around 40 Mbps with Lochnair’s builds (on EdgeOS 1.9.1) and it does a great job.

- It’s an interesting question: what can be done as an ISP? Essentially it boils down to the fundamentals of deploying AQM- finding where the queues are forming and placing fq_codel or Cake at the bottleneck links, preferring “hardware” queue management like BQL or in the case of WiFi the ath9k’s driver in LEDE, over soft rate limiting, where possible. When soft rate limiting, the rate limiting strategy and chosen rate are the most CPU intensive and finicky parts of deploying AQM.

- I don't understand why ADSL modem vendors don’t just bake BQL-like functionality right into their devices so they can ship AQM without the need for soft rate limiting. AQM is so effective on ADSL's upstream that it seems it would just make a lot of sense. For that matter, why not on the DSLAM as well to shape the customer’s downstream, if that’s also a bottleneck?

- There seems to be a bit of upheaval now with BBR. If every host had BBR deployed, that would theoretically mitigate the need for AQM, but it’s a) it’s going to be years before that happens and b) I’m not sure all the BBR corner cases have been found yet. There are far more knowledgeable people than me on this and already more detailed discussions about it.