[Bloat] SQM tuning question

Sun Jun 4 15:55:41 EDT 2023

Hi Aaron,

> On Jun 3, 2023, at 20:04, Aaron Wood via Bloat <bloat at lists.bufferbloat.net> wrote:
> 
> I’ve found that _usually_ I can set cake’s bandwidth limits to 90-95% of the advertised bandwidth, and everything “just works”.  So long as you’re routinely able to achieve the bandwidth, it tends to work.
> 
> I’ve found in my testing over the years (I’ve been a user of fq_codel since 2013) that limiting the upstream bandwidth is the most important one to do.  Downstream bandwidth limits, especially if you have well above 100Mbps downstream aren’t as critical, but it still can make a big difference if you routinely saturate your downstream connection.
> 
> It’s especially important to manage the upstream bandwidth when the connection is asymmetric.  For instance my 1Gbps Comcast internet service only has a 35Mbps upstream limit, which is just a bit more than you need for the acks that go back upstream in response to 1Gbps of downstream traffic.
> 
> But if your connection quality suffers with time of day (congestion on shared local ISP infrastructure), or due to weather (radio links can degrade during the rain), then that can be a bit more annoying.
> 
> But as I said, I’ve found with my Comcast connection that I can just set my bandwidth limits to 90-95% of advertised and it just works.
> 
> The other thing to look into is DNS.  Most of the weird pauses that I see are, DNS related.  I e switches to using a local DNS cache/lookup-forwarding server, and use DNS over HTTPS to parallel requests to both cloudflare and google’s HTTPS DNS proxies.  That seems to be more stable (long-lived connections with faster recovery from a dropped packed than UDP has).
> 
> (More continued below)
> 
> On Sat, Jun 3, 2023 at 10:17 AM John D via Bloat <bloat at lists.bufferbloat.net> wrote:
> Thanks for the detail. It makes sense but it kind of feels like in some (maybe many) cases the router could know the internet link performance. Particularly home router-modems often monitor this already. Maybe that's just not exposed in any standardised way?
> 
> There are two issues I’ve seen here:
> 
> 1) there’s not a standardized way (or even usually any api) for getting the provisioned rate, or the current link speed, from the modem portion of things.
> 
> 2) local congestion at the ISP can degrade things below the provisioned rates, and that’s even harder to detect or have available.
> 
> If your upstream connection is DOCSIS 3.1, you should have the PIE AQM running in the modem, which _should_ help considerably.  But at least at my cable headend, while downstream runs DOCSIS 3.1, upstream is only DOCSIS 3.0, and definitely isn’t using PIE.
> 
> The PIE AQM, running on the modem itself has the advantage of running on what is almost always the bottle neck for upstream traffic in my experience:  the provisioned upstream bandwidth in the modem.

	[SM] Even better, the cable modem requests mini-slots from the scheduler and really has control over the true upstream queue, so even in an overloaded segment (where mini-slot scheduling gets slow) the PIE instance sees the relevant queueing delay to do its thing even if the current rate is well below the "provisioned" rate (which is just a maximum peak rate anyway). Now this will only help for congestion in the DOCSIS segment, as you said that is likely to be the most likely cause for upstream delay anyways...

> 
> I'm guessing if I was into openwrt I could maybe do something, but I prefer just to find something off the shelf with half decent SQM... If "auto configuration" isn't a feature then that answers my question and I can get on choosing the best option.
> 
> I’m not aware of any decent “off the shelf” solutions that can track bandwidth correctly, aside from DOCSIS 3.1 modems (which still requires the cable company to provision it correctly to enable it)
> 
> But, as I said, if your connection is stable, you can get the majority of the benefits just by setting it up once and leaving it be.
> 
> I’ve only tweaked things 1-2 times a year, for the last three years, and that was only because I was moving.  Once I had setup a router that could run cake at my line rates, I’ve just left it there and it’s been fine.
> 
> Especially since for me, WiFi is my usual bottleneck (my APs only do about 250-300Mbps)
> 
> -Aaron
> 
> 
> On Sat, Jun 3, 2023, 16:44 Jonathan Morton <chromatix99 at gmail.com> wrote:
> > On 3 Jun, 2023, at 4:56 pm, John D via Bloat <bloat at lists.bufferbloat.net> wrote:
> > 
> > On the website it says the following:
> > 
> > CoDel is a novel “no knobs”, “just works”, “handles variable bandwidth and RTT”, and simple AQM algorithm.
> > 
> >       • It is parameterless — no knobs are required for operators, users, or implementers to adjust.
> >       • It treats good queue and bad queue differently - that is, it keeps the delays low while permitting bursts of traffic.
> >       • It controls delay, while insensitive to round-trip delays, link rates, and traffic loads.
> >       • It adapts to dynamically changing link rates with no negative impact on utilization.
> > 
> > But everywhere I have read about about hardware which implements SQM (including the bufferbloat website) it describes the need to tune based on actual internet connection speed.
> > These seem to conflict especially that "handles variable bandwidth" bit. Have I misunderstood or do the algorithms used in modern hardware just not provide this part typically? My connection performance is quite variable and I'm worried about crippling SQM to the lowest speed seen.
> 
> SQM in practice requires three components:
> 
> 1: Flow isolation, so that different flows don't affect each others' latency and are delivered fairly;
> 
> 2: Active Queue Management (AQM) to signal flows to slow down transmissions when link capacity is exceeded;
> 
> 3: Bandwidth shaping to match the queue to the available capacity.
> 
> CoDel is, in itself, only the AQM component.  It does indeed work pretty well with no additional tuning - but only in combination with the other two components, or when applied directly to the actual bottleneck.  Unfortunately in most consumer internet links, the actual bottleneck is inaccessible for this purpose.  Thus an artificial bottleneck must be introduced, at which SQM is applied.
> 
> The most convenient tool for applying all three SQM components at once is Cake.  This includes implementations of advanced flow isolation, CoDel AQM, and a deficit-mode bandwidth shaper.  All you really need to do is to tell it how much bandwidth you have in each direction, minus a small margin to ensure it becomes the actual bottleneck and can exert the necessary control.
> 
> When your available bandwidth varies over time, that can be inconvenient.  There are methods, however, of observing how available capacity tends to change over time (typically on diurnal and weekly patterns, if the variations are due to congestion in the ISP backhaul or peering) and scheduling adjustments on that basis.  If you have more information on your situation, we might be able to give more detailed advice.
> 
>  - Jonathan Morton
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
> -- 
> - Sent from my iPhone.
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat