From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oo1-xc36.google.com (mail-oo1-xc36.google.com [IPv6:2607:f8b0:4864:20::c36]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id A54043B29D for ; Sat, 3 Jun 2023 14:18:11 -0400 (EDT) Received: by mail-oo1-xc36.google.com with SMTP id 006d021491bc7-5556e2bddf9so1692403eaf.1 for ; Sat, 03 Jun 2023 11:18:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685816291; x=1688408291; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=+Fg3BrizuCkG4tatZ0eD9xOeeIKAzTOpQdt0Hr6Q36g=; b=K7QtS4dU4zo94KxzOOI/28e+x6EDGnSJbfF59Zx0+qwoVZxIPalopkBcK+2ILeGzaX h9zdH57fs3FYk8Ri/TjIxF6zTyCveor1Uv4/QiJcnUOzz/goXIz24sMd7QmoqHo6ORjw 1M/6tWBlN6hBugPc1QAulKN5ILEW4RFtRdNQqDoNnW2oC6Bit18AjqSk23Ly2xkQUvqp iHCbvgqNnShLtCVjuS7j1EJbpmk02EDwSdBmROhfUPTSduW9ErWhybNL1FvQE70zEaXs nP1R7GP9lD4KetQRINzDhO/Gnzd0lE2bOKmB/NpOQ/7C6AtA7i+jrVQ5XfEsRGAinA+E wJKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685816291; x=1688408291; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=+Fg3BrizuCkG4tatZ0eD9xOeeIKAzTOpQdt0Hr6Q36g=; b=ROEyd+XH+8bMT8IGTMeKUcRRFXKQfUHKZGsBv/SlEPzFBzXD+Cubp5+JkVKDPICxDa svX8nPBLBMtC/72ZjLA6FccnXlWReWLFGfj5NcsXmvyGfcPCQOwMNHCdUJ0h3TGekGFO RJEgIN8aOTRvmfW4BanV4VRzdz2iG55ThtRiRPCiw+Eia88EbWuiM2giAJgL/1ttGXk7 ROWHsZMspqjZwtVcUaeeWQ3uzd8iOiRJKc3ipvcqBaT/yv6xX/yECqUU9tyv+yeFHzmO mtdQyZ0O+xNUCX6xMdMYPrAlCDHjwLbOLNfjLNHlMixpnckTzGqXKKkSX9M/Lz9nsTMN L8Tg== X-Gm-Message-State: AC+VfDw/2oEIZsAwBw7yQSmRpiJq7wk1tkPeDNu1qdtTfVm7A2vf7prv QihUIpCpiomyvCsY77tgyboFXNpzdwSaak1Y3vfHXVD3 X-Google-Smtp-Source: ACHHUZ6hQaJhA0yRYKuvuXt6wOcX7uUNsNT1BYtvdegEA8PfYVXbe34Bt+Ks2Lykui0BTTSbcBFqiPzR9HPjYmMJg7k= X-Received: by 2002:a4a:374c:0:b0:547:5593:d6 with SMTP id r73-20020a4a374c000000b00547559300d6mr7597617oor.5.1685816290524; Sat, 03 Jun 2023 11:18:10 -0700 (PDT) MIME-Version: 1.0 References: <39DED14A-AACF-4C45-9834-C295F92E8800@gmail.com> In-Reply-To: From: Aaron Wood Date: Sat, 3 Jun 2023 11:04:56 -0700 Message-ID: To: John D Cc: bloat@lists.bufferbloat.net Content-Type: multipart/alternative; boundary="00000000000027441105fd3db132" Subject: Re: [Bloat] SQM tuning question X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jun 2023 18:18:11 -0000 --00000000000027441105fd3db132 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I=E2=80=99ve found that _usually_ I can set cake=E2=80=99s bandwidth limits= to 90-95% of the advertised bandwidth, and everything =E2=80=9Cjust works=E2=80=9D. So = long as you=E2=80=99re routinely able to achieve the bandwidth, it tends to work. I=E2=80=99ve found in my testing over the years (I=E2=80=99ve been a user o= f fq_codel since 2013) that limiting the upstream bandwidth is the most important one to do. Downstream bandwidth limits, especially if you have well above 100Mbps downstream aren=E2=80=99t as critical, but it still can make a big differen= ce if you routinely saturate your downstream connection. It=E2=80=99s especially important to manage the upstream bandwidth when the connection is asymmetric. For instance my 1Gbps Comcast internet service only has a 35Mbps upstream limit, which is just a bit more than you need for the acks that go back upstream in response to 1Gbps of downstream traffic. But if your connection quality suffers with time of day (congestion on shared local ISP infrastructure), or due to weather (radio links can degrade during the rain), then that can be a bit more annoying. But as I said, I=E2=80=99ve found with my Comcast connection that I can jus= t set my bandwidth limits to 90-95% of advertised and it just works. The other thing to look into is DNS. Most of the weird pauses that I see are, DNS related. I e switches to using a local DNS cache/lookup-forwarding server, and use DNS over HTTPS to parallel requests to both cloudflare and google=E2=80=99s HTTPS DNS proxies. That seems to b= e more stable (long-lived connections with faster recovery from a dropped packed than UDP has). (More continued below) On Sat, Jun 3, 2023 at 10:17 AM John D via Bloat < bloat@lists.bufferbloat.net> wrote: > Thanks for the detail. It makes sense but it kind of feels like in some > (maybe many) cases the router could know the internet link performance. > Particularly home router-modems often monitor this already. Maybe that's > just not exposed in any standardised way? > There are two issues I=E2=80=99ve seen here: 1) there=E2=80=99s not a standardized way (or even usually any api) for get= ting the provisioned rate, or the current link speed, from the modem portion of things. 2) local congestion at the ISP can degrade things below the provisioned rates, and that=E2=80=99s even harder to detect or have available. If your upstream connection is DOCSIS 3.1, you should have the PIE AQM running in the modem, which _should_ help considerably. But at least at my cable headend, while downstream runs DOCSIS 3.1, upstream is only DOCSIS 3.0, and definitely isn=E2=80=99t using PIE. The PIE AQM, running on the modem itself has the advantage of running on what is almost always the bottle neck for upstream traffic in my experience: the provisioned upstream bandwidth in the modem. I'm guessing if I was into openwrt I could maybe do something, but I prefer > just to find something off the shelf with half decent SQM... If "auto > configuration" isn't a feature then that answers my question and I can ge= t > on choosing the best option. > I=E2=80=99m not aware of any decent =E2=80=9Coff the shelf=E2=80=9D solutio= ns that can track bandwidth correctly, aside from DOCSIS 3.1 modems (which still requires the cable company to provision it correctly to enable it) But, as I said, if your connection is stable, you can get the majority of the benefits just by setting it up once and leaving it be. I=E2=80=99ve only tweaked things 1-2 times a year, for the last three years= , and that was only because I was moving. Once I had setup a router that could run cake at my line rates, I=E2=80=99ve just left it there and it=E2=80=99s= been fine. Especially since for me, WiFi is my usual bottleneck (my APs only do about 250-300Mbps) -Aaron > On Sat, Jun 3, 2023, 16:44 Jonathan Morton wrote: > >> > On 3 Jun, 2023, at 4:56 pm, John D via Bloat < >> bloat@lists.bufferbloat.net> wrote: >> > >> > On the website it says the following: >> > >> > CoDel is a novel =E2=80=9Cno knobs=E2=80=9D, =E2=80=9Cjust works=E2=80= =9D, =E2=80=9Chandles variable bandwidth >> and RTT=E2=80=9D, and simple AQM algorithm. >> > >> > =E2=80=A2 It is parameterless =E2=80=94 no knobs are required fo= r operators, >> users, or implementers to adjust. >> > =E2=80=A2 It treats good queue and bad queue differently - that = is, it >> keeps the delays low while permitting bursts of traffic. >> > =E2=80=A2 It controls delay, while insensitive to round-trip del= ays, link >> rates, and traffic loads. >> > =E2=80=A2 It adapts to dynamically changing link rates with no n= egative >> impact on utilization. >> > >> > But everywhere I have read about about hardware which implements SQM >> (including the bufferbloat website) it describes the need to tune based = on >> actual internet connection speed. >> > These seem to conflict especially that "handles variable bandwidth" >> bit. Have I misunderstood or do the algorithms used in modern hardware j= ust >> not provide this part typically? My connection performance is quite >> variable and I'm worried about crippling SQM to the lowest speed seen. >> >> SQM in practice requires three components: >> >> 1: Flow isolation, so that different flows don't affect each others' >> latency and are delivered fairly; >> >> 2: Active Queue Management (AQM) to signal flows to slow down >> transmissions when link capacity is exceeded; >> >> 3: Bandwidth shaping to match the queue to the available capacity. >> >> CoDel is, in itself, only the AQM component. It does indeed work pretty >> well with no additional tuning - but only in combination with the other = two >> components, or when applied directly to the actual bottleneck. >> Unfortunately in most consumer internet links, the actual bottleneck is >> inaccessible for this purpose. Thus an artificial bottleneck must be >> introduced, at which SQM is applied. >> >> The most convenient tool for applying all three SQM components at once i= s >> Cake. This includes implementations of advanced flow isolation, CoDel A= QM, >> and a deficit-mode bandwidth shaper. All you really need to do is to te= ll >> it how much bandwidth you have in each direction, minus a small margin t= o >> ensure it becomes the actual bottleneck and can exert the necessary cont= rol. >> >> When your available bandwidth varies over time, that can be >> inconvenient. There are methods, however, of observing how available >> capacity tends to change over time (typically on diurnal and weekly >> patterns, if the variations are due to congestion in the ISP backhaul or >> peering) and scheduling adjustments on that basis. If you have more >> information on your situation, we might be able to give more detailed >> advice. >> >> - Jonathan Morton > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > --=20 - Sent from my iPhone. --00000000000027441105fd3db132 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I=E2=80=99ve found that _usually_ I can set cake=E2=80=99= s bandwidth limits to 90-95% of the advertised bandwidth, and everything = =E2=80=9Cjust works=E2=80=9D.=C2=A0 So long as you=E2=80=99re routinely abl= e to achieve the bandwidth, it tends to work.

I=E2=80=99ve found in my testing over the years (I=E2= =80=99ve been a user of fq_codel since 2013) that limiting the upstream ban= dwidth is the most important one to do.=C2=A0 Downstream bandwidth limits, = especially if you have well above 100Mbps downstream aren=E2=80=99t as crit= ical, but it still can make a big difference if you routinely saturate your= downstream connection.

= It=E2=80=99s especially important to manage the upstream bandwidth when the= connection is asymmetric.=C2=A0 For instance my 1Gbps Comcast internet ser= vice only has a 35Mbps upstream limit, which is just a bit more than you ne= ed for the acks that go back upstream in response to 1Gbps of downstream tr= affic.

But if your conne= ction quality suffers with time of day (congestion on shared local ISP infr= astructure), or due to weather (radio links can degrade during the rain), t= hen that can be a bit more annoying.

But as I said, I=E2=80=99ve found with my Comcast connection t= hat I can just set my bandwidth limits to 90-95% of advertised and it just = works.

The other thing t= o look into is DNS.=C2=A0 Most of the weird pauses that I see are, DNS rela= ted.=C2=A0 I e switches to using a local DNS cache/lookup-forwarding server= , and use DNS over HTTPS to parallel requests to both cloudflare and google= =E2=80=99s HTTPS DNS proxies.=C2=A0 That seems to be more stable (long-live= d connections with faster recovery from a dropped packed than UDP has).

(More continued below)

On Sat, Jun 3, 2023 at 10:17= AM John D via Bloat <blo= at@lists.bufferbloat.net> wrote:
Thanks for the detail. It makes se= nse but it kind of feels like in some (maybe many) cases the router could k= now the internet link performance. Particularly home router-modems often mo= nitor this already. Maybe that's just not exposed in any standardised w= ay?

There = are two issues I=E2=80=99ve seen here:

1) there=E2=80=99s not a standardized way (or even usually a= ny api) for getting the provisioned rate, or the current link speed, from t= he modem portion of things.

2) local congestion at the ISP can degrade things below the provisioned= rates, and that=E2=80=99s even harder to detect or have available.

If your upstream connection is = DOCSIS 3.1, you should have the PIE AQM running in the modem, which _should= _ help considerably.=C2=A0 But at least at my cable headend, while downstre= am runs DOCSIS 3.1, upstream is only DOCSIS 3.0, and definitely isn=E2=80= =99t using PIE.

The PIE = AQM, running on the modem itself has the advantage of running on what is al= most always the bottle neck for upstream traffic in my experience: =C2=A0th= e provisioned upstream bandwidth in the modem.

<= /div>
I'm guessing if I was into open= wrt I could maybe do something, but I prefer just to find something off the= shelf with half decent SQM... If "auto configuration" isn't = a feature then that answers my question and I can get on choosing the best = option.

I= =E2=80=99m not aware of any decent =E2=80=9Coff the shelf=E2=80=9D solution= s that can track bandwidth correctly, aside from DOCSIS 3.1 modems (which s= till requires the cable company to provision it correctly to enable it)

But, as I said, if your con= nection is stable, you can get the majority of the benefits just by setting= it up once and leaving it be.

I=E2=80=99ve only tweaked things 1-2 times a year, for the last thre= e years, and that was only because I was moving.=C2=A0 Once I had setup a r= outer that could run cake at my line rates, I=E2=80=99ve just left it there= and it=E2=80=99s been fine.

Especially since for me, WiFi is my usual bottleneck (my APs only do a= bout 250-300Mbps)

-Aaron=


On S= at, Jun 3, 2023, 16:44 Jonathan Morton <chromatix99@gmail.com>= wrote:
> On 3 Jun, 2023, at 4:56 pm, John D v= ia Bloat <bloat@lists.bufferbloat.net> wrote:=
>
> On the website it says the following:
>
> CoDel is a novel =E2=80=9Cno knobs=E2=80=9D, =E2=80=9Cjust works=E2=80= =9D, =E2=80=9Chandles variable bandwidth and RTT=E2=80=9D, and simple AQM a= lgorithm.
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0=E2=80=A2 It is parameterless =E2=80=94 no k= nobs are required for operators, users, or implementers to adjust.
>=C2=A0 =C2=A0 =C2=A0 =C2=A0=E2=80=A2 It treats good queue and bad queue= differently - that is, it keeps the delays low while permitting bursts of = traffic.
>=C2=A0 =C2=A0 =C2=A0 =C2=A0=E2=80=A2 It controls delay, while insensiti= ve to round-trip delays, link rates, and traffic loads.
>=C2=A0 =C2=A0 =C2=A0 =C2=A0=E2=80=A2 It adapts to dynamically changing = link rates with no negative impact on utilization.
>
> But everywhere I have read about about hardware which implements SQM (= including the bufferbloat website) it describes the need to tune based on a= ctual internet connection speed.
> These seem to conflict especially that "handles variable bandwidt= h" bit. Have I misunderstood or do the algorithms used in modern hardw= are just not provide this part typically? My connection performance is quit= e variable and I'm worried about crippling SQM to the lowest speed seen= .

SQM in practice requires three components:

1: Flow isolation, so that different flows don't affect each others'= ; latency and are delivered fairly;

2: Active Queue Management (AQM) to signal flows to slow down transmissions= when link capacity is exceeded;

3: Bandwidth shaping to match the queue to the available capacity.

CoDel is, in itself, only the AQM component.=C2=A0 It does indeed work pret= ty well with no additional tuning - but only in combination with the other = two components, or when applied directly to the actual bottleneck.=C2=A0 Un= fortunately in most consumer internet links, the actual bottleneck is inacc= essible for this purpose.=C2=A0 Thus an artificial bottleneck must be intro= duced, at which SQM is applied.

The most convenient tool for applying all three SQM components at once is C= ake.=C2=A0 This includes implementations of advanced flow isolation, CoDel = AQM, and a deficit-mode bandwidth shaper.=C2=A0 All you really need to do i= s to tell it how much bandwidth you have in each direction, minus a small m= argin to ensure it becomes the actual bottleneck and can exert the necessar= y control.

When your available bandwidth varies over time, that can be inconvenient.= =C2=A0 There are methods, however, of observing how available capacity tend= s to change over time (typically on diurnal and weekly patterns, if the var= iations are due to congestion in the ISP backhaul or peering) and schedulin= g adjustments on that basis.=C2=A0 If you have more information on your sit= uation, we might be able to give more detailed advice.

=C2=A0- Jonathan Morton
_______________________________________________
Bloat mailing list
Bloat@list= s.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat
-- <= br>
- Sent from my iPhone.
--00000000000027441105fd3db132--