From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 7753F3B29D for ; Sat, 7 Dec 2024 17:57:48 -0500 (EST) Received: by mail-pj1-x102c.google.com with SMTP id 98e67ed59e1d1-2ef718cb473so323629a91.1 for ; Sat, 07 Dec 2024 14:57:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733612267; x=1734217067; darn=lists.bufferbloat.net; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=zXTz3ODbfn/I9/WvqFExq/TAbpYfRIl3E5mUOhAUPr0=; b=Cs0LVLjy4nSxK3Kvfn8Kba6xMqt+boEJQaW7q0QqjMQakSnXpfp2qu3zKyQcKfIvbG 5M5xKeCT2FYQ2YO7ATt42hbRN1oW5Oh1+1WhXDXyfrII61vOqn1DV+S6Stn2hlJf8iFL FGThYCPoZSOVAePzPaBPyWe7SPmIWHWuCCBSwCb1mEkzjHKEGGcmymJ4hfC/h2luhj6A YNTAuo5erCK9h+LNpZuIfNgKoVID0qCsgfLQVBBvROH9hz4ysV7O6bxngcd36NJZ1Byn /lZ5QUecBnigEGy8iWHQDavhwUBe/2LSV3GMDliUixkzal6UpnpzroIt5kcqxouLL7Pv wIEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733612267; x=1734217067; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zXTz3ODbfn/I9/WvqFExq/TAbpYfRIl3E5mUOhAUPr0=; b=jkocmnaHYf/8Wr8FYY97KgjdV4LhCIF4fTM8Aqt0SnK8ofi3ZJtlonXd5VN3fkGL6x kXZF/AHwKYQNecWVIRwevlL+yRVuNOu6YDFGcuPtXCocffaY4ao1RecZS1VRW4LanhM3 +UDpLmfY5+UV54wn/a4Xerfyrh2iV7jGJIJL/HtEczGq8sSn6Tmx8PMlCv4GyOrsIeh8 E56ltyaRBvkr30l8wzbKVoa8G3YjNXVa82gJIoWZeCXPHZLx352HE0bNbACmMYeFZUS3 HAvokPBAlUy+74zeZNhC6/16WOLyYkUThqa83SVo1xbY4f0TP87+BMjJuNsBgF2uZepz nDPQ== X-Gm-Message-State: AOJu0YzPfgkc4asJx4DtHMugvLBKpTXqEzi4/8m6rSLkUlEVVtDHwvuq u+nJCALjy+iax8kQqHIDQMlragf/ao3f7kwO5R+cp1d1CU5diTWuZhIZmQ9kqoevBZjdDPqRg7y yTxI30dwdNrvEeCdz8D7l9lRmVus= X-Gm-Gg: ASbGnct4+z93R/we7Qk98nzyMoOeMBiGvMd+Saupiy8TsSlxdPSw6UjHMOTn/RiwC8O RwfBTENyaTVNv/+zF9YS/Q0edwhIV6f9bYO8rB2B+ubd7uHVYahaUso4wnTv0EYV9 X-Google-Smtp-Source: AGHT+IGgZj7WlOX83OcbRYzR8svsMS/gSVPA1GdyxgtMFqHnhQ3vMpcBJpra56BRvj12jeJVF2b+EMTpBUOGBuZ308A= X-Received: by 2002:a17:90b:33c4:b0:2ee:e518:c1d4 with SMTP id 98e67ed59e1d1-2ef69462617mr4503450a91.1.1733612267398; Sat, 07 Dec 2024 14:57:47 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Evan Mesterhazy Date: Sat, 7 Dec 2024 17:57:36 -0500 Message-ID: To: Dave Taht Cc: codel@lists.bufferbloat.net Content-Type: multipart/alternative; boundary="000000000000606eaa0628b60fcc" Subject: Re: [Codel] Do codel or fq-codel need to know the link bandwidth? X-BeenThere: codel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: CoDel AQM discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Dec 2024 22:57:48 -0000 --000000000000606eaa0628b60fcc Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks Dave, your reply is very informative. I definitely understand that FreeBSD is somewhat less sophisticated than Linux when it comes to this since I've read through a few of the bugs and other messages you've posted regarding its implementation. However the Unifi routers run on Linux, and they still require the user to specify the ISP's service bandwidth. I think this paragraph from your reply may be the crux of it: > Further most ISPs use a non-native rate for their customer interfaces, > either using a policer or FIFO shaper and thus the rise of combatting > that with shaping via fq_codel to slightly below the ISPs' rate to > move the bottleneck to your own hardware. If the router is able to keep pushing bytes over the ISP's link even in excess of the allowed bandwidth, then the local router's packets probably aren't sitting in their queues long enough to be dropped by fq-codel. It sounds like proper traffic shaping needs to happen wherever the bottleneck is (I suppose this should have been obvious to me), which in the case of a 300 Mbp/s connection over GPON is somewhere at the ISP. I guess if the ISP does proper traffic shaping the end user shouldn't need to do anything. Assuming I didn't say anything inaccurate in my reply, I think I understand why setting a bandwidth limit is necessary to make fq-codel work properly on a router with a less than line speed connection from the ISP. Thank you! On Sat, Dec 7, 2024 at 5:32=E2=80=AFPM Dave Taht wrot= e: > "variable bandwidth" requires that the queuing below fq_codel be > minimal. For example 5g has enormous device buffers all over the place > which we haven't found an answer to aside from nagging the 5g folk to > fix it, and "cake-autorate" > > Linux WiFi had similar problems, fixed now. > > https://www.cs.kau.se/tohojo/airtime-fairness/ > > > > On Sat, Dec 7, 2024 at 2:25=E2=80=AFPM Dave Taht wr= ote: > > > > Concurrent with the rise of codel was the bql algorithm in linux, and > > the combination of the two led to sufficient backpressure for it to > > operate at the native rate of the interface. (AQL does a similar job > > for wifi). This keeps queuing in the device ring down to what can be > > serviced in a single interrupt. > > > > "Bufferbloat Systematic Analysis": > > https://d1wqtxts1xzle7.cloudfront.net/109517412/its2014_bb-libre.pdf > > Byte queue limits: https://lwn.net/Articles/469652/ > > > > BSD has not evolved an equivalent mechanism. Theoretically you could > > run it natively but it would have a lot of jitter from a per-packet > > device ring. > > > > Further most ISPs use a non-native rate for their customer interfaces, > > either using a policer or FIFO shaper and thus the rise of combatting > > that with shaping via fq_codel to slightly below the ISPs' rate to > > move the bottleneck to your own hardware. > > > > We designed CAKE to use a deficit based shaper to be (roughly) the > > inverse of a token bucket shaper to get very close to the ISP rate yet > > shape well > > > > LibreQos (Preseem, Paraqum, Bequant) give ISPs a fq_codel or cake based > shaper. > > The rightest answer was to have the isp correctly shape the download, > > and the uplink device shape the upload. > > > > The bufferbloat project has never had the resources to tackle BSD. > > > > These days all of Linux (and OSX) run fq_codel natively. The FIFO is > dead. > > > > -- > Dave T=C3=A4ht CSO, LibreQos > --000000000000606eaa0628b60fcc Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks Dave, your reply is very informative. I definitely = understand that FreeBSD is somewhat less sophisticated than Linux when it c= omes to this since I've read through a few of the bugs and other messag= es you've posted regarding its implementation. However the Unifi router= s run on Linux, and they still require the user to specify the ISP's se= rvice bandwidth. I think this paragraph from your reply may be the crux of = it:

> Further most I= SPs use a non-native rate for their customer interfaces,
> either using a pol= icer or FIFO shaper and thus the rise of combatting
> that with shaping via f= q_codel to slightly below the ISPs' rate to
> move the bottleneck to your= own hardware.

If the router is able to keep pushing bytes over the ISP's link even = in excess of the allowed bandwidth, then the local router's packets pro= bably aren't sitting in their queues long enough to be dropped by fq-co= del. It sounds like proper traffic shaping needs to happen wherever the bot= tleneck is (I suppose this should have been obvious to me), which in the ca= se of a 300 Mbp/s connection over GPON is somewhere at the ISP. I guess if = the ISP does proper traffic shaping the end user shouldn't need to do a= nything.

Assuming I didn't say anything i= naccurate in my reply, I think I understand why setting a bandwidth limit i= s necessary to make fq-codel work properly on a router with a less than lin= e speed connection from the ISP.

Thank you!

On Sat, Dec 7, 2024 at 5:32=E2=80=AFPM Dav= e Taht <dave.taht@gmail.com&g= t; wrote:
"= variable bandwidth" requires that the queuing below fq_codel be
minimal. For example 5g has enormous device buffers all over the place
which we haven't found an answer to aside from nagging the 5g folk to fix it, and "cake-autorate"

Linux WiFi had similar problems, fixed now.

https://www.cs.kau.se/tohojo/airtime-fairness/



On Sat, Dec 7, 2024 at 2:25=E2=80=AFPM Dave Taht <dave.taht@gmail.com> wrote:
>
> Concurrent with the rise of codel was the bql algorithm in linux, and<= br> > the combination of the two led to sufficient backpressure for it to > operate at the native rate of the interface. (AQL does a similar job > for wifi). This keeps queuing in the device ring down to what can be > serviced in a single interrupt.
>
> "Bufferbloat Systematic Analysis":
> https://d1wqtxts1xzle7.clou= dfront.net/109517412/its2014_bb-libre.pdf
> Byte queue limits: https://lwn.net/Articles/469652/
>
> BSD has not evolved an equivalent mechanism. Theoretically you could > run it natively but it would have a lot of jitter from a per-packet > device ring.
>
> Further most ISPs use a non-native rate for their customer interfaces,=
> either using a policer or FIFO shaper and thus the rise of combatting<= br> > that with shaping via fq_codel to slightly below the ISPs' rate to=
> move the bottleneck to your own hardware.
>
> We designed CAKE to use a deficit based shaper to be (roughly) the
> inverse of a token bucket shaper to get very close to the ISP rate yet=
> shape well
>
> LibreQos (Preseem, Paraqum, Bequant) give ISPs a fq_codel or cake base= d shaper.
> The rightest answer was to have the isp correctly shape the download,<= br> > and the uplink device shape the upload.
>
> The bufferbloat project has never had the resources to tackle BSD.
>
> These days all of Linux (and OSX) run fq_codel natively. The FIFO is d= ead.



--
Dave T=C3=A4ht CSO, LibreQos
--000000000000606eaa0628b60fcc--