[Bloat] CAKE-MQ merged to OpenWrt 25.12 today (February 15)

[Bloat] CAKE-MQ merged to OpenWrt 25.12 today (February 15)

[Frantisek Borsik]

https://forum.openwrt.org/t/cake-mq-backport-of-multi-core-capable-cake-implementation-to-25-12-branch/246349/37

https://github.com/openwrt/packages/pull/28569

All the best,

Frank

Frantisek (Frank) Borsik


*In loving memory of Dave Täht: *1965-2025

https://libreqos.io/2025/04/01/in-loving-memory-of-dave/


https://www.linkedin.com/in/frantisekborsik

Signal, Telegram, WhatsApp: +421919416714

iMessage, mobile: +420775230885

Skype: casioa5302ca

frantisek.borsik@gmail.com

[Bloat] Consumer CPE with Modern AQM?

[Livingood, Jason]

Any recent news on home router CPE that supports fq_codel or cake, etc.? I see a lot of end users on our network that would benefit from this but do not want to run/manage OpenWrt.

Thanks
Jason

[Bloat] Re: Consumer CPE with Modern AQM?

[Frantisek Borsik]

Hey Jason,

MikroTik, eero 6/6E/7 or the best OpenWrt implementation with great GUI (no
need to deal with CLI or LuCI) https://www.gl-inet.com - their mesh should
be coming in Q2. Dave was working with GL-iNet quite a lot.

Alta Labs added CAKE last year, but it's not in their APs yet. Firewalla
has CAKE, too.

These guys from Germany, FRITZ!
<https://fritz.com/en/apps/knowledge-base/FRITZ-Box-7590/228_Prioritizing-internet-access-for-important-network-applications-and-devices-in-the-FRITZ-Box>,
have CAKE, but with some limitations because of their underpowered CPUs.

All the best,

Frank

Frantisek (Frank) Borsik


*In loving memory of Dave Täht: *1965-2025

https://libreqos.io/2025/04/01/in-loving-memory-of-dave/


https://www.linkedin.com/in/frantisekborsik

Signal, Telegram, WhatsApp: +421919416714

iMessage, mobile: +420775230885

Skype: casioa5302ca

frantisek.borsik@gmail.com


On Mon, Feb 16, 2026 at 5:51 PM Livingood, Jason via Bloat <
bloat@lists.bufferbloat.net> wrote:

> Any recent news on home router CPE that supports fq_codel or cake, etc.? I
> see a lot of end users on our network that would benefit from this but do
> not want to run/manage OpenWrt.
>
> Thanks
> Jason
> _______________________________________________
> Bloat mailing list -- bloat@lists.bufferbloat.net
> To unsubscribe send an email to bloat-leave@lists.bufferbloat.net
>

[Bloat] Re: Consumer CPE with Modern AQM?

[Daniel Sterling]

I mean no disrespect, but surely large ISPs should be pushing their
CPE vendors for this functionality? I don't know what the general
community (represented on this list) might be able to do to help here
--

or are you hinting that someone should start a company / release a
product with good SQM support? :)

I for one (as a home user) would be glad to buy some hardware that
advertised support for cake (and IPv6) -- right now I just run a linux
box plugged into my PON ONT (spoofing the MAC address of the ISP's
CPE, so I can fully bypass it). but getting IPv6 working properly was
a huge pain and I'm afeared to upgrade the linux version lest I break
it

-- Dan

On Mon, Feb 16, 2026 at 11:51 AM Livingood, Jason via Bloat
<bloat@lists.bufferbloat.net> wrote:
>
> Any recent news on home router CPE that supports fq_codel or cake, etc.? I see a lot of end users on our network that would benefit from this but do not want to run/manage OpenWrt.
>
> Thanks
> Jason
> _______________________________________________
> Bloat mailing list -- bloat@lists.bufferbloat.net
> To unsubscribe send an email to bloat-leave@lists.bufferbloat.net

[Bloat] Re: Consumer CPE with Modern AQM?

[David Collier-Brown]

A less expensive alternative for ISPs is LibreQOS, rather than new 
end-user devices.
And yet, I think the home router folks are really missing the boat!

--dave

On 2/16/26 12:45, Daniel Sterling wrote:
> I mean no disrespect, but surely large ISPs should be pushing their
> CPE vendors for this functionality? I don't know what the general
> community (represented on this list) might be able to do to help here
> --
>
> or are you hinting that someone should start a company / release a
> product with good SQM support? :)
>
> I for one (as a home user) would be glad to buy some hardware that
> advertised support for cake (and IPv6) -- right now I just run a linux
> box plugged into my PON ONT (spoofing the MAC address of the ISP's
> CPE, so I can fully bypass it). but getting IPv6 working properly was
> a huge pain and I'm afeared to upgrade the linux version lest I break
> it
>
> -- Dan
>
> On Mon, Feb 16, 2026 at 11:51 AM Livingood, Jason via Bloat
> <bloat@lists.bufferbloat.net> wrote:
>> Any recent news on home router CPE that supports fq_codel or cake, etc.? I see a lot of end users on our network that would benefit from this but do not want to run/manage OpenWrt.
>>
>> Thanks
>> Jason
>> _______________________________________________
>> Bloat mailing list -- bloat@lists.bufferbloat.net
>> To unsubscribe send an email to bloat-leave@lists.bufferbloat.net
> _______________________________________________
> Bloat mailing list -- bloat@lists.bufferbloat.net
> To unsubscribe send an email to bloat-leave@lists.bufferbloat.net

-- 
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
davecb@spamcop.net           |              -- Mark Twain

[Bloat] Re: Consumer CPE with Modern AQM?

[Jim Gettys]

On Mon, Feb 16, 2026, 12:46 PM Daniel Sterling <sterling.daniel@gmail.com>
wrote:

> I mean no disrespect, but surely large ISPs should be pushing their
> CPE vendors for this functionality? I don't know what the general
> community (represented on this list) might be able to do to help here
>


Jason, I have said this to your face before at least 10 years ago, and now
I will say it again on the mailing list. One of the pieces of leverage that
a major ISP such as Comcast has is its recommended hardware list, not just
the hardware that you guys distribute directly. Best of course if you
arrange a somewhat neutral third party to pick the winners, but somebody's
got to start picking the hardware on an ongoing basis that actually works
well. As you know because you've been one of the people enabling testing in
the first place, it can be impossible to easily tell the difference between
buffer bloat and the last mile hop of the ISP, or the last Wi-Fi hop from
the CPE. In fact it can easily switch from one to the other on a second by
second basis.

 Even 10 years ago we lacked even the basic tests to expose the problem.
That's been a solved problem now for quite a while, though testing needs to
be done carefully, And doing reproducible Wi-Fi testing is definitely a
challenge due to radio interference and the shared nature of the
transmission channel.

If Comcast and other major ISP's made it clear to the vendors of home
routers and other cpe equipment that they would be penalized explicitly if
they failed Wi-Fi bufferbloat or other buffer bloat testing, that is
probably the fastest way to get the Wi-Fi side of these CPE devices fixed,
or at least reduced to a semi-manageable level.

You and other ISP's have more leverage on what happens in Taiwan and China
by the market power and influence that your companies hold then all of us
geeks do on this mailing list.

A volunteer organization like bufferbloat.net is not in a position to run
such tests in a unbiased way. There are various organizations in the
internet ecosystem which do such testing, and can charge money to get the
testing done for such equipment on an ongoing basis. We have the tests in
hand these days, and by systematically encouraging and publishing the test
results, we can get the chip vendors, which is typically the hold up with
Wi-Fi these days, to pay attention as it will start costing them money, And
ensure that their firmware and device drivers do not just work but don't
add tons of bloat buried in places where we have no chance of touching it.

 The same thing happens with the cable chipsets buried in the integrated
equipment Comcast and others provide, but there are just a few chipset
vendors you need to coerce into proper buffer management.  And typically as
I understand it, you retain control on the firmware that often runs on the
cable chipsets, though I gather even you sometimes need to take cudgels to
certain chip vendors for the broadband chips.

Each generation of these WIFi chipsets of each WiFi chip vendor usually
needs work in their firmware and or their device drivers, BUT you, the ISP
will continue to take service calls from your customers due to buffer
bloat, until this problem is exposed and something that people can vote
with their dollars about, and the people who build the silicon understand
that they can't get away with crap firmware and device drivers and the
current dismal ecosystem found on these devices.

The worst thing about the Chinese ecosystem of CPE equipment is generally
that it's the cheapest and often is running truly antique and insecure
versions of Linux under the covers, and they actually lack the ability to
do more than take what the chipset vendors give them in a sample Linux,
often many years out of date, as that is the running reference software In
the boards that the chip vendors make has samples for the high volume
manufacturers. There is typically nobody home they're they're capable of
that engineering if indeed they have access to the firmware that runs in
some of these chips or in the device drivers. That has meant that only a
small fraction of the overall ecosystem have enabled it to be even possible
to do the necessary technical work in the bowels of the system.

There are a few exceptions to this generality as in the bufferbloat.net
list of things that have debloated code, and many of them ultimately trace
their upstreams to OpenWrt or other open software In some form, but the
reality is that these board support packages are generally seldom updated
to again become current with OpenWrt and other open er source development
occurs. Often half a decade goes by without a chip vender bothering to
update to what we consider up-to-date software.

So even if we got things fixed and working well in OpenWrt, before that
work hands off in consumer gear can actually be many years, if ever, In the
box you can buy at Best buy or online for the least money.  And you guys
get it on the chin with service calls has people plug this junk into their
home network.

I'm happy to chat at length with you by phone about how this dysfunctional
ecosystem functions, to the detriment of all.

Essentially, until we name and shame and make it hurt in the CPE vendors
pocketbooks, by making the problem visible for customers and make vendor
choice active in the economic equation, we will continue to fight buffer
bloat not just for the last 15 years but for the next 15 or 50 years. This
has not actually been a technology problem for quite a while. This is a
market failure and unless we treat it that way, we will live with
bufferbloat indefinitely.

All the best, and my thanks for all the support that Comcast, more than
most/all other major ISP's, have given this project from very early on. It
is truly appreciated by us here, myself maybe even more than most on the
list can know. And you were the person who has made that support happen.

You know how to call me! I'm in your contacts list. I'm happy to explain
how this dysfunctional mess of an ecosystem actually works.

All the best and again all my thanks,

Jim Gettys

or are you hinting that someone should start a company / release a
> product with good SQM support? :)
>
> I for one (as a home user) would be glad to buy some hardware that
> advertised support for cake (and IPv6) -- right now I just run a linux
> box plugged into my PON ONT (spoofing the MAC address of the ISP's
> CPE, so I can fully bypass it). but getting IPv6 working properly was
> a huge pain and I'm afeared to upgrade the linux version lest I break
> it
>
> -- Dan
>
> On Mon, Feb 16, 2026 at 11:51 AM Livingood, Jason via Bloat
> <bloat@lists.bufferbloat.net> wrote:
> >
> > Any recent news on home router CPE that supports fq_codel or cake, etc.?
> I see a lot of end users on our network that would benefit from this but do
> not want to run/manage OpenWrt.
> >
> > Thanks
> > Jason
> > _______________________________________________
> > Bloat mailing list -- bloat@lists.bufferbloat.net
> > To unsubscribe send an email to bloat-leave@lists.bufferbloat.net
> _______________________________________________
> Bloat mailing list -- bloat@lists.bufferbloat.net
> To unsubscribe send an email to bloat-leave@lists.bufferbloat.net
>

[Bloat] Re: Consumer CPE with Modern AQM?

[Jan Ceuleers]

On 17/02/2026 05:12, Jim Gettys wrote:
> If Comcast and other major ISP's made it clear to the vendors of home
> routers and other cpe equipment that they would be penalized explicitly if
> they failed Wi-Fi bufferbloat or other buffer bloat testing, that is
> probably the fastest way to get the Wi-Fi side of these CPE devices fixed,
> or at least reduced to a semi-manageable level.

(Not replying specifically to what is quoted above).

I have a question. Please feel free to refer me to some resource I
should go and read.

The consumer CPE business is extremely cost-sensitive, and so such
devices must rely on hardware forwarding, because it would be
unaffordable to equip them with enough cores tobe able to lift all
packets to the CPU. How does one implement AQM in such an architecture?

[Bloat] Re: [EXTERNAL] Re: Re: Consumer CPE with Modern AQM?

[Livingood, Jason]

> One of the pieces of leverage that a major ISP such as Comcast has is its recommended hardware list, not just the hardware that you guys distribute directly.

Sure - and we did that for all the COAM cable modems years ago. We now have a subset of end users that run their own router, which they have purchased. Some of these are shockingly old. I’m working on an outreach campaign to those users now - with the idea being to offer advice on how to improve latency by replacing device X with a new device from a list at some TBD URL.

The motivation for an ISP is clear - we see complaints from users with these old routers saying service is poor. But their DOCSIS or EPON connection quality is great. The root cause is the router and/or AP - so working to help them understand and address this.

> If Comcast and other major ISP's made it clear to the vendors of home routers and other cpe equipment that they would be penalized explicitly if they failed Wi-Fi bufferbloat or other buffer bloat testing, that is probably the fastest way to get the Wi-Fi side of these CPE devices fixed, or at least reduced to a semi-manageable level.

We already do this for cable modems that connect to and authenticate on the network. AQM is build into every D3.1 CM that ships today - whether we provide it or a user buys it in retail.

But we do not have a way to qualify COAM routers and APs. But I suppose a company may ask how to get on the ‘suggested list’ of routers - in which case it is to implement AQM and/or other latency-reducing technologies. ;-)

JL

[Bloat] Re: [EXTERNAL] Re: Consumer CPE with Modern AQM?

[Livingood, Jason]

Certainly our​ vendors already support this and for several years. I am trying to find ways to improve things for end users that buy and manage their own CPE router.

From: Daniel Sterling <sterling.daniel@gmail.com>
Date: Monday, February 16, 2026 at 12:46
To: Livingood, Jason <Jason_Livingood@comcast.com>
Cc: bloat <bloat@lists.bufferbloat.net>
Subject: [EXTERNAL] Re: [Bloat] Consumer CPE with Modern AQM?

I mean no disrespect, but surely large ISPs should be pushing their
CPE vendors for this functionality? I don't know what the general
community (represented on this list) might be able to do to help here
--

or are you hinting that someone should start a company / release a
product with good SQM support? :)

I for one (as a home user) would be glad to buy some hardware that
advertised support for cake (and IPv6) -- right now I just run a linux
box plugged into my PON ONT (spoofing the MAC address of the ISP's
CPE, so I can fully bypass it). but getting IPv6 working properly was
a huge pain and I'm afeared to upgrade the linux version lest I break
it

-- Dan

On Mon, Feb 16, 2026 at 11:51 AM Livingood, Jason via Bloat
<bloat@lists.bufferbloat.net> wrote:
>
> Any recent news on home router CPE that supports fq_codel or cake, etc.? I see a lot of end users on our network that would benefit from this but do not want to run/manage OpenWrt.
>
> Thanks
> Jason
> _______________________________________________
> Bloat mailing list -- bloat@lists.bufferbloat.net
> To unsubscribe send an email to bloat-leave@lists.bufferbloat.net

[Bloat] Re: [Cake] CAKE-MQ merged to OpenWrt 25.12 today (February 15)

[dave seddon]

Woot woot!

Thanks for the mq-cake patches. !

I got them working on NixOS 6.12 ( and next-net patches on 6.18.8 )

[das@l2:~/nixos/desktop/l2/mq-cake-orchestrator]$ uname -a
Linux l2 6.12.68 #1-NixOS SMP PREEMPT_DYNAMIC Fri Jan 30 09:28:49 UTC
2026 x86_64 GNU/Linux


I'm currently setting up a load-testing harness.  The idea will be to
generate MANY 5-tuple flows to really stress out the qdiscs

Early results, but they look great!

[das@l2:~/nixos/desktop/l2/mq-cake-orchestrator]$ sudo
./mq-cake-orchestrator --config config-test.yaml
mq-cake-orchestrator starting...
Config: 1 tools, 3 qdiscs, 2 flow counts

=== Pre-flight Checks ===
Pre-flight: namespaces exist... OK
Pre-flight: interfaces present... OK
Pre-flight: forwarding enabled... OK
Pre-flight: end-to-end ping... OK
Pre-flight: tools available... OK
Pre-flight: baseline latency... OK

=== Pre-flight Complete ===
Running 6 test points

[1/6] qdisc=fq_codel flows=1 tool=iperf2
  Switching qdisc to fq_codel...
  Throughput: 9.41 Gbps
[2/6] qdisc=fq_codel flows=10 tool=iperf2
  Throughput: 9.43 Gbps
[3/6] qdisc=cake flows=1 tool=iperf2
  Switching qdisc to cake...
  Throughput: 6.93 Gbps
[4/6] qdisc=cake flows=10 tool=iperf2
  Throughput: 4.37 Gbps                                 <---- cake
[5/6] qdisc=mq-cake flows=1 tool=iperf2
  Switching qdisc to mq-cake...
  Throughput: 7.17 Gbps
[6/6] qdisc=mq-cake flows=10 tool=iperf2
  Throughput: 9.44 Gbps                                   <-----
mq-cake ... Actually, that's interesting.  higher than fq_codel

=== Test Complete ===
Completed 6 test points
Wrote /tmp/mq-cake-results/results-20260216-220443.json
Wrote /tmp/mq-cake-results/results-20260216-220443.csv
Results saved to /tmp/mq-cake-results
Shutdown complete

[das@l2:~/nixos/desktop/l2/mq-cake-orchestrator]$

I'm using some intel cards, which I've adjusted to x8 queues each

[das@l2:~/nixos/desktop/l2/mq-cake-orchestrator]$ lspci | grep -i eth
| grep -i intel
23:00.0 Ethernet controller: Intel Corporation Ethernet Controller
X710 for 10GbE SFP+ (rev 01)
23:00.1 Ethernet controller: Intel Corporation Ethernet Controller
X710 for 10GbE SFP+ (rev 01)
42:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
42:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit
SFI/SFP+ Network Connection (rev 01)

On Sun, Feb 15, 2026 at 9:38 AM Frantisek Borsik
<frantisek.borsik@gmail.com> wrote:
>
> https://forum.openwrt.org/t/cake-mq-backport-of-multi-core-capable-cake-implementation-to-25-12-branch/246349/37
>
> https://github.com/openwrt/packages/pull/28569
>
> All the best,
>
> Frank
>
> Frantisek (Frank) Borsik
>
>
> *In loving memory of Dave Täht: *1965-2025
>
> https://libreqos.io/2025/04/01/in-loving-memory-of-dave/
>
>
> https://www.linkedin.com/in/frantisekborsik
>
> Signal, Telegram, WhatsApp: +421919416714
>
> iMessage, mobile: +420775230885
>
> Skype: casioa5302ca
>
> frantisek.borsik@gmail.com
> _______________________________________________
> Cake mailing list -- cake@lists.bufferbloat.net
> To unsubscribe send an email to cake-leave@lists.bufferbloat.net



-- 
Regards,
Dave Seddon
+1 415 857 5102

[Bloat] Re: [Cake] CAKE-MQ merged to OpenWrt 25.12 today (February 15)

[Stephen Hemminger]

On Mon, 16 Feb 2026 22:10:31 -0800
dave seddon <dave.seddon.ca@gmail.com> wrote:

> Woot woot!
> 
> Thanks for the mq-cake patches. !
> 
> I got them working on NixOS 6.12 ( and next-net patches on 6.18.8 )
> 
> [das@l2:~/nixos/desktop/l2/mq-cake-orchestrator]$ uname -a
> Linux l2 6.12.68 #1-NixOS SMP PREEMPT_DYNAMIC Fri Jan 30 09:28:49 UTC
> 2026 x86_64 GNU/Linux
> 
> 
> I'm currently setting up a load-testing harness.  The idea will be to
> generate MANY 5-tuple flows to really stress out the qdiscs
> 
> Early results, but they look great!

How fast is the CPU, because on current Arm Cortex-A53 cake is cpu
limited and max's out around 500Mbps but can do fq-codel at 900.
Could be fixed by tuning?

[Bloat] Re: [Cake] CAKE-MQ merged to OpenWrt 25.12 today (February 15)

[Toke Høiland-Jørgensen]

dave seddon <dave.seddon.ca@gmail.com> writes:

> === Pre-flight Complete ===
> Running 6 test points
>
> [1/6] qdisc=fq_codel flows=1 tool=iperf2
>   Switching qdisc to fq_codel...
>   Throughput: 9.41 Gbps
> [2/6] qdisc=fq_codel flows=10 tool=iperf2
>   Throughput: 9.43 Gbps
> [3/6] qdisc=cake flows=1 tool=iperf2
>   Switching qdisc to cake...
>   Throughput: 6.93 Gbps
> [4/6] qdisc=cake flows=10 tool=iperf2
>   Throughput: 4.37 Gbps                                 <---- cake
> [5/6] qdisc=mq-cake flows=1 tool=iperf2
>   Switching qdisc to mq-cake...
>   Throughput: 7.17 Gbps
> [6/6] qdisc=mq-cake flows=10 tool=iperf2
>   Throughput: 9.44 Gbps                                   <-----
> mq-cake ... Actually, that's interesting.  higher than fq_codel

Are you running fq_codel as the root qdisc? Because in that case you're
running through the single qdisc lock, which could explain the
difference. Try running separate fq_codel instances beneath an 'mq'
qdisc as the root.

Also, if you're not setting a shaping rate, cake_mq is basically the
same as just installing an mq qdisc at the root and having separate cake
instances beneath that. So to test the multi-core shaper algorithm
you'll need to set a rate ('bandwidth' parameter).

-Toke

[Bloat] Re: [Cake] Re: Re: CAKE-MQ merged to OpenWrt 25.12 today (February 15)

[Stephen Hemminger]

On Tue, 17 Feb 2026 14:23:52 +0100
Toke Høiland-Jørgensen via Cake <cake@lists.bufferbloat.net> wrote:

> dave seddon <dave.seddon.ca@gmail.com> writes:
> 
> > === Pre-flight Complete ===
> > Running 6 test points
> >
> > [1/6] qdisc=fq_codel flows=1 tool=iperf2
> >   Switching qdisc to fq_codel...
> >   Throughput: 9.41 Gbps
> > [2/6] qdisc=fq_codel flows=10 tool=iperf2
> >   Throughput: 9.43 Gbps
> > [3/6] qdisc=cake flows=1 tool=iperf2
> >   Switching qdisc to cake...
> >   Throughput: 6.93 Gbps
> > [4/6] qdisc=cake flows=10 tool=iperf2
> >   Throughput: 4.37 Gbps                                 <---- cake
> > [5/6] qdisc=mq-cake flows=1 tool=iperf2
> >   Switching qdisc to mq-cake...
> >   Throughput: 7.17 Gbps
> > [6/6] qdisc=mq-cake flows=10 tool=iperf2
> >   Throughput: 9.44 Gbps                                   <-----
> > mq-cake ... Actually, that's interesting.  higher than fq_codel  
> 
> Are you running fq_codel as the root qdisc? Because in that case you're
> running through the single qdisc lock, which could explain the
> difference. Try running separate fq_codel instances beneath an 'mq'
> qdisc as the root.
> 
> Also, if you're not setting a shaping rate, cake_mq is basically the
> same as just installing an mq qdisc at the root and having separate cake
> instances beneath that. So to test the multi-core shaper algorithm
> you'll need to set a rate ('bandwidth' parameter).
> 

This is what the OpenWrt SQM scripts seem to favor.
I probably need to tune the adaption overhead

# tc qdisc show dev wan
qdisc htb 1: root refcnt 2 r2q 10 default 0x12 direct_packets_stat 0 direct_qlen 1000
qdisc fq_codel 120: parent 1:12 limit 1001p flows 1024 quantum 300 target 5ms interval 100ms memory_limit 4Mb drop_batch 64 
qdisc fq_codel 130: parent 1:13 limit 1001p flows 1024 quantum 300 target 5ms interval 100ms memory_limit 4Mb drop_batch 64 
qdisc fq_codel 110: parent 1:11 limit 1001p flows 1024 quantum 300 target 5ms interval 100ms memory_limit 4Mb drop_batch 64 

It is dumb that in US it is standard to offer 1G down and 10M up!

[Bloat] Re: [Cake] Re: Re: CAKE-MQ merged to OpenWrt 25.12 today (February 15)

[Toke Høiland-Jørgensen]

Stephen Hemminger <stephen@networkplumber.org> writes:

> On Tue, 17 Feb 2026 14:23:52 +0100
> Toke Høiland-Jørgensen via Cake <cake@lists.bufferbloat.net> wrote:
>
>> dave seddon <dave.seddon.ca@gmail.com> writes:
>> 
>> > === Pre-flight Complete ===
>> > Running 6 test points
>> >
>> > [1/6] qdisc=fq_codel flows=1 tool=iperf2
>> >   Switching qdisc to fq_codel...
>> >   Throughput: 9.41 Gbps
>> > [2/6] qdisc=fq_codel flows=10 tool=iperf2
>> >   Throughput: 9.43 Gbps
>> > [3/6] qdisc=cake flows=1 tool=iperf2
>> >   Switching qdisc to cake...
>> >   Throughput: 6.93 Gbps
>> > [4/6] qdisc=cake flows=10 tool=iperf2
>> >   Throughput: 4.37 Gbps                                 <---- cake
>> > [5/6] qdisc=mq-cake flows=1 tool=iperf2
>> >   Switching qdisc to mq-cake...
>> >   Throughput: 7.17 Gbps
>> > [6/6] qdisc=mq-cake flows=10 tool=iperf2
>> >   Throughput: 9.44 Gbps                                   <-----
>> > mq-cake ... Actually, that's interesting.  higher than fq_codel  
>> 
>> Are you running fq_codel as the root qdisc? Because in that case you're
>> running through the single qdisc lock, which could explain the
>> difference. Try running separate fq_codel instances beneath an 'mq'
>> qdisc as the root.
>> 
>> Also, if you're not setting a shaping rate, cake_mq is basically the
>> same as just installing an mq qdisc at the root and having separate cake
>> instances beneath that. So to test the multi-core shaper algorithm
>> you'll need to set a rate ('bandwidth' parameter).
>> 
>
> This is what the OpenWrt SQM scripts seem to favor.
> I probably need to tune the adaption overhead
>
> # tc qdisc show dev wan
> qdisc htb 1: root refcnt 2 r2q 10 default 0x12 direct_packets_stat 0 direct_qlen 1000
> qdisc fq_codel 120: parent 1:12 limit 1001p flows 1024 quantum 300 target 5ms interval 100ms memory_limit 4Mb drop_batch 64 
> qdisc fq_codel 130: parent 1:13 limit 1001p flows 1024 quantum 300 target 5ms interval 100ms memory_limit 4Mb drop_batch 64 
> qdisc fq_codel 110: parent 1:11 limit 1001p flows 1024 quantum 300
> target 5ms interval 100ms memory_limit 4Mb drop_batch 64

If you set 'script' to 'layer_cake.qos' or 'piece_of_cake.qos' in the
config, you'll get a CAKE setup instead. On OpenWrt 25.12 (still in -rc)
this will automatically select cake_mq where the system supports it.

> It is dumb that in US it is standard to offer 1G down and 10M up!

Yes, very. Don't think it's actually possible to use that 1G link with
TCP without ACK thinning of some kind. The full-MTU-to-ACK ratio is ~47
(1500/32; this is assuming one 64-byte ACK for every two 1500-byte
packets), so you can only use ~470 Mbps downstream with 10M up. CAKE's
ACK thinning should help here, but you need to turn that on explicitly
by adding 'ack-filter' to eqdisc_opts in the config.

-Toke

[Bloat] Re: [Cake] Re: Re: CAKE-MQ merged to OpenWrt 25.12 today (February 15)

[Jonas Köppeler]

On 2/17/26 5:32 PM, Toke Høiland-Jørgensen via Bloat wrote:
> Stephen Hemminger <stephen@networkplumber.org> writes:
>
>> On Tue, 17 Feb 2026 14:23:52 +0100
>> Toke Høiland-Jørgensen via Cake <cake@lists.bufferbloat.net> wrote:
>>
>>> dave seddon <dave.seddon.ca@gmail.com> writes:
>>>
>>>> === Pre-flight Complete ===
>>>> Running 6 test points
>>>>
>>>> [1/6] qdisc=fq_codel flows=1 tool=iperf2
>>>>    Switching qdisc to fq_codel...
>>>>    Throughput: 9.41 Gbps
>>>> [2/6] qdisc=fq_codel flows=10 tool=iperf2
>>>>    Throughput: 9.43 Gbps
>>>> [3/6] qdisc=cake flows=1 tool=iperf2
>>>>    Switching qdisc to cake...
>>>>    Throughput: 6.93 Gbps
>>>> [4/6] qdisc=cake flows=10 tool=iperf2
>>>>    Throughput: 4.37 Gbps                                 <---- cake
>>>> [5/6] qdisc=mq-cake flows=1 tool=iperf2
>>>>    Switching qdisc to mq-cake...
>>>>    Throughput: 7.17 Gbps
>>>> [6/6] qdisc=mq-cake flows=10 tool=iperf2
>>>>    Throughput: 9.44 Gbps                                   <-----
>>>> mq-cake ... Actually, that's interesting.  higher than fq_codel
>>> Are you running fq_codel as the root qdisc? Because in that case you're
>>> running through the single qdisc lock, which could explain the
>>> difference. Try running separate fq_codel instances beneath an 'mq'
>>> qdisc as the root.
>>>
>>> Also, if you're not setting a shaping rate, cake_mq is basically the
>>> same as just installing an mq qdisc at the root and having separate cake
>>> instances beneath that. So to test the multi-core shaper algorithm
>>> you'll need to set a rate ('bandwidth' parameter).
>>>
>> This is what the OpenWrt SQM scripts seem to favor.
>> I probably need to tune the adaption overhead
>>
>> # tc qdisc show dev wan
>> qdisc htb 1: root refcnt 2 r2q 10 default 0x12 direct_packets_stat 0 direct_qlen 1000
>> qdisc fq_codel 120: parent 1:12 limit 1001p flows 1024 quantum 300 target 5ms interval 100ms memory_limit 4Mb drop_batch 64
>> qdisc fq_codel 130: parent 1:13 limit 1001p flows 1024 quantum 300 target 5ms interval 100ms memory_limit 4Mb drop_batch 64
>> qdisc fq_codel 110: parent 1:11 limit 1001p flows 1024 quantum 300
>> target 5ms interval 100ms memory_limit 4Mb drop_batch 64
> If you set 'script' to 'layer_cake.qos' or 'piece_of_cake.qos' in the
> config, you'll get a CAKE setup instead. On OpenWrt 25.12 (still in -rc)
> this will automatically select cake_mq where the system supports it.
Actually on that note: I was playing around with the latest OpenWrt 
(25.12 snapshot from today). I can successfully install cake_mq 
manually. However, the sqm script does not install cake_mq (to be more 
specific using the web interface luci-sqm-script). I opened a github 
issue providing more details: 
https://github.com/tohojo/sqm-scripts/issues/182

Jonas

[Bloat] Re: [Cake] Re: Re: CAKE-MQ merged to OpenWrt 25.12 today (February 15)

[dave seddon]

[-- Attachment #1: Type: text/plain, Size: 8798 bytes --]

Thanks for the replies guys.

I'm using a desktop class machine with a Ryzen Threadripper PRO 3945WX
(12 cores/24 threads).  - It's really not fast enough for what I was
attempting.  When you are used to using more powerful machines at
work, it's easy to forget how powerful server class machines are.

I tried creating many flows using a combination of tools, but this
just saturates all the cores, causing RTTs to spike due to CPU
contention.  The idea was to simulate lots of flows like you might
have at a conference, but I'm going to need more and more powerful
machines.

This is the mq-cake config

  Flows: fping=1, iperf2=300, wrk=100, dnsperf=20, flent=1, crusader=1
  Qdisc: mq-cake
    ixgbe0:
      qdisc mq 1: root
      qdisc cake 8005: parent 1:3 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
      qdisc cake 8003: parent 1:1 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
      qdisc cake 8007: parent 1:5 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
      qdisc cake 8009: parent 1:7 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
      qdisc cake 8004: parent 1:2 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
      qdisc cake 8006: parent 1:4 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
      qdisc cake 800a: parent 1:8 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
      qdisc cake 8008: parent 1:6 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
    ixgbe1:
      qdisc mq 1: root
      qdisc cake 800b: parent 1:1 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
      qdisc cake 800f: parent 1:5 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
      qdisc cake 800d: parent 1:3 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
      qdisc cake 8011: parent 1:7 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
      qdisc cake 800c: parent 1:2 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
      qdisc cake 800e: parent 1:4 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
      qdisc cake 8010: parent 1:6 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0
      qdisc cake 8012: parent 1:8 bandwidth 10Gbit diffserv4
triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
0

[das@l2:~/nixos/desktop/l2]$ uname -a
Linux l2 6.12.68 #1-NixOS SMP PREEMPT_DYNAMIC Fri Jan 30 09:28:49 UTC
2026 x86_64 GNU/Linux

Thanks for the idea of using mq and fq_codel, that's a very interesting idea.

Not sure if this diagram will come through in email, cos it doesn't
look good in this interface:

┌─────────────────────────┐ ┌─────────────────────────┐
┌─────────────────────────┐
│ ns-gen-a │ │ ns-dut │ │ ns-gen-b │
│ (Load Generator) │ │ (Device Under Test) │ │ (Server) │
│ │ │ │ │ │
│ ┌───────────────────┐ │ │ ┌───────────────────┐ │ │ ┌───────────────────┐ │
│ │ enp35s0f0np0 │ │ │ │ ixgbe0 │ │ │ │ enp35s0f1np1 │ │
│ │ Intel X710 p0 │──╋───────╋──│ Intel 82599 p0 │ │ │ │ Intel X710 p1 │ │
│ │ 10.1.0.2/24 │ │ SFP+ │ │ 10.1.0.1/24 │ │ │ │ 10.2.0.2/24 │ │
│ └───────────────────┘ │ Cable │ └─────────┬─────────┘ │ │
└─────────┬─────────┘ │
│ │ │ │ │ │ │ │
│ netem: 30ms ±3ms │ │ ┌──────┴──────┐ │ │ netem: 30ms ±3ms │
│ │ │ │ Forwarding │ │ │ │ │
│ Tools: │ │ │ (ip_forward)│ │ │ Services: │ │
│ - iperf2 client │ │ └──────┬──────┘ │ │ - iperf2 │ │
│ - iperf3 client │ │ │ │ │ - iperf3 │ │
│ - wrk (HTTP) │ │ ┌─────────┴─────────┐ │ │ - flent │ │
│ - dnsperf │ │ │ ixgbe1 │ │ │ - crusader│ │
│ - flent │ │ │ Intel 82599 p1 │──╋───────╋──- nginx │ │
│ - crusader client │ │ │ 10.2.0.1/24 │ │ SFP+ │ - PowerDNS│ │
│ - fping │ │ └───────────────────┘ │ Cable │ │ │
│ │ │ │ │ │ │
└─────────────────────────┘ │ Qdisc under test: │ └────────────┴────────────┘
│ - fq_codel │
│ - cake │
│ - mq-cake (mq+cake) │
│ │
└─────────────────────────┘

On Tue, Feb 17, 2026 at 8:32 AM Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>
> Stephen Hemminger <stephen@networkplumber.org> writes:
>
> > On Tue, 17 Feb 2026 14:23:52 +0100
> > Toke Høiland-Jørgensen via Cake <cake@lists.bufferbloat.net> wrote:
> >
> >> dave seddon <dave.seddon.ca@gmail.com> writes:
> >>
> >> > === Pre-flight Complete ===
> >> > Running 6 test points
> >> >
> >> > [1/6] qdisc=fq_codel flows=1 tool=iperf2
> >> >   Switching qdisc to fq_codel...
> >> >   Throughput: 9.41 Gbps
> >> > [2/6] qdisc=fq_codel flows=10 tool=iperf2
> >> >   Throughput: 9.43 Gbps
> >> > [3/6] qdisc=cake flows=1 tool=iperf2
> >> >   Switching qdisc to cake...
> >> >   Throughput: 6.93 Gbps
> >> > [4/6] qdisc=cake flows=10 tool=iperf2
> >> >   Throughput: 4.37 Gbps                                 <---- cake
> >> > [5/6] qdisc=mq-cake flows=1 tool=iperf2
> >> >   Switching qdisc to mq-cake...
> >> >   Throughput: 7.17 Gbps
> >> > [6/6] qdisc=mq-cake flows=10 tool=iperf2
> >> >   Throughput: 9.44 Gbps                                   <-----
> >> > mq-cake ... Actually, that's interesting.  higher than fq_codel
> >>
> >> Are you running fq_codel as the root qdisc? Because in that case you're
> >> running through the single qdisc lock, which could explain the
> >> difference. Try running separate fq_codel instances beneath an 'mq'
> >> qdisc as the root.
> >>
> >> Also, if you're not setting a shaping rate, cake_mq is basically the
> >> same as just installing an mq qdisc at the root and having separate cake
> >> instances beneath that. So to test the multi-core shaper algorithm
> >> you'll need to set a rate ('bandwidth' parameter).
> >>
> >
> > This is what the OpenWrt SQM scripts seem to favor.
> > I probably need to tune the adaption overhead
> >
> > # tc qdisc show dev wan
> > qdisc htb 1: root refcnt 2 r2q 10 default 0x12 direct_packets_stat 0 direct_qlen 1000
> > qdisc fq_codel 120: parent 1:12 limit 1001p flows 1024 quantum 300 target 5ms interval 100ms memory_limit 4Mb drop_batch 64
> > qdisc fq_codel 130: parent 1:13 limit 1001p flows 1024 quantum 300 target 5ms interval 100ms memory_limit 4Mb drop_batch 64
> > qdisc fq_codel 110: parent 1:11 limit 1001p flows 1024 quantum 300
> > target 5ms interval 100ms memory_limit 4Mb drop_batch 64
>
> If you set 'script' to 'layer_cake.qos' or 'piece_of_cake.qos' in the
> config, you'll get a CAKE setup instead. On OpenWrt 25.12 (still in -rc)
> this will automatically select cake_mq where the system supports it.
>
> > It is dumb that in US it is standard to offer 1G down and 10M up!
>
> Yes, very. Don't think it's actually possible to use that 1G link with
> TCP without ACK thinning of some kind. The full-MTU-to-ACK ratio is ~47
> (1500/32; this is assuming one 64-byte ACK for every two 1500-byte
> packets), so you can only use ~470 Mbps downstream with 10M up. CAKE's
> ACK thinning should help here, but you need to turn that on explicitly
> by adding 'ack-filter' to eqdisc_opts in the config.
>
> -Toke



-- 
Regards,
Dave Seddon
+1 415 857 5102

[-- Attachment #2: test-diagram.txt --]
[-- Type: text/plain, Size: 3583 bytes --]

  ┌─────────────────────────┐       ┌─────────────────────────┐       ┌─────────────────────────┐
  │      ns-gen-a           │       │        ns-dut           │       │      ns-gen-b           │
  │    (Load Generator)     │       │   (Device Under Test)   │       │       (Server)          │
  │                         │       │                         │       │                         │
  │  ┌───────────────────┐  │       │  ┌───────────────────┐  │       │  ┌───────────────────┐  │
  │  │   enp35s0f0np0    │  │       │  │      ixgbe0       │  │       │  │   enp35s0f1np1    │  │
  │  │   Intel X710 p0   │──╋───────╋──│  Intel 82599 p0   │  │       │  │   Intel X710 p1   │  │
  │  │   10.1.0.2/24     │  │  SFP+ │  │   10.1.0.1/24     │  │       │  │   10.2.0.2/24     │  │
  │  └───────────────────┘  │ Cable │  └─────────┬─────────┘  │       │  └─────────┬─────────┘  │
  │                         │       │            │            │       │            │            │
  │   netem: 30ms ±3ms      │       │     ┌──────┴──────┐     │       │   netem: 30ms ±3ms      │
  │                         │       │     │  Forwarding │     │       │            │            │
  │  Tools:                 │       │     │ (ip_forward)│     │       │  Services: │            │
  │  - iperf2 client        │       │     └──────┬──────┘     │       │  - iperf2  │            │
  │  - iperf3 client        │       │            │            │       │  - iperf3  │            │
  │  - wrk (HTTP)           │       │  ┌─────────┴─────────┐  │       │  - flent   │            │
  │  - dnsperf              │       │  │      ixgbe1       │  │       │  - crusader│            │
  │  - flent                │       │  │  Intel 82599 p1   │──╋───────╋──- nginx   │            │
  │  - crusader client      │       │  │   10.2.0.1/24     │  │  SFP+ │  - PowerDNS│            │
  │  - fping                │       │  └───────────────────┘  │ Cable │            │            │
  │                         │       │                         │       │            │            │
  └─────────────────────────┘       │  Qdisc under test:      │       └────────────┴────────────┘
                                    │  - fq_codel             │
                                    │  - cake                 │
                                    │  - mq-cake (mq+cake)    │
                                    │                         │
                                    └─────────────────────────┘

[Bloat] Re: [Cake] Re: Re: CAKE-MQ merged to OpenWrt 25.12 today (February 15)

[Toke Høiland-Jørgensen]

dave seddon <dave.seddon.ca@gmail.com> writes:

> Thanks for the replies guys.
>
> I'm using a desktop class machine with a Ryzen Threadripper PRO 3945WX
> (12 cores/24 threads).  - It's really not fast enough for what I was
> attempting.  When you are used to using more powerful machines at
> work, it's easy to forget how powerful server class machines are.
>
> I tried creating many flows using a combination of tools, but this
> just saturates all the cores, causing RTTs to spike due to CPU
> contention.  The idea was to simulate lots of flows like you might
> have at a conference, but I'm going to need more and more powerful
> machines.
>
> This is the mq-cake config
>
>   Flows: fping=1, iperf2=300, wrk=100, dnsperf=20, flent=1, crusader=1
>   Qdisc: mq-cake
>     ixgbe0:
>       qdisc mq 1: root
>       qdisc cake 8005: parent 1:3 bandwidth 10Gbit diffserv4
> triple-isolate nat wash no-ack-filter split-gso rtt 100ms raw overhead
> 0

[...]

Erm, why this setup? You're asking each instance of cake to shape at
10Gbit (on a single CPU), for a total bandwidth of 80 Gbit. Multi-queue
shaping with cake is exactly the use case that the cake_mq qdisc is
meant to be used for, so try that?

-Toke