[Starlink] Bufferbloat cure question

[Starlink] Bufferbloat cure question

[Colin_Higbie]

I have a question for the experts here. This is not necessarily limited to Starlink, but at Starlink's latest bandwidth capabilites, pushing past 300Mbps, this becomes relevant to Starlink. 

In my simple tests at https://www.waveform.com/tools/bufferbloat, using both Starlink and a new 1Gbps symmetric fiber connection (we're keeping Starlink for backup), I see decent but not great bufferbloat and jitter values of about 16-25ms by setting my router to a standard Adaptive QoS setting (an Asus consumer-grade router that supports 1Gbps WAN connection, RT-AX86U-Pro). When I use CAKE at lower speeds, it impressively crushes the impact under load down to about 0-1ms. But that only holds at lower speeds, meaning a bad day for Starlink where it's not pushing 300Mbps or if I set my router to limit bandwidth to about 280Mbps. On fiber, I have to limit the 1Gbps connection to 280Mbps to run CAKE and eliminate the bufferbloat.

At 300Mbps or higher, bufferbloat and jitter skyrocket to hundreds of ms, much worse than not using CAKE at all. I can see one of the 4 router CPU cores spikes to 100% at those transfer rates whenever I don't limit bandwidth to something below 280Mbps. My crude understanding is that CAKE can only run on a single CPU core (can't be multithreaded to take advantage of the other 3 cores) and becomes CPU bound at higher transfer rates. Then, when the CPU is maxed, CAKE will do more harm than good, which is exactly why bufferbloat actually increases with CAKE above 280Mbps. I understand that FQ-CoDel is similar.

All those tests are on wired connections, so these are not Wi-Fi effects.

Presumably, a faster CPU would increase the rate before CAKE becomes the bottleneck. But to my surprise, I can't find anything on the Internet that addresses solutions to bufferbloat at higher bandwidths or confirms that some modern routers have fast enough CPU to run at the full bandwidth capability of their WAN port.

I had been told that a simple Ubiquiti Gateway-Lite between the Internet and my wireless router could take on this role, that it had a sufficiently powerful CPU to handle up to about 1Gbps (or very close to that), but in my testing, first, it doesn't appear to include CAKE, and turning on FQ-CoDel, as its closest alternative, requires disabling its hardware acceleration, which dramatically harms both throughput and latency under load.

I'm willing to go up to something more expensive and powerful (like the Ubiquiti UDR7, where I like Ubiquiti for the group management of all the networking hardware from one interface with our existing Ubiquiti Wi-Fi APs, but that's not a requirement, and I don't think any of the Ubiquiti routers include CAKE), but given how hard it is to find anything on the Internet about this, it makes me suspicious that I'm chasing the wrong problem.

My chief concern is keeping jitter to a level where it will not cause any issues for VoIP or video calls. At 16-25ms, that's not bad, but it can lead to some stuttering and glitching on calls, shifting QoS to prioritize voice and video calls helps, but if I could just eliminate the bufferbloat, seems that would be a better solution.

Questions:

1. If CAKE is not viable due to CPU constraints on most modern routers as their WAN port transfer rate, then what is the recommended solution? 

2. Are there any routers that can handle CAKE at 1Gbps or 2Gbps?

3. Is there a site that provides ratings for different routers at these high bandwidths or lists MODERN 1Gbps+ routers that support CAKE up to their full bandwidth capacity? 

4. Or, at bandwidths above 300Mbps, are these 16-25ms increases considered acceptable, meaning I'm chasing perfection when I should just be happy with 16-25ms latency impact and jitter under load and accept that occasionally any Internet connection will have some problems when the connection gets crowded? 

How should I tackle finding a solution? And if I'm asking the wrong questions, please feel free to mock my ignorance in your response. :-) Let my foolishness be used educate others (and myself). 

Many thanks,
Colin

[Starlink] Re: Bufferbloat cure question

[Sebastian Moeller]

Hi Colin,


> On 31. Dec 2025, at 23:19, Colin_Higbie via Starlink <starlink@lists.bufferbloat.net> wrote:
> 
> I have a question for the experts here. This is not necessarily limited to Starlink, but at Starlink's latest bandwidth capabilites, pushing past 300Mbps, this becomes relevant to Starlink. 
> 
> In my simple tests at https://www.waveform.com/tools/bufferbloat, using both Starlink and a new 1Gbps symmetric fiber connection (we're keeping Starlink for backup), I see decent but not great bufferbloat and jitter values of about 16-25ms by setting my router to a standard Adaptive QoS setting (an Asus consumer-grade router that supports 1Gbps WAN connection, RT-AX86U-Pro). When I use CAKE at lower speeds, it impressively crushes the impact under load down to about 0-1ms. But that only holds at lower speeds, meaning a bad day for Starlink where it's not pushing 300Mbps or if I set my router to limit bandwidth to about 280Mbps. On fiber, I have to limit the 1Gbps connection to 280Mbps to run CAKE and eliminate the bufferbloat.
> 
> At 300Mbps or higher, bufferbloat and jitter skyrocket to hundreds of ms, much worse than not using CAKE at all. I can see one of the 4 router CPU cores spikes to 100% at those transfer rates whenever I don't limit bandwidth to something below 280Mbps. My crude understanding is that CAKE can only run on a single CPU core (can't be multithreaded to take advantage of the other 3 cores) and becomes CPU bound at higher transfer rates. Then, when the CPU is maxed, CAKE will do more harm than good, which is exactly why bufferbloat actually increases with CAKE above 280Mbps.

In deed cake's shaper and complicated scheduler do not like running out of timely CPU access... (the solution is simple, 1at gigabit rates get a real CPU with a solid IO and memory system, quite a lot of all in one routers are simply not designed for that load)

> I understand that FQ-CoDel is similar.

Which is incorrect... fq-codel has a simpler scheduler than cake and no traffic shaper, combined with e.g. HTB you can build a shaper that fails differently than cake on CPU overload... HTB+fq-codel as in sqm-scripts simple.qos will not show increased latency but simply a reduced rate.

> 
> All those tests are on wired connections, so these are not Wi-Fi effects.
> 
> Presumably, a faster CPU would increase the rate before CAKE becomes the bottleneck.

Indeed.

> But to my surprise, I can't find anything on the Internet that addresses solutions to bufferbloat at higher bandwidths or confirms that some modern routers have fast enough CPU to run at the full bandwidth capability of their WAN port.

Have a look at LibreQoS...

> 
> I had been told that a simple Ubiquiti Gateway-Lite between the Internet and my wireless router could take on this role, that it had a sufficiently powerful CPU to handle up to about 1Gbps (or very close to that), but in my testing, first, it doesn't appear to include CAKE, and turning on FQ-CoDel, as its closest alternative, requires disabling its hardware acceleration, which dramatically harms both throughput and latency under load.

A lot of cheapish x86 machines will do a gigabit with cake quite easily...

> 
> I'm willing to go up to something more expensive and powerful (like the Ubiquiti UDR7, where I like Ubiquiti for the group management of all the networking hardware from one interface with our existing Ubiquiti Wi-Fi APs, but that's not a requirement, and I don't think any of the Ubiquiti routers include CAKE), but given how hard it is to find anything on the Internet about this, it makes me suspicious that I'm chasing the wrong problem.
> 
> My chief concern is keeping jitter to a level where it will not cause any issues for VoIP or video calls. At 16-25ms, that's not bad, but it can lead to some stuttering and glitching on calls, shifting QoS to prioritize voice and video calls helps, but if I could just eliminate the bufferbloat, seems that would be a better solution.
> 
> Questions:
> 
> 1. If CAKE is not viable due to CPU constraints on most modern routers as their WAN port transfer rate, then what is the recommended solution?

Get a router with a beefy enough CPU... many of the commercial offerings for laymen pair an relative weak CPU with some offload engines to make up for the weak CPU, but traffic shaping typically can not be off loaded, so this simply shows routers that are offered for gigabit rates, but are really not up to the job. 

> 2. Are there any routers that can handle CAKE at 1Gbps or 2Gbps?

I hear the minisforum ms-01 is quite popular with the LibreQoS crowd (albeit not cheap). Or if you are willing to twiddle a bit, even a raspberry pi5 with an additional USB3 gigabit ethernet dongle should work at 1 Gbps.

> 
> 3. Is there a site that provides ratings for different routers at these high bandwidths or lists MODERN 1Gbps+ routers that support CAKE up to their full bandwidth capacity?

Not that I know off, but e.g the OpenWrt forum has some benchmark threads that might at least give an expectation.

> 
> 4. Or, at bandwidths above 300Mbps, are these 16-25ms increases considered acceptable, meaning I'm chasing perfection when I should just be happy with 16-25ms latency impact and jitter under load and accept that occasionally any Internet connection will have some problems when the connection gets crowded? 
> 
> How should I tackle finding a solution? And if I'm asking the wrong questions, please feel free to mock my ignorance in your response. :-) Let my foolishness be used educate others (and myself). 
> 
> Many thanks,
> Colin
> _______________________________________________
> Starlink mailing list -- starlink@lists.bufferbloat.net
> To unsubscribe send an email to starlink-leave@lists.bufferbloat.net

[Starlink] Re: Bufferbloat cure question

[David Lang]

Colin_Higbie wrote:

> At 300Mbps or higher, bufferbloat and jitter skyrocket to hundreds of ms, much 
> worse than not using CAKE at all. I can see one of the 4 router CPU cores 
> spikes to 100% at those transfer rates whenever I don't limit bandwidth to 
> something below 280Mbps.

I know I've seen some patches going past in the last couple of months to let 
cake use more than one CPU, so you may want to look at testing those.

it's not uncommon for access points to run out of CPU when processing large 
packet volumes (with or without cake), the wndr3700/3800 series that we started 
with on the bufferbloat project would run out of steam at about that speed 
without any traffic shaping at all.

it's not just a matter of cpu cycles, it also starts to hit memory and MIC 
connection limits.

you may be able to get to high speeds if you don't try to have Cake completely 
eliminate the effect of bloat, just mitigate it to an acceptale level.

Unfortunantly, I don't know of anyone doing any benchmark testing of routers. 
This isn't just a problem at the low end, It's an industry joke that Cisco speed 
ratings are "we guarantee that no matter what you do you will never go faster 
than this", and that real performance can be as bad as 10% of what the speed 
ratings list.

David Lang

[Starlink] Re: Bufferbloat cure question

[Toke Høiland-Jørgensen]

David Lang via Starlink <starlink@lists.bufferbloat.net> writes:

> Colin_Higbie wrote:
>
>> At 300Mbps or higher, bufferbloat and jitter skyrocket to hundreds of ms, much 
>> worse than not using CAKE at all. I can see one of the 4 router CPU cores 
>> spikes to 100% at those transfer rates whenever I don't limit bandwidth to 
>> something below 280Mbps.
>
> I know I've seen some patches going past in the last couple of months to let 
> cake use more than one CPU, so you may want to look at testing those.

The current version of those patches are here:
https://git.kernel.org/pub/scm/linux/kernel/git/toke/linux.git/log/?h=mq-cake-sub-qdisc

Testing very welcome! I'm planning to resubmit them for upstream
inclusion on Monday :)

-Toke