[Cake] Re: help request for cake on a large network

[Sebastian Moeller <moeller0@gmx.de>]

Hi David,


> On 30. Sep 2025, at 14:37, David Lang <david@lang.hm> wrote:
> 
> Jonathan Morton wrote:
> 
>> On Sunday, 28 September 2025, David Lang wrote:
>>> I'm starting to prepare for the next Scale conference and we are switching from
>>> Juniper routers to Linux routers. This gives me the ability to implement cake.
>>> 
>>> One problem we have is classes that tell everyone 'go download this' that
>>> trigger hundreds of people to hammer the network at the same time (this is both
>>> a wifi and a network bandwidth issue, wifi is being worked on)
>>> 
>>> The network is pretty flat, a couple of subnets each on ipv4 and ipv6.
>>> 
>>> Any suggestions on how to configure cake for this sort of environment where
>>> there are so many devices?
>> 
>> So far as Cake is concerned, the normal setup should work fine even under stress conditions.  If there are too many simultaneous flows to achieve full flow isolation, it degrades gracefully to statistical multiplexing, and you still have the AQM working to keep everything responsive.  Or rather, a thousand AQMs working in parallel...
> 
> what would need to be done to increase resources to allow for full isolation of more flows?

I believe editing net/sched/sch_cake.c and increase the number of CAKE_QUEUES:
#define CAKE_QUEUES (1024)

to something larger...

however I do not know what a realistic maximum is... and IIRC cake will search through this repeatedly, 1024 is one full memory page (4k pages) anything larger will require multiple pages (likely as single contiguous allocation). I remember dimmly someone ages ago playing with this and finding that the number can be increased, but not arbitrarily so at least on your typical somewhat memory-poor router. But that should be easy to figure out.
Mind you, given Toke's reminder on the small number of concurrently active flows and cake's 8-way associativity, maybe this is not necessary at all.


> 
>> Of course, this only matters when Cake is set up to be the bottleneck.  If wifi is the bottleneck, you'll want a wifi stack with integrated fq_codel, which I believe still applies to only some hardware since it needs to manage the MAC queue which some devices don't expose.  This has the extra smarts needed to adapt gracefully to wifi's foibles, and might already be enough to convert an effectively nonfunctional wifi into one that feels, if not fast, then at least reliable.
> 
> I am doing everything I can to eliminate wifi as the bottleneck and overload our uplink ;-p
> 
> mostly the wifi bottleneck is being eliminated by having lots of separate APs for people to connect to.
> 
> David Lang
> _______________________________________________
> Cake mailing list -- cake@lists.bufferbloat.net
> To unsubscribe send an email to cake-leave@lists.bufferbloat.net