[Cake] Re: help request for cake on a large network

["Toke Høiland-Jørgensen" <toke@toke.dk>]

David Lang <david@lang.hm> writes:

> Sebastian Moeller wrote:
>
>> Hi David,
>>
>>
>>> On 30. Sep 2025, at 14:37, David Lang <david@lang.hm> wrote:
>>>
>>> Jonathan Morton wrote:
>>>
>>>> On Sunday, 28 September 2025, David Lang wrote:
>>>>> I'm starting to prepare for the next Scale conference and we are switching from
>>>>> Juniper routers to Linux routers. This gives me the ability to implement cake.
>>>>>
>>>>> One problem we have is classes that tell everyone 'go download this' that
>>>>> trigger hundreds of people to hammer the network at the same time (this is both
>>>>> a wifi and a network bandwidth issue, wifi is being worked on)
>>>>>
>>>>> The network is pretty flat, a couple of subnets each on ipv4 and ipv6.
>>>>>
>>>>> Any suggestions on how to configure cake for this sort of environment where
>>>>> there are so many devices?
>>>>
>>>> So far as Cake is concerned, the normal setup should work fine even under stress conditions.  If there are too many simultaneous flows to achieve full flow isolation, it degrades gracefully to statistical multiplexing, and you still have the AQM working to keep everything responsive.  Or rather, a thousand AQMs working in parallel...
>>>
>>> what would need to be done to increase resources to allow for full isolation of more flows?
>>
>> I believe editing net/sched/sch_cake.c and increase the number of CAKE_QUEUES:
>> #define CAKE_QUEUES (1024)
>>
>> to something larger...
>>
>> however I do not know what a realistic maximum is... and IIRC cake will search through this repeatedly, 1024 is one full memory page (4k pages) anything larger will require multiple pages (likely as single contiguous allocation). I remember dimmly someone ages ago playing with this and finding that the number can be increased, but not arbitrarily so at least on your typical somewhat memory-poor router. But that should be easy to figure out.
>
> In this case, we are using Ryzon systems with at least 32G of ram (need to look 
> up the detailed specs)

Other than the memory usage, the number of queues are constrained by the
need to avoid overflows. Also, it has to be a power of 2. Anything up to
0x8000 (32k) should be fine, AFAICT from a quick glance through the code...

-Toke