[Cake] help request for cake on a large network

[Cake] help request for cake on a large network

[David Lang]

I'm starting to prepare for the next Scale conference and we are switching from 
Juniper routers to Linux routers. This gives me the ability to implement cake.

One problem we have is classes that tell everyone 'go download this' that 
trigger hundreds of people to hammer the network at the same time (this is both 
a wifi and a network bandwidth issue, wifi is being worked on)

The network is pretty flat, a couple of subnets each on ipv4 and ipv6.

Any suggestions on how to configure cake for this sort of environment where 
there are so many devices?

David Lang

[Cake] Re: help request for cake on a large network

[Sebastian Moeller]

Hi David,

while I have no real answer for your questions (due to never having had that kind of load in my home network ;) ) I would like to ask you to make take scripted captures of tc -s qdisc for the wan interface is reasonable short intervals (say every 10 minutes?) as that might be just what we need to actually answer your question.



> On 28. Sep 2025, at 13:06, David Lang <david@lang.hm> wrote:
> 
> I'm starting to prepare for the next Scale conference and we are switching from Juniper routers to Linux routers. This gives me the ability to implement cake.
> 
> One problem we have is classes that tell everyone 'go download this' that trigger hundreds of people to hammer the network at the same time (this is both a wifi and a network bandwidth issue, wifi is being worked on)

So one issue might be that with several 100 users the default compile-time size of queues (1024, IIRC) that cake will entertain might be too little, even in light of the 8 way assoziative hashing design. I believe this can be changed (within limits) only by modifying at source and recompilation of the kernel, if that should be needed at all.

I wonder whether multi-queue cake would not solve this to some degree, as I assume each queue's instance would bring its own independent set of 1024 bins?


> The network is pretty flat, a couple of subnets each on ipv4 and ipv6.
> 
> Any suggestions on how to configure cake for this sort of environment where there are so many devices?

Maybe switch to a simpler pure per-flow isolation mode than the default triple-isolate?

BTW what kind of uplink capacity will you use?

> 
> David Lang
> _______________________________________________
> Cake mailing list -- cake@lists.bufferbloat.net
> To unsubscribe send an email to cake-leave@lists.bufferbloat.net

[Cake] Re: help request for cake on a large network

[David Lang]

Sebastian Moeller wrote:

> Hi David,
>
> while I have no real answer for your questions (due to never having had that kind of load in my home network ;) ) I would like to ask you to make take scripted captures of tc -s qdisc for the wan interface is reasonable short intervals (say every 10 minutes?) as that might be just what we need to actually answer your question.

I will do that, however the network is only up under load for 4 days a year, so 
it's a slow feedback loop :-)

I would welcome any other suggestions for data to gather.

>> On 28. Sep 2025, at 13:06, David Lang <david@lang.hm> wrote:
>>
>> I'm starting to prepare for the next Scale conference and we are switching from Juniper routers to Linux routers. This gives me the ability to implement cake.
>>
>> One problem we have is classes that tell everyone 'go download this' that trigger hundreds of people to hammer the network at the same time (this is both a wifi and a network bandwidth issue, wifi is being worked on)
>

> So one issue might be that with several 100 users the default compile-time 
> size of queues (1024, IIRC) that cake will entertain might be too little, even 
> in light of the 8 way assoziative hashing design. I believe this can be 
> changed (within limits) only by modifying at source and recompilation of the 
> kernel, if that should be needed at all.

custom compiling a kernel is very much an option (and this sort of tweaking is 
the sort of thing I'm expecting to need to do)

The conference is in March, so we have some time to think about this and 
customize things, just no chance to test before the show.

> I wonder whether multi-queue cake would not solve this to some degree, as I 
> assume each queue's instance would bring its own independent set of 1024 bins?

good thought


>> The network is pretty flat, a couple of subnets each on ipv4 and ipv6.
>>
>> Any suggestions on how to configure cake for this sort of environment where there are so many devices?
>
> Maybe switch to a simpler pure per-flow isolation mode than the default triple-isolate?
>
> BTW what kind of uplink capacity will you use?

last year we had something around 500Mb, they are in the process of changing 
ISPs and we may get a bit more, but probably still under 1G

hotel and conference center Internet connections are far smaller than most 
people think.

David Lang

[Cake] Re: help request for cake on a large network

[Jaap de Vos]

Hi David,

I haven't tried it myself yet, but this sounds like a use case for which LibreQoS is suitable. However, LibreQoS would fit best if you can build something in line with the design assumptions: https://libreqos.readthedocs.io/en/latest/docs/v2.0/design.html
Specifically the part about putting it in between other routers and running a routing protocol with those.

There are some good recommendations around suitable hardware and sizing/scaling too: https://libreqos.readthedocs.io/en/latest/docs/v2.0/requirements.html

What kind of upstream bandwidth are you working with at the conference? I run the Mikrotik implementation of CAKE on a campsite. It really helps to get the most out of our asymmetrical gigabit coax line over there, no fiber available yet. So far, we got close to 500 devices on our outdoor Wi-Fi network with mostly good experiences, amazing. Before that, FQ_CoDel on PfSense worked well too, but the network wasn't as busy back then. I'd like to see CAKE on BSD some day.

Kind regards,
Jaap de Vos


> On 09/28/2025 1:06 PM CEST David Lang <david@lang.hm> wrote:
> 
>  
> I'm starting to prepare for the next Scale conference and we are switching from 
> Juniper routers to Linux routers. This gives me the ability to implement cake.
> 
> One problem we have is classes that tell everyone 'go download this' that 
> trigger hundreds of people to hammer the network at the same time (this is both 
> a wifi and a network bandwidth issue, wifi is being worked on)
> 
> The network is pretty flat, a couple of subnets each on ipv4 and ipv6.
> 
> Any suggestions on how to configure cake for this sort of environment where 
> there are so many devices?
> 
> David Lang
> _______________________________________________
> Cake mailing list -- cake@lists.bufferbloat.net
> To unsubscribe send an email to cake-leave@lists.bufferbloat.net

[Cake] Re: help request for cake on a large network

[David Lang]

I guess I didn't give enough info on our network architecture.

The upstream ISP is between 500Mb and 1G.

We disconnect the conference center firewall and connect our firewall in it's 
place (Linux system)

This border router connect to the conference center firewall (so their offices 
still get connectivity), and to two routers (Linux, one per building) that are 
also direct connected (a triangle between the three routers. The border router 
is doing NAT for IPv4 and MSS squashing for IPV6 that goes out through a tunnel 
(google doesn't do PMTU discovery on IPv6)

(we are currently debating between deploying OSPF between the three routers, or 
just giving them static routes, high priority direct and low priority for the 
indirect path)

Each building router then connects to one or a few switches in different closets 
that then go to a switch in each room (trunked ports).
We hve a handful of wired users, 120APs for the users, and AV equipment for each 
of the 16 tracks that are recorded and streamed.

a quick look at libreqos and it's pushing for a more complex network layout that 
will be a hard sell at least this year (last year our juniper routers fell over 
under the MSS squashing load for IPv6, so we are going to Linux systems this 
year, getting people to consider cake is hard enough, let alone another 
middleware box)

David Lang

On Sun, 28 Sep 2025, Jaap de Vos wrote:

> Date: Sun, 28 Sep 2025 14:12:24 +0200 (CEST)
> From: Jaap de Vos <m@jaap.pro>
> Reply-To: "cake@lists.bufferbloat.net" <cake@lists.bufferbloat.net>
> To: David Lang <david@lang.hm>, cake@lists.bufferbloat.net
> Subject: Re: [Cake] help request for cake on a large network
> 
> Hi David,
>
> I haven't tried it myself yet, but this sounds like a use case for which LibreQoS is suitable. However, LibreQoS would fit best if you can build something in line with the design assumptions: https://libreqos.readthedocs.io/en/latest/docs/v2.0/design.html
> Specifically the part about putting it in between other routers and running a routing protocol with those.
>
> There are some good recommendations around suitable hardware and sizing/scaling too: https://libreqos.readthedocs.io/en/latest/docs/v2.0/requirements.html
>
> What kind of upstream bandwidth are you working with at the conference? I run the Mikrotik implementation of CAKE on a campsite. It really helps to get the most out of our asymmetrical gigabit coax line over there, no fiber available yet. So far, we got close to 500 devices on our outdoor Wi-Fi network with mostly good experiences, amazing. Before that, FQ_CoDel on PfSense worked well too, but the network wasn't as busy back then. I'd like to see CAKE on BSD some day.
>
> Kind regards,
> Jaap de Vos
>
>
>> On 09/28/2025 1:06 PM CEST David Lang <david@lang.hm> wrote:
>>
>>
>> I'm starting to prepare for the next Scale conference and we are switching from
>> Juniper routers to Linux routers. This gives me the ability to implement cake.
>>
>> One problem we have is classes that tell everyone 'go download this' that
>> trigger hundreds of people to hammer the network at the same time (this is both
>> a wifi and a network bandwidth issue, wifi is being worked on)
>>
>> The network is pretty flat, a couple of subnets each on ipv4 and ipv6.
>>
>> Any suggestions on how to configure cake for this sort of environment where
>> there are so many devices?
>>
>> David Lang
>> _______________________________________________
>> Cake mailing list -- cake@lists.bufferbloat.net
>> To unsubscribe send an email to cake-leave@lists.bufferbloat.net
>

[Cake] Re: help request for cake on a large network

[Frantisek Borsik]

Jaap - thank you very much for recommending LibreQoS.

Yes, besides a typical last-mile ISP scenario, LibreQoS is being used in
various other setups, like to help with event Wi-Fi - here is one of the
deployments https://libreqos.io/2023/11/13/success-story-raceway/

To brag a little, other deployment scenarios we see are data centers,
cruise ships, university campuses, even mines :)

All the best,

Frank

Frantisek (Frank) Borsik


*In loving memory of Dave Täht: *1965-2025

https://libreqos.io/2025/04/01/in-loving-memory-of-dave/


https://www.linkedin.com/in/frantisekborsik

Signal, Telegram, WhatsApp: +421919416714

iMessage, mobile: +420775230885

Skype: casioa5302ca

frantisek.borsik@gmail.com


On Sun, Sep 28, 2025 at 2:38 PM David Lang <david@lang.hm> wrote:

> I guess I didn't give enough info on our network architecture.
>
> The upstream ISP is between 500Mb and 1G.
>
> We disconnect the conference center firewall and connect our firewall in
> it's
> place (Linux system)
>
> This border router connect to the conference center firewall (so their
> offices
> still get connectivity), and to two routers (Linux, one per building) that
> are
> also direct connected (a triangle between the three routers. The border
> router
> is doing NAT for IPv4 and MSS squashing for IPV6 that goes out through a
> tunnel
> (google doesn't do PMTU discovery on IPv6)
>
> (we are currently debating between deploying OSPF between the three
> routers, or
> just giving them static routes, high priority direct and low priority for
> the
> indirect path)
>
> Each building router then connects to one or a few switches in different
> closets
> that then go to a switch in each room (trunked ports).
> We hve a handful of wired users, 120APs for the users, and AV equipment
> for each
> of the 16 tracks that are recorded and streamed.
>
> a quick look at libreqos and it's pushing for a more complex network
> layout that
> will be a hard sell at least this year (last year our juniper routers fell
> over
> under the MSS squashing load for IPv6, so we are going to Linux systems
> this
> year, getting people to consider cake is hard enough, let alone another
> middleware box)
>
> David Lang
>
> On Sun, 28 Sep 2025, Jaap de Vos wrote:
>
> > Date: Sun, 28 Sep 2025 14:12:24 +0200 (CEST)
> > From: Jaap de Vos <m@jaap.pro>
> > Reply-To: "cake@lists.bufferbloat.net" <cake@lists.bufferbloat.net>
> > To: David Lang <david@lang.hm>, cake@lists.bufferbloat.net
> > Subject: Re: [Cake] help request for cake on a large network
> >
> > Hi David,
> >
> > I haven't tried it myself yet, but this sounds like a use case for which
> LibreQoS is suitable. However, LibreQoS would fit best if you can build
> something in line with the design assumptions:
> https://libreqos.readthedocs.io/en/latest/docs/v2.0/design.html
> > Specifically the part about putting it in between other routers and
> running a routing protocol with those.
> >
> > There are some good recommendations around suitable hardware and
> sizing/scaling too:
> https://libreqos.readthedocs.io/en/latest/docs/v2.0/requirements.html
> >
> > What kind of upstream bandwidth are you working with at the conference?
> I run the Mikrotik implementation of CAKE on a campsite. It really helps to
> get the most out of our asymmetrical gigabit coax line over there, no fiber
> available yet. So far, we got close to 500 devices on our outdoor Wi-Fi
> network with mostly good experiences, amazing. Before that, FQ_CoDel on
> PfSense worked well too, but the network wasn't as busy back then. I'd like
> to see CAKE on BSD some day.
> >
> > Kind regards,
> > Jaap de Vos
> >
> >
> >> On 09/28/2025 1:06 PM CEST David Lang <david@lang.hm> wrote:
> >>
> >>
> >> I'm starting to prepare for the next Scale conference and we are
> switching from
> >> Juniper routers to Linux routers. This gives me the ability to
> implement cake.
> >>
> >> One problem we have is classes that tell everyone 'go download this'
> that
> >> trigger hundreds of people to hammer the network at the same time (this
> is both
> >> a wifi and a network bandwidth issue, wifi is being worked on)
> >>
> >> The network is pretty flat, a couple of subnets each on ipv4 and ipv6.
> >>
> >> Any suggestions on how to configure cake for this sort of environment
> where
> >> there are so many devices?
> >>
> >> David Lang
> >> _______________________________________________
> >> Cake mailing list -- cake@lists.bufferbloat.net
> >> To unsubscribe send an email to cake-leave@lists.bufferbloat.net
> >
> _______________________________________________
> Cake mailing list -- cake@lists.bufferbloat.net
> To unsubscribe send an email to cake-leave@lists.bufferbloat.net
>

[Cake] Re: help request for cake on a large network

[dave seddon]

G'day David,

This sounds great and I'd be very happy to discuss this in real time
to give you a hand.  The SCALE team does a lot of Nix, right?  I think
all the OpenWrt One routers were configured with nix, and so I assume
you're going to go with Nix for this?

I think the TLDR is that cake will mostly do what you want.  The idea
being that "it is a piece of cake" to use cake. ;)


I was recently configuring an active/standby firewall, including
configuring cake, with Nix, which I really need to 1) put config on
github 2) document 3) and make a video about.


One nice thing I worked out is that the systemd networkd makes it
super easy to apply the qdisc of choice.

This isn't the whole config, but hopefully gives you the vague idea

  cakeConfig = {
    Bandwidth = "990M";  # We currently have 1Gb/s, so this is our limit
    OverheadBytes = 38;  # Ethernet overhead (preamble + inter-frame gap + FCS)
    MPUBytes = 84;       # Minimum packet unit for Ethernet
    NAT = true;
    FlowIsolationMode = "triple";
    PriorityQueueingPreset = "besteffort";
  };

systemd.network.networks = {

    # Bond interface configuration
    "bond0" = {
      matchConfig.Name = bondConfig.Name;
      networkConfig = {
        Address = [
          (constructIP bondConfig.subnet4 (getNodeAddress bondConfig))
          #bondConfig.vrrp_ip # secondary IP
          #fd00::1/64" #FIXME!!
        ];
        LinkLocalAddressing = "no";
        # Note: Gateway is configured in routes section with high metric
        # to make it less preferred than DHCP route (metric 1024)
      };
      routes = [
        {
          Gateway = bondConfig.gateway_ip;
          Destination = "0.0.0.0/0"; # Default route (all destinations)
          Metric = 2000; # Higher than DHCP metric (1024) to make it
less preferred
        }
      ];
      # IMPORTANT: systemd-networkd Route syntax requirements:
      # - Use "0.0.0.0/0" for IPv4 default routes (NOT "default")
      # - Use "::/0" for IPv6 default routes (NOT "default")
      # - "default" is NOT a valid Destination value in systemd-networkd
      # - Reference:
https://www.freedesktop.org/software/systemd/man/latest/systemd.network.html
      linkConfig = {
        #RequiredForOnline = "carrier";
        MTUBytes = MTUBytes;
      };
      vlan = vlanNames;
      inherit cakeConfig;
<------------------------ CAKE
    };

  # Bond device configuration
  systemd.network.netdevs = {
    "bond0" = {
      netdevConfig = {
        Name = bondConfig.Name;
        Kind = "bond";
        MTUBytes = MTUBytes;
      };
      bondConfig = {
        Mode = "802.3ad";
        # MIIMonitorSec = "100ms";
        LACPTransmitRate = "fast"; # fast is only 1 second, so it's
not really very fast :)
        TransmitHashPolicy = "layer3+4";
      };
    };

    # VLAN devices - generated from vlanConfigs
  } // lib.foldl' (acc: name: acc // createVlanNetdev name
vlanConfigs.${name}) {} (lib.attrNames vlanConfigs);

}

Regards,
Dave Seddon

On Sun, Sep 28, 2025 at 5:55 AM Frantisek Borsik
<frantisek.borsik@gmail.com> wrote:
>
> Jaap - thank you very much for recommending LibreQoS.
>
> Yes, besides a typical last-mile ISP scenario, LibreQoS is being used in
> various other setups, like to help with event Wi-Fi - here is one of the
> deployments https://libreqos.io/2023/11/13/success-story-raceway/
>
> To brag a little, other deployment scenarios we see are data centers,
> cruise ships, university campuses, even mines :)
>
> All the best,
>
> Frank
>
> Frantisek (Frank) Borsik
>
>
> *In loving memory of Dave Täht: *1965-2025
>
> https://libreqos.io/2025/04/01/in-loving-memory-of-dave/
>
>
> https://www.linkedin.com/in/frantisekborsik
>
> Signal, Telegram, WhatsApp: +421919416714
>
> iMessage, mobile: +420775230885
>
> Skype: casioa5302ca
>
> frantisek.borsik@gmail.com
>
>
> On Sun, Sep 28, 2025 at 2:38 PM David Lang <david@lang.hm> wrote:
>
> > I guess I didn't give enough info on our network architecture.
> >
> > The upstream ISP is between 500Mb and 1G.
> >
> > We disconnect the conference center firewall and connect our firewall in
> > it's
> > place (Linux system)
> >
> > This border router connect to the conference center firewall (so their
> > offices
> > still get connectivity), and to two routers (Linux, one per building) that
> > are
> > also direct connected (a triangle between the three routers. The border
> > router
> > is doing NAT for IPv4 and MSS squashing for IPV6 that goes out through a
> > tunnel
> > (google doesn't do PMTU discovery on IPv6)
> >
> > (we are currently debating between deploying OSPF between the three
> > routers, or
> > just giving them static routes, high priority direct and low priority for
> > the
> > indirect path)
> >
> > Each building router then connects to one or a few switches in different
> > closets
> > that then go to a switch in each room (trunked ports).
> > We hve a handful of wired users, 120APs for the users, and AV equipment
> > for each
> > of the 16 tracks that are recorded and streamed.
> >
> > a quick look at libreqos and it's pushing for a more complex network
> > layout that
> > will be a hard sell at least this year (last year our juniper routers fell
> > over
> > under the MSS squashing load for IPv6, so we are going to Linux systems
> > this
> > year, getting people to consider cake is hard enough, let alone another
> > middleware box)
> >
> > David Lang
> >
> > On Sun, 28 Sep 2025, Jaap de Vos wrote:
> >
> > > Date: Sun, 28 Sep 2025 14:12:24 +0200 (CEST)
> > > From: Jaap de Vos <m@jaap.pro>
> > > Reply-To: "cake@lists.bufferbloat.net" <cake@lists.bufferbloat.net>
> > > To: David Lang <david@lang.hm>, cake@lists.bufferbloat.net
> > > Subject: Re: [Cake] help request for cake on a large network
> > >
> > > Hi David,
> > >
> > > I haven't tried it myself yet, but this sounds like a use case for which
> > LibreQoS is suitable. However, LibreQoS would fit best if you can build
> > something in line with the design assumptions:
> > https://libreqos.readthedocs.io/en/latest/docs/v2.0/design.html
> > > Specifically the part about putting it in between other routers and
> > running a routing protocol with those.
> > >
> > > There are some good recommendations around suitable hardware and
> > sizing/scaling too:
> > https://libreqos.readthedocs.io/en/latest/docs/v2.0/requirements.html
> > >
> > > What kind of upstream bandwidth are you working with at the conference?
> > I run the Mikrotik implementation of CAKE on a campsite. It really helps to
> > get the most out of our asymmetrical gigabit coax line over there, no fiber
> > available yet. So far, we got close to 500 devices on our outdoor Wi-Fi
> > network with mostly good experiences, amazing. Before that, FQ_CoDel on
> > PfSense worked well too, but the network wasn't as busy back then. I'd like
> > to see CAKE on BSD some day.
> > >
> > > Kind regards,
> > > Jaap de Vos
> > >
> > >
> > >> On 09/28/2025 1:06 PM CEST David Lang <david@lang.hm> wrote:
> > >>
> > >>
> > >> I'm starting to prepare for the next Scale conference and we are
> > switching from
> > >> Juniper routers to Linux routers. This gives me the ability to
> > implement cake.
> > >>
> > >> One problem we have is classes that tell everyone 'go download this'
> > that
> > >> trigger hundreds of people to hammer the network at the same time (this
> > is both
> > >> a wifi and a network bandwidth issue, wifi is being worked on)
> > >>
> > >> The network is pretty flat, a couple of subnets each on ipv4 and ipv6.
> > >>
> > >> Any suggestions on how to configure cake for this sort of environment
> > where
> > >> there are so many devices?
> > >>
> > >> David Lang
> > >> _______________________________________________
> > >> Cake mailing list -- cake@lists.bufferbloat.net
> > >> To unsubscribe send an email to cake-leave@lists.bufferbloat.net
> > >
> > _______________________________________________
> > Cake mailing list -- cake@lists.bufferbloat.net
> > To unsubscribe send an email to cake-leave@lists.bufferbloat.net
> >
> _______________________________________________
> Cake mailing list -- cake@lists.bufferbloat.net
> To unsubscribe send an email to cake-leave@lists.bufferbloat.net



-- 
Regards,
Dave Seddon
+1 415 857 5102

[Cake] Re: help request for cake on a large network

[David Lang]

dave seddon wrote:

> G'day David,
>
> This sounds great and I'd be very happy to discuss this in real time
> to give you a hand.  The SCALE team does a lot of Nix, right?

Yes

> I think all the OpenWrt One routers were configured with nix, and so I assume 
> you're going to go with Nix for this?

they openwrt one routers were configured with a custom build of openwrt (we do 
have someone working on bringing nix up on them)

these routers will probably be running nix

> I think the TLDR is that cake will mostly do what you want.  The idea
> being that "it is a piece of cake" to use cake. ;)

that's what I expect, I just expect to need tuning of things to handle the large 
number of endpoints, what is appropriate for a dozen users will probably need 
larger tables to handle 3000 users :-)

> I was recently configuring an active/standby firewall, including
> configuring cake, with Nix, which I really need to 1) put config on
> github 2) document 3) and make a video about.

looking forward to it.

David Lang

>
> One nice thing I worked out is that the systemd networkd makes it
> super easy to apply the qdisc of choice.
>
> This isn't the whole config, but hopefully gives you the vague idea
>
>  cakeConfig = {
>    Bandwidth = "990M";  # We currently have 1Gb/s, so this is our limit
>    OverheadBytes = 38;  # Ethernet overhead (preamble + inter-frame gap + FCS)
>    MPUBytes = 84;       # Minimum packet unit for Ethernet
>    NAT = true;
>    FlowIsolationMode = "triple";
>    PriorityQueueingPreset = "besteffort";
>  };
>
> systemd.network.networks = {
>
>    # Bond interface configuration
>    "bond0" = {
>      matchConfig.Name = bondConfig.Name;
>      networkConfig = {
>        Address = [
>          (constructIP bondConfig.subnet4 (getNodeAddress bondConfig))
>          #bondConfig.vrrp_ip # secondary IP
>          #fd00::1/64" #FIXME!!
>        ];
>        LinkLocalAddressing = "no";
>        # Note: Gateway is configured in routes section with high metric
>        # to make it less preferred than DHCP route (metric 1024)
>      };
>      routes = [
>        {
>          Gateway = bondConfig.gateway_ip;
>          Destination = "0.0.0.0/0"; # Default route (all destinations)
>          Metric = 2000; # Higher than DHCP metric (1024) to make it
> less preferred
>        }
>      ];
>      # IMPORTANT: systemd-networkd Route syntax requirements:
>      # - Use "0.0.0.0/0" for IPv4 default routes (NOT "default")
>      # - Use "::/0" for IPv6 default routes (NOT "default")
>      # - "default" is NOT a valid Destination value in systemd-networkd
>      # - Reference:
> https://www.freedesktop.org/software/systemd/man/latest/systemd.network.html
>      linkConfig = {
>        #RequiredForOnline = "carrier";
>        MTUBytes = MTUBytes;
>      };
>      vlan = vlanNames;
>      inherit cakeConfig;
> <------------------------ CAKE
>    };
>
>  # Bond device configuration
>  systemd.network.netdevs = {
>    "bond0" = {
>      netdevConfig = {
>        Name = bondConfig.Name;
>        Kind = "bond";
>        MTUBytes = MTUBytes;
>      };
>      bondConfig = {
>        Mode = "802.3ad";
>        # MIIMonitorSec = "100ms";
>        LACPTransmitRate = "fast"; # fast is only 1 second, so it's
> not really very fast :)
>        TransmitHashPolicy = "layer3+4";
>      };
>    };
>
>    # VLAN devices - generated from vlanConfigs
>  } // lib.foldl' (acc: name: acc // createVlanNetdev name
> vlanConfigs.${name}) {} (lib.attrNames vlanConfigs);
>
> }
>
> Regards,
> Dave Seddon
>
> On Sun, Sep 28, 2025 at 5:55 AM Frantisek Borsik
> <frantisek.borsik@gmail.com> wrote:
>>
>> Jaap - thank you very much for recommending LibreQoS.
>>
>> Yes, besides a typical last-mile ISP scenario, LibreQoS is being used in
>> various other setups, like to help with event Wi-Fi - here is one of the
>> deployments https://libreqos.io/2023/11/13/success-story-raceway/
>>
>> To brag a little, other deployment scenarios we see are data centers,
>> cruise ships, university campuses, even mines :)
>>
>> All the best,
>>
>> Frank
>>
>> Frantisek (Frank) Borsik
>>
>>
>> *In loving memory of Dave Täht: *1965-2025
>>
>> https://libreqos.io/2025/04/01/in-loving-memory-of-dave/
>>
>>
>> https://www.linkedin.com/in/frantisekborsik
>>
>> Signal, Telegram, WhatsApp: +421919416714
>>
>> iMessage, mobile: +420775230885
>>
>> Skype: casioa5302ca
>>
>> frantisek.borsik@gmail.com
>>
>>
>> On Sun, Sep 28, 2025 at 2:38 PM David Lang <david@lang.hm> wrote:
>>
>>> I guess I didn't give enough info on our network architecture.
>>>
>>> The upstream ISP is between 500Mb and 1G.
>>>
>>> We disconnect the conference center firewall and connect our firewall in
>>> it's
>>> place (Linux system)
>>>
>>> This border router connect to the conference center firewall (so their
>>> offices
>>> still get connectivity), and to two routers (Linux, one per building) that
>>> are
>>> also direct connected (a triangle between the three routers. The border
>>> router
>>> is doing NAT for IPv4 and MSS squashing for IPV6 that goes out through a
>>> tunnel
>>> (google doesn't do PMTU discovery on IPv6)
>>>
>>> (we are currently debating between deploying OSPF between the three
>>> routers, or
>>> just giving them static routes, high priority direct and low priority for
>>> the
>>> indirect path)
>>>
>>> Each building router then connects to one or a few switches in different
>>> closets
>>> that then go to a switch in each room (trunked ports).
>>> We hve a handful of wired users, 120APs for the users, and AV equipment
>>> for each
>>> of the 16 tracks that are recorded and streamed.
>>>
>>> a quick look at libreqos and it's pushing for a more complex network
>>> layout that
>>> will be a hard sell at least this year (last year our juniper routers fell
>>> over
>>> under the MSS squashing load for IPv6, so we are going to Linux systems
>>> this
>>> year, getting people to consider cake is hard enough, let alone another
>>> middleware box)
>>>
>>> David Lang
>>>
>>> On Sun, 28 Sep 2025, Jaap de Vos wrote:
>>>
>>>> Date: Sun, 28 Sep 2025 14:12:24 +0200 (CEST)
>>>> From: Jaap de Vos <m@jaap.pro>
>>>> Reply-To: "cake@lists.bufferbloat.net" <cake@lists.bufferbloat.net>
>>>> To: David Lang <david@lang.hm>, cake@lists.bufferbloat.net
>>>> Subject: Re: [Cake] help request for cake on a large network
>>>>
>>>> Hi David,
>>>>
>>>> I haven't tried it myself yet, but this sounds like a use case for which
>>> LibreQoS is suitable. However, LibreQoS would fit best if you can build
>>> something in line with the design assumptions:
>>> https://libreqos.readthedocs.io/en/latest/docs/v2.0/design.html
>>>> Specifically the part about putting it in between other routers and
>>> running a routing protocol with those.
>>>>
>>>> There are some good recommendations around suitable hardware and
>>> sizing/scaling too:
>>> https://libreqos.readthedocs.io/en/latest/docs/v2.0/requirements.html
>>>>
>>>> What kind of upstream bandwidth are you working with at the conference?
>>> I run the Mikrotik implementation of CAKE on a campsite. It really helps to
>>> get the most out of our asymmetrical gigabit coax line over there, no fiber
>>> available yet. So far, we got close to 500 devices on our outdoor Wi-Fi
>>> network with mostly good experiences, amazing. Before that, FQ_CoDel on
>>> PfSense worked well too, but the network wasn't as busy back then. I'd like
>>> to see CAKE on BSD some day.
>>>>
>>>> Kind regards,
>>>> Jaap de Vos
>>>>
>>>>
>>>>> On 09/28/2025 1:06 PM CEST David Lang <david@lang.hm> wrote:
>>>>>
>>>>>
>>>>> I'm starting to prepare for the next Scale conference and we are
>>> switching from
>>>>> Juniper routers to Linux routers. This gives me the ability to
>>> implement cake.
>>>>>
>>>>> One problem we have is classes that tell everyone 'go download this'
>>> that
>>>>> trigger hundreds of people to hammer the network at the same time (this
>>> is both
>>>>> a wifi and a network bandwidth issue, wifi is being worked on)
>>>>>
>>>>> The network is pretty flat, a couple of subnets each on ipv4 and ipv6.
>>>>>
>>>>> Any suggestions on how to configure cake for this sort of environment
>>> where
>>>>> there are so many devices?
>>>>>
>>>>> David Lang
>>>>> _______________________________________________
>>>>> Cake mailing list -- cake@lists.bufferbloat.net
>>>>> To unsubscribe send an email to cake-leave@lists.bufferbloat.net
>>>>
>>> _______________________________________________
>>> Cake mailing list -- cake@lists.bufferbloat.net
>>> To unsubscribe send an email to cake-leave@lists.bufferbloat.net
>>>
>> _______________________________________________
>> Cake mailing list -- cake@lists.bufferbloat.net
>> To unsubscribe send an email to cake-leave@lists.bufferbloat.net
>
>
>
>