From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mail.toke.dk; spf=pass smtp.mailfrom=lang.hm; dkim=fail; arc=none (Message is not ARC signed); dmarc=none Received: from mail.lang.hm (wsip-70-167-213-146.ph.ph.cox.net [70.167.213.146]) by mail.toke.dk (Postfix) with ESMTPS id 09C46701CFB for ; Sun, 28 Sep 2025 14:18:01 +0200 (CEST) Received: from asgard.lang.hm (syslog [10.0.0.100]) by mail.lang.hm (Postfix) with ESMTP id C7FF920A6F2; Sun, 28 Sep 2025 05:17:59 -0700 (PDT) Date: Sun, 28 Sep 2025 05:17:59 -0700 (PDT) From: David Lang X-X-Sender: dlang@asgard.lang.hm To: Sebastian Moeller cc: cake@lists.bufferbloat.net In-Reply-To: <06DAA0CB-70EF-4FDD-BD3D-16A4FC28AD12@gmx.de> Message-ID: References: <06DAA0CB-70EF-4FDD-BD3D-16A4FC28AD12@gmx.de> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Message-ID-Hash: KCON4DWT2RRYUHT2WBF3Z4IQMT7A7DVC X-Message-ID-Hash: KCON4DWT2RRYUHT2WBF3Z4IQMT7A7DVC X-MailFrom: david@lang.hm X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list Subject: [Cake] Re: help request for cake on a large network List-Id: Cake - FQ_codel the next generation Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Sebastian Moeller wrote: > Hi David, > > while I have no real answer for your questions (due to never having had that kind of load in my home network ;) ) I would like to ask you to make take scripted captures of tc -s qdisc for the wan interface is reasonable short intervals (say every 10 minutes?) as that might be just what we need to actually answer your question. I will do that, however the network is only up under load for 4 days a year, so it's a slow feedback loop :-) I would welcome any other suggestions for data to gather. >> On 28. Sep 2025, at 13:06, David Lang wrote: >> >> I'm starting to prepare for the next Scale conference and we are switching from Juniper routers to Linux routers. This gives me the ability to implement cake. >> >> One problem we have is classes that tell everyone 'go download this' that trigger hundreds of people to hammer the network at the same time (this is both a wifi and a network bandwidth issue, wifi is being worked on) > > So one issue might be that with several 100 users the default compile-time > size of queues (1024, IIRC) that cake will entertain might be too little, even > in light of the 8 way assoziative hashing design. I believe this can be > changed (within limits) only by modifying at source and recompilation of the > kernel, if that should be needed at all. custom compiling a kernel is very much an option (and this sort of tweaking is the sort of thing I'm expecting to need to do) The conference is in March, so we have some time to think about this and customize things, just no chance to test before the show. > I wonder whether multi-queue cake would not solve this to some degree, as I > assume each queue's instance would bring its own independent set of 1024 bins? good thought >> The network is pretty flat, a couple of subnets each on ipv4 and ipv6. >> >> Any suggestions on how to configure cake for this sort of environment where there are so many devices? > > Maybe switch to a simpler pure per-flow isolation mode than the default triple-isolate? > > BTW what kind of uplink capacity will you use? last year we had something around 500Mb, they are in the process of changing ISPs and we may get a bit more, but probably still under 1G hotel and conference center Internet connections are far smaller than most people think. David Lang