From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 3B9333B29D for ; Fri, 22 Nov 2019 07:10:20 -0500 (EST) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 56F81128A for ; Fri, 22 Nov 2019 07:10:19 -0500 (EST) Received: from imap2 ([10.202.2.52]) by compute4.internal (MEProxy); Fri, 22 Nov 2019 07:10:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=althea.net; h= mime-version:message-id:in-reply-to:references:date:from:to :subject:content-type:content-transfer-encoding; s=fm2; bh=e+lyg fx29d/A6np3z+Q1Y+D8g5REWvz6HoHtr9vRMC0=; b=uGvXHROrANsHmZo5ZAoTN xiGm8AOKzh6s7PjB11titQUkP78wTi+6yXn/FoabcD8z1CF/dfeR4EYc54rcMays daTblWZzNikMKG2f0Uz1rOxNsELXX7NJZQVmf/mXl0OiRk3JnHr8HSNSxIH2VZpT zhVMRQJomeJNsXyN0H0xbhNhejnMDfck/lDW1GhvWENU8k5bSEsIhCPNy440/qs8 BTV4IwfZrQqvpYQWn8ZT0XB0UtxQeijuC6zplnE9hUb/q+esVPBWQl9EnvJyC5UH Qfksz4qyGyOLyvqzBZnfSx7YlFToAgIE0RhtSeUB0wLBUrU6lloeJM9Eg2oLD/QN g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=e+lygfx29d/A6np3z+Q1Y+D8g5REWvz6HoHtr9vRM C0=; b=Dyuo7qz8c7IRVSy7In9AWBRCDp0xexcmd7Q4wDaLTGKLqGlM5lILCcbsL 9ORPz6K9010qf76rNHxe+QgyYyZCTnCeXPzKKqA1/rEAQyx6KeWjH214DJzCt/1d 45gazKyJw7r4fBvRaw8/OZVkn5aBZuSQz0nBQXJDT+758A0oNhDRJyGGjp56ahpp wXZTWZ+4pEZKlY1d7m6R68oNK6/ZR+ArP3i2s64gZ84EryYII1ZZ1IzQbSZXzMB4 0JMopv2Gq2QQrLZBpP5IVFivqyPKp0ZMemZGCk6hBPCZieNWMKZv1GjsL8yWD1Bt MrZ0ZMCAR79kCIXAUpACA1TRpKt3w== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrudehgedgfeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtgfesth hqredtreerjeenucfhrhhomhepfdfluhhsthhinhcumfhilhhprghtrhhitghkfdcuoehj uhhsthhinhesrghlthhhvggrrdhnvghtqeenucffohhmrghinhepsghufhhfvghrsghloh grthdrnhgvthdpghhithhhuhgsrdgtohhmpdgrlhhthhgvrgdrnhgvthenucfrrghrrghm pehmrghilhhfrhhomhepjhhushhtihhnsegrlhhthhgvrgdrnhgvthenucevlhhushhtvg hrufhiiigvpedt X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id B19AFE00A3; Fri, 22 Nov 2019 07:10:18 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.1.7-578-g826f590-fmstable-20191119v1 Mime-Version: 1.0 Message-Id: In-Reply-To: <878so85e2d.fsf@toke.dk> References: <878so85e2d.fsf@toke.dk> Date: Fri, 22 Nov 2019 07:09:51 -0500 From: "Justin Kilpatrick" To: cake@lists.bufferbloat.net Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] Cake implementations X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Nov 2019 12:10:20 -0000 > An idea which was floated was to experiment with routing ISP customer= =20 > >traffic through a Linux server using cake to improve customer=20 > experience. Basically like Preseem. My colleague has toyed with it = a=20 > >bit in small test cases and was impressed with the outcomes. Althea.net runs Cake at every hop. Since we use Babel to dynamically gen= erate the network topology we rely on it's neighbor RTT extension and ba= ndwidth counters to reduce the bandwidth over a link when an intermediat= e device has decided to become bloaty.=20 It's been a very effective system that results in happier customers even= when we have weak leaks or crazy failover situations. Usually wireless = links have to be over provisioned by a factor of 2ish to keep latency un= der control in the face of dynamic interference and other external facto= rs. Instead Althea networks have more smaller links then back links betw= een each pop. Traffic is then load balanced based on latency and when an antenna start= s to really go crazy due to overloading the auto tuner clamps down Cake = until the experience is good again.=20 --=20 Justin Kilpatrick justin@althea.net On Fri, Nov 22, 2019, at 6:46 AM, Toke H=C3=B8iland-J=C3=B8rgensen wrote= : > "Adam Moffett" writes: >=20 > >>> > >>> Are there any commercial products already using Cake? > >> > >>Evenroute, eero, ubnt top that list. Evenroute's implementation is > >>superb, the first one that used active line measurements to handle > >>"sag". Anything derived from openwrt (somewhere between 10-30% of th= e > >>home router market). I'm not sure if preseem is using it or not. > >>dd-wrt. Most other things doing "SQM" are doing it via htb + fq_code= l. > >> > >> > > An idea which was floated was to experiment with routing ISP custome= r=20 > > traffic through a Linux server using cake to improve customer=20 > > experience. Basically like Preseem. My colleague has toyed with it= a=20 > > bit in small test cases and was impressed with the outcomes. > > > > He's looked closer than I have, but I'm trying to picture how his id= ea=20 > > would scale. I believe I'm seeing a CLI tool for configuring polici= es. =20 > > It seems like we'd have to create a middle layer to create/update=20= > > policies for customer's IP address based on information obtained fro= m=20 > > our AAA and CRM systems. I can picture some shapes that might take,= but=20 > > I think it would ultimately have to revolve around scripting the tc=20= > > command. There would be thousands of policies and a policy would be= =20 > > created/updated whenever a subscriber reconnects (e.g. when a DHCP l= ease=20 > > renews or a RADIUS auth event happens or similar). >=20 > I know several ISPs that do this (route traffic through a Linux box an= d > shape there). This deployment mode has not been the primary focus of > CAKE, though; the "standard" way to do it is with HTB+FQ-CoDel, which > allows you to set up arbitrarily complex configurations on a single > interface. This can also be made to scale, but there's a central qdisc= > lock in Linux that you have to get around to scale to multiple cores. > Fortunately, Jesper has already solved this particular issue: >=20 > https://github.com/netoptimizer/xdp-cpumap-tc >=20 > > Should we even pursue this idea? >=20 > In my own totally non-biased opinion: Yes! :) >=20 > > Although most staff who would touch this will have studied programmi= ng=20 > > in college, I would not qualify any of us as "programmers" per se. = My=20 > > biggest concern would be hitting a service affecting problem that we= =20 > > can't solve. >=20 > One way to go about this would be to open source the entire solution. > There are already bits and pieces available as open source (such as > Jesper's repository above, and sqm-scripts[0]) that you can build on, > and this way you could enlist community help to solve any issues with > the Linux side. You'd still need to get the data from your internal > systems, of course, but you could define a simple configuration format= > that was part of the open source code; then you'll just need to write = a > script that grabs customer info from your CRM and outputs the config > format... >=20 > > Second concern is that many of our equipment vendors already use > > Linux. Even Cisco now in some products. Maybe we'll waste our time > > trying to roll our own solution and then find that a software update= > > from a vendor next year gives us everything we needed anyway. >=20 > This would be great, of course, and do go and bug your vendors to solv= e > this problem! Note, however, that just because a system is running > Linux on the control plane, it may be using a hardware-offloaded data > plane that does not have any of the bufferbloat mitigation features > (unless the vendor specifically implemented them). I'm hoping that > *eventually* these things will be ubiquitous across the industry, but > thus far this has seemed to be an "any decade now" kind of proposition= :/ >=20 > -Toke >=20 > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake >