From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ot1-x329.google.com (mail-ot1-x329.google.com [IPv6:2607:f8b0:4864:20::329]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id E43C03B2A4 for ; Tue, 11 Aug 2020 08:43:53 -0400 (EDT) Received: by mail-ot1-x329.google.com with SMTP id x24so9977211otp.3 for ; Tue, 11 Aug 2020 05:43:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=4gh5vpu4lLoKzZQEm1BYMd7rYPHLgs91HseCEKLID74=; b=Efa4k0Ne+MH1ra1WwIH7c2WW4J0GfbuZI4i/0Qq23giMK5EVCFZxHODs2HJZk5UJz3 ZueT9YkXeSkrvIDa7HNzd8SmZPdwi3wRWEZXNfkz61pSqUACd7IOhRmCEq9vXA+vvLu0 uHmi08yJM7gco5hKC11ocN4bBKQdFhfzUEnHjeZRm7A9IKTAV1Dc/8sri0sYIIJNlUIr JLYzSxU6U8N7RYRHKlRtXWoVIZA2quJSuMLfoCYALyTA0iQdVAkZ0lKzBhqSzihrVceS LB7qBs5KRzErMsqY7cVTDUcgLu5FEwUlHN/vQpmYWfiLKGt+KXLhjvUXCqPO8H9Qa0t3 Ulcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=4gh5vpu4lLoKzZQEm1BYMd7rYPHLgs91HseCEKLID74=; b=XAW9GMhGD8xNcIPTKmgDuHzG5IjgQEDTwDoCBdBdx2LUztb7YfenoiGGzpILbCP9wa xe6PvAtNziGuAlXzlnIUuWAzZwTFA0EmABX2tqlPgn9djJZuEli4ZQturOonojSdqeAr +06yNK1sa/do/3tq0sriv+RKPHt6G2VlJSrGkn1TRJThQAm0cSszE+w+wfCf6vn7FtXi hyQlX8+s1RIi7DatIWIGtpfQh/j1bdQwytgFDuR38D82sfbYBibSESjlggNFUkdklnln VIoxWSIpPisgZfdD35AUy+9ENQWcxLoM38dwSvfd4up0Ly/47aZQ/jhScx47byHB81uB cnOw== X-Gm-Message-State: AOAM533v1jj+x9DM3mD50gq+cPZeqjGlnnYHBQzEGe1aZc8y4KCQ4mDB FzSJldUZ94j+OLOWeUl/yDRR+JJhOV9Dwa26vzk= X-Google-Smtp-Source: ABdhPJzLrFQcxouW2Yzb5DuQm2Srmh5AoTyUM8sQQDDcb/yWeMqcUSRwXLzC8geBQ4ikzqI8S+Q7IvQB2D4vSmzlm8I= X-Received: by 2002:a9d:7f0b:: with SMTP id j11mr4584494otq.291.1597149832793; Tue, 11 Aug 2020 05:43:52 -0700 (PDT) MIME-Version: 1.0 References: <225a9c89-ac76-f21e-1450-5deeb3cd23eb@tomh.org> <04949cee-c4de-900c-e1b1-4b1f227933eb@rogers.com> In-Reply-To: From: Daniel Sterling Date: Tue, 11 Aug 2020 08:43:40 -0400 Message-ID: To: davecb@spamcop.net Cc: Jonathan Morton , tomh@tomh.org, "dave.collier-brown@indexexchange.com" , bloat Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Bloat] How about a topical LWN article on demonstrating the real-world goodness of CAKE? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Aug 2020 12:43:54 -0000 as promised here is the script I run after rebooting my openwrt box, to set up cake https://gist.github.com/eqhmcow/c378c46a41aa5716767a0da811087dd4 On Mon, Aug 10, 2020 at 10:00 AM Daniel Sterling wrote: > > So I've been wanting to write up what I did to improve my home network > for a while. > > Here's a quick overview: > > I'm running a small laptop-class sandy bridge CPU in a small desktop > computer, running openwrt, running cake. It has two NICs -- the > built-in realtek NIC, and an old Intel gigabit NIC in the PCI slot. > > Internet goes into the realtek NIC and out the Intel NIC. (WAN / LAN in o= penwrt) > > my internet is AT&T gigabit fiber, but I throttle that heavily with > cake (see below) > > I manually apply cake with my own scripts. I'll post those on gist and > reply to this email with that info, just wanted to write this up > quickly this morning. but it's basically just, apply two simple cake > tc lines to the NICs. > > For wifi I use UBNT's SOHO line -- Amplifi HD units. > > it works really rather well; after some tweaking I've managed to > essentially get rid of the things that I've empirically found really > hurt home network performance: > > 1. wifi dead zones -- solved by using as many amplifi HD units as you > like, meshed or wired together. obviously wires are better than mesh > and a dedicated backhaul set of APs is better than mesh but mesh works > too. > > 2. wifi trying to use 5ghz when it's too slow and refusing to switch > to 2ghz -- solved by amplifi AP having a setting where it kicks > devices off the 5ghz network proactively to convice them to switch to > 2ghz. thank you UBNT! > > 3. TCP not dropping enough packets. (or rather, not having good queue > management) > > 4. TCP (or rather, the network) dropping too many TCP packets -- > streams / apps / web sites will get "stuck" > > so after much tweaking, I've got cake set to 40mbit down, 20mbit up, > enforced by two cakes (one for each NIC). that's fairly low -- > > it's low to highly throttle bulk streams so that I can play > latency-sensitive games with basically no jitter and low latency, even > if other people are using the wifi. even if I can't wire an xbox, I > can still get low latency gaming on wifi > > but it's still high enough that we can stream HD video. > > and of course low jitter and low latency across the board means good > ssh and video cal performance. > > just wanted to write this up quickly to reply to this thread -- cake > really is amazing and I'd bet people would be willing to pay for a > magic box like I've set up that they can stick in between their > existing CPE and a decent AP that applies cake. or if AP vendors would > put cake in their APs themselves, that would be good too. > > but as you note #1 and #2 on my list are important, even before queue > management comes into play. you have to be willing to buy a good AP > before cake really starts to matter, I think > > -- Dan > > On Mon, Aug 10, 2020 at 8:57 AM David Collier-Brown = wrote: > > > > On 2020-08-09 5:35 p.m., Jonathan Morton wrote: > > > > Are the risks and tradeoffs well enough understood (and visible enough > > for troubleshooting) to recommend broader deployment? > > > > I recently gave openwrt a try on some hardware that I ultimately > > concluded was insufficient for the job. Fairly soon after changing out > > my access point, I started getting complaints of Wi-Fi dropping in my > > household, especially when someone was trying to videoconference. I > > discovered that my AP was spontaneously rebooting, and the box was > > getting hot. > > > > Most CPE devices these days rely on hardware accelerated packet forward= ing to achieve their published specs. That's all about taking packets in o= ne side and pushing them out the other as quickly as possible, with only mi= nimal support from the CPU (likely, new connections get a NAT/firewall look= up, that's all). It has the advantages of speed and power efficiency, but = unfortunately it is also incompatible with our debloating efforts. So debl= oated CPE will tend to run hotter and with lower peak throughput, which may= be noticeable to cable and fibre users; VDSL (FTTC) users might have servi= ce of 80Mbps or less where this effect is less likely to matter. > > > > It sounds like that AP had a very marginal thermal design which caused = the hardware to overheat as soon as the CPU was under significant load, whi= ch it can easily be when a shaper and AQM are running on it at high through= put. The cure is to use better designed hardware, though you could also co= ntemplate breaking the case open to cure the thermal problem directly. The= re are some known reliable models which could be collected into a list. As= a rule of thumb, the ones based on ARM cores are likely to be designed wit= h CPU performance more in mind than those with MIPS. > > > > Cake has some features which can be used to support explicit classifica= tion and (de)prioritisation of traffic via firewall marking rules, either b= y rewriting the Diffserv field or by associating metadata with packets with= in the network stack (fwmark). This can be very useful for pushing Bittorr= ent or WinUpdate swarm traffic out of the way. But for most situations, th= e default flow-isolating behaviour already works pretty well, especially fo= r ensuring that one computer's network load has only a bounded effect on an= y other. We can discuss that in more detail if that would be helpful. > > > > I'm primarily thinking of this week's version of the home router proble= m (;-)) > > > > Because of the degree to which we're working from home and videoconfere= ncing, a lot of low-price, medium-performance devices are suddenly too wimp= y for their new role. > > > > A (very!) draft version is up in Google docs, at https://docs.google.co= m/document/d/1gWKp9HqTbuHLfgD59WU4KJ8Og3eHuBtIeC7BUK0Ju9w/edit?usp=3Dsharin= g > > > > Using myself as the guinea-pig, running pfifo-fast was clearly bad, fq_= codel was better, and cake was good with a newish Fedora and the stock Roge= rs router. It's been a while since I did rrul tests, and in any case, I th= ink that to convince readers we need a very practical way of making it clea= r that they have a problem. I'm thinking that making VOIP fail might do the= trick (;-)) > > > > The hard part, IMHO, is constructing a test that immediately communicat= es the idea that the reader has a problem, and that CAKE addresses it. > > > > Returning to the hardware question, https://evenroute.com/iqrv3 seems t= o be capable of handling up to ~300 Mbit/S connections, and my ISP only del= ivers 170 (and advertises 150, which is mildly surprising!) > > > > I just ordered one, so I'll have a 'plug in" example, along with reflas= hing my linksys for the umpty-thousandth time. > > > > --dave > > > > I suspect not enough people are aware of the later efforts of the buff= erbloat team, so I'm thinking of one or two articles, starting with LWN and= an audience of aficionados. > > > > The core community is aware of what we've done, but in my view we haven= 't converted "grandma". Grandma, as well as a whole bunch of ordinary engin= eers and partners of engineers, are dependent on debloated performance beca= use they're working at home now, and competing with granddaughter playing v= ideo games while they're trying to hold a video call. > > > > Right now, my colleagues at work suffer from more than a second of bloa= t-related lag. They therefore tend to speak over each other on con-calls, a= pologize, start again and talk over each other, again. After a little while= , the picture becomes a distinctly silly one: a bunch of grown adults putti= ng their hands up and waving, like little kids in school. No-one has called= out =E2=80=9Cme, me, teacher=E2=80=9D yet, but I expect it any time. > > > > I propose we show the results in terms that we can explain to Grandma, = specifically concentrating on functioning VOIP. I just upgraded to Fedora 3= 1, and the networking is absolutely stock, so I make a perfect victim/guine= a-pig (;-)) > > > > Who's interested? > > > > > > > > > > -- > > David Collier-Brown, | Always do right. This will gratify > > System Programmer and Author | some people and astonish the rest > > davecb@spamcop.net | -- Mark Twain > > > > _______________________________________________ > > Bloat mailing list > > Bloat@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/bloat