From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ot1-x331.google.com (mail-ot1-x331.google.com [IPv6:2607:f8b0:4864:20::331]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 5AAEE3CB35 for ; Mon, 10 Aug 2020 10:00:41 -0400 (EDT) Received: by mail-ot1-x331.google.com with SMTP id h22so7302061otq.11 for ; Mon, 10 Aug 2020 07:00:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=zs7eF+rrHTFvJ37fYcLuqOx48+ugE8lzR75E8vV1txI=; b=sQ2b//S5FeqTYNc07Pq67WhM9YMfjxNt3WH5yCU6eApWHSC/2SA7gFmW91y4ugmArV eJeZeTUSSY4UkphlkrRokQvPf05zwUzoKxk2sNAe0CN3XCy7W1zb6GG7FbJ3u0zVD6iL 3TODsYDvBb3zYXXUfTUmCyNCoIZZU4jzPGqRnKS8pmasTsaH4aHxWg01PC9+fRMPpPvf Mo8B48YgS0XhoVD/RUbuHB6DetybI2Vg1APagnB08tCHmvEfSet2HnPbUFsICfjVokpW vzkVlUIpwWE6zDn8shDuXu7K0yXeISB8bLGEx8UBEPKFX8d/+iErwA6gtGQ3ZGkYGzFu m8mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=zs7eF+rrHTFvJ37fYcLuqOx48+ugE8lzR75E8vV1txI=; b=cDVzaykzWI/axTV6j+l65QUh/Ao0VLcLKI/KHgcKjs+Cvzm66ugK1AtIpoZkfhXWox 2HORGdfg1xWkRsJlgBD5jHIkgN6BLTSAdzojodDdu3QOSGZOeiI+5kfnx/XIiLNv8OYB KOHA1g6qwnQJAV+YFx+y+jgr2vfFrO6RRPA44DsNkwIgOqsp1US/zIx+R6o2P38324N/ Vw58yPPhCFNasTlAB2UIjEAd0oeVBwNtWcUz+pQf95ST2zg38Q78MdeF1ZSrrll7+REK Zjh55Mu1EUzhgpUoTOtGaEd1uw2EwpiNWGd1BRE/imXhz6lM7BuIZaiVNkVe0RgbOHVs Eqiw== X-Gm-Message-State: AOAM530UIoDHMfVakkXpSYnoKtY5SoaSC1he/p7NDFqbsDY/9RxKtsii vaHn7KT1DX1Y8OxoFEczULb2G2NLZmmR4DaSmQ4= X-Google-Smtp-Source: ABdhPJyXlvZCJb9p5wCcVWw/NpnFcnQsjXMy5NoidNa6XmBtoINiy0tJNqNGEzJ1Ah0dUj8E1RoLCHCFmu9aQNzGu7Y= X-Received: by 2002:a05:6830:15ca:: with SMTP id j10mr846178otr.2.1597068040176; Mon, 10 Aug 2020 07:00:40 -0700 (PDT) MIME-Version: 1.0 References: <225a9c89-ac76-f21e-1450-5deeb3cd23eb@tomh.org> <04949cee-c4de-900c-e1b1-4b1f227933eb@rogers.com> In-Reply-To: <04949cee-c4de-900c-e1b1-4b1f227933eb@rogers.com> From: Daniel Sterling Date: Mon, 10 Aug 2020 10:00:28 -0400 Message-ID: To: davecb@spamcop.net Cc: Jonathan Morton , tomh@tomh.org, "dave.collier-brown@indexexchange.com" , bloat Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Bloat] How about a topical LWN article on demonstrating the real-world goodness of CAKE? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Aug 2020 14:00:41 -0000 So I've been wanting to write up what I did to improve my home network for a while. Here's a quick overview: I'm running a small laptop-class sandy bridge CPU in a small desktop computer, running openwrt, running cake. It has two NICs -- the built-in realtek NIC, and an old Intel gigabit NIC in the PCI slot. Internet goes into the realtek NIC and out the Intel NIC. (WAN / LAN in ope= nwrt) my internet is AT&T gigabit fiber, but I throttle that heavily with cake (see below) I manually apply cake with my own scripts. I'll post those on gist and reply to this email with that info, just wanted to write this up quickly this morning. but it's basically just, apply two simple cake tc lines to the NICs. For wifi I use UBNT's SOHO line -- Amplifi HD units. it works really rather well; after some tweaking I've managed to essentially get rid of the things that I've empirically found really hurt home network performance: 1. wifi dead zones -- solved by using as many amplifi HD units as you like, meshed or wired together. obviously wires are better than mesh and a dedicated backhaul set of APs is better than mesh but mesh works too. 2. wifi trying to use 5ghz when it's too slow and refusing to switch to 2ghz -- solved by amplifi AP having a setting where it kicks devices off the 5ghz network proactively to convice them to switch to 2ghz. thank you UBNT! 3. TCP not dropping enough packets. (or rather, not having good queue management) 4. TCP (or rather, the network) dropping too many TCP packets -- streams / apps / web sites will get "stuck" so after much tweaking, I've got cake set to 40mbit down, 20mbit up, enforced by two cakes (one for each NIC). that's fairly low -- it's low to highly throttle bulk streams so that I can play latency-sensitive games with basically no jitter and low latency, even if other people are using the wifi. even if I can't wire an xbox, I can still get low latency gaming on wifi but it's still high enough that we can stream HD video. and of course low jitter and low latency across the board means good ssh and video cal performance. just wanted to write this up quickly to reply to this thread -- cake really is amazing and I'd bet people would be willing to pay for a magic box like I've set up that they can stick in between their existing CPE and a decent AP that applies cake. or if AP vendors would put cake in their APs themselves, that would be good too. but as you note #1 and #2 on my list are important, even before queue management comes into play. you have to be willing to buy a good AP before cake really starts to matter, I think -- Dan On Mon, Aug 10, 2020 at 8:57 AM David Collier-Brown w= rote: > > On 2020-08-09 5:35 p.m., Jonathan Morton wrote: > > Are the risks and tradeoffs well enough understood (and visible enough > for troubleshooting) to recommend broader deployment? > > I recently gave openwrt a try on some hardware that I ultimately > concluded was insufficient for the job. Fairly soon after changing out > my access point, I started getting complaints of Wi-Fi dropping in my > household, especially when someone was trying to videoconference. I > discovered that my AP was spontaneously rebooting, and the box was > getting hot. > > Most CPE devices these days rely on hardware accelerated packet forwardin= g to achieve their published specs. That's all about taking packets in one= side and pushing them out the other as quickly as possible, with only mini= mal support from the CPU (likely, new connections get a NAT/firewall lookup= , that's all). It has the advantages of speed and power efficiency, but un= fortunately it is also incompatible with our debloating efforts. So debloa= ted CPE will tend to run hotter and with lower peak throughput, which may b= e noticeable to cable and fibre users; VDSL (FTTC) users might have service= of 80Mbps or less where this effect is less likely to matter. > > It sounds like that AP had a very marginal thermal design which caused th= e hardware to overheat as soon as the CPU was under significant load, which= it can easily be when a shaper and AQM are running on it at high throughpu= t. The cure is to use better designed hardware, though you could also cont= emplate breaking the case open to cure the thermal problem directly. There= are some known reliable models which could be collected into a list. As a= rule of thumb, the ones based on ARM cores are likely to be designed with = CPU performance more in mind than those with MIPS. > > Cake has some features which can be used to support explicit classificati= on and (de)prioritisation of traffic via firewall marking rules, either by = rewriting the Diffserv field or by associating metadata with packets within= the network stack (fwmark). This can be very useful for pushing Bittorren= t or WinUpdate swarm traffic out of the way. But for most situations, the = default flow-isolating behaviour already works pretty well, especially for = ensuring that one computer's network load has only a bounded effect on any = other. We can discuss that in more detail if that would be helpful. > > I'm primarily thinking of this week's version of the home router problem = (;-)) > > Because of the degree to which we're working from home and videoconferenc= ing, a lot of low-price, medium-performance devices are suddenly too wimpy = for their new role. > > A (very!) draft version is up in Google docs, at https://docs.google.com/= document/d/1gWKp9HqTbuHLfgD59WU4KJ8Og3eHuBtIeC7BUK0Ju9w/edit?usp=3Dsharing > > Using myself as the guinea-pig, running pfifo-fast was clearly bad, fq_co= del was better, and cake was good with a newish Fedora and the stock Rogers= router. It's been a while since I did rrul tests, and in any case, I thin= k that to convince readers we need a very practical way of making it clear = that they have a problem. I'm thinking that making VOIP fail might do the t= rick (;-)) > > The hard part, IMHO, is constructing a test that immediately communicates= the idea that the reader has a problem, and that CAKE addresses it. > > Returning to the hardware question, https://evenroute.com/iqrv3 seems to = be capable of handling up to ~300 Mbit/S connections, and my ISP only deliv= ers 170 (and advertises 150, which is mildly surprising!) > > I just ordered one, so I'll have a 'plug in" example, along with reflashi= ng my linksys for the umpty-thousandth time. > > --dave > > I suspect not enough people are aware of the later efforts of the buffer= bloat team, so I'm thinking of one or two articles, starting with LWN and a= n audience of aficionados. > > The core community is aware of what we've done, but in my view we haven't= converted "grandma". Grandma, as well as a whole bunch of ordinary enginee= rs and partners of engineers, are dependent on debloated performance becaus= e they're working at home now, and competing with granddaughter playing vid= eo games while they're trying to hold a video call. > > Right now, my colleagues at work suffer from more than a second of bloat-= related lag. They therefore tend to speak over each other on con-calls, apo= logize, start again and talk over each other, again. After a little while, = the picture becomes a distinctly silly one: a bunch of grown adults putting= their hands up and waving, like little kids in school. No-one has called o= ut =E2=80=9Cme, me, teacher=E2=80=9D yet, but I expect it any time. > > I propose we show the results in terms that we can explain to Grandma, sp= ecifically concentrating on functioning VOIP. I just upgraded to Fedora 31,= and the networking is absolutely stock, so I make a perfect victim/guinea-= pig (;-)) > > Who's interested? > > > > > -- > David Collier-Brown, | Always do right. This will gratify > System Programmer and Author | some people and astonish the rest > davecb@spamcop.net | -- Mark Twain > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat