From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-x234.google.com (mail-qt0-x234.google.com [IPv6:2607:f8b0:400d:c0d::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 9FB203B29F for ; Wed, 19 Oct 2016 09:44:50 -0400 (EDT) Received: by mail-qt0-x234.google.com with SMTP id m5so19764634qtb.3 for ; Wed, 19 Oct 2016 06:44:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=uIQzejmt+Bt91V/Q4wntb4Z6LE68YBmu2nKEZrgv5b8=; b=HfXsHFPT96C5qwY8bu9K91U+y2RebrZvKFMa7Hx/jsD+6MMnvDhA0B3ChG/lyt97sy 7bWRBBGAr0CBr72EKZyxyqAtX10oOr7XaUgBrn522kMc1lst/QPpgrR15m7cM/mgp45D Ik1BNH+C3o3i+UEdIuoKK5C8SOVb+x85eVPkX9nrRhB8iFj4i9k9QY6WVt61bB2/dhHD qb0nGiKeOOvVtOTwUZlTO5uJke3gv5NJ9eUfPQfbH52F2dAlVsVEIFIj+gwaSrVSYxz0 3uwBf94AYSMN7ABYdEJRkA34/Rd6K8PR8a9jNV4dubQrkYbe1xTzPoMlyvskoe3X8iG+ 1U/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=uIQzejmt+Bt91V/Q4wntb4Z6LE68YBmu2nKEZrgv5b8=; b=F37kdMcNUc3Bc2BL1/4kyfnJmpLy3EP8soevsvBTIw1TMVBwKgY8klphG9MuolX7it Vkuk37BM9A3O02T7EudFzs+8mIRyJNhJy/+AhVWaJBfjTHnvyA6Rqdj51BxSiW4fIkwZ 6BqCH0OiJIsQ5NJJf+l7ofIV/jH7S7kVdci8eg/YfvF+Su45scy55BnCS/w3rqoi3EVi jYJmc/f3u7PSUEdthKdz7u1oRKfd+FjA3CD/55yw3rvzaxEqV5R1iiTzm5QceykFe7tF pWuyYwQhjRgNPLYP84rMUjF7pNZG9neb7QVZZxGVWmK7+psZNFxj24ZT0Z3NYGST0hkT oK0g== X-Gm-Message-State: AA6/9RkCHE5iUjrwci3+6mEzsxdrHnOQdZ+MQXYFj0KtgAKjGr9cEKUTrkw8GPtXwxCgyy87f+IpMg48mtVn4g== X-Received: by 10.200.52.75 with SMTP id v11mr5869913qtb.137.1476884689847; Wed, 19 Oct 2016 06:44:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.12.146.164 with HTTP; Wed, 19 Oct 2016 06:44:48 -0700 (PDT) In-Reply-To: <5754D14C-6A14-4100-947C-CAE5C7449F69@gmail.com> References: <8943993b-cc1a-4e57-be52-b2096e7dda22@googlegroups.com> <5754D14C-6A14-4100-947C-CAE5C7449F69@gmail.com> From: Dave Taht Date: Wed, 19 Oct 2016 06:44:48 -0700 Message-ID: To: Jonathan Morton , make-wifi-fast@lists.bufferbloat.net Cc: Simon Barber , BBR Development Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Make-wifi-fast] [bbr-dev] WiFi aggregation X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 13:44:50 -0000 On Tue, Oct 18, 2016 at 5:12 PM, Jonathan Morton wr= ote: > >> On 19 Oct, 2016, at 02:24, 'Simon Barber' via BBR Development wrote: >> >> To maintain high efficiency filling the aggregates is required. To maint= ain low latency a fq_codel like approach can be used, where mice get schedu= led first. > > But that behaviour will obviously lead to the high variation in RTT which= was mentioned earlier in this thread, in the context of BBR having to cope= with it. > > When BBR probes for minRTT, it will intentionally become a sparse flow an= d, if it is the only flow on that station, the resultant preferential airti= me scheduling will yield a considerably lower latency than the optimal-thro= ughput steady state. Depending on the overall RTT ratio, BBR may then be u= nable to discover the true maximum bandwidth before the RTT climbs above it= s threshold. > > This high variation in RTT is also likely to affect conventional TCPs=E2= =80=99 RTO calculations, since unlike a wired link, the acks will come in b= ursts each corresponding to one MAC grant. > > It also means that latency of a genuinely sparse, latency-sensitive flow = is strongly affected by the presence of a bulk flow on the same station, ev= en if flow isolation within stations is in effect. > > These are all effects peculiar to aggregation (which is itself a reaction= to wifi=E2=80=99s extremely high MAC overhead), and are not present on a w= ell-behaved wired link. In the latter case, doling out one packet per host= per cycle still results in good throughput and the best possible latency. > > All of this leads back to airtime as the correct measure of per-station f= airness - but how much airtime per station? > > From a pure throughput perspective, the answer is =E2=80=9Cthe airtime of= a max-size aggregate to the fastest station=E2=80=9D, with slower stations= getting either correspondingly smaller aggregates, or proportionally fewer= TXOPs, or a combination of the two. (This assumes we don=E2=80=99t starve= slower stations entirely in the holy name of global throughput.) > > But I (and Dave) firmly believe you have to consider latency in the equat= ion as well, as the world is not all about bulk TCP transfers (any more tha= n it=E2=80=99s all about UDP floods, which are often used in PMPO-style ben= chmarks). Latency is also a measure of airtime, but it=E2=80=99s airtime *= not* given to a station which has data ready for it, and there must be a ma= ximum acceptable value which the granted airtime per station must be dynami= cally tuned to fit. > > In the limit, global optimisation for latency results in one packet sent = per TXOP. At high SNR, this will give much lower throughput than large agg= regates, but networks have an annoying habit of continuing to function at l= ow throughput and low latencies, in a way they just don=E2=80=99t at high t= hroughput but unacceptable latencies. With Cake, I can still have a well-f= unctioning Internet connection at 64Kbps symmetric, with the exception of e= ssentially CBR services like YouTube. > > There is of course a middle ground. Let=E2=80=99s find it. > > A good starting point might be to aim for 50% airtime efficiency, rather = than 100%. That is, aggregates are allowed to grow until they occupy as mu= ch airtime as the MAC overhead. Discuss. Well, I think the issues a small number of people are having with fixing wifi and BBR's potential impact on it probably belongs more to the make-wifi-fast list. The BBR community is focused on fixing a portion of the internet as it is today. Merely getting a cubic replacement out there is a big job... I could see one day trying to write a roadmap document about what else is still needed in BBR and elsewhere, for a better internet. FQ at the ISP would remain one of them. Getting ISP buffering *universally* down to no more than 100ms would be nice also. Unless that breaks Africa. Converging on the right way to do ECN could be another, but I'm not looking forward to all the nicotine patches that would take. Maybe someone at google already has got that roadmap? (Matt?) I've enjoyed thinking about BBR, as the first TCP designed with big data, actual application behavior, and the user experience in mind, it's reset my thinking in multiple ways, but I have to admit my own foci for the next while are, on my last fumes of funding, finishing a talk for linuxplumbers nov 3, and trying to find and fix the latest "last bugs" in the fq-wifi-mac code, and get that successfully upstream, while making a bit of progress on the airtime fair code. Merely having BBR around to check against is an enormous help in thinking about futures, but I'm not sure if any insights will emerge with mere discussion. I've got tons more data to collect and think about, and a couple tweaks here and there, and overall I think I'm marching in a better direction. I hopefully succeeded in demolishing the "always fill a full txop even with tons of stations" argument earlier in this thread. I'm compelled to point out that the numbers I used throughout that post were pretty bogus (I can try to refine the argument for a wider audience), and that in the real world, when a station pops up for a 2 second transaction, the actual bandwidth available to it tends to vary massively as the rate controller (minstrel in that case) struggles to find the right wifi rate. I like this new math floating about, I hope to get spare brain cells to think about the minstrel part of the problem... after some more stuff gets off my plate! as for aiming for 50% airtime usage, I think that's too narrow a goal. My own at the moment is merely to keep ripping out all the excess latency and jitter in wifi and to find more ways to resist putting any back in except where absolutely necessary. I generally think that the more we do that, the better TCPs and most applications will behave. Although we are getting great aggregation results now, I also tend to think of aggregation as a "shock absorber" all it's own. --=20 Dave T=C3=A4ht Let's go make home routers and wifi faster! With better software! http://blog.cerowrt.org