From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 2F0DB21F36E for ; Sun, 1 Feb 2015 15:34:11 -0800 (PST) Received: by mail-qg0-f42.google.com with SMTP id q107so45335805qgd.1 for ; Sun, 01 Feb 2015 15:34:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=BacxEfJwRrCukTnzZtcx3hqVi6A+298PY1cpsFIZpxs=; b=NXrSQTR7gvLfmBy6lvUIOLJKCmMWDPQibD+OSkWfY78T19Otw1WEOHkY/9kCpAb/lq Q5aqhMffyr0rbq3mRIGun+r0LtYjeeRdYvtF9HcDd9uXIbWwixmCtHrMUtBAbMS0+qww BA48aH1MGWoIIGvRrXMZ5UszrWUh0+8D7iNTeEdGm+kzUwJT6akLJJf/vsWfKJNw+NgP IqKEZejw+vVkDLXbhAV855HxIJf/+zATZ95g05Qg5HyG88QLUWU1ibsawY8BvO1+yMMv Gaf+xyiXVAN9MR3Hc9x89CzjHfDCazH19EP7/eLCnGLTTSAfUllQ5if8mYe+7dbS0X/y 8mIQ== MIME-Version: 1.0 X-Received: by 10.140.43.194 with SMTP id e60mr34459928qga.60.1422833650441; Sun, 01 Feb 2015 15:34:10 -0800 (PST) Received: by 10.140.48.7 with HTTP; Sun, 1 Feb 2015 15:34:10 -0800 (PST) In-Reply-To: <1422801814.796219699@apps.rackspace.com> References: <1422537297.21689.15.camel@edumazet-glaptop2.roam.corp.google.com> <54CB5D08.2070906@broadcom.com> <1422623975.21689.77.camel@edumazet-glaptop2.roam.corp.google.com> <54CB8B69.1070807@broadcom.com> <1422741065.199624134@apps.rackspace.com> <1422801814.796219699@apps.rackspace.com> Date: Mon, 2 Feb 2015 10:34:10 +1100 Message-ID: From: Andrew McGregor To: dpreed@reed.com Content-Type: multipart/alternative; boundary=001a113a5fb04f657b050e0f46d0 Cc: dstanley@arubanetworks.com, Stig Thormodsrud , netdev@vger.kernel.org, linux-wireless , Jesper Dangaard Brouer , cerowrt-devel@lists.bufferbloat.net, Matt Mathis , Derrick Pallas , Kathy Giori , Mahesh Paolini-Subramanya , Jonathan Morton , Tim Shepard , Avery Pennarun Subject: Re: [Cerowrt-devel] Fwd: Throughput regression with `tcp: refine TSO autosizing` X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Feb 2015 23:34:40 -0000 --001a113a5fb04f657b050e0f46d0 Content-Type: text/plain; charset=UTF-8 So far as I'm concerned, Minstrel is as widely adopted as we could want; it could go further, but it doesn't need any help with that. It could be better, however, and that's a fairly straightforward change. One of the problems with rate selection is that the standard itself has a bit to say on what rates to use for what packets, and what it says is braindead. So if you want to be standards compliant you are also leaving quite a bit of potential performance on the table. I missed one item in my list of potential improvements: the most braindead thing 802.11 has to say about rates is that broadcast and multicast packets should be sent at 'the lowest basic rate in the current supported rate set', which is really wasteful. There are a couple of ways of dealing with this: one, ignore the standard and pick the rate that is most likely to get the frame to as many neighbours as possible (by a scan of the Minstrel tables). Or two, fan it out as unicast, which might well take less airtime (due to aggregation) as well as being much more likely to be delivered, since you get ACKs and retries by doing that. As for the industry... sure, there are those forces, but then again there's a non-trivial amount of influence in this audience. On Mon, Feb 2, 2015 at 1:43 AM, wrote: > Just to clarify, managing queueing in a single access point WiFi network > is only a small part of the problem of fixing the rapidly degrading > performance of WiFi based systems. Similarly, mesh routing is only a small > part of the problem with the scalability of cooperative meshes based on the > WiFi MAC. > > > > So I don't disagree with work on queue management (which is not good in > any commercial product). > > > > But Dave T has done some great talks on "fixing WiFi" that don't have to > do with queueing very much at all. > > > > For example, rate selection for various packets is terrible. When you > have nearly 1000:1 ratios of transmission rates and codes that are not > backward compatible, there's a huge opportunity for improvement. > Similarly, choice of frequency bandwidth and center frequency at each > station offers huge opportunities for practical scalability of systems. > Also, as we noted earlier, "handoff" from one next hop to another is a huge > problem with performance in practical deployments (a factor of 10x at > least, just in that). > > > > Propagation information is not used at all when 802.11 systems share a > channel, even in single AP deployments, yet all stations can measure > propagation quite accurately in their hardware. > > > > Finally, Listen-before-talk is highly wasteful for two reasons: 1) any > random radio noise from other sources unnecessarily degrades communications > (and in the 5.8 MHz band, the rule about "radar avoidance" requires > treating very low level noise as a "signal to shut the net down by law", > but there is a loophole if you can tell that it's not actually "radar" (the > technique requires two or more stations to measure the same noise event, > and if the power is significantly different - more than a few dB - then it > can't possibly be due to a distant transmitter, and therefore can be > ignored). 2) the transmitter cannot tell when the intended receiver will be > perfectly able to decode the signal without interference with the station > it hears (this second point is actually proven in theory in a paper by Jon > Peha that argued against trivial "etiquettes" as a mechanism for sharing > among uncooperative and non-interoperable stations). > > > > Dave T has discussed more, as have I in other venues. > > > > The reason no one is making progress on any of these particular issues is > that there is no coordination at the "systems level" around creating rising > tides that lift all boats in the WiFi-ish space. It's all about ripping > the competition by creating stuff that can sell better than the other guys' > stuff, and avoiding cooperation at all costs. > > > > I agree that, to the extent that managing queues in a single box or a > single operating system doesn't require cooperation, it's much easier to > get such things into the market. That's why CeroWRT has been as effective > as it has been. But has Microsoft done anything at all about it? Do the > better ECN signals that can arise from good queue management get used by > the TCP endpoints, or for that matter UDP-based protocol endpoints? > > > > But the big wins in making WiFi better are going begging. As WiFi becomes > more closed, as it will as the major Internet Access Providers and Gadget > builders (Google, Apple) start excluding innovators in wireless from the > market by closed, proprietary solutions, the problem WILL get worse. You > won't be able to fix those problems at all. If you have a solution you > will have to convince the oligopoly to even bother trying it. > > > > So, let me reiterate. The problem is not just "getting Minstrel adopted", > though I have nothing against that as a subgoal. The problem is to find > good systems-level answers, and to find a strategy to deliver those answers > to a WiFi ecology that spans the planet, and where the marketing > value-story focuses on things one can measure between two stations in a > Faraday cage, and never on any systems-level issues. > > > > > > I personally think that things like promoting semi-closed, essentially > proprietary ESSID-based bridged distribution systems as "good ideas" are > counterproductive to this goal. But that's perhaps too radical for this > crowd. It reminds me of Cisco's attempt to create a proprietary Internet > technology with IOS, which fortunately was not the success Cisco hoped for, > or Juniper would not have existed. Maybe IOS would have been a fine > standard, but it would have killed the evolution of the Internet as we know > it. > > > On Sunday, February 1, 2015 5:47am, "Jonathan Morton" < > chromatix99@gmail.com> said: > > Since this is going to be a big job, it's worth prioritising parts of it > appropriately. > > Minstrel is probably already the single best feature of the Linux Wi-Fi > stack. AFAIK it still outperforms any other rate selector we know about. So > I don't consider improving it further to be a high priority, although that > trick of using it as a sneaky random packet loss inducer is intriguing. > > Much more important and urgent is getting some form of functioning SQM > closer to the hardware, where the information is. I don't think we need to > get super fancy here to do some real good, in the same way that PIE is a > major improvement over drop-tail. I'd settle for a variant of fq_codel that > gets and uses information about whether the current packet request might be > aggregated with the previous packet provided, and adjusts its choice of > packet accordingly. > > At the same time, models would undoubtedly be useful to help test and > repeatably demonstrate the advantages of both simple and more sophisticated > solutions. Ns3 allows laying out a reasonably complex radio environment, > which is great for this. To counter the prevalence of one-station Faraday > cage tests in the industry, the simulated environments should represent > realistic, challenging use cases: > > 1) the family home, with half a dozen client devices competing with > several interference sources (Bluetooth, RC toys, microwave oven, etc). > This is a relatively easy environment, representing the expected > environment for consumer equipment. > > 2) the apartment block, with fewer clients per AP but lots of APs > distributed throughout a large building. Walls and floors may provide > valuable attenuation here - unless you're in Japan, where they can be > notoriously thin. > > 3) the railway carriage, consisting of eighty passengers in a 20x3 m > space, and roughly the same number of client devices. The uplink is 3G > based and has some inherent latency. Add some Bluetooth for flavour, stir > gently. This one is rather challenging, but there is scope to optimise AP > antenna placement, and to scale the test down slightly by reducing seat > occupancy. > > 4) the jumbo jet, consisting of several hundred passengers crammed in like > sardines. The uplink has satellite latencies built in. Good luck. > > 5) the business hotel. Multiple APs will be needed to provide adequate > coverage for this environment, which should encompass the rooms as well as > lounge, conference and dining areas. Some visitors may bring their own APs, > and the system must be able to cope with this without seriously degrading > performance. > > 6) the trade conference. A large arena filled with thousands of people. > Multiple APs required. Good luck. > > I also feel that ultimately we're going to have to get industry on board. > Not just passively letting us play around as with ath9k, but actively > taking note of our findings and implementing at least a few of our ideas > themselves. Of course, tools, models and real-world results are likely to > make that easier. > > - Jonathan Morton > --001a113a5fb04f657b050e0f46d0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
So far as I'm concerned, Minstrel is as widely adopted= as we could want; it could go further, but it doesn't need any help wi= th that.=C2=A0 It could be better, however, and that's a fairly straigh= tforward change.=C2=A0 One of the problems with rate selection is that the = standard itself has a bit to say on what rates to use for what packets, and= what it says is braindead.=C2=A0 So if you want to be standards compliant = you are also leaving quite a bit of potential performance on the table.
I missed one item in my list of potential improvements: the= most braindead thing 802.11 has to say about rates is that broadcast and m= ulticast packets should be sent at 'the lowest basic rate in the curren= t supported rate set', which is really wasteful.=C2=A0 There are a coup= le of ways of dealing with this: one, ignore the standard and pick the rate= that is most likely to get the frame to as many neighbours as possible (by= a scan of the Minstrel tables).=C2=A0 Or two, fan it out as unicast, which= might well take less airtime (due to aggregation) as well as being much mo= re likely to be delivered, since you get ACKs and retries by doing that.

As for the industry... sure, there are those forces,= but then again there's a non-trivial amount of influence in this audie= nce.

O= n Mon, Feb 2, 2015 at 1:43 AM, <dpreed@reed.com> wrote:

Just to clar= ify, managing queueing in a single access point WiFi network is only a smal= l part of the problem of fixing the rapidly degrading performance of WiFi b= ased systems.=C2=A0 Similarly, mesh routing is only a small part of the pro= blem with the scalability of cooperative meshes based on the WiFi MAC.

=C2=A0

So I don't disagree with work on queue management (which is= not good in any commercial product).

=C2=A0

But Dave T has done some great talks on "fixing WiFi"= that don't have to do with queueing very much at all.

=C2=A0

For example, rate selection for various packets is terrible.=C2= =A0 When you have nearly 1000:1 ratios of transmission rates and codes that= are not backward compatible, there's a huge opportunity for improvemen= t.=C2=A0 Similarly, choice of frequency bandwidth and center frequency at e= ach station offers huge opportunities for practical scalability of systems.= =C2=A0 Also, as we noted earlier, "handoff" from one next hop to = another is a huge problem with performance in practical deployments (a fact= or of 10x at least, just in that).

=C2=A0

Propagation information is not used at all when 802.11 systems = share a channel, even in single AP deployments, yet all stations can measur= e propagation quite accurately in their hardware.

=C2=A0

Finally, Listen-before-talk is highly wasteful for two reasons:= 1) any random radio noise from other sources unnecessarily degrades commun= ications (and in the 5.8 MHz band, the rule about "radar avoidance&quo= t; requires treating very low level noise as a "signal to shut the net= down by law", but there is a loophole if you can tell that it's n= ot actually "radar" (the technique requires two or more stations = to measure the same noise event, and if the power is significantly differen= t - more than a few dB - then it can't possibly be due to a distant tra= nsmitter, and therefore can be ignored). 2) the transmitter cannot tell whe= n the intended receiver will be perfectly able to decode the signal without= interference with the station it hears (this second point is actually prov= en in theory in a paper by Jon Peha that argued against trivial "etiqu= ettes" as a mechanism for sharing among uncooperative and non-interope= rable stations).

=C2=A0

Dave T has discussed more, as have I in other venues.

=C2=A0

The reason no one is making progress on any of these particular= issues is that there is no coordination at the "systems level" a= round creating rising tides that lift all boats in the WiFi-ish space.=C2= =A0 It's all about ripping the competition by creating stuff that can s= ell better than the other guys' stuff, and avoiding cooperation at all = costs.

=C2=A0

I agree that, to the extent that managing queues in a single bo= x or a single operating system doesn't require cooperation, it's mu= ch easier to get such things into the market.=C2=A0 That's why CeroWRT = has been as effective as it has been.=C2=A0 But has Microsoft done anything= at all about it? =C2=A0 Do the better ECN signals that can arise from good= queue management get used by the TCP endpoints, or for that matter UDP-bas= ed protocol endpoints?

=C2=A0

But the big wins in making WiFi better are going begging.=C2=A0= As WiFi becomes more closed, as it will as the major Internet Access Provi= ders and Gadget builders (Google, Apple) start excluding innovators in wire= less from the market by closed, proprietary solutions, the problem WILL get= worse.=C2=A0 You won't be able to fix those problems at all.=C2=A0 If = you have a solution you will have to convince the oligopoly to even bother = trying it.

=C2=A0

So, let me reiterate.=C2=A0 The problem is not just "getti= ng Minstrel adopted", though I have nothing against that as a subgoal.= =C2=A0 The problem is to find good systems-level answers, and to find a str= ategy to deliver those answers to a WiFi ecology that spans the planet, and= where the marketing value-story focuses on things one can measure between = two stations in a Faraday cage, and never on any systems-level issues.

=C2=A0

=C2=A0

I personally think that things like promoting semi-closed, esse= ntially proprietary ESSID-based bridged distribution systems as "good = ideas" are counterproductive to this goal.=C2=A0 But that's perhap= s too radical for this crowd.=C2=A0 It reminds me of Cisco's attempt to= create a proprietary Internet technology with IOS, which fortunately was n= ot the success Cisco hoped for, or Juniper would not have existed. Maybe IO= S would have been a fine standard, but it would have killed the evolution o= f the Internet as we know it.


On Sunday, February 1, 2015 5:47am, "Jonathan Morton&q= uot; <chromat= ix99@gmail.com> said:

Since this is going to be a big job, it's worth= prioritising parts of it appropriately.

Minstrel is probably already the single best featur= e of the Linux Wi-Fi stack. AFAIK it still outperforms any other rate selec= tor we know about. So I don't consider improving it further to be a hig= h priority, although that trick of using it as a sneaky random packet loss = inducer is intriguing.

Much more important and urgent is getting some form= of functioning SQM closer to the hardware, where the information is. I don= 't think we need to get super fancy here to do some real good, in the s= ame way that PIE is a major improvement over drop-tail. I'd settle for = a variant of fq_codel that gets and uses information about whether the curr= ent packet request might be aggregated with the previous packet provided, a= nd adjusts its choice of packet accordingly.

At the same time, models would undoubtedly be usefu= l to help test and repeatably demonstrate the advantages of both simple and= more sophisticated solutions. Ns3 allows laying out a reasonably complex r= adio environment, which is great for this. To counter the prevalence of one= -station Faraday cage tests in the industry, the simulated environments sho= uld represent realistic, challenging use cases:

1) the family home, with half a dozen client device= s competing with several interference sources (Bluetooth, RC toys, microwav= e oven, etc). This is a relatively easy environment, representing the expec= ted environment for consumer equipment.

2) the apartment block, with fewer clients per AP b= ut lots of APs distributed throughout a large building. Walls and floors ma= y provide valuable attenuation here - unless you're in Japan, where the= y can be notoriously thin.

3) the railway carriage, consisting of eighty passe= ngers in a 20x3 m space, and roughly the same number of client devices. The= uplink is 3G based and has some inherent latency. Add some Bluetooth for f= lavour, stir gently. This one is rather challenging, but there is scope to = optimise AP antenna placement, and to scale the test down slightly by reduc= ing seat occupancy.

4) the jumbo jet, consisting of several hundred pas= sengers crammed in like sardines. The uplink has satellite latencies built = in. Good luck.

5) the business hotel. Multiple APs will be needed = to provide adequate coverage for this environment, which should encompass t= he rooms as well as lounge, conference and dining areas. Some visitors may = bring their own APs, and the system must be able to cope with this without = seriously degrading performance.

6) the trade conference. A large arena filled with = thousands of people. Multiple APs required. Good luck.

I also feel that ultimately we're going to have= to get industry on board. Not just passively letting us play around as wit= h ath9k, but actively taking note of our findings and implementing at least= a few of our ideas themselves. Of course, tools, models and real-world res= ults are likely to make that easier.

- Jonathan Morton


--001a113a5fb04f657b050e0f46d0--