From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp97.iad3a.emailsrvr.com (smtp97.iad3a.emailsrvr.com [173.203.187.97]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by huchra.bufferbloat.net (Postfix) with ESMTPS id 7320821F2A5 for ; Sat, 31 Jan 2015 13:51:07 -0800 (PST) Received: from smtp21.relay.iad3a.emailsrvr.com (localhost.localdomain [127.0.0.1]) by smtp21.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id B87371803A6; Sat, 31 Jan 2015 16:51:05 -0500 (EST) Received: from app44.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by smtp21.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 48D98180399; Sat, 31 Jan 2015 16:51:05 -0500 (EST) X-Sender-Id: dpreed@reed.com Received: from app44.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by 0.0.0.0:25 (trex/5.4.2); Sat, 31 Jan 2015 21:51:05 GMT Received: from reed.com (localhost.localdomain [127.0.0.1]) by app44.wa-webapps.iad3a (Postfix) with ESMTP id 33D6018004A; Sat, 31 Jan 2015 16:51:05 -0500 (EST) Received: by apps.rackspace.com (Authenticated sender: dpreed@reed.com, from: dpreed@reed.com) with HTTP; Sat, 31 Jan 2015 16:51:05 -0500 (EST) Date: Sat, 31 Jan 2015 16:51:05 -0500 (EST) From: dpreed@reed.com To: "Dave Taht" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20150131165105000000_31330" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: References: <1422537297.21689.15.camel@edumazet-glaptop2.roam.corp.google.com> <54CB5D08.2070906@broadcom.com> <1422623975.21689.77.camel@edumazet-glaptop2.roam.corp.google.com> <54CB8B69.1070807@broadcom.com> X-Auth-ID: dpreed@reed.com Message-ID: <1422741065.199624134@apps.rackspace.com> X-Mailer: webmail/11.3.11-RC Cc: Andrew McGregor , Jesper Dangaard Brouer , Matt Mathis , "cerowrt-devel@lists.bufferbloat.net" , Jonathan Morton , Tim Shepard , Avery Pennarun Subject: Re: [Cerowrt-devel] Fwd: Throughput regression with `tcp: refine TSO autosizing` X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Jan 2015 21:51:36 -0000 ------=_20150131165105000000_31330 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0AI think we need to create an Internet focused 802.11 working group that = would be to the "OS wireless designers and IEEE 802.11 standards groups" as= the WHATML group was to W3C.=0A =0AW3C was clueless about the real world a= t the point WHATML was created. And WHATML was a "revenge of the real" aga= inst W3C - advancing a wide variety of important practical innovations rath= er than attending endless standards meetings with people who were not focus= ed on solving actually important problems.=0A =0AIt took a bunch of work to= get WHATML going, and it offended W3C, who became unhelpful. But the appr= oach actually worked - we now have a Web that really uses browser-side expr= essivity and that would never have happened if W3C were left to its own dev= ices.=0A =0AThe WiFi consortium was an attempt to wrest control of pragmati= c direction from 802.11 and the proprietary-divergence folks at Qualcomm, B= roadcom, Cisco, etc. But it failed, because it became thieves on a raft, m= ore focused on picking each others' pockets than on actually addressing the= big issues.=0A =0AJim has seen this play out in the Linux community around= X. Though there are lots of interests who would benefit by moving the eng= ineering ball forward, everyone resists action because it means giving up t= he chance at dominance, and the central group is far too weak to do anythin= g beyond adjudicating the worst battles.=0A =0AWhen I say "we" I definitely= include myself (though my time is limited due to other commitments and the= need to support my family), but I would only play with people who actually= are committed to making stuff happen - which includes raising hell with th= e vendors if need be, but also effective engineering steps that can achieve= quick adoption.=0A =0ASadly, and I think it is manageable at the moment, t= here are moves out there being made to get the FCC to "protect" WiFi from "= interference". The current one was Marriott, who requested the FCC for a r= ule to make it legal to disrupt and block use of WiFi in people's rooms in = their hotels, except with their access points. This also needs some techni= cal defense. I believe any issues with WiFi performance in actual Marriott= hotels are due to bufferbloat in their hotel-wide systems, just as the iss= ues with GoGo are the same. But it's possible that queueing problems in th= eir own WiFi gear are bad as well.=0A =0AI mention this because it is relat= ed, and to the layperson, or non-radio-knowledgeable executive, indistingui= shable. It will take away the incentive to actually fix the 802.11 impleme= ntations to be better performing, making the problem seem to be a "manageme= nt" issue that can be solved by making WiFi less interoperable and less fle= xible by rules, rather than by engineering.=0A =0AHowever, solving the prob= lems of hotspot networks and hotel networks are definitely "real world" iss= ues, and quite along the same lines you mention, Dave. FQ is almost certai= nly a big deal both in WiFi and in the distribution networks behind WiFi. C= o-existence is also a big deal (RTS/CTS-like mechanisms can go a long way t= o remediate hidden-terminal disruption of the basic protocols). Roaming and= scaling need work as well.=0A =0AIt would even be a good thing to invent p= ragmatic ways to provide "low rate" subnets and "high rate" subnets that ca= n coexist, so that compatibility with ancient "b" networks need not be main= tained on all nets, at great cost - just send beacons at a high rate, so th= at the "b" NICs can't see them.... but you need pragmatic stack implementat= ions.=0A =0ABut the engineering is not the only challenge. The other challe= nge is to take the initiative and get stuff deployed. In the case of buffe= rbloat, the grade currently is a "D" for deployments, maybe a "D-". Beauti= ful technical work, but the economic/business/political side of things has = been poor. Look at how slow IETF has been to achieve anything (the perfect= is truly the enemy of the good, and Dave Clark's "rough consensus and work= ing code" has been replaced by technocratic malaise, and what appears to me= to be a class of people who love traveling the world to a floating cocktai= l party without getting anything important done).=0A =0AThe problem with co= mmunications is that you can't just ship a product with a new "feature", be= cause the innovation only works if widely adopted. Since there is no "Linu= x Desktop" (and Linus hates the idea, to a large extent) Linux can't be the= sole carrier of the idea. You pretty much need iOS and Android both to bu= y in or to provide a path for easy third-party upgrades. How do you do tha= t? Well, that's where the WHATML-type approach is necessary.=0A =0AI don't= know if this can be achieved, and there are lots of details to be worked o= ut. But I'll play.=0A =0A =0A=0A=0AOn Saturday, January 31, 2015 4:05pm, "= Dave Taht" said:=0A=0A=0A=0AI would like to have some= how assembled all the focused resources to make a go at fixing wifi, or at = least having a f2f with a bunch of people in the late march timeframe. This= message of mine to linux-wireless bounced for some reason and I am off to = log out for 10 days, so...=0Asee relevant netdev thread also for ore detail= s.=0A=0A=0A---------- Forwarded message ----------=0AFrom: Dave Taht <[ dav= e.taht@gmail.com ]( mailto:dave.taht@gmail.com )>=0ADate: Sat, Jan 31, 2015= at 12:29 PM=0ASubject: Re: Throughput regression with `tcp: refine TSO aut= osizing`=0ATo: Arend van Spriel <[ arend@broadcom.com ]( mailto:arend@broad= com.com )>=0ACc: linux-wireless <[ linux-wireless@vger.kernel.org ]( mailto= :linux-wireless@vger.kernel.org )>, Michal Kazior <[ michal.kazior@tieto.co= m ]( mailto:michal.kazior@tieto.com )>, Eyal Perry <[ eyalpe@dev.mellanox.c= o.il ]( mailto:eyalpe@dev.mellanox.co.il )>, Network Development <[ netdev@= vger.kernel.org ]( mailto:netdev@vger.kernel.org )>, Eric Dumazet <[ eric.d= umazet@gmail.com ]( mailto:eric.dumazet@gmail.com )>=0A=0A=0A=0A=0AThe wifi= industry as a whole has vastly bigger problems than achieving 1500Mbits in= a faraday cage on a single flow.=0AI encourage you to try tests in netperf= -wrapper that explicitly test for latency under load, and in particular, th= e RTT_FAIR tests against 4 or more stations on a single wifi AP. You will f= ind the results very depressing. Similarly, on your previous test series, a= latency figure would have been nice to have. I just did a talk at nznog, w= here I tested the local wifi with less than ambits of throughput, and 3 sec= onds of latency, filmed here: =0A[ https://plus.google.com/u/0/107942175615= 993706558/posts/CY8ew8MPnMt ]( https://plus.google.com/u/0/1079421756159937= 06558/posts/CY8ew8MPnMt )=0ADo wish more folk were testing in the busy real= world environments, like coffee shops, cities... really, anywhere outside = a faraday cage!=0AI am not attending netconf - I was unable to raise funds = to go, and the program committee wanted something "new",=0Ainstead of the p= reso I gave the IEEE 802.11 working group back in september. ( [ http://sna= pon.lab.bufferbloat.net/~d/ieee802.11-sept-17-2014/11-14-1265-00-0wng-More-= on-Bufferbloat.pdf ]( http://snapon.lab.bufferbloat.net/~d/ieee802.11-sept-= 17-2014/11-14-1265-00-0wng-More-on-Bufferbloat.pdf ) )=0AI was very pleased= with the results of that talk - the day after I gave it, the phrase "test = for latency" showed up in a bunch of 802.11ax (the next generation after ac= ) documents. :) Still, we are stuck with the train wreck that is 802.11ac g= lommed on top of 802.11n, glommed on top of 802.11g, in terms of queue mana= gement, terrible uses of airtime, rate control and other stuff. Aruba and M= eraki, in particular took a big interest in what I'd outlined in the preso = above (we have a half dozen less well baked ideas - that's just the easy st= uff that can be done to improve wifi). I gave a followup at meraki but I d= on't think that's online.=0AFelix (nbd) is on vacation right now, as I am I= . In fact I am going somewhere for a week totally lacking internet access.= =0APresently the plan, with what budget (none) we have and time (very littl= e) we have is to produce a pair of proof of concept implementations for per= tid queuing (see relevant commit by nbd), leveraging the new minstrel sta= ts, the new minstrel-blues stuff, and an aggregation aware codel with a cal= culated target based on the most recently active stations, and a bunch of t= he other stuff outlined above at IEEE.=0AIt is my hope that this will start= to provide accurate back pressure (or sufficient lack thereof for TSQ), to= also improve throughput while still retaining low latency. But it is a cer= tainty that we will run into more cross layer issues that will be a pita to= resolve.=0AIf we can put together a meet up around or during ELC in califo= rnia in march? =0AI am really not terribly optimistic on anything other tha= n the 2 chipsets we can hack on (ath9k, mt76). Negotiations to get qualcomm= to open up their ath10k firmware have thus far failed, nor has a ath10k-li= te got anywhere. Perhaps broadcom would be willing to open up their firmwar= e sufficiently to build in a better API?=0AA bit more below.=0A=0A On Jan 3= 0, 2015 5:59 AM, "Arend van Spriel" <[ arend@broadcom.com ]( mailto:arend@b= roadcom.com )> wrote:=0A >=0A > On 01/30/15 14:19, Eric Dumazet wrote:=0A >= >=0A >> On Fri, 2015-01-30 at 11:29 +0100, Arend van Spriel wrote:=0A >>=0A= >>> Hi Eric,=0A >>>=0A >>> Your suggestions are still based on the fact th= at you consider wireless=0A >>> networking to be similar to ethernet, but a= s Michal indicated there are=0A >>> some fundamental differences starting w= ith CSMA/CD versus CSMA/CA. Also=0A >>> the medium conditions are far from = comparable. =0AThe analogy i now use for it is that switched ethernet is ge= nerally your classic "dumbbell"=0Atopology. Wifi is more like a "taxi-stand= " topology. If you think about how people=0Aqueue up at a taxi stand (and s= ometimes agree to share a ride), the inter arrival=0Aand departure times of= a taxi stand make for a better mental model. =0AAdmittedly, I seem to spen= d a lot of time, waiting for taxies, thinking about=0Awifi.=0A>> There is n= o shielding so=0A >>> it needs to deal with interference and dynamically dr= ops the link rate=0A >>> so transmission of packets can take several millis= econds. Then with 11n=0A >>> they came up with aggregation with sends up to= 64 packets in a single=0A >>> transmit over the air at worst case 6.5 Mbps= (if I am not mistaken). The=0A >>> parameter value for tcp_limit_output_by= tes of 131072 means that it=0A >>> allows queuing for about 1ms on a 1Gbps = link, but I hope you can see=0A >>> this is not realistic for dealing with = all variances of the wireless=0A >>> medium/standard. I suggested this as t= opic for the wireless workshop in=0A >>> Otawa [1], but I can not attend th= ere. Still hope that there will be=0A >>> some discussions to get more awar= eness.=0A=0AI have sometimes hoped that TSQ could be made more a function o= f the=0Anumber of active flows exiting an interface, but eric tells me that= 's impossible.=0AThis is possibly another case where TSQ could use to be a = callback function...=0Abut frankly I care not a whit about maximizing singl= e flow tcp throughput on wifi=0Ain a faraday cage.=0A=0A >>=0A >> Ever hear= d about bufferbloat ?=0A >=0A >=0A > Sure. I am trying to get awareness abo= ut that in our wireless driver/firmware development teams. So bear with me.= =0A >=0A >=0A >> Have you read my suggestions and tried them ?=0A >>=0A >> = You can adjust the limit per flow to pretty much you want. If you need=0A >= > 64 packets, just do the math. If in 2018 you need 128 packets, do the=0A = >> math again.=0A >>=0A >> I am very well aware that wireless wants aggrega= tion, thank you.=0A=0AI note that a lot of people testing this are getting = it backwards. Usually it is the AP that is sending lots and lots of big pac= kets, where the return path is predominately acks from the station. =0AI am= not a huge fan of stretch acks, but certainly a little bit of thinning doe= sn't bother me on the return path there.=0AGoing the other way, particularl= y in a wifi world that insists on treating every packet as sacred (which I = don't agree with at all), thinning acks can help, but single stream through= put is of interest only on benchmarks, FQing as much as possible all the fl= ows destined the station in each aggregate masks loss and reduces the need = to protect everything so much.=0A >=0A > Sorry if I offended you. I was jus= t giving these as example combined with effective rate usable on the medium= to say that the bandwidth is more dynamic in wireless and as such need dyn= amic change of queue depth. Now this can be done by making the fraction siz= e as used in your suggestion adaptive to these conditions.=0A=0AWell... see= above. Maybe this technique will do more of the right thing, but... go tes= t.=0A=0A >=0A >> 131072 bytes of queue on 40Gbit is not 1ms, but 26 usec of= queueing, and=0A >> we get line rate nevertheless.=0A >=0A >=0A > I was sa= ying it was about 1ms on *1Gbit* as the wireless TCP rates are moving into = that direction in 11ac.=0A >=0A >=0A >> We need this level of shallow queue= s (BQL, TSQ), to get very precise rtt=0A >> estimations so that TCP has goo= d entropy for its pacing, even in the 50=0A >> usec rtt ranges.=0A >>=0A >>= If we allowed 1ms of queueing, then a 40Gbit flow would queue 5 MBytes.=0A= >>=0A >> This was terrible, because it increased cwnd and all sender queue= s to=0A >> insane levels.=0A >=0A >=0A > Indeed and that is what we would l= ike to address in our wireless drivers. I will setup some experiments using= the fraction sizing and post my findings. Again sorry if I offended you.= =0A=0A=0AYou really, really, really need to test at rates below 50mbit and = with other stations, also while doing this. It's not going to be a linear c= urve.=0A =0A>=0A > Regards,=0A > Arend=0A >=0A > --=0A > To unsubscribe fro= m this list: send the line "unsubscribe netdev" in=0A > the body of a messa= ge to [ majordomo@vger.kernel.org ]( mailto:majordomo@vger.kernel.org )=0A = > More majordomo info at [ http://vger.kernel.org/majordomo-info.html ]( h= ttp://vger.kernel.org/majordomo-info.html )=0A=0A-- =0A=0ADave T=C3=A4ht=0A= =0Athttp://[ www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks ]( http= ://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks ) ------=_20150131165105000000_31330 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

I think we need to creat= e an Internet focused 802.11 working group that would be to the "OS wireles= s designers and IEEE 802.11 standards groups" as the WHATML group was to W3= C.

=0A

 

=0A

W3C was clueless = about the real world at the point WHATML was created.  And WHATML was = a "revenge of the real" against W3C - advancing a wide variety of important= practical innovations rather than attending endless standards meetings wit= h people who were not focused on solving actually important problems.

= =0A

 

=0A

It took a bunch of work= to get WHATML going, and it offended W3C, who became unhelpful.  But = the approach actually worked - we now have a Web that really uses browser-s= ide expressivity and that would never have happened if W3C were left to its= own devices.

=0A

 

=0A

The Wi= Fi consortium was an attempt to wrest control of pragmatic direction from 8= 02.11 and the proprietary-divergence folks at Qualcomm, Broadcom, Cisco, et= c.  But it failed, because it became thieves on a raft, more focused o= n picking each others' pockets than on actually addressing the big issues.<= /p>=0A

 

=0A

Jim has seen this pl= ay out in the Linux community around X.  Though there are lots of inte= rests who would benefit by moving the engineering ball forward, everyone re= sists action because it means giving up the chance at dominance, and the ce= ntral group is far too weak to do anything beyond adjudicating the worst ba= ttles.

=0A

 

=0A

When I say "w= e" I definitely include myself (though my time is limited due to other comm= itments and the need to support my family), but I would only play with peop= le who actually are committed to making stuff happen - which includes raisi= ng hell with the vendors if need be, but also effective engineering steps t= hat can achieve quick adoption.

=0A

 

=0A

Sadly, and I think it is manageable at the moment, there are mo= ves out there being made to get the FCC to "protect" WiFi from "interferenc= e".  The current one was Marriott, who requested the FCC for a rule to= make it legal to disrupt and block use of WiFi in people's rooms in t= heir hotels, except with their access points.  This also needs some te= chnical defense.  I believe any issues with WiFi performance in actual= Marriott hotels are due to bufferbloat in their hotel-wide systems, just a= s the issues with GoGo are the same.  But it's possible that queueing = problems in their own WiFi gear are bad as well.

=0A

&n= bsp;

=0A

I mention this because it is related, and to t= he layperson, or non-radio-knowledgeable executive, indistinguishable. &nbs= p;It will take away the incentive to actually fix the 802.11 implementation= s to be better performing, making the problem seem to be a "management" iss= ue that can be solved by making WiFi less interoperable and less flexible b= y rules, rather than by engineering.

=0A

 

=0A<= p style=3D"margin:0;padding:0;font-family: tahoma; font-size: 10pt; word-wr= ap: break-word;">However, solving the problems of hotspot networks and hote= l networks are definitely "real world" issues, and quite along the same lin= es you mention, Dave.  FQ is almost certainly a big deal both in WiFi = and in the distribution networks behind WiFi. Co-existence is also a big de= al (RTS/CTS-like mechanisms can go a long way to remediate hidden-terminal = disruption of the basic protocols). Roaming and scaling need work as well.<= /p>=0A

 

=0A

It would even be a g= ood thing to invent pragmatic ways to provide "low rate" subnets and "high = rate" subnets that can coexist, so that compatibility with ancient "b" netw= orks need not be maintained on all nets, at great cost - just send beacons = at a high rate, so that the "b" NICs can't see them.... but you need p= ragmatic stack implementations.

=0A

 

=0A

But the engineering is not the only challenge. The other challe= nge is to take the initiative and get stuff deployed.  In the case of = bufferbloat, the grade currently is a "D" for deployments, maybe a "D-". &n= bsp;Beautiful technical work, but the economic/business/political side of t= hings has been poor.  Look at how slow IETF has been to achieve anythi= ng (the perfect is truly the enemy of the good, and Dave Clark's "rough con= sensus and working code" has been replaced by technocratic malaise, and wha= t appears to me to be a class of people who love traveling the world to a f= loating cocktail party without getting anything important done).

=0A

 

=0A

The problem with communication= s is that you can't just ship a product with a new "feature", because the i= nnovation only works if widely adopted.  Since there is no "Linux Desk= top" (and Linus hates the idea, to a large extent) Linux can't be the sole = carrier of the idea.  You pretty much need iOS and Android both to buy= in or to provide a path for easy third-party upgrades.  How do you do= that?  Well, that's where the WHATML-type approach is necessary.

= =0A

 

=0A

I don't know if this ca= n be achieved, and there are lots of details to be worked out.  But I'= ll play.

=0A

 

=0A

 

= =0A=0A



On Saturday, January 31, 2015 4:05pm, "Dave Taht" = <dave.taht@gmail.com> said:

=0A
=0A
I would like to have somehow assembled all th= e focused resources to make a go at fixing wifi, or at least having a f2f w= ith a bunch of people in the late march timeframe. This message of mine to = linux-wireless bounced for some reason and I am off to log out for 10 days,= so...=0A
see relevant netdev thread also for ore details.
=0A
=0A
---------- Forwarded message ---------= -
From: Dave Taht <dave.taht@gmail.com>
Date: Sat, Jan 31, 2015 at 12:29 PM
Subject: Re: Thr= oughput regression with `tcp: refine TSO autosizing`
To: Arend van Spr= iel <arend@broadcom.com>Cc: linux-wireless <linux-wireless@vger.kernel.org>, Michal Kazior <michal.kazior@tieto.com>, Eyal Perry <= eyalpe@dev.mellanox.co.il&= gt;, Network Development <netd= ev@vger.kernel.org>, Eric Dumazet <eric.dumazet@gmail.com>


=0A
=0A

The wifi industry as a whole has vast= ly bigger problems than achieving 1500Mbits in a faraday cage on a single f= low.

=0A

I encourage you to try tests in netperf-wrappe= r that explicitly test for latency under load, and in particular, the RTT_F= AIR tests against 4 or more stations on a single wifi AP. You will find the= results very depressing. Similarly, on your previous test series, a latenc= y figure would have been nice to have. I just did a talk at nznog, where I = tested the local wifi with less than ambits of throughput, and 3 seconds of= latency, filmed here: 

=0A

https://plus.google.com/u/0/107942175615993706558/posts/CY8ew8MPnMt=0A

Do wish more folk were testing in the busy real world= environments, like coffee shops, cities... really, anywhere outside a fara= day cage!

=0A

I am not attending netconf - I was unable= to raise funds to go, and the program committee wanted something "new",=0A

instead of the preso I gave the IEEE 802.11 working g= roup back in september. ( http://snapon.lab.bufferbloat.net/~d/ieee802.11-sept-17-2014= /11-14-1265-00-0wng-More-on-Bufferbloat.pdf )

=0A

I= was very pleased with the results of that talk - the day after I gave it, = the phrase "test for latency" showed up in a bunch of 802.11ax (the next ge= neration after ac) documents. :) Still, we are stuck with the train wreck t= hat is 802.11ac glommed on top of 802.11n, glommed on top of 802.11g, in te= rms of queue management, terrible uses of airtime, rate control and other s= tuff. Aruba and Meraki, in particular took a big interest in what I'd outli= ned in the preso above (we have a half dozen less well baked ideas - that's= just the easy stuff that can be done to improve wifi).  I gave a foll= owup at meraki but I don't think that's online.

=0A

Fel= ix (nbd) is on vacation right now, as I am I. In fact I am going somewhere = for a week totally lacking internet access.

=0A

Present= ly the plan, with what budget (none) we have and time (very little) we have= is to produce a pair of proof of concept implementations for per tid queui= ng (see relevant commit by nbd),  leveraging the new minstrel stats, t= he new minstrel-blues stuff, and an aggregation aware codel with a calculat= ed target based on the most recently active stations, and a bunch of the ot= her stuff outlined above at IEEE.

=0A

It is my hope tha= t this will start to provide accurate back pressure (or sufficient lack the= reof for TSQ), to also improve throughput while still retaining low latency= . But it is a certainty that we will run into more cross layer issues that = will be a pita to resolve.

=0A

If we can put together a= meet up around or during ELC in california in march? 

=0A

I am really not terribly optimistic on anything other than the 2 = chipsets we can hack on (ath9k, mt76). Negotiations to get qualcomm to open= up their ath10k firmware have thus far failed, nor has a ath10k-lite got a= nywhere. Perhaps broadcom would be willing to open up their firmware suffic= iently to build in a better API?

=0A

A bit = more below.

=0A


On Jan 30, 2015 5:59= AM, "Arend van Spriel" <arend@broadcom.com> wrote:
>
> On 01/30/1= 5 14:19, Eric Dumazet wrote:
>>
>> On Fri, 2015-01-= 30 at 11:29 +0100, Arend van Spriel wrote:
>>
>>>= ; Hi Eric,
>>>
>>> Your suggestions are still= based on the fact that you consider wireless
>>> networking= to be similar to ethernet, but as Michal indicated there are
>>= ;> some fundamental differences starting with CSMA/CD versus CSMA/CA. Al= so
>>> the medium conditions are far from comparable. <= /p>=0A

The analogy i now use for it is that switched ether= net is generally your classic "dumbbell"

=0A

topology. = Wifi is more like a "taxi-stand" topology. If you think about how people=0A

queue up at a taxi stand (and sometimes agree to shar= e a ride), the inter arrival

=0A

and departure times of= a taxi stand make for a better mental model. 

=0A

Admittedly, I seem to spend a lot of time, waiting for taxies, thinking ab= out

=0A

wifi.

=0A

>> There is no shielding so
>>> it needs t= o deal with interference and dynamically drops the link rate
>>= > so transmission of packets can take several milliseconds. Then with 11= n
>>> they came up with aggregation with sends up to 64 pack= ets in a single
>>> transmit over the air at worst case 6.5 = Mbps (if I am not mistaken). The
>>> parameter value for tcp= _limit_output_bytes of 131072 means that it
>>> allows queui= ng for about 1ms on a 1Gbps link, but I hope you can see
>>>= this is not realistic for dealing with all variances of the wireless
= >>> medium/standard. I suggested this as topic for the wireless w= orkshop in
>>> Otawa [1], but I can not attend there. Still = hope that there will be
>>> some discussions to get more awa= reness.

I have sometimes hoped that TSQ could be made mor= e a function of the

=0A

number of active fl= ows exiting an interface, but eric tells me that's impossible.

=0A

This is possibly another case where TSQ could use t= o be a callback function...

=0A

but frankly= I care not a whit about maximizing single flow tcp throughput on wifi

= =0A

in a faraday cage.

=0A


>>
>> Ever heard abo= ut bufferbloat ?
>
>
> Sure. I am trying to get = awareness about that in our wireless driver/firmware development teams. So = bear with me.
>
>
>> Have you read my suggest= ions and tried them ?
>>
>> You can adjust the limi= t per flow to pretty much you want. If you need
>> 64 packets, = just do the math. If in 2018 you need 128 packets, do the
>> ma= th again.
>>
>> I am very well aware that wireless = wants aggregation, thank you.

I note that a lot of people= testing this are getting it backwards. Usually it is the AP that is sendin= g lots and lots of big packets, where the return path is predominately acks= from the station. 

=0A

I am not a hug= e fan of stretch acks, but certainly a little bit of thinning doesn't bothe= r me on the return path there.

=0A

Going the other way,= particularly in a wifi world that insists on treating every packet as sacr= ed (which I don't agree with at all), thinning acks can help, but single st= ream throughput is of interest only on benchmarks, FQing as much as possibl= e all the flows destined the station in each aggregate masks loss and reduc= es the need to protect everything so much.

=0A

>
> Sorry if I offended you. I was just g= iving these as example combined with effective rate usable on the medium to= say that the bandwidth is more dynamic in wireless and as such need dynami= c change of queue depth. Now this can be done by making the fraction size a= s used in your suggestion adaptive to these conditions.

W= ell... see above. Maybe this technique will do more of the right thing, but= ... go test.

=0A


>
>>= 131072 bytes of queue on 40Gbit is not 1ms, but 26 usec of queueing, and >> we get line rate nevertheless.
>
>
&g= t; I was saying it was about 1ms on *1Gbit* as the wireless TCP rates are m= oving into that direction in 11ac.
>
>
>> We = need this level of shallow queues (BQL, TSQ), to get very precise rtt
= >> estimations so that TCP has good entropy for its pacing, even in = the 50
>> usec rtt ranges.
>>
>> If we = allowed 1ms of queueing, then a 40Gbit flow would queue 5 MBytes.
>= ;>
>> This was terrible, because it increased cwnd and all s= ender queues to
>> insane levels.
>
>
&= gt; Indeed and that is what we would like to address in our wireless driver= s. I will setup some experiments using the fraction sizing and post my find= ings. Again sorry if I offended you.

=0A

Yo= u really, really, really need to test at rates below 50mbit and with other = stations, also while doing this. It's not going to be a linear curve.

= =0A

 

=0A

>
> Regards,
> Arend
>
> --
&= gt; To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majordomo@vger.kernel.org
> More majordo= mo info at  http://vger.kernel.org/majordomo-info.html

=0A
= =0A
=0A

--
=0A
Dave T= =C3=A4ht

thttp://www.bufferbloat.net/projects/b= loat/wiki/Upcoming_Talks
=0A
=0A
=0A ------=_20150131165105000000_31330--