[Make-wifi-fast] On the 802.11 performance anomaly and an airtime fairness scheduler to fix it

[Make-wifi-fast] On the 802.11 performance anomaly and an airtime fairness scheduler to fix it

[Toke Høiland-Jørgensen]

I've previously posted my airtime fairness scheduler patch here. When I
did I promised a more thorough performance evaluation of it. This took a
bit longer to get done than I had anticipated, but I have now put up a
blog post with some pretty graphs.

The post also explains the background of the 802.11 performance anomaly
to those not familiar with it; feel free to skip to the results if you
are already familiar with it.

Link: https://blog.tohojo.dk/2016/06/fixing-the-wifi-performance-anomaly-on-ath9k.html

-Toke

Re: [Make-wifi-fast] On the 802.11 performance anomaly and an airtime fairness scheduler to fix it

[Sebastian Moeller]

Hi Toke,

On June 30, 2016 11:06:40 PM GMT+02:00, "Toke Høiland-Jørgensen" <toke@toke.dk> wrote:
>I've previously posted my airtime fairness scheduler patch here. When I
>did I promised a more thorough performance evaluation of it. This took
>a
>bit longer to get done than I had anticipated, but I have now put up a
>blog post with some pretty graphs.
>
>The post also explains the background of the 802.11 performance anomaly
>to those not familiar with it; feel free to skip to the results if you
>are already familiar with it.
>
>Link:
>https://blog.tohojo.dk/2016/06/fixing-the-wifi-performance-anomaly-on-ath9k.html
>
>-Toke


Great work, I note that in figure 5, the downstream gdrom stations to AP seems to be almost wiped out. Was this using airtime Fairness for tx only?

Best Regards 
        Sebastian 



>_______________________________________________
>Make-wifi-fast mailing list
>Make-wifi-fast@lists.bufferbloat.net
>https://lists.bufferbloat.net/listinfo/make-wifi-fast

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [Make-wifi-fast] On the 802.11 performance anomaly and an airtime fairness scheduler to fix it

[Dave Taht]

You are a master of understatement...

couldn't you sneak in there a mention of the 30x reduction in latency,
and the 5fold improvement in throughput, on these tests with the new
patches... in addition to the note about the 4th station's performance
needing a bit more research?



On Thu, Jun 30, 2016 at 10:06 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> I've previously posted my airtime fairness scheduler patch here. When I
> did I promised a more thorough performance evaluation of it. This took a
> bit longer to get done than I had anticipated, but I have now put up a
> blog post with some pretty graphs.
>
> The post also explains the background of the 802.11 performance anomaly
> to those not familiar with it; feel free to skip to the results if you
> are already familiar with it.
>
> Link: https://blog.tohojo.dk/2016/06/fixing-the-wifi-performance-anomaly-on-ath9k.html
>
> -Toke
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Make-wifi-fast] On the 802.11 performance anomaly and an airtime fairness scheduler to fix it

[Dave Taht]

Comments:

(my overall take on things is these patches should all go upstream
yesterday, we get some .debs built for others to use, and iterate from
there)

(some of the below has already been discussed but I've lost track on
what was discussed privately or not)

A) I would put down the issues you had in fig 3, 4, to the enormous
client latencies still apparent in fig 5.

While proving that these patches help an AP's performance enormously,
and are thus valuable there, clients should show an enormous benefit
as well. I am dying for a "full enchalada" test, as "The stations are
running a stock Ubuntu 16.04, while the access point is running a
4.6-based kernel with the appropriate patches applied"... but maybe
not with 30 iterations? 5? :)

the benefit on the client is mostly covered by the switch to the soft
queues and fq_codel at the mac80211 layer.

B) Similarly, in fig 6, I would put down to the clients using slightly
more TXOPs to do what they are doing, due to the existing (borken)
structure of the ath9k driver, primarily.  Or - what was the volume of
the ping traffic? isochronous (voip-like) testing is better...

Clamping the txop size further under contention would be
"interesting". A possible synergetic enhancement to the fast/slow
queue concept is to more aggressively cut the max txop size for all
"slow queue" stations, when a new station appears. A simpler
enhancement to test is to cut the max txop size as a function of the
number of stations with backlogs to something smaller than the max,
say 2ms. (my mental vision was max(2ms,5.6/stations))

C) What was the performance loss, with airtime fairness achieved, for
the slow station? 3x?

Was ecn enabled?

D) Fig 2 is in 10^8, it's easier to read if not in that notation

E) The download/upload asymmetry is a bit bothersome, there are
multiple potential root causes. It's not clear which way the upload
was?

1) Usually the AP has much better antennas, but I don't think this is
the case here
2) rx accounting not correct?
3) codel breakage? (what was the target and quantum?). We can drop
acks all day long but dropping tcp data too early is problematic. The
codel target is currently set to 20ms in the ath10k patchset, I think.
Hope is to make that dynamic based on the aftereffects of fiddling on
B.

One of the experiments I did was to clamp the max txop size as
distributed in the beacon to "94" in hostapd. This led to much better
symmetry on some tests of mine, and cost throughput. Again this could
one day be a dynamically set option based on the workload, if it
helps.

I was told recently that the beacon interval is set to 3 or more in
some APs nowadays, notably apple's, but that's probably not relevant.


Longer term tests:

F) Differences in ad-hoc mode?

G) Hardware mq tests

H) Minstrel-blues integration

Random tests

Reducing retries, reducing the size of the retry chain, etc.


On Thu, Jun 30, 2016 at 10:06 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> I've previously posted my airtime fairness scheduler patch here. When I
> did I promised a more thorough performance evaluation of it. This took a
> bit longer to get done than I had anticipated, but I have now put up a
> blog post with some pretty graphs.
>
> The post also explains the background of the 802.11 performance anomaly
> to those not familiar with it; feel free to skip to the results if you
> are already familiar with it.
>
> Link: https://blog.tohojo.dk/2016/06/fixing-the-wifi-performance-anomaly-on-ath9k.html
>
> -Toke
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

[Make-wifi-fast] On the 802.11 performance anomaly and an airtime fairness scheduler to fix it

[Luca Muscariello]

[-- Attachment #1: Type: text/plain, Size: 1557 bytes --]

Toke,

great piece of work.
UDP tests show that the scheduler is working quite well.
TCP tests: no news that ACK streams are a problem.
This is another well known issue  in wifi.

Every data packet is ACKed already at L2 and L4
sends an ACK that is non contention free.

There is an NSDI paper in 2014 that proposes to use the L2 ACK to carry the
L4 ACK.
It works well but purists might hate that approach.

BTW the gains you show are huge.
If you add multiple stations uniformly distributed around the AP you'll see
tremendous gains.

The delay gain is also huge and poorly explored so far.

On Thu, Jun 30, 2016 at 11:06 PM, Toke Høiland-Jørgensen <toke@toke.dk
<javascript:_e(%7B%7D,'cvml','toke@toke.dk');>> wrote:

> I've previously posted my airtime fairness scheduler patch here. When I
> did I promised a more thorough performance evaluation of it. This took a
> bit longer to get done than I had anticipated, but I have now put up a
> blog post with some pretty graphs.
>
> The post also explains the background of the 802.11 performance anomaly
> to those not familiar with it; feel free to skip to the results if you
> are already familiar with it.
>
> Link:
> https://blog.tohojo.dk/2016/06/fixing-the-wifi-performance-anomaly-on-ath9k.html
>
> -Toke
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> <javascript:_e(%7B%7D,'cvml','Make-wifi-fast@lists.bufferbloat.net');>
> https://lists.bufferbloat.net/listinfo/make-wifi-fast
>

[-- Attachment #2: Type: text/html, Size: 2337 bytes --]

Re: [Make-wifi-fast] On the 802.11 performance anomaly and an airtime fairness scheduler to fix it

[David Lang]

[-- Attachment #1: Type: TEXT/Plain, Size: 651 bytes --]

On Sun, 3 Jul 2016, Luca Muscariello wrote:

> Toke,
>
> great piece of work.
> UDP tests show that the scheduler is working quite well.
> TCP tests: no news that ACK streams are a problem.
> This is another well known issue  in wifi.
>
> Every data packet is ACKed already at L2 and L4
> sends an ACK that is non contention free.
>
> There is an NSDI paper in 2014 that proposes to use the L2 ACK to carry the
> L4 ACK.
> It works well but purists might hate that approach.

do they delay the L2 Ack until the L4 ack comes back? If so, how does that work 
on long-latency connections where it takes a long time for the L4 ack to show 
up?

David Lang

[-- Attachment #2: Type: TEXT/PLAIN, Size: 167 bytes --]

_______________________________________________
Make-wifi-fast mailing list
Make-wifi-fast@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/make-wifi-fast

Re: [Make-wifi-fast] On the 802.11 performance anomaly and an airtime fairness scheduler to fix it

[Jonathan Morton]

> On 3 Jul, 2016, at 10:06, David Lang <david@lang.hm> wrote:
> 
> do they delay the L2 Ack until the L4 ack comes back? If so, how does that work on long-latency connections where it takes a long time for the L4 ack to show up?

I’m pretty sure it’s only meant to work when the TCP endpoint is local to the receiving station, assuring low turnaround latency.  This is the typical case, so it’s a win.

With that said, there’s no fundamental reason why the piggybacked L4 ack need be the one corresponding to the L2 ack.  It just needs to be a small packet that won’t unduly extend the airtime occupied by the ack anyway, and which won’t mind being lost if the L2 ack gets squashed.  A scheme allowing a certain amount of slop in this way would accommodate remote TCP endpoints as well as local ones.

 - Jonathan Morton

Re: [Make-wifi-fast] On the 802.11 performance anomaly and an airtime fairness scheduler to fix it

[Luca Muscariello]

[-- Attachment #1: Type: text/plain, Size: 1121 bytes --]

https://www.usenix.org/system/files/conference/atc14/atc14-paper-salameh.pdf

The paper I mentioned.
Usenix ATC'14 not NSDI.




On Sunday, 3 July 2016, Jonathan Morton <chromatix99@gmail.com> wrote:

>
> > On 3 Jul, 2016, at 10:06, David Lang <david@lang.hm <javascript:;>>
> wrote:
> >
> > do they delay the L2 Ack until the L4 ack comes back? If so, how does
> that work on long-latency connections where it takes a long time for the L4
> ack to show up?
>
> I’m pretty sure it’s only meant to work when the TCP endpoint is local to
> the receiving station, assuring low turnaround latency.  This is the
> typical case, so it’s a win.
>
> With that said, there’s no fundamental reason why the piggybacked L4 ack
> need be the one corresponding to the L2 ack.  It just needs to be a small
> packet that won’t unduly extend the airtime occupied by the ack anyway, and
> which won’t mind being lost if the L2 ack gets squashed.  A scheme allowing
> a certain amount of slop in this way would accommodate remote TCP endpoints
> as well as local ones.
>
>  - Jonathan Morton
>
>

[-- Attachment #2: Type: text/html, Size: 1551 bytes --]

Re: [Make-wifi-fast] On the 802.11 performance anomaly and an airtime fairness scheduler to fix it

[David Lang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1495 bytes --]

On Sun, 3 Jul 2016, Jonathan Morton wrote:

>> On 3 Jul, 2016, at 10:06, David Lang <david@lang.hm> wrote:
>>
>> do they delay the L2 Ack until the L4 ack comes back? If so, how does that 
>> work on long-latency connections where it takes a long time for the L4 ack to 
>> show up?
>
> I’m pretty sure it’s only meant to work when the TCP endpoint is local to the 
> receiving station, assuring low turnaround latency.  This is the typical case, 
> so it’s a win.

how is it the typical case that a wifi connection it to a local system not to 
something over the Internet? Even in business settings, Internet bound traffic 
can be the majority (cloud based e-mail, google docs, etc)

> With that said, there’s no fundamental reason why the piggybacked L4 ack need 
> be the one corresponding to the L2 ack.  It just needs to be a small packet 
> that won’t unduly extend the airtime occupied by the ack anyway, and which 
> won’t mind being lost if the L2 ack gets squashed.  A scheme allowing a 
> certain amount of slop in this way would accommodate remote TCP endpoints as 
> well as local ones.

Given the normal overhead of any txop, being able to piggy back a small amount 
of real data at high speed with the L2 ack would be a significant win in many 
cases.

For the common case of downloading from the Internet, the endpoint system should 
be able to return a real L4 ack fast enough to piggy back it on the L2 ack.

If that what is meant by the 'typical case'?

David Lang

Re: [Make-wifi-fast] On the 802.11 performance anomaly and an airtime fairness scheduler to fix it

[Jonathan Morton]

> On 3 Jul, 2016, at 11:03, David Lang <david@lang.hm> wrote:
> 
>>> do they delay the L2 Ack until the L4 ack comes back? If so, how does that work on long-latency connections where it takes a long time for the L4 ack to show up?
>> 
>> I’m pretty sure it’s only meant to work when the TCP endpoint is local to the receiving station, assuring low turnaround latency.  This is the typical case, so it’s a win.
> 
> how is it the typical case that a wifi connection it to a local system not to something over the Internet? Even in business settings, Internet bound traffic can be the majority (cloud based e-mail, google docs, etc)
> 
>> With that said, there’s no fundamental reason why the piggybacked L4 ack need be the one corresponding to the L2 ack.  It just needs to be a small packet that won’t unduly extend the airtime occupied by the ack anyway, and which won’t mind being lost if the L2 ack gets squashed.  A scheme allowing a certain amount of slop in this way would accommodate remote TCP endpoints as well as local ones.
> 
> Given the normal overhead of any txop, being able to piggy back a small amount of real data at high speed with the L2 ack would be a significant win in many cases.
> 
> For the common case of downloading from the Internet, the endpoint system should be able to return a real L4 ack fast enough to piggy back it on the L2 ack.
> 
> If that what is meant by the 'typical case'?

Yes.  I meant that, for example, a laptop performing a download would both be the receiving station and the receiving TCP endpoint, which is where TCP acks are generated.

The fact that the *other* TCP endpoint is typically remote doesn’t matter, because that is mostly sending data, not acks.

 - Jonathan Morton