[Make-wifi-fast] Thoughts on tackling airtime fairness

[Make-wifi-fast] Thoughts on tackling airtime fairness

[Toke Høiland-Jørgensen]

So one of the components of tackling WiFi performance in Linux is (I
believe) achieving airtime fairness (which turns out to be equivalent to
proportional throughput fairness [1]). This is my attempt to summarise
some thoughts on how to achieve this. I am by no means claiming these
thoughts are necessarily (all) original, a lot of it has been floating
around for a while; I'm simply trying to separate out airtime fairness
as a separate issue that can be tackled orthogonally from the other
parts of the stack (and that hopefully will be complementary). Feedback
welcome, obviously :)


In the literature, there are basically three approaches to achieving
airtime fairness: Varying the MAC protocol timing parameters (e.g. TXOP
duration or CWmin; see e.g. [2] for an over-engineered "optimal" way of
doing this), varying the size of the aggregate (see [3]) and scheduling
at the access point (see [4]). The two former approaches are distributed
in nature (i.e. each node adjusts its behaviour in a way that will
result in aggregate airtime fairness for the whole network) while the
latter is access point-centric.

Improving the access point behaviour is definitely the straight-forward
way, and there are many obvious improvements that can result from that,
as we've already seen in e.g. Michal's patch set. So looking at airtime
fairness from the point of view of the access point makes sense.

Basically, on a high level I think there are two things that we need to
achieve good airtime fairness: Building the right aggregates (like in
[3]), and scheduling (like in [4]) to compensate for when we get it
wrong or when we can't control the aggregation because the firmware
handles it.

Now, of course the academic publications tend to be some way from
something that can be implemented in practice; I simply refer to them
here as concepts worth pursuing, and to shape thought. And, well,
because I happen to be in academia myself ;)


Getting closer to the practical level, what I plan to go poke at is:

(1) The ath9k aggregation building logic, to try to achieve
  constant-time aggregate size (for a suitable time quantum). Not sure
  how much actually needs to change here; the standard already mandates
  that an aggregate cannot be longer than 4ms, so the code already takes
  time into account. And not sure how this interacts with rate
  selection.

(2) Adding a time-based scheduler to enforce airtime fairness. Building
  this on FQ-CoDel seems like the sensible thing to do, to get the same
  latency bonus for stations that only transmit occasionally. It would
  be nice to be able to stick this in the mac80211 layer to make it
  shared between drivers, but not sure the required timing information
  is available at that layer.


A non-exhaustive list of issues that need to be resolved one way or
another:

- What to account to each station's airtime? My thought would be
  retransmissions, but not time spent waiting for other stations.

- What about management, control and multicast frames? I'd say just
  ignore it (at least for now).

- How to measure the results? Can we dump the information from the
  driver (and is it accurate)? Do we need to parse aircaps? Something
  else?


Hope this all makes sense. Comments very welcome! :)

-Toke


[1] Li Bin Jiang and Soung Chang Liew, "Proportional fairness in
wireless LANs and ad hoc networks"
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1424745&isnumber=30730

[2] P. Lin, W. I. Chou and T. Lin, "Achieving airtime fairness of
delay-sensitive applications in multirate IEEE 802.11 wireless LANs,"
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6011749&isnumber=6011722

[3] Minho Kim, Eun-Chan Park, and Chong-Ho Choi, “Adaptive Two-Level
Frame Aggregation for Fairness and Efficiency in IEEE 802.11n Wireless LANs,”
http://www.hindawi.com/journals/misy/2015/548109/

[4] Li Bin Jiang and Soung Chang Liew, "Proportional fairness in
wireless LANs and ad hoc networks,"
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1424745&isnumber=30730

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Luca Muscariello]

[-- Attachment #1: Type: text/plain, Size: 5613 bytes --]

Toke,

I'd suggest to add this in you list of references:

Godfrey Tan and John Guttag, Time-based fairness improves performance in
multi-rate WLANs. In Proc of USENIX 2004
https://www.usenix.org/legacy/publications/library/proceedings/usenix04/tech/general/full_papers/tan/tan.pdf

It's worth having a look to the APware project for freeBSD and Godfrey Tan
PhD thesis at MIT.

http://nms.csail.mit.edu/projects/apware/

this work predates 802.11n and aggregation.

Ten years ago I played with SFQ and madwifi for 802.11g
to get max-min time fairness (and so proportional rate fairness) with
excellent results.  The hacking I made was based on
using time quanta instead of bytes. Which required me to
get the current PHY rates (AP to all STAtions) and dynamically
compute/update SFQ time quanta.

It's surprising that 802.11 standard never considered time fairness in the
EDCF. A reason might be the time fairness might be enforced using the PCF.

IMO, It's a very good topic.
Thanks for bringing this up.

Luca


On Wed, May 11, 2016 at 2:19 PM, Toke Høiland-Jørgensen <toke@toke.dk>
wrote:

> So one of the components of tackling WiFi performance in Linux is (I
> believe) achieving airtime fairness (which turns out to be equivalent to
> proportional throughput fairness [1]). This is my attempt to summarise
> some thoughts on how to achieve this. I am by no means claiming these
> thoughts are necessarily (all) original, a lot of it has been floating
> around for a while; I'm simply trying to separate out airtime fairness
> as a separate issue that can be tackled orthogonally from the other
> parts of the stack (and that hopefully will be complementary). Feedback
> welcome, obviously :)
>
>
> In the literature, there are basically three approaches to achieving
> airtime fairness: Varying the MAC protocol timing parameters (e.g. TXOP
> duration or CWmin; see e.g. [2] for an over-engineered "optimal" way of
> doing this), varying the size of the aggregate (see [3]) and scheduling
> at the access point (see [4]). The two former approaches are distributed
> in nature (i.e. each node adjusts its behaviour in a way that will
> result in aggregate airtime fairness for the whole network) while the
> latter is access point-centric.
>
> Improving the access point behaviour is definitely the straight-forward
> way, and there are many obvious improvements that can result from that,
> as we've already seen in e.g. Michal's patch set. So looking at airtime
> fairness from the point of view of the access point makes sense.
>
> Basically, on a high level I think there are two things that we need to
> achieve good airtime fairness: Building the right aggregates (like in
> [3]), and scheduling (like in [4]) to compensate for when we get it
> wrong or when we can't control the aggregation because the firmware
> handles it.
>
> Now, of course the academic publications tend to be some way from
> something that can be implemented in practice; I simply refer to them
> here as concepts worth pursuing, and to shape thought. And, well,
> because I happen to be in academia myself ;)
>
>
> Getting closer to the practical level, what I plan to go poke at is:
>
> (1) The ath9k aggregation building logic, to try to achieve
>   constant-time aggregate size (for a suitable time quantum). Not sure
>   how much actually needs to change here; the standard already mandates
>   that an aggregate cannot be longer than 4ms, so the code already takes
>   time into account. And not sure how this interacts with rate
>   selection.
>
> (2) Adding a time-based scheduler to enforce airtime fairness. Building
>   this on FQ-CoDel seems like the sensible thing to do, to get the same
>   latency bonus for stations that only transmit occasionally. It would
>   be nice to be able to stick this in the mac80211 layer to make it
>   shared between drivers, but not sure the required timing information
>   is available at that layer.
>
>
> A non-exhaustive list of issues that need to be resolved one way or
> another:
>
> - What to account to each station's airtime? My thought would be
>   retransmissions, but not time spent waiting for other stations.
>
> - What about management, control and multicast frames? I'd say just
>   ignore it (at least for now).
>
> - How to measure the results? Can we dump the information from the
>   driver (and is it accurate)? Do we need to parse aircaps? Something
>   else?
>
>
> Hope this all makes sense. Comments very welcome! :)
>
> -Toke
>
>
> [1] Li Bin Jiang and Soung Chang Liew, "Proportional fairness in
> wireless LANs and ad hoc networks"
>
> http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1424745&isnumber=30730
>
> [2] P. Lin, W. I. Chou and T. Lin, "Achieving airtime fairness of
> delay-sensitive applications in multirate IEEE 802.11 wireless LANs,"
>
> http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6011749&isnumber=6011722
>
> [3] Minho Kim, Eun-Chan Park, and Chong-Ho Choi, “Adaptive Two-Level
> Frame Aggregation for Fairness and Efficiency in IEEE 802.11n Wireless
> LANs,”
> http://www.hindawi.com/journals/misy/2015/548109/
>
> [4] Li Bin Jiang and Soung Chang Liew, "Proportional fairness in
> wireless LANs and ad hoc networks,"
>
> http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1424745&isnumber=30730
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast
>

[-- Attachment #2: Type: text/html, Size: 9739 bytes --]

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Toke Høiland-Jørgensen]

Luca Muscariello <luca.muscariello@gmail.com> writes:

> Toke,
>
> I'd suggest to add this in you list of references:
>
> Godfrey Tan and John Guttag, Time-based fairness improves performance in multi-rate WLANs. In Proc of USENIX 2004
> https://www.usenix.org/legacy/publications/library/proceedings/usenix04/tech/general/full_papers/tan/tan.pdf

Awesome, was not aware of that. Thanks!

> It's worth having a look to the APware project for freeBSD and Godfrey Tan PhD thesis at MIT.
>
> http://nms.csail.mit.edu/projects/apware/

Hmm, that link is not working for me right now; will try again later.

> this work predates 802.11n and aggregation.

Yeah, I'm aware that there is a lot of stuff that predates 802.11n. In
fact the article I linked (Kim et al) is the only one I've found that
talks about 802.11n. We also had some people at my uni doing stuff with
802.11g.

> Ten years ago I played with SFQ and madwifi for 802.11g to get max-min
> time fairness (and so proportional rate fairness) with excellent
> results. The hacking I made was based on using time quanta instead of
> bytes. Which required me to get the current PHY rates (AP to all
> STAtions) and dynamically compute/update SFQ time quanta.

Do you happen to recall what precision you achieved or how much the
precision was really important? Several papers seem to assume that very
high precision is not terribly important since it all evens out in the
end, and I can see how that could be true; but would like to have it
confirmed :)

> It's surprising that 802.11 standard never considered time fairness in
> the EDCF. A reason might be the time fairness might be enforced using
> the PCF.

Might be. Might also be that no one thought to measure for that? A lot
of vendors seem to only test single-station raw throughput...

Are you aware of any open source 802.11 stuff that uses PCF at all?

> IMO, It's a very good topic.
> Thanks for bringing this up.

You're very welcome! ;)

-Toke

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Luca Muscariello]

[-- Attachment #1: Type: text/plain, Size: 3267 bytes --]

On Wed, May 11, 2016 at 3:45 PM, Toke Høiland-Jørgensen <toke@toke.dk>
wrote:

> Luca Muscariello <luca.muscariello@gmail.com> writes:
>
> > Toke,
> >
> > I'd suggest to add this in you list of references:
> >
> > Godfrey Tan and John Guttag, Time-based fairness improves performance in
> multi-rate WLANs. In Proc of USENIX 2004
> >
> https://www.usenix.org/legacy/publications/library/proceedings/usenix04/tech/general/full_papers/tan/tan.pdf
>
> Awesome, was not aware of that. Thanks!
>
> > It's worth having a look to the APware project for freeBSD and Godfrey
> Tan PhD thesis at MIT.
> >
> > http://nms.csail.mit.edu/projects/apware/
>
> Hmm, that link is not working for me right now; will try again later.
>
> > this work predates 802.11n and aggregation.
>
> Yeah, I'm aware that there is a lot of stuff that predates 802.11n. In
> fact the article I linked (Kim et al) is the only one I've found that
> talks about 802.11n. We also had some people at my uni doing stuff with
> 802.11g.
>
> > Ten years ago I played with SFQ and madwifi for 802.11g to get max-min
> > time fairness (and so proportional rate fairness) with excellent
> > results. The hacking I made was based on using time quanta instead of
> > bytes. Which required me to get the current PHY rates (AP to all
> > STAtions) and dynamically compute/update SFQ time quanta.
>
> Do you happen to recall what precision you achieved or how much the
> precision was really important? Several papers seem to assume that very
> high precision is not terribly important since it all evens out in the
> end, and I can see how that could be true; but would like to have it
> confirmed :)
>

what do you mean with precision?
Do you mean in measuring the PHY rate?
Short term vs long term measurements? else?

The hard part was adaptiveness. Correlation between the speed of the STA,
the
PHY rate controller and quanta updates in SFQ.
Ideally we were trying to approach something like
downlink channel scheduling in HSDPA where you have slotted time TX
and polling. A slot in HSDPA (but also LTE) is a burst  of fixed time size
of several packets.
Similar to what you want to achieve in your email point 1.
in HSDPA/LTE you have UEs feedback every ms in a side channel. So rate
adaptation
is a lot simpler.

As a side note,
there are several differences in aggregates in 802.11n and HSDPA/LTE
as in the latter the scheduler can send an aggregate containing packets
using different modulation/coding schemes to reach different stations with
a single aggregate transmission.
For the records, this feature was rejected in the 802.11n amendement but
discussed  by the group as it makes the chip more expensive.




>
> > It's surprising that 802.11 standard never considered time fairness in
> > the EDCF. A reason might be the time fairness might be enforced using
> > the PCF.
>
> Might be. Might also be that no one thought to measure for that? A lot
> of vendors seem to only test single-station raw throughput...
>
> Are you aware of any open source 802.11 stuff that uses PCF at all?
>

No. Sorry.


>
> > IMO, It's a very good topic.
> > Thanks for bringing this up.
>
> You're very welcome! ;)
>
> -Toke
>

[-- Attachment #2: Type: text/html, Size: 4705 bytes --]

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Toke Høiland-Jørgensen]

Luca Muscariello <luca.muscariello@gmail.com> writes:

>  Do you happen to recall what precision you achieved or how much the
>  precision was really important? Several papers seem to assume that very
>  high precision is not terribly important since it all evens out in the
>  end, and I can see how that could be true; but would like to have it
>  confirmed :)
>
> what do you mean with precision?
> Do you mean in measuring the PHY rate? 
> Short term vs long term measurements? else?

Yes, in measuring the rate. Was this a per-packet thing, and were you
actually able to get information sufficiently accurate to achieve the
desired level of fairness? And by what mechanism? Was this in the driver
or higher up in the stack?

> The hard part was adaptiveness. Correlation between the speed of the
> STA, the PHY rate controller and quanta updates in SFQ.

Yes, that's what I though :)

> Ideally we were trying to approach something like 
> downlink channel scheduling in HSDPA where you have slotted time TX
> and polling. A slot in HSDPA (but also LTE) is a burst  of fixed time size of several packets.
> Similar to what you want to achieve in your email point 1.

Yeah, if we could just switch to TDMA a lot of things would be much
simpler...

> As a side note, there are several differences in aggregates in 802.11n
> and HSDPA/LTE as in the latter the scheduler can send an aggregate
> containing packets using different modulation/coding schemes to reach
> different stations with a single aggregate transmission.

Neat.

> For the records, this feature was rejected in the 802.11n amendement
> but discussed by the group as it makes the chip more expensive.

Right; not terribly surprised, but still a shame.

-Toke

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[David Lang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 949 bytes --]

On Wed, 11 May 2016, Toke Høiland-Jørgensen wrote:

> Luca Muscariello <luca.muscariello@gmail.com> writes:
>
>>  Do you happen to recall what precision you achieved or how much the
>>  precision was really important? Several papers seem to assume that very
>>  high precision is not terribly important since it all evens out in the
>>  end, and I can see how that could be true; but would like to have it
>>  confirmed :)
>>
>> what do you mean with precision?
>> Do you mean in measuring the PHY rate? 
>> Short term vs long term measurements? else?
>
> Yes, in measuring the rate. Was this a per-packet thing, and were you
> actually able to get information sufficiently accurate to achieve the
> desired level of fairness? And by what mechanism? Was this in the driver
> or higher up in the stack?

I expect that if you were able to change this even once/sec and account for the 
rate you would be far better than what we have now.

David Lang

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[David Lang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1543 bytes --]

On Wed, 11 May 2016, David Lang wrote:

> On Wed, 11 May 2016, Toke Høiland-Jørgensen wrote:
>
>> Luca Muscariello <luca.muscariello@gmail.com> writes:
>>
>>>  Do you happen to recall what precision you achieved or how much the
>>>  precision was really important? Several papers seem to assume that very
>>>  high precision is not terribly important since it all evens out in the
>>>  end, and I can see how that could be true; but would like to have it
>>>  confirmed :)
>>> 
>>> what do you mean with precision?
>>> Do you mean in measuring the PHY rate? Short term vs long term 
>>> measurements? else?
>> 
>> Yes, in measuring the rate. Was this a per-packet thing, and were you
>> actually able to get information sufficiently accurate to achieve the
>> desired level of fairness? And by what mechanism? Was this in the driver
>> or higher up in the stack?
>
> I expect that if you were able to change this even once/sec and account for 
> the rate you would be far better than what we have now.

by the way, I have logs from the last two scale conferences of the /sys data 
per-station showing the rate info. if you give me a way to send you the multi-G 
file you can look through it to see how rapidly the rate changes for a given 
station under real-world high-user-density conditions.

I suspect that the biggest problem right now is that the higher level scheduling 
isn't accounting for "station X is at rate 1, station Y is at rate 100" and is 
trying to be 'fair' by sending the same amount of data to each of them.

David Lang

[-- Attachment #2: Type: TEXT/PLAIN, Size: 167 bytes --]

_______________________________________________
Make-wifi-fast mailing list
Make-wifi-fast@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/make-wifi-fast

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Toke Høiland-Jørgensen]

David Lang <david@lang.hm> writes:

>> I expect that if you were able to change this even once/sec and
>> account for the rate you would be far better than what we have now.

I suspect so too, but would like to have someone who actually tried it
before confirm it ;)

> by the way, I have logs from the last two scale conferences of the
> /sys data per-station showing the rate info. if you give me a way to
> send you the multi-G file you can look through it to see how rapidly
> the rate changes for a given station under real-world
> high-user-density conditions.

Hmm, can probably give you and sftp dump space; will set it up and
message you off-list :)

> I suspect that the biggest problem right now is that the higher level
> scheduling isn't accounting for "station X is at rate 1, station Y is
> at rate 100" and is trying to be 'fair' by sending the same amount of
> data to each of them.

Well, from a scheduling perspective, there really is no "higher level
scheduling". There's just a FIFO queue.

However, the throughput-based fairness you describe is an intrinsic
property of the 802.11 DCF. This is known in the academic literature as
"the 802.11 performance anomaly", It was first described in 2003(!) for
802.11b [1]. There has been several academic publications on ways to fix
this (just finished the one Lucas linked, which was pretty good), but
none of this seems to have percolated into mainline Linux. Surprise,
surprise.

-Toke

[1] M. Heusse, F. Rousseau, G. Berger-Sabbatel and A. Duda, "Performance
anomaly of 802.11b,"
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1208921&isnumber=27206

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Luca Muscariello]

[-- Attachment #1: Type: text/plain, Size: 2324 bytes --]

What I did was just a hack to demonstrate the principle. Today 802.11n is
very different assuming you only use HT.
The APware project at MIT will give you some good hints.

Short term fairness was bad, long term fairness very good.

Short term I believe is very difficult to achieve because of the EDCF.
Short term fairness can be very good only using time slotted MAC.


On Wednesday, 11 May 2016, Toke Høiland-Jørgensen <toke@toke.dk> wrote:

> Luca Muscariello <luca.muscariello@gmail.com <javascript:;>> writes:
>
> >  Do you happen to recall what precision you achieved or how much the
> >  precision was really important? Several papers seem to assume that very
> >  high precision is not terribly important since it all evens out in the
> >  end, and I can see how that could be true; but would like to have it
> >  confirmed :)
> >
> > what do you mean with precision?
> > Do you mean in measuring the PHY rate?
> > Short term vs long term measurements? else?
>
> Yes, in measuring the rate. Was this a per-packet thing, and were you
> actually able to get information sufficiently accurate to achieve the
> desired level of fairness? And by what mechanism? Was this in the driver
> or higher up in the stack?
>
> > The hard part was adaptiveness. Correlation between the speed of the
> > STA, the PHY rate controller and quanta updates in SFQ.
>
> Yes, that's what I though :)
>
> > Ideally we were trying to approach something like
> > downlink channel scheduling in HSDPA where you have slotted time TX
> > and polling. A slot in HSDPA (but also LTE) is a burst  of fixed time
> size of several packets.
> > Similar to what you want to achieve in your email point 1.
>
> Yeah, if we could just switch to TDMA a lot of things would be much
> simpler...
>
> > As a side note, there are several differences in aggregates in 802.11n
> > and HSDPA/LTE as in the latter the scheduler can send an aggregate
> > containing packets using different modulation/coding schemes to reach
> > different stations with a single aggregate transmission.
>
> Neat.
>
> > For the records, this feature was rejected in the 802.11n amendement
> > but discussed by the group as it makes the chip more expensive.
>
> Right; not terribly surprised, but still a shame.
>
> -Toke
>

[-- Attachment #2: Type: text/html, Size: 2821 bytes --]

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Dave Taht]

My own foci are going to be around trying to rip every source of
potential latency out of the current system: be it deferred
interrupts, bad rate control information, overlong txops, excessive
retries, insufficient packet loss, busting the block ack window, and
quashing stations grabbing too much airtime...

and then adding back in "bandwidth" from there. We have enough
bandwidth in wifi nowadays, just now narrow enough time slices to feed
many stations sanely.

a somewhat subtle distinction is that aiming for airtime fairness
independently of the behaviors of real traffic is not a goal (for me).
A system handing out 8 stations 8ms each of airtime is "fair", but
handing out 1ms each - or just enough, for example, for my
videoconferencing flow to handle each frame with a single txop - or
getting a new station started faster on some web traffic - is better.

Certainly there is a ton of low hanging fruit to excise, and achieving
something closer to but we ignore multicast, channel scans, and other
oddities to our peril.

I don't, for example, think that aiming for airtime fairness over 1sec
intervals is good, 20-50ms would be way more desirable. And so on.
Getting a good rate control estimate by the second txop used by a
previously idle station would be good. And so on.

...

this message brought to you from the association for more coffee in the morning.

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Dave Taht]

On Wed, May 11, 2016 at 9:19 AM, Dave Taht <dave.taht@gmail.com> wrote:
> My own foci are going to be around trying to rip every source of
> potential latency out of the current system: be it deferred
> interrupts, bad rate control information, overlong txops, excessive
> retries, insufficient packet loss, busting the block ack window, and
> quashing stations grabbing too much airtime...

and oh, yea, queuing delay. :)

> and then adding back in "bandwidth" from there. We have enough
> bandwidth in wifi nowadays, just now narrow enough time slices to feed
> many stations sanely.

Bandwidth = rate/interval. Humans have a terrible tendency to using
big intervals, like seconds... I'd like to focus on calculating
bandwidth as rate/(minimal achievable txop under contention) rather
than maximal.

>
> a somewhat subtle distinction is that aiming for airtime fairness
> independently of the behaviors of real traffic is not a goal (for me).
> A system handing out 8 stations 8ms each of airtime is "fair", but
> handing out 1ms each - or just enough, for example, for my
> videoconferencing flow to handle each frame with a single txop - or
> getting a new station started faster on some web traffic - is better.
>
> Certainly there is a ton of low hanging fruit to excise, and achieving
> something closer to

airtime fairness is *great*

 but we ignore multicast, channel scans, and other
> oddities to our peril.
>
> I don't, for example, think that aiming for airtime fairness over 1sec
> intervals is good, 20-50ms would be way more desirable. And so on.
> Getting a good rate control estimate by the second txop used by a
> previously idle station would be good. And so on.
>
> ...
>
> this message brought to you from the association for more coffee in the morning.

losing the www.bufferbloat.net server was not helpful, I'd just booked
a vacation ticket.

-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Toke Høiland-Jørgensen]

>> My own foci are going to be around trying to rip every source of
>> potential latency out of the current system: be it deferred
>> interrupts, bad rate control information, overlong txops, excessive
>> retries, insufficient packet loss, busting the block ack window, and
>> quashing stations grabbing too much airtime...
>
> and oh, yea, queuing delay. :)

What's that? ;)

>> and then adding back in "bandwidth" from there. We have enough
>> bandwidth in wifi nowadays, just now narrow enough time slices to feed
>> many stations sanely.
>
> Bandwidth = rate/interval. Humans have a terrible tendency to using
> big intervals, like seconds... I'd like to focus on calculating
> bandwidth as rate/(minimal achievable txop under contention) rather
> than maximal.

Yes, well, it's a tradeoff to a certain extent. But sure, latency is
key; hence the idea to use an FQ-CoDel-based scheduler rather than
simply do round-robin. And I figure getting the aggregate size down to a
manageable size is another angle of attack.

-Toke

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Luca Muscariello]

[-- Attachment #1: Type: text/plain, Size: 1853 bytes --]

What is worse is that I agree entirely.

Short time fairness is important and more important is a fairness criterion
that enables expedited forwarding for RTC.
 And for RTC
what you say is key.

However, EDCF is not going to help when you have many stations and low PHY
rates should be disabled
to get something that approaches to what you have in mind.

On Wednesday, 11 May 2016, Dave Taht <dave.taht@gmail.com> wrote:

> My own foci are going to be around trying to rip every source of
> potential latency out of the current system: be it deferred
> interrupts, bad rate control information, overlong txops, excessive
> retries, insufficient packet loss, busting the block ack window, and
> quashing stations grabbing too much airtime...
>
> and then adding back in "bandwidth" from there. We have enough
> bandwidth in wifi nowadays, just now narrow enough time slices to feed
> many stations sanely.
>
> a somewhat subtle distinction is that aiming for airtime fairness
> independently of the behaviors of real traffic is not a goal (for me).
> A system handing out 8 stations 8ms each of airtime is "fair", but
> handing out 1ms each - or just enough, for example, for my
> videoconferencing flow to handle each frame with a single txop - or
> getting a new station started faster on some web traffic - is better.
>
> Certainly there is a ton of low hanging fruit to excise, and achieving
> something closer to but we ignore multicast, channel scans, and other
> oddities to our peril.
>
> I don't, for example, think that aiming for airtime fairness over 1sec
> intervals is good, 20-50ms would be way more desirable. And so on.
> Getting a good rate control estimate by the second txop used by a
> previously idle station would be good. And so on.
>
> ...
>
> this message brought to you from the association for more coffee in the
> morning.
>

[-- Attachment #2: Type: text/html, Size: 2234 bytes --]

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[David Lang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1216 bytes --]

On Wed, 11 May 2016, Toke Høiland-Jørgensen wrote:

>> Ten years ago I played with SFQ and madwifi for 802.11g to get max-min
>> time fairness (and so proportional rate fairness) with excellent
>> results. The hacking I made was based on using time quanta instead of
>> bytes. Which required me to get the current PHY rates (AP to all
>> STAtions) and dynamically compute/update SFQ time quanta.
>
> Do you happen to recall what precision you achieved or how much the
> precision was really important? Several papers seem to assume that very
> high precision is not terribly important since it all evens out in the
> end, and I can see how that could be true; but would like to have it
> confirmed :)

I think the point here is that perfect is the enemy of good enough. Right now 
the only attempt at fairness is packet counts. Anything that improved this would 
be a win. We can probably get several orders of magnatude improvement from even 
the most crude measurements.

So start simple/cheap and see how much improvement there is, only go to more 
precision (more expensive) if you find there's a need to.

for that matter, retries may be something that you start off ignoring, just like 
multicast

David Lang

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Dave Taht]

On Wed, May 11, 2016 at 6:45 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> Luca Muscariello <luca.muscariello@gmail.com> writes:
>
>> Toke,
>>
>> I'd suggest to add this in you list of references:
>>
>> Godfrey Tan and John Guttag, Time-based fairness improves performance in multi-rate WLANs. In Proc of USENIX 2004
>> https://www.usenix.org/legacy/publications/library/proceedings/usenix04/tech/general/full_papers/tan/tan.pdf
>
> Awesome, was not aware of that. Thanks!
>
>> It's worth having a look to the APware project for freeBSD and Godfrey Tan PhD thesis at MIT.
>>
>> http://nms.csail.mit.edu/projects/apware/
>
> Hmm, that link is not working for me right now; will try again later.

http://web.archive.org/web/20150911055537/http://nms.csail.mit.edu/projects/apware/


>> this work predates 802.11n and aggregation.
>
> Yeah, I'm aware that there is a lot of stuff that predates 802.11n. In
> fact the article I linked (Kim et al) is the only one I've found that
> talks about 802.11n. We also had some people at my uni doing stuff with
> 802.11g.
>
>> Ten years ago I played with SFQ and madwifi for 802.11g to get max-min
>> time fairness (and so proportional rate fairness) with excellent
>> results. The hacking I made was based on using time quanta instead of
>> bytes. Which required me to get the current PHY rates (AP to all
>> STAtions) and dynamically compute/update SFQ time quanta.
>
> Do you happen to recall what precision you achieved or how much the
> precision was really important? Several papers seem to assume that very
> high precision is not terribly important since it all evens out in the
> end, and I can see how that could be true; but would like to have it
> confirmed :)
>
>> It's surprising that 802.11 standard never considered time fairness in
>> the EDCF. A reason might be the time fairness might be enforced using
>> the PCF.
>
> Might be. Might also be that no one thought to measure for that? A lot
> of vendors seem to only test single-station raw throughput...
>
> Are you aware of any open source 802.11 stuff that uses PCF at all?
>
>> IMO, It's a very good topic.
>> Thanks for bringing this up.
>
> You're very welcome! ;)
>
> -Toke
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[David Lang]

[-- Attachment #1: Type: TEXT/Plain, Size: 320 bytes --]

On Wed, 11 May 2016, Luca Muscariello wrote:

> It's surprising that 802.11 standard never considered time fairness in the
> EDCF. A reason might be the time fairness might be enforced using the PCF.

to be fair, at that point the rate variation was 1Mb - 11Mb and wasn't expected 
to change much during use.

David Lang

[-- Attachment #2: Type: TEXT/PLAIN, Size: 167 bytes --]

_______________________________________________
Make-wifi-fast mailing list
Make-wifi-fast@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/make-wifi-fast

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Luca Muscariello]

[-- Attachment #1: Type: text/plain, Size: 621 bytes --]

Correct, but in between that time and now a lot has been done in different
areas but not much on this point.
The fact that some part of the industry is looking at LTE-U is also because
802.11 standard is not good enough.



On Wednesday, 11 May 2016, David Lang <david@lang.hm> wrote:

> On Wed, 11 May 2016, Luca Muscariello wrote:
>
> It's surprising that 802.11 standard never considered time fairness in the
>> EDCF. A reason might be the time fairness might be enforced using the PCF.
>>
>
> to be fair, at that point the rate variation was 1Mb - 11Mb and wasn't
> expected to change much during use.
>
> David Lang

[-- Attachment #2: Type: text/html, Size: 979 bytes --]

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Dave Taht]

On Wed, May 11, 2016 at 9:09 AM, Luca Muscariello
<luca.muscariello@gmail.com> wrote:
> Correct, but in between that time and now a lot has been done in different
> areas but not much on this point.
> The fact that some part of the industry is looking at LTE-U is also because
> 802.11 standard is not good enough.

What do you think of LTE-LAA?

I do think very strongly that actual usage of 802.11 can be made
vastly more efficient, that we can use up a great deal of the mac
currently being left unused, and schedule txops way more efficiently -
and that I'd love to test with michal's patch set against the LTE-U
tests cablelabs, etc which did

100 stations before (stock):

http://blog.cerowrt.org/flent/drr/10tothe5.svg

after

http://blog.cerowrt.org/flent/drr/newcode.svg

I became mortally opposed to LTE-U (lacking exponential backoff and
ignoring sparse station behavior, as well as today's crappy wifi
drivers - along with some very dubious benchmarks), but have not poked
much into LTE-LAA.

I freely admit to loathing the 802.11 mac, and IF LTE-LAA could be as
open, accessible and usable to ordinary users as wifi was, would be
more embracing of it.

>
>
> On Wednesday, 11 May 2016, David Lang <david@lang.hm> wrote:
>>
>> On Wed, 11 May 2016, Luca Muscariello wrote:
>>
>>> It's surprising that 802.11 standard never considered time fairness in
>>> the
>>> EDCF. A reason might be the time fairness might be enforced using the
>>> PCF.
>>
>>
>> to be fair, at that point the rate variation was 1Mb - 11Mb and wasn't
>> expected to change much during use.
>>
>> David Lang
>
>
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast
>



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Dave Taht]

On Wed, May 11, 2016 at 9:41 AM, Dave Taht <dave.taht@gmail.com> wrote:
> On Wed, May 11, 2016 at 9:09 AM, Luca Muscariello
> <luca.muscariello@gmail.com> wrote:
>> Correct, but in between that time and now a lot has been done in different
>> areas but not much on this point.
>> The fact that some part of the industry is looking at LTE-U is also because
>> 802.11 standard is not good enough.
>
> What do you think of LTE-LAA?
>
> I do think very strongly that actual usage of 802.11 can be made
> vastly more efficient, that we can use up a great deal of the mac
> currently being left unused, and schedule txops way more efficiently -
> and that I'd love to test with michal's patch set against the LTE-U
> tests cablelabs, etc which did
>
> 100 stations before (stock):
>
> http://blog.cerowrt.org/flent/drr/10tothe5.svg
>
> after
>
> http://blog.cerowrt.org/flent/drr/newcode.svg

Seeing "only" 250ms worth of delay for 100 stations here was what
kicked off a prior thread, my understanding of a theoretical base
number here would be about 70ms. (?)

...

Adding in mu-mimo to the picture makes my head hurt. My understanding
of how mu-mimo is supposed to work is you have to have accumulated
2-3ms worth of packets for the number of stations you are going to
schedule before it's worthwhile at all.

The stations are going to typically be limited to 1 antenna (most
laptops have 2), I think. So a 4 antenna system *could* send to 4
stations if all have traffic pending... at a cost of a (proposed, I
don't agree with it) 500 usec sounding phase every 10ms. My take on
that is you should only sound when you actually have some potential to
share that many flows to that many stations, sounding being more of an
aspect of rate control, also.

Having only 2 stations that you can mu-mimo to seems like a lose generally.

Based on normal traffic behaviors the stations that could be sent to
varies, and gang scheduling with lots of stations would require even
more soundings...

...

I don't have a lot of hope for mu-mimo, although what I kind of expect
is the work done here will end up marketed as due to that feature in
the wave2 stuff...

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Michal Kazior]

On 11 May 2016 at 20:13, Dave Taht <dave.taht@gmail.com> wrote:
[...]
> Adding in mu-mimo to the picture makes my head hurt. My understanding
> of how mu-mimo is supposed to work is you have to have accumulated
> 2-3ms worth of packets for the number of stations you are going to
> schedule before it's worthwhile at all.
>
> The stations are going to typically be limited to 1 antenna (most
> laptops have 2), I think. So a 4 antenna system *could* send to 4
> stations if all have traffic pending...

4 antenna system can beamform 3 independent spatial streams that can
be used for MU-MIMO. Depending on connected station capabilities with
4 antenna AP you could do 1+1 (some stations may not be able to
participate in 4-antenna system), 1+1+1 or 1+2 groupings.


Michał

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Luca Muscariello]

[-- Attachment #1: Type: text/plain, Size: 3001 bytes --]

I share your skepticism about beam forming. It depends of course on the kind
of usage you make of wifi. If it's to cover a city in a small cell
optimized deployment
I don't think beam forming is going to help. When cell traffic is high only
TDMA can help.

If you use beam forming to reach non mobile users and traffic is not to
high it's going to
give best performance.

Both are valuable use cases with economic incentives behind.
The first one is more difficult and time fairness is gonna help a lot there
as the average cell throughput is gonna be a lot better.



On Wed, May 11, 2016 at 8:13 PM, Dave Taht <dave.taht@gmail.com> wrote:

> On Wed, May 11, 2016 at 9:41 AM, Dave Taht <dave.taht@gmail.com> wrote:
> > On Wed, May 11, 2016 at 9:09 AM, Luca Muscariello
> > <luca.muscariello@gmail.com> wrote:
> >> Correct, but in between that time and now a lot has been done in
> different
> >> areas but not much on this point.
> >> The fact that some part of the industry is looking at LTE-U is also
> because
> >> 802.11 standard is not good enough.
> >
> > What do you think of LTE-LAA?
> >
> > I do think very strongly that actual usage of 802.11 can be made
> > vastly more efficient, that we can use up a great deal of the mac
> > currently being left unused, and schedule txops way more efficiently -
> > and that I'd love to test with michal's patch set against the LTE-U
> > tests cablelabs, etc which did
> >
> > 100 stations before (stock):
> >
> > http://blog.cerowrt.org/flent/drr/10tothe5.svg
> >
> > after
> >
> > http://blog.cerowrt.org/flent/drr/newcode.svg
>
> Seeing "only" 250ms worth of delay for 100 stations here was what
> kicked off a prior thread, my understanding of a theoretical base
> number here would be about 70ms. (?)
>

I miss many details about these tests. And probably I missed the thread...
Can you give me pointers?



>
> ...
>
> Adding in mu-mimo to the picture makes my head hurt. My understanding
> of how mu-mimo is supposed to work is you have to have accumulated
> 2-3ms worth of packets for the number of stations you are going to
> schedule before it's worthwhile at all.
>
> The stations are going to typically be limited to 1 antenna (most
> laptops have 2), I think. So a 4 antenna system *could* send to 4
> stations if all have traffic pending... at a cost of a (proposed, I
> don't agree with it) 500 usec sounding phase every 10ms. My take on
> that is you should only sound when you actually have some potential to
> share that many flows to that many stations, sounding being more of an
> aspect of rate control, also.
>
> Having only 2 stations that you can mu-mimo to seems like a lose generally.
>
> Based on normal traffic behaviors the stations that could be sent to
> varies, and gang scheduling with lots of stations would require even
> more soundings...
>
> ...
>
> I don't have a lot of hope for mu-mimo, although what I kind of expect
> is the work done here will end up marketed as due to that feature in
> the wave2 stuff...
>

[-- Attachment #2: Type: text/html, Size: 4189 bytes --]

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[David Lang]

I actually have hopes that it will help in the (fairly common) case of multiple 
stations streaming video from the Internet at the same time. If the AP -> 
endpoint traffic can send to multiple stations in the same txop, it can greatly 
reduce the amount of airtime needed.

unfortunantly, from what I hear, the factory AP firmware breaks down if there 
are more than about a dozen stations connected, just as it becomes the most 
valuable.

But to make it work, we can't have two layers of buffering, we need the firmware 
to be able to grab several aggregates worth of data from different queues, and 
some of the data grabbed may be an 'unfair' share for that source, but it should 
still be grabbed because there is nothing else that could be sent at the same 
time, so the transmission is effectively free.

but to do that sort of thing, we really need to be operating close to the 
hardware, not with some firmware buffers between us.

David Lang

On Thu, 12 May 2016, Luca Muscariello wrote:

> I share your skepticism about beam forming. It depends of course on the kind
> of usage you make of wifi. If it's to cover a city in a small cell
> optimized deployment
> I don't think beam forming is going to help. When cell traffic is high only
> TDMA can help.
>
> If you use beam forming to reach non mobile users and traffic is not to
> high it's going to
> give best performance.
>
> Both are valuable use cases with economic incentives behind.
> The first one is more difficult and time fairness is gonna help a lot there
> as the average cell throughput is gonna be a lot better.
>
>
>
> On Wed, May 11, 2016 at 8:13 PM, Dave Taht <dave.taht@gmail.com> wrote:
>
>> On Wed, May 11, 2016 at 9:41 AM, Dave Taht <dave.taht@gmail.com> wrote:
>>> On Wed, May 11, 2016 at 9:09 AM, Luca Muscariello
>>> <luca.muscariello@gmail.com> wrote:
>>>> Correct, but in between that time and now a lot has been done in
>> different
>>>> areas but not much on this point.
>>>> The fact that some part of the industry is looking at LTE-U is also
>> because
>>>> 802.11 standard is not good enough.
>>>
>>> What do you think of LTE-LAA?
>>>
>>> I do think very strongly that actual usage of 802.11 can be made
>>> vastly more efficient, that we can use up a great deal of the mac
>>> currently being left unused, and schedule txops way more efficiently -
>>> and that I'd love to test with michal's patch set against the LTE-U
>>> tests cablelabs, etc which did
>>>
>>> 100 stations before (stock):
>>>
>>> http://blog.cerowrt.org/flent/drr/10tothe5.svg
>>>
>>> after
>>>
>>> http://blog.cerowrt.org/flent/drr/newcode.svg
>>
>> Seeing "only" 250ms worth of delay for 100 stations here was what
>> kicked off a prior thread, my understanding of a theoretical base
>> number here would be about 70ms. (?)
>>
>
> I miss many details about these tests. And probably I missed the thread...
> Can you give me pointers?
>
>
>
>>
>> ...
>>
>> Adding in mu-mimo to the picture makes my head hurt. My understanding
>> of how mu-mimo is supposed to work is you have to have accumulated
>> 2-3ms worth of packets for the number of stations you are going to
>> schedule before it's worthwhile at all.
>>
>> The stations are going to typically be limited to 1 antenna (most
>> laptops have 2), I think. So a 4 antenna system *could* send to 4
>> stations if all have traffic pending... at a cost of a (proposed, I
>> don't agree with it) 500 usec sounding phase every 10ms. My take on
>> that is you should only sound when you actually have some potential to
>> share that many flows to that many stations, sounding being more of an
>> aspect of rate control, also.
>>
>> Having only 2 stations that you can mu-mimo to seems like a lose generally.
>>
>> Based on normal traffic behaviors the stations that could be sent to
>> varies, and gang scheduling with lots of stations would require even
>> more soundings...
>>
>> ...
>>
>> I don't have a lot of hope for mu-mimo, although what I kind of expect
>> is the work done here will end up marketed as due to that feature in
>> the wave2 stuff...
>>
>

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Michal Kazior]

On 12 May 2016 at 10:40, David Lang <david@lang.hm> wrote:
> I actually have hopes that it will help in the (fairly common) case of
> multiple stations streaming video from the Internet at the same time. If the
> AP -> endpoint traffic can send to multiple stations in the same txop, it
> can greatly reduce the amount of airtime needed.
>
> unfortunantly, from what I hear, the factory AP firmware breaks down if
> there are more than about a dozen stations connected, just as it becomes the
> most valuable.

FWIW There are some firmware changes in some recent ath10k for qca99x0
hardware which involves firmware performing more sophisticated
scheduling coordinated with the host driver (not sure how much of it
is available publicly, yet). The host driver exposes queue depth of
each station/tid and firmware asks host to submit frames to given
station/tid within advertised byte/packet limits.

This is supposed to be "fast" even with up to 50 MU-MIMO clients. I
didn't really get to test it fully from a latency point of view with
flent yet (and I only have 10 MU clients at my disposal currently).


Michał

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Luca Muscariello]

[-- Attachment #1: Type: text/plain, Size: 2479 bytes --]

LTE-U and LTE-LAA are basically the same thing.
They require a licensed anchor.
MuLTEFire does not.

All needs to have a listen before talk and some level of fairness.

All these are gonna give a lot better quality and capacity than 802.11.
Enough to push 802.11 improvement in the standard?

On Wednesday, 11 May 2016, Dave Taht <dave.taht@gmail.com> wrote:

> On Wed, May 11, 2016 at 9:09 AM, Luca Muscariello
> <luca.muscariello@gmail.com <javascript:;>> wrote:
> > Correct, but in between that time and now a lot has been done in
> different
> > areas but not much on this point.
> > The fact that some part of the industry is looking at LTE-U is also
> because
> > 802.11 standard is not good enough.
>
> What do you think of LTE-LAA?
>
> I do think very strongly that actual usage of 802.11 can be made
> vastly more efficient, that we can use up a great deal of the mac
> currently being left unused, and schedule txops way more efficiently -
> and that I'd love to test with michal's patch set against the LTE-U
> tests cablelabs, etc which did
>
> 100 stations before (stock):
>
> http://blog.cerowrt.org/flent/drr/10tothe5.svg
>
> after
>
> http://blog.cerowrt.org/flent/drr/newcode.svg
>
> I became mortally opposed to LTE-U (lacking exponential backoff and
> ignoring sparse station behavior, as well as today's crappy wifi
> drivers - along with some very dubious benchmarks), but have not poked
> much into LTE-LAA.
>
> I freely admit to loathing the 802.11 mac, and IF LTE-LAA could be as
> open, accessible and usable to ordinary users as wifi was, would be
> more embracing of it.
>
> >
> >
> > On Wednesday, 11 May 2016, David Lang <david@lang.hm <javascript:;>>
> wrote:
> >>
> >> On Wed, 11 May 2016, Luca Muscariello wrote:
> >>
> >>> It's surprising that 802.11 standard never considered time fairness in
> >>> the
> >>> EDCF. A reason might be the time fairness might be enforced using the
> >>> PCF.
> >>
> >>
> >> to be fair, at that point the rate variation was 1Mb - 11Mb and wasn't
> >> expected to change much during use.
> >>
> >> David Lang
> >
> >
> > _______________________________________________
> > Make-wifi-fast mailing list
> > Make-wifi-fast@lists.bufferbloat.net <javascript:;>
> > https://lists.bufferbloat.net/listinfo/make-wifi-fast
> >
>
>
>
> --
> Dave Täht
> Let's go make home routers and wifi faster! With better software!
> http://blog.cerowrt.org
>

[-- Attachment #2: Type: text/html, Size: 3564 bytes --]

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Luca Muscariello]

[-- Attachment #1: Type: text/plain, Size: 2753 bytes --]

to be fair I should say that these technologies are gonna be a lot more
expensive than wifi.
There is a good reason to keep wifi simple.

On Wed, May 11, 2016 at 8:28 PM, Luca Muscariello <
luca.muscariello@gmail.com> wrote:

> LTE-U and LTE-LAA are basically the same thing.
> They require a licensed anchor.
> MuLTEFire does not.
>
> All needs to have a listen before talk and some level of fairness.
>
> All these are gonna give a lot better quality and capacity than 802.11.
> Enough to push 802.11 improvement in the standard?
>
>
> On Wednesday, 11 May 2016, Dave Taht <dave.taht@gmail.com> wrote:
>
>> On Wed, May 11, 2016 at 9:09 AM, Luca Muscariello
>> <luca.muscariello@gmail.com> wrote:
>> > Correct, but in between that time and now a lot has been done in
>> different
>> > areas but not much on this point.
>> > The fact that some part of the industry is looking at LTE-U is also
>> because
>> > 802.11 standard is not good enough.
>>
>> What do you think of LTE-LAA?
>>
>> I do think very strongly that actual usage of 802.11 can be made
>> vastly more efficient, that we can use up a great deal of the mac
>> currently being left unused, and schedule txops way more efficiently -
>> and that I'd love to test with michal's patch set against the LTE-U
>> tests cablelabs, etc which did
>>
>> 100 stations before (stock):
>>
>> http://blog.cerowrt.org/flent/drr/10tothe5.svg
>>
>> after
>>
>> http://blog.cerowrt.org/flent/drr/newcode.svg
>>
>> I became mortally opposed to LTE-U (lacking exponential backoff and
>> ignoring sparse station behavior, as well as today's crappy wifi
>> drivers - along with some very dubious benchmarks), but have not poked
>> much into LTE-LAA.
>>
>> I freely admit to loathing the 802.11 mac, and IF LTE-LAA could be as
>> open, accessible and usable to ordinary users as wifi was, would be
>> more embracing of it.
>>
>> >
>> >
>> > On Wednesday, 11 May 2016, David Lang <david@lang.hm> wrote:
>> >>
>> >> On Wed, 11 May 2016, Luca Muscariello wrote:
>> >>
>> >>> It's surprising that 802.11 standard never considered time fairness in
>> >>> the
>> >>> EDCF. A reason might be the time fairness might be enforced using the
>> >>> PCF.
>> >>
>> >>
>> >> to be fair, at that point the rate variation was 1Mb - 11Mb and wasn't
>> >> expected to change much during use.
>> >>
>> >> David Lang
>> >
>> >
>> > _______________________________________________
>> > Make-wifi-fast mailing list
>> > Make-wifi-fast@lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/make-wifi-fast
>> >
>>
>>
>>
>> --
>> Dave Täht
>> Let's go make home routers and wifi faster! With better software!
>> http://blog.cerowrt.org
>>
>

[-- Attachment #2: Type: text/html, Size: 3870 bytes --]

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[David Lang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3631 bytes --]

On Wed, 11 May 2016, Toke Høiland-Jørgensen wrote:

> Improving the access point behaviour is definitely the straight-forward
> way, and there are many obvious improvements that can result from that,
> as we've already seen in e.g. Michal's patch set. So looking at airtime
> fairness from the point of view of the access point makes sense.

As a practical matter, we are not going to be able to change the code on the 
nodes. There are just too many closed source, and never-to-be-updated devices 
out there (IoT)

So focusing on the AP is the right thing to do.

> Basically, on a high level I think there are two things that we need to
> achieve good airtime fairness: Building the right aggregates (like in
> [3]), and scheduling (like in [4]) to compensate for when we get it
> wrong or when we can't control the aggregation because the firmware
> handles it.
>
> Getting closer to the practical level, what I plan to go poke at is:
>
> (1) The ath9k aggregation building logic, to try to achieve
>  constant-time aggregate size (for a suitable time quantum). Not sure
>  how much actually needs to change here; the standard already mandates
>  that an aggregate cannot be longer than 4ms, so the code already takes
>  time into account. And not sure how this interacts with rate
>  selection.

I think the problem  is that the aggregates are created too soon, and as a 
result the rate can change drastically between the time the aggreagate is formed 
and when it is sent. I don't believe that they are really limited to 4ms in 
practice.

> (2) Adding a time-based scheduler to enforce airtime fairness. Building
>  this on FQ-CoDel seems like the sensible thing to do, to get the same
>  latency bonus for stations that only transmit occasionally. It would
>  be nice to be able to stick this in the mac80211 layer to make it
>  shared between drivers, but not sure the required timing information
>  is available at that layer.

Right now, most of the stuff is still doing packet-count based queues. On wired 
networks where the rate was constant, moving to BQL was a huge win. When the 
rate can vary so drastically, we need to be factoring rate into the picture 
somehow.

> A non-exhaustive list of issues that need to be resolved one way or
> another:
>
> - What to account to each station's airtime? My thought would be
>  retransmissions, but not time spent waiting for other stations.

Possibly, or possibly have retransmissions change the effective rate.

> - What about management, control and multicast frames? I'd say just
>  ignore it (at least for now).

I agree.

> - How to measure the results? Can we dump the information from the
>  driver (and is it accurate)? Do we need to parse aircaps? Something
>  else?

we need to be able to instrament the driver (I don't know if we can get enough 
data there or not currently)

trying to work with aircaps has the problem that what you see in the aircap 
doesn't match what either the sender or receiver sees

Take retransmissions as an example. They only happen because the receiver didn't 
see them. If you were to get an aircap off the same antenna as the receiver, you 
also wouldn't see them and therefor could not account for them. In the real 
world, you are doing the aircap from a different device, with a different 
antenna so what you see will be even more different. Now think about the normal 
case where you have two stations taking in two very different locations and one 
device to do the aircap.

If we don't have anything else, aircaps are what we have to fall back on, but we 
need to realize how much we can't see at that point.

David Lang

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[Toke Høiland-Jørgensen]

David Lang <david@lang.hm> writes:

>> Improving the access point behaviour is definitely the straight-forward
>> way, and there are many obvious improvements that can result from that,
>> as we've already seen in e.g. Michal's patch set. So looking at airtime
>> fairness from the point of view of the access point makes sense.
>
> As a practical matter, we are not going to be able to change the code
> on the nodes. There are just too many closed source, and
> never-to-be-updated devices out there (IoT)

Yes, there's that too. Besides, if we assume most traffic is TCP (or has
similar ack-type feedback), we can to a large extent control what the
client does by scheduling in the downstream.

> I think the problem is that the aggregates are created too soon, and
> as a result the rate can change drastically between the time the
> aggreagate is formed and when it is sent. I don't believe that they
> are really limited to 4ms in practice.

Ah, right, hadn't thought of that. Another good reason to reduce
queueing in the lower layers I suppose...

> Right now, most of the stuff is still doing packet-count based queues.
> On wired networks where the rate was constant, moving to BQL was a
> huge win. When the rate can vary so drastically, we need to be
> factoring rate into the picture somehow.

Yes, my thought was to just switch entirely to time-based accounting in
the scheduler. Which means that the scheduler has to have this
information available to it. Not sure whether that is practical at the
mac80211 layer (but suppose it could be done by adding appropriate
driver hooks if things are missing).

>> - How to measure the results? Can we dump the information from the
>>  driver (and is it accurate)? Do we need to parse aircaps? Something
>>  else?
>
> we need to be able to instrament the driver (I don't know if we can
> get enough data there or not currently)
>
> trying to work with aircaps has the problem that what you see in the
> aircap doesn't match what either the sender or receiver sees
>
> Take retransmissions as an example. They only happen because the
> receiver didn't see them. If you were to get an aircap off the same
> antenna as the receiver, you also wouldn't see them and therefor could
> not account for them. In the real world, you are doing the aircap from
> a different device, with a different antenna so what you see will be
> even more different. Now think about the normal case where you have
> two stations taking in two very different locations and one device to
> do the aircap.
>
> If we don't have anything else, aircaps are what we have to fall back
> on, but we need to realize how much we can't see at that point.

Hmm, hadn't thought of that. Damn. Well, guess we'll have to trust the
driver (or make it trustworthy if we can't).

-Toke

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[David Lang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1858 bytes --]

On Wed, 11 May 2016, Toke Høiland-Jørgensen wrote:

> David Lang <david@lang.hm> writes:
>
>>> - How to measure the results? Can we dump the information from the
>>>  driver (and is it accurate)? Do we need to parse aircaps? Something
>>>  else?
>>
>> we need to be able to instrament the driver (I don't know if we can
>> get enough data there or not currently)
>>
>> trying to work with aircaps has the problem that what you see in the
>> aircap doesn't match what either the sender or receiver sees
>>
>> Take retransmissions as an example. They only happen because the
>> receiver didn't see them. If you were to get an aircap off the same
>> antenna as the receiver, you also wouldn't see them and therefor could
>> not account for them. In the real world, you are doing the aircap from
>> a different device, with a different antenna so what you see will be
>> even more different. Now think about the normal case where you have
>> two stations taking in two very different locations and one device to
>> do the aircap.
>>
>> If we don't have anything else, aircaps are what we have to fall back
>> on, but we need to realize how much we can't see at that point.
>
> Hmm, hadn't thought of that. Damn. Well, guess we'll have to trust the
> driver (or make it trustworthy if we can't).

the good news is that getting this information out of the driver should not be 
hard. It knows when it does a retransmit.

I also suspect that we could get away without explictly tracking retransmits for 
now. As it retransmits to a station, it's also going to end up slowing down if 
it's a long-term problem.

we need to think of the scale we are trying to be fair on. I think that if we 
are fair on the scale of seconds we will gain 80% of the effect that we would 
get if we were trying to be fair on the order of ms, and it's much easier to do 
:-)

David Lang

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[moeller0]

> On May 11, 2016, at 17:15 , Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> David Lang <david@lang.hm> writes:
>> 
>> Take retransmissions as an example. They only happen because the
>> receiver didn't see them. If you were to get an aircap off the same
>> antenna as the receiver, you also wouldn't see them and therefor could
>> not account for them. In the real world, you are doing the aircap from
>> a different device, with a different antenna so what you see will be
>> even more different. Now think about the normal case where you have
>> two stations taking in two very different locations and one device to
>> do the aircap.
>> 
>> If we don't have anything else, aircaps are what we have to fall back
>> on, but we need to realize how much we can't see at that point.
> 
> Hmm, hadn't thought of that. Damn. Well, guess we'll have to trust the
> driver (or make it trustworthy if we can’t).

	This issue seems surmountable, just make sure the air-capping machine is a) located halfway between the other two hosts in question and b) has better antennas ;) . It should be possible to “degrade” the antenna quality of the two hosts to make sure the air-capper (sounds like a fancy whale species) has better RF visibility…

Best Regards
	Sebastian


> 
> -Toke
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[David Lang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1924 bytes --]

On Wed, 11 May 2016, moeller0 wrote:

>> On May 11, 2016, at 17:15 , Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>
>> David Lang <david@lang.hm> writes:
>>>
>>> Take retransmissions as an example. They only happen because the
>>> receiver didn't see them. If you were to get an aircap off the same
>>> antenna as the receiver, you also wouldn't see them and therefor could
>>> not account for them. In the real world, you are doing the aircap from
>>> a different device, with a different antenna so what you see will be
>>> even more different. Now think about the normal case where you have
>>> two stations taking in two very different locations and one device to
>>> do the aircap.
>>>
>>> If we don't have anything else, aircaps are what we have to fall back
>>> on, but we need to realize how much we can't see at that point.
>>
>> Hmm, hadn't thought of that. Damn. Well, guess we'll have to trust the
>> driver (or make it trustworthy if we can’t).
>
> 	This issue seems surmountable, just make sure the air-capping machine is a) located halfway between the other two hosts in question and b) has better antennas ;) . It should be possible to “degrade” the antenna quality of the two hosts to make sure the air-capper (sounds like a fancy whale species) has better RF visibility…

better antennas on the aircap machine doesn't solve the problem for two reasons.

1. 'better' antennas are almost never better in every direction, they tend to 
change the pattern (for omnidirectional antennas, they trade vertical 
sensitivity for better horizontal sensitivity), so your 'better' antenna may not 
hear interference from the floor above you.

2. your 'better' antenna will pick up interference from stations further away 
which the receiving stations don't hear, so something they hear you may not be 
able to make out.

There's a reason why RF engineering is half black magic and experience :-(

David Lang

Re: [Make-wifi-fast] Thoughts on tackling airtime fairness

[moeller0]

Hi David,

> On May 12, 2016, at 01:25 , David Lang <david@lang.hm> wrote:
> 
> On Wed, 11 May 2016, moeller0 wrote:
> 
>>> On May 11, 2016, at 17:15 , Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>> 
>>> David Lang <david@lang.hm> writes:
>>>> 
>>>> Take retransmissions as an example. They only happen because the
>>>> receiver didn't see them. If you were to get an aircap off the same
>>>> antenna as the receiver, you also wouldn't see them and therefor could
>>>> not account for them. In the real world, you are doing the aircap from
>>>> a different device, with a different antenna so what you see will be
>>>> even more different. Now think about the normal case where you have
>>>> two stations taking in two very different locations and one device to
>>>> do the aircap.
>>>> 
>>>> If we don't have anything else, aircaps are what we have to fall back
>>>> on, but we need to realize how much we can't see at that point.
>>> 
>>> Hmm, hadn't thought of that. Damn. Well, guess we'll have to trust the
>>> driver (or make it trustworthy if we can’t).
>> 
>> 	This issue seems surmountable, just make sure the air-capping machine is a) located halfway between the other two hosts in question and b) has better antennas ;) . It should be possible to “degrade” the antenna quality of the two hosts to make sure the air-capper (sounds like a fancy whale species) has better RF visibility…
> 
> better antennas on the aircap machine doesn't solve the problem for two reasons.
> 
> 1. ‘better' antennas are almost never better in every direction, they tend to change the pattern (for omnidirectional antennas, they trade vertical sensitivity for better horizontal sensitivity), so your 'better' antenna may not hear interference from the floor above you.

	Please read what I wrote again; the better “better antennas” requirement can be fulfilled by making the antennas of the to be tested device pair worse… that should take care of your concern pretty much. Plus locating the device halfway between the pair will also give it better RF visibility to signals from both…

> 
> 2. your ‘better' antenna will pick up interference from stations further away which the receiving stations don't hear, so something they hear you may not be able to make out.

	Which is totally fine to a degree as long as the aircap device will pick up all transmissions/retransmissions of the device pair under test. Also placing the air-capper at the midpoint will also ameliorate this concern.

> 
> There’s a reason why RF engineering is half black magic and experience :-(

	Oh, if you require perfect I am sure you are right, if we aim for good enough I have a hunch we can avoid the requirement for at least the “black” part of the magic component ;)

Best Regards
	Sebastian

> 
> David Lang