[Make-wifi-fast] How to figure out Linux wifi queue and ath10k queue scheduling?

[Make-wifi-fast] How to figure out Linux wifi queue and ath10k queue scheduling?

[Louie Lu]

Hi all,

I'm a graduate student in NCTU, Taiwan.

I'm working on making VoWIFI performance more better on ath10k driver,
my first thought is that I can modify queue scheduling algorithm
inside ath10k.

After tracking the commit log, it seems currently ath10k is using
mac80211 as an intermidieate queue[1][2], and as [2] indicate, ath10k
is not able to do airtime fairness now.

After reading the source code and [2], what I think about the queue in
Linux to ath10k is:

1. Mac layer queue (mac80211)
2. ath10k 802.11e priority queues (4 queues, with different 802.11e ToS level)
3. ath10k firmware (HW) queue

Am I right on this?

Also, I saw the make-wifi-fast google docs[3], it has a plan for
obsolete VO queue, is that still in the roadmap, or it is implemented,
if not, where can I help for it?


Thanks,
Louie.

[1]: https://www.linuxplumbersconf.org/2016/ocw/system/presentations/3963/original/linuxplumbers_wifi_latency-3Nov.pdf
[2]: https://arxiv.org/abs/1703.00064
[3]: https://docs.google.com/document/d/1Se36svYE1Uzpppe1HWnEyat_sAGghB3kE285LElJBW4/edit#

Re: [Make-wifi-fast] How to figure out Linux wifi queue and ath10k queue scheduling?

[Toke Høiland-Jørgensen]

Hi Louie

> I'm working on making VoWIFI performance more better on ath10k driver,
> my first thought is that I can modify queue scheduling algorithm
> inside ath10k.

Sounds awesome! What kind of VoIP traffic are you working with, and what
aspect of performance are you looking to improve? :)

> After tracking the commit log, it seems currently ath10k is using
> mac80211 as an intermidieate queue[1][2], and as [2] indicate, ath10k
> is not able to do airtime fairness now.

Yup, that's about right. We're working on the airtime fairness part,
though; see my recent presentation at Netdev a few weeks ago for an
updated status on previous and ongoing work [1].

> After reading the source code and [2], what I think about the queue in
> Linux to ath10k is:
>
> 1. Mac layer queue (mac80211)
> 2. ath10k 802.11e priority queues (4 queues, with different 802.11e ToS level)
> 3. ath10k firmware (HW) queue
>
> Am I right on this?

More or less (I don't know enough about how ath10k works internally to
tell you exactly how the internal queueing in the driver works).
Ideally, the lower-level queues (2 and 3) should be as empty as
possible, but no one ever got around to working on improving that. There
was an aborted attempt to apply the dynamic queue limit code[2] to
ath10k a while ago, but it was never completed. The WiP commit is here:
[3].

> Also, I saw the make-wifi-fast google docs[3], it has a plan for
> obsolete VO queue, is that still in the roadmap, or it is implemented,
> if not, where can I help for it?

The work on the different QoS levels has mostly been on the idea stage
so far. My presentation[1] has some ideas related to this as well, but
nothing has been implemented thus far. What's on the roadmap depends
mostly on what people go ahead and implement ;)

-Toke

[1] https://www.netdevconf.org/2.2/session.html?jorgensen-wifistack-talk
[2] https://lwn.net/Articles/454390/
[3] https://github.com/kazikcz/linux/commit/1486ffbafdb4dd433203c35c694b7443fa769210

Re: [Make-wifi-fast] How to figure out Linux wifi queue and ath10k queue scheduling?

[Louie Lu]

2017-11-22 22:25 GMT+08:00 Toke Høiland-Jørgensen <toke@toke.dk>:
> Hi Louie
>
>> I'm working on making VoWIFI performance more better on ath10k driver,
>> my first thought is that I can modify queue scheduling algorithm
>> inside ath10k.
>
> Sounds awesome! What kind of VoIP traffic are you working with, and what
> aspect of performance are you looking to improve? :)

We are working on 802.11ac with telecom operator's VoWIFI (through ePDG),
but much more focus on the phone to AP jitter, latency and TXOP when
AP has multiple stations (10+) traffic.


>> After tracking the commit log, it seems currently ath10k is using
>> mac80211 as an intermidieate queue[1][2], and as [2] indicate, ath10k
>> is not able to do airtime fairness now.
>
> Yup, that's about right. We're working on the airtime fairness part,
> though; see my recent presentation at Netdev a few weeks ago for an
> updated status on previous and ongoing work [1].

Thanks for the update, I'll take a look at this. Also, is "airetime
fairness" a concept as fair TXOP
to each station? I'm not sure if I get this right.

>> After reading the source code and [2], what I think about the queue in
>> Linux to ath10k is:
>>
>> 1. Mac layer queue (mac80211)
>> 2. ath10k 802.11e priority queues (4 queues, with different 802.11e ToS level)
>> 3. ath10k firmware (HW) queue
>>
>> Am I right on this?
>
> More or less (I don't know enough about how ath10k works internally to
> tell you exactly how the internal queueing in the driver works).
> Ideally, the lower-level queues (2 and 3) should be as empty as
> possible, but no one ever got around to working on improving that. There
> was an aborted attempt to apply the dynamic queue limit code[2] to
> ath10k a while ago, but it was never completed. The WiP commit is here:
> [3].

Thanks for the information, I'll take a look at it.

>> Also, I saw the make-wifi-fast google docs[3], it has a plan for
>> obsolete VO queue, is that still in the roadmap, or it is implemented,
>> if not, where can I help for it?
>
> The work on the different QoS levels has mostly been on the idea stage
> so far. My presentation[1] has some ideas related to this as well, but
> nothing has been implemented thus far. What's on the roadmap depends
> mostly on what people go ahead and implement ;)

Got it, but are there any udpate about the make-wifi-fast google docs?
Or this is
the latest version of it. Beside patchwork, is there an issue tracker
or just mailing list
to get the latest information?

Thanks,
Louie.

> -Toke
>
> [1] https://www.netdevconf.org/2.2/session.html?jorgensen-wifistack-talk
> [2] https://lwn.net/Articles/454390/
> [3] https://github.com/kazikcz/linux/commit/1486ffbafdb4dd433203c35c694b7443fa769210

Re: [Make-wifi-fast] How to figure out Linux wifi queue and ath10k queue scheduling?

[Toke Høiland-Jørgensen]

Louie Lu <nctu@louie.lu> writes:

> 2017-11-22 22:25 GMT+08:00 Toke Høiland-Jørgensen <toke@toke.dk>:
>> Hi Louie
>>
>>> I'm working on making VoWIFI performance more better on ath10k driver,
>>> my first thought is that I can modify queue scheduling algorithm
>>> inside ath10k.
>>
>> Sounds awesome! What kind of VoIP traffic are you working with, and what
>> aspect of performance are you looking to improve? :)
>
> We are working on 802.11ac with telecom operator's VoWIFI (through
> ePDG),

Ah, right, so this is the same technology as VoLTE, but offloaded to
WiFi? Is that a weird proprietary protocol, or is it similar to standard
ones like RTP?

> but much more focus on the phone to AP jitter, latency and TXOP when
> AP has multiple stations (10+) traffic.

Right. Well, with the 802.11 MAC there's only so much you can do about
that (since there's no central controller); especially jitter is
probably hard to kill entirely. What bounds on latency and jitter are
you targeting?

>>> After tracking the commit log, it seems currently ath10k is using
>>> mac80211 as an intermidieate queue[1][2], and as [2] indicate, ath10k
>>> is not able to do airtime fairness now.
>>
>> Yup, that's about right. We're working on the airtime fairness part,
>> though; see my recent presentation at Netdev a few weeks ago for an
>> updated status on previous and ongoing work [1].
>
> Thanks for the update, I'll take a look at this. Also, is "airetime
> fairness" a concept as fair TXOP to each station? I'm not sure if I
> get this right.

By "airtime fairness" we simply mean a network state where each station
gets the same total amount of transmission time (as opposed to the
default 802.11 behaviour which gives "throughput fairness", where slow
stations take up more airtime than fast stations, slowing the whole
network down). This is explained in detail in section 2.2 of the paper
you already found on arxiv (the final version is published in USENIX
ATC[1]; I don't recall if that is one of the sections we updated). This
general behaviour of the 802.11 MAC was first described way back in 2001
and is referred to as the "performance anomaly"[2].

Since there's no central scheduling in 802.11, the way we implement
airtime fairness is basically by throttling stations that take up too
much airtime. This is fairly straight forward to do in the AP->station
direction, since the AP controls transmissions. In the other direction
there is no direct control, but by counting the receive airtime towards
that station and then throttling its traffic from the AP. This works
because most traffic is TCP.

Currently, throttling takes place after the fact (i.e., we don't adjust
the length of the transmissions). The idea of making TXOPs smaller as
the number of stations goes up has been discussed on this list before
(and is mentioned on slide 15 of the talk I linked to before), but we
haven't gotten around to implementing it yet. And since ath10k does
aggregation and rate control in firmware, I'm not sure if it's possible
to do at the driver level there...

>> The work on the different QoS levels has mostly been on the idea stage
>> so far. My presentation[1] has some ideas related to this as well, but
>> nothing has been implemented thus far. What's on the roadmap depends
>> mostly on what people go ahead and implement ;)
>
> Got it, but are there any udpate about the make-wifi-fast google docs?
> Or this is the latest version of it. Beside patchwork, is there an
> issue tracker or just mailing list to get the latest information?

Not really, no. You've already found the mailing list; most things take
place here (and on linux-wireless, but I at least try to CC both lists
in the stuff I post). Looking through the mailing list archive can be a
source of ideas, or just posting and soliciting feedback :)

-Toke

[1] https://www.usenix.org/conference/atc17/technical-sessions/presentation/hoilan-jorgesen
[2] http://ieeexplore.ieee.org/abstract/document/1208921/?reload=true