[Make-wifi-fast] bloated ath10k, extra latency at lower rates?

[Make-wifi-fast] bloated ath10k, extra latency at lower rates?

[Jon Pike]

[-- Attachment #1: Type: text/plain, Size: 1143 bytes --]

This might be a good time to mention an observation I've seen that's
related to this.

I have noticed about the same as reported,  10's down to single digits on
my ath9k radios, and a similar 20ms-30ms on the ath10k's.  Thats at high
signal strength and the highest data rates.  But, I've also noticed that
it's different if you get farther out in range, and you end up on the
slower signaling speeds.

Under those conditions, I'm seeing the ath9k latency rise only slightly,
maybe to 20-30ms, while the ath10k will go up more drastically,  like a
continuous 100-150ms or more.  Been meaning to mention it to see if it
should be considered as an issue.

Haven't dug too deeply to quantify this much yet,  and have been going thru
different setup changes, versions of OpenWrt, etc which complicates
things.

Current setup is C7's as dumb AP's, with a Zotac x86 box doing routing.
Have had 17.01.6, 18.06.1, 18.06.2, and recent snapshot on these, with
their various vintages of drivers and associated code.

Couldnt tell you as of now if all of the above versions have the behavior
or not.  Willing to investigate more if its thought important.

[-- Attachment #2: Type: text/html, Size: 1457 bytes --]

Re: [Make-wifi-fast] bloated ath10k, extra latency at lower rates?

[Toke Høiland-Jørgensen]

Jon Pike <jonpike54@gmail.com> writes:

> This might be a good time to mention an observation I've seen that's
> related to this.
>
> I have noticed about the same as reported, 10's down to single digits
> on my ath9k radios, and a similar 20ms-30ms on the ath10k's. Thats at
> high signal strength and the highest data rates. But, I've also
> noticed that it's different if you get farther out in range, and you
> end up on the slower signaling speeds.
>
> Under those conditions, I'm seeing the ath9k latency rise only
> slightly, maybe to 20-30ms, while the ath10k will go up more
> drastically, like a continuous 100-150ms or more. Been meaning to
> mention it to see if it should be considered as an issue.

Well yeah, I guess it should. To answer both you and Adrian: The
debloating of ath10k is only partial currently. It uses the intermediate
queueing system, but it still has quite a bit of buffering in the
firmware, which is not controlled by the driver. The actual latency
induced by this buffering varies by firmware version; but it sounds
quite likely that it would be worse as the rates drop.

The fix for this is to control the queue length in the firmware, similar
to what BQL does for Ethernet drivers. The ChromeOS people at Google
already has this as a patch for ath10k, which we are planning to port
upstream to live inside mac80211 on top of the airtime fairness
mechanism that is in the process of being merged.

So there is some hope that this situation will improve in the future :)

-Toke

Re: [Make-wifi-fast] bloated ath10k, extra latency at lower rates?

[Dave Taht]

Toke Høiland-Jørgensen <toke@redhat.com> writes:

> Jon Pike <jonpike54@gmail.com> writes:
>
>> This might be a good time to mention an observation I've seen that's
>> related to this.
>>
>> I have noticed about the same as reported, 10's down to single digits
>> on my ath9k radios, and a similar 20ms-30ms on the ath10k's. Thats at
>> high signal strength and the highest data rates. But, I've also
>> noticed that it's different if you get farther out in range, and you
>> end up on the slower signaling speeds.
>>
>> Under those conditions, I'm seeing the ath9k latency rise only
>> slightly, maybe to 20-30ms, while the ath10k will go up more
>> drastically, like a continuous 100-150ms or more. Been meaning to
>> mention it to see if it should be considered as an issue.

The C7 has an older version of firmware. It would be interesting to
benchmark the -ct version of the firmware. 

>
> Well yeah, I guess it should. To answer both you and Adrian: The
> debloating of ath10k is only partial currently. It uses the intermediate
> queueing system, but it still has quite a bit of buffering in the
> firmware, which is not controlled by the driver. The actual latency
> induced by this buffering varies by firmware version; but it sounds
> quite likely that it would be worse as the rates drop.
>
> The fix for this is to control the queue length in the firmware, similar
> to what BQL does for Ethernet drivers. The ChromeOS people at Google
> already has this as a patch for ath10k, which we are planning to port
> upstream to live inside mac80211 on top of the airtime fairness
> mechanism that is in the process of being merged.
>
> So there is some hope that this situation will improve in the future :)
>
> -Toke
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast

Re: [Make-wifi-fast] bloated ath10k, extra latency at lower rates?

[Jon Pike]

[-- Attachment #1: Type: text/plain, Size: 1249 bytes --]

Somehow, the prior msg didnt get to the list...    I've had a lot going
on,  so havent done detailed testing.  Did a brief test that resulted in a
lot of weird results, which gave the impression the -ct FW had problems and
worse performance than the basic.  Unexpected.

But,  too many variables...   different router FW versions, different C7 HW
versions, etc,  so not conclusive at this point.  I'll get back to it later.

On Sun, Feb 10, 2019, 3:08 PM Jon Pike <jonpike54@gmail.com> wrote:

> Well,  OK to know that it's more expected behavior rather than a bug,  and
> great that there sounds like an improvement is on the way..
>
> And to Dave's comment, have to check my notes,  but am pretty sure the
> master snapshot branch I ran a few weeks ago had a -ct firmware.  Can't
> remember now if I saw the large distance/latency change on that one.  When
> I get a chance I'll try comparing them. Might take a bit, home internet
> access depends on the HW!
>
> On Sun, Feb 10, 2019, 12:40 PM Dave Taht <dave@taht.net wrote:
>
>> Toke Høiland-Jørgensen <toke@redhat.com> writes:
>> >
>> >
>> The C7 has an older version of firmware. It would be interesting to
>> benchmark the -ct version of the firmware.
>>
>>
>>

[-- Attachment #2: Type: text/html, Size: 2398 bytes --]

Re: [Make-wifi-fast] bloated ath10k, extra latency at lower rates?

[Adrian Popescu]

[-- Attachment #1: Type: text/plain, Size: 1906 bytes --]

Hello,

Lower rates and transitions between rates were things I didn't think of
before.

Something worth a shot is to set DSCP to 0 for all packets which go
out on wlan0 to see if this has any impact on those spikes:
iptables -t mangle -A POSTROUTING -o wlan0 -j DSCP --set-dscp 0

The rule didn't seem to work at all for me. No packets were matching
this.

On Wed, Feb 27, 2019 at 8:24 AM Jon Pike <jonpike54@gmail.com> wrote:

> Somehow, the prior msg didnt get to the list...    I've had a lot going
> on,  so havent done detailed testing.  Did a brief test that resulted in a
> lot of weird results, which gave the impression the -ct FW had problems and
> worse performance than the basic.  Unexpected.
>
> But,  too many variables...   different router FW versions, different C7
> HW versions, etc,  so not conclusive at this point.  I'll get back to it
> later.
>
> On Sun, Feb 10, 2019, 3:08 PM Jon Pike <jonpike54@gmail.com> wrote:
>
>> Well,  OK to know that it's more expected behavior rather than a bug,
>> and great that there sounds like an improvement is on the way..
>>
>> And to Dave's comment, have to check my notes,  but am pretty sure the
>> master snapshot branch I ran a few weeks ago had a -ct firmware.  Can't
>> remember now if I saw the large distance/latency change on that one.  When
>> I get a chance I'll try comparing them. Might take a bit, home internet
>> access depends on the HW!
>>
>> On Sun, Feb 10, 2019, 12:40 PM Dave Taht <dave@taht.net wrote:
>>
>>> Toke Høiland-Jørgensen <toke@redhat.com> writes:
>>> >
>>> >
>>> The C7 has an older version of firmware. It would be interesting to
>>> benchmark the -ct version of the firmware.
>>>
>>>
>>> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast

[-- Attachment #2: Type: text/html, Size: 3579 bytes --]