[Make-wifi-fast] ath9k airtime fairness stabiity issues?

[Make-wifi-fast] ath9k airtime fairness stabiity issues?

[Dave Taht]

Felix:

Was there a bugreport?  (don't see one)

Do you have a specific device or behavior triggering this revert?


On Thu, Jan 5, 2017 at 4:42 AM, Dave Taht <dave.taht@gmail.com> wrote:
> https://github.com/lede-project/source/commit/c296ba834db4ce8c71e0ad7030aab188fe60b27b






-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Make-wifi-fast] ath9k airtime fairness stabiity issues?

[Felix Fietkau]

On 2017-01-05 13:59, Dave Taht wrote:
> Felix:
> 
> Was there a bugreport?  (don't see one)
> 
> Do you have a specific device or behavior triggering this revert?
Not yet. I'm still collecting information, but since we want to release
soon, and I trust at least some of the users that reported these issues
to me, I decided that airtime fairness is not ready for this release yet.

- Felix

Re: [Make-wifi-fast] ath9k airtime fairness stabiity issues?

[Dave Taht]

On Thu, Jan 5, 2017 at 5:02 AM, Felix Fietkau <nbd@nbd.name> wrote:
> On 2017-01-05 13:59, Dave Taht wrote:
>> Felix:
>>
>> Was there a bugreport?  (don't see one)
>>
>> Do you have a specific device or behavior triggering this revert?
> Not yet. I'm still collecting information, but since we want to release
> soon, and I trust at least some of the users that reported these issues
> to me, I decided that airtime fairness is not ready for this release yet.

Well in my case I am seeing what I'd been putting down to
some ipv6 multicast weirdness (using babel and dnsmasq), particularly
over a bridge
(I see several bugs on this filed, I'm still scratching my head on
what I'm seeing)

but otherwise ATF seeeeemed OK on the devices I was testing. (uap-lite, archer)

I guess it would be too much to hope that some builds are archived
enough to a/b, and can assume post jan 4 builds are "- atf"?

(I was *just* rolling out a sizeable test on 12 routers, sigh)


> - Felix



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Make-wifi-fast] ath9k airtime fairness stabiity issues?

[Loganaden Velvindron]

On Thu, Jan 5, 2017 at 4:59 PM, Dave Taht <dave.taht@gmail.com> wrote:
> Felix:
>
> Was there a bugreport?  (don't see one)
>
> Do you have a specific device or behavior triggering this revert?
>
>
> On Thu, Jan 5, 2017 at 4:42 AM, Dave Taht <dave.taht@gmail.com> wrote:
>> https://github.com/lede-project/source/commit/c296ba834db4ce8c71e0ad7030aab188fe60b27b
>
>

Hi nbd & Toke,

Would it be possible to enable it only on platforms like the tp-link
archer c7 v2 and the ubnt, where we have confirmed test reports for
the upcoming release ?

Re: [Make-wifi-fast] ath9k airtime fairness stabiity issues?

[Felix Fietkau]

On 2017-01-05 14:22, Loganaden Velvindron wrote:
> On Thu, Jan 5, 2017 at 4:59 PM, Dave Taht <dave.taht@gmail.com> wrote:
>> Felix:
>>
>> Was there a bugreport?  (don't see one)
>>
>> Do you have a specific device or behavior triggering this revert?
>>
>>
>> On Thu, Jan 5, 2017 at 4:42 AM, Dave Taht <dave.taht@gmail.com> wrote:
>>> https://github.com/lede-project/source/commit/c296ba834db4ce8c71e0ad7030aab188fe60b27b
>>
>>
> 
> Hi nbd & Toke,
> 
> Would it be possible to enable it only on platforms like the tp-link
> archer c7 v2 and the ubnt, where we have confirmed test reports for
> the upcoming release ?
I think it's quite unlikely that these issues are hardware specific.
It's probably more related to the environment, types of clients, or even
traffic patterns.

- Felix

Re: [Make-wifi-fast] ath9k airtime fairness stabiity issues?

[Michal Kazior]

On 5 January 2017 at 14:23, Felix Fietkau <nbd@nbd.name> wrote:
> On 2017-01-05 14:22, Loganaden Velvindron wrote:
>> On Thu, Jan 5, 2017 at 4:59 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>> Felix:
>>>
>>> Was there a bugreport?  (don't see one)
>>>
>>> Do you have a specific device or behavior triggering this revert?
>>>
>>>
>>> On Thu, Jan 5, 2017 at 4:42 AM, Dave Taht <dave.taht@gmail.com> wrote:
>>>> https://github.com/lede-project/source/commit/c296ba834db4ce8c71e0ad7030aab188fe60b27b
>>>
>>>
>>
>> Hi nbd & Toke,
>>
>> Would it be possible to enable it only on platforms like the tp-link
>> archer c7 v2 and the ubnt, where we have confirmed test reports for
>> the upcoming release ?
> I think it's quite unlikely that these issues are hardware specific.
> It's probably more related to the environment, types of clients, or even
> traffic patterns.

Some people are complaining ath10k is unstable for them when
wake_tx_queue is enabled. I suspect the ATF problem in ath9k might be
providing extra opportunities to hit the same bug.

I think RCU is not properly handled. txq_info shares lifecycle of
sta_info and should therefore be protected in the same manner. When
you queue up ieee80211_txq in a driver and use it later you
effectively break RCU. Grabbing rcu_read_lock() *later*, e.g. when
re-scheduling tx is not sufficient to protect from the possible race
of part1/part2 of station destroying logic and driver accessing its
internal txq list.

There seems to be a mechanism to hook up with to fix that already -
drv_sta_pre_rcu_remove().

I've been seldom looking at the ath10k problem and noticed this bit. I
didn't get a chance (and probably won't, any time soon) to take a
closer look, nor test/verify it for that matter.


Michał

Re: [Make-wifi-fast] ath9k airtime fairness stabiity issues?

[Dave Taht]

On Thu, Jan 5, 2017 at 5:23 AM, Felix Fietkau <nbd@nbd.name> wrote:
> On 2017-01-05 14:22, Loganaden Velvindron wrote:
>> On Thu, Jan 5, 2017 at 4:59 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>> Felix:
>>>
>>> Was there a bugreport?  (don't see one)
>>>
>>> Do you have a specific device or behavior triggering this revert?
>>>
>>>
>>> On Thu, Jan 5, 2017 at 4:42 AM, Dave Taht <dave.taht@gmail.com> wrote:
>>>> https://github.com/lede-project/source/commit/c296ba834db4ce8c71e0ad7030aab188fe60b27b
>>>
>>>
>>
>> Hi nbd & Toke,
>>
>> Would it be possible to enable it only on platforms like the tp-link
>> archer c7 v2 and the ubnt, where we have confirmed test reports for
>> the upcoming release ?
> I think it's quite unlikely that these issues are hardware specific.
> It's probably more related to the environment, types of clients, or even
> traffic patterns.

OK, well, I filed a bug. Can I encourage your reporters to chime in on
their issues and what hardware they see them on?

https://bugs.lede-project.org/index.php?do=details&task_id=368

We have extensive testbeds available, but my big one is 2 hours away
right now (and running a version of lede with this patch from october
or so), and hard to upgrade remotely. Been building up a new one in SF
(warmer) this week, just deployed a bunch of "c.h.i.p"s to stress it
out, but now I have to go check to see which builds of lede I
installed where. I'm pretty sure the UAP-lites are running jan 2...
the wndr is jan 4....

(And here I was dreaming up ways to seriously stress out ipv6
multicast (adding tons of routes to babel,
and/or using "uftp" heavily) (see
https://bugs.lede-project.org/index.php?do=details&task_id=253 ))

I'm going to finish getting something up that works and then go blow
up multicast harder, I think, while scratching my head on ATF. At
least I can get some non-atf data on how the chips behave, I guess.


-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Make-wifi-fast] ath9k airtime fairness stabiity issues?

[Dave Taht]

On Thu, Jan 5, 2017 at 6:03 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
> On 5 January 2017 at 14:23, Felix Fietkau <nbd@nbd.name> wrote:
>> On 2017-01-05 14:22, Loganaden Velvindron wrote:
>>> On Thu, Jan 5, 2017 at 4:59 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>>> Felix:
>>>>
>>>> Was there a bugreport?  (don't see one)
>>>>
>>>> Do you have a specific device or behavior triggering this revert?
>>>>
>>>>
>>>> On Thu, Jan 5, 2017 at 4:42 AM, Dave Taht <dave.taht@gmail.com> wrote:
>>>>> https://github.com/lede-project/source/commit/c296ba834db4ce8c71e0ad7030aab188fe60b27b
>>>>
>>>>
>>>
>>> Hi nbd & Toke,
>>>
>>> Would it be possible to enable it only on platforms like the tp-link
>>> archer c7 v2 and the ubnt, where we have confirmed test reports for
>>> the upcoming release ?
>> I think it's quite unlikely that these issues are hardware specific.
>> It's probably more related to the environment, types of clients, or even
>> traffic patterns.
>
> Some people are complaining ath10k is unstable for them when
> wake_tx_queue is enabled. I suspect the ATF problem in ath9k might be
> providing extra opportunities to hit the same bug.

Hmm. I would assume most ath10k users are on a multi-core?

> I think RCU is not properly handled. txq_info shares lifecycle of
> sta_info and should therefore be protected in the same manner. When
> you queue up ieee80211_txq in a driver and use it later you
> effectively break RCU. Grabbing rcu_read_lock() *later*, e.g. when
> re-scheduling tx is not sufficient to protect from the possible race
> of part1/part2 of station destroying logic and driver accessing its
> internal txq list.

Sounds like a promising theory. Most of our testing was on single-core
devices, with the multi-core x86 version being kernel mainline
(4.8ish), and not the lede backport.

I long had mildly poor results in terms of throughput on the apu2 (x86
dual core), but assumed it was due to poor antennas. (no crashes)

The omnia is a dual core arm, but I don't have one of those.

As it turns out the UAP-lite I flashed ~2 days back is crashed right
now, and another box was failing to get dhcp addresses (why I was
looking at multicast), not even over ethernet.

(someone remind me to not take a vacation over the holidays, next time
there's holidays)

> There seems to be a mechanism to hook up with to fix that already -
> drv_sta_pre_rcu_remove().
>
> I've been seldom looking at the ath10k problem and noticed this bit. I
> didn't get a chance (and probably won't, any time soon) to take a
> closer look, nor test/verify it for that matter.
>
>
> Michał



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Make-wifi-fast] [LEDE-DEV] ath9k airtime fairness stabiity issues?

[L. D. Pinney]

[-- Attachment #1: Type: text/plain, Size: 3302 bytes --]

Hello Dave :
Can you please stop cross posting?
It's quite annoying to see this unrelated babble about UN-related ath10k when subject is ath9k
Felix says it a no go on ath9k airtime fairness for the release....AND that's all we really need to know :)

Larry 

    On Thursday, January 5, 2017 11:08 PM, Dave Taht <dave.taht@gmail.com> wrote:
 

 On Thu, Jan 5, 2017 at 6:03 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
> On 5 January 2017 at 14:23, Felix Fietkau <nbd@nbd.name> wrote:
>> On 2017-01-05 14:22, Loganaden Velvindron wrote:
>>> On Thu, Jan 5, 2017 at 4:59 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>>> Felix:
>>>>
>>>> Was there a bugreport?  (don't see one)
>>>>
>>>> Do you have a specific device or behavior triggering this revert?
>>>>
>>>>
>>>> On Thu, Jan 5, 2017 at 4:42 AM, Dave Taht <dave.taht@gmail.com> wrote:
>>>>> https://github.com/lede-project/source/commit/c296ba834db4ce8c71e0ad7030aab188fe60b27b
>>>>
>>>>
>>>
>>> Hi nbd & Toke,
>>>
>>> Would it be possible to enable it only on platforms like the tp-link
>>> archer c7 v2 and the ubnt, where we have confirmed test reports for
>>> the upcoming release ?
>> I think it's quite unlikely that these issues are hardware specific.
>> It's probably more related to the environment, types of clients, or even
>> traffic patterns.
>
> Some people are complaining ath10k is unstable for them when
> wake_tx_queue is enabled. I suspect the ATF problem in ath9k might be
> providing extra opportunities to hit the same bug.

Hmm. I would assume most ath10k users are on a multi-core?

> I think RCU is not properly handled. txq_info shares lifecycle of
> sta_info and should therefore be protected in the same manner. When
> you queue up ieee80211_txq in a driver and use it later you
> effectively break RCU. Grabbing rcu_read_lock() *later*, e.g. when
> re-scheduling tx is not sufficient to protect from the possible race
> of part1/part2 of station destroying logic and driver accessing its
> internal txq list.

Sounds like a promising theory. Most of our testing was on single-core
devices, with the multi-core x86 version being kernel mainline
(4.8ish), and not the lede backport.

I long had mildly poor results in terms of throughput on the apu2 (x86
dual core), but assumed it was due to poor antennas. (no crashes)

The omnia is a dual core arm, but I don't have one of those.

As it turns out the UAP-lite I flashed ~2 days back is crashed right
now, and another box was failing to get dhcp addresses (why I was
looking at multicast), not even over ethernet.

(someone remind me to not take a vacation over the holidays, next time
there's holidays)

> There seems to be a mechanism to hook up with to fix that already -
> drv_sta_pre_rcu_remove().
>
> I've been seldom looking at the ath10k problem and noticed this bit. I
> didn't get a chance (and probably won't, any time soon) to take a
> closer look, nor test/verify it for that matter.
>
>
> Michał



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

_______________________________________________
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


   

[-- Attachment #2: Type: text/html, Size: 6987 bytes --]

Re: [Make-wifi-fast] [LEDE-DEV] ath9k airtime fairness stabiity issues?

[Felix Fietkau]

On 2017-01-05 16:17, L. D. Pinney wrote:
> Hello Dave :
> 
> Can you please stop cross posting?
> 
> It's quite annoying to see this unrelated babble about UN-related ath10k
> when subject is ath9k
> 
> Felix says it a no go on ath9k airtime fairness for the release....AND
> that's all we really need to know :)
Larry, please calm down and read the thread more carefully before
getting annoyed :)

Michal brought up ath10k first, because he thinks the issues might be
related to what we're seeing with ath9k + ATF.

Also, the mailing lists that Dave Cc'd make sense to me and any
discussion that might lead to resolving these issues is highly appreciated.

- Felix

Re: [Make-wifi-fast] [LEDE-DEV] ath9k airtime fairness stabiity issues?

[Weedy]

I just want to chime in that I have felt no negatives and arguable
positives since deploying v1 of the airtime fairness patch.

I'm currently on git master from Dec 29th (so after the compat bump
and v2 fairness merge) and I want to say my Wi-Fi feels better then my
build from v1 timeframe. I got a few laptops (mines ath9k, pretty sure
ones intel, and the last ones broadcom), a few phones (qualcomm
CPUs/radios), and a couple tablets (broadcom). I even deployed a
wireless repeater I made with 2 ath9k cards so wds is turned on too.

I would think airtime fairness is not going to be pulled out of
master, but I really hope we can still ship it to people in 17.01. If
there is anything you want me to try I can do my best to test things.

Re: [Make-wifi-fast] [LEDE-DEV] ath9k airtime fairness stabiity issues?

[Jaap Buurman]

[-- Attachment #1: Type: text/plain, Size: 875 bytes --]

Hi all,

According to Dave Täht the following code disables Airtime Fairness:

for i in /sys/kernel/debug/ieee80211/phy*/*/airtime_flags

do
echo 0 > $i
done


Wouldn't it be a good idea to ship Lede with ATF baked in, but disabled at
default? That would allow end-users to easily enable ATF. Since enabling
ATF takes some work, people without the knowledge wouldn't accidentally
enable this feature and run into possible instability issues. But at the
same time, this would enable tech savvy people to experiment.

There are reports of people running their networks with ATF enabled for
weeks with no issues whatsoever, so if there are instability issues it
might not be as widespread as one would think. There is an interesting
discussion going on here:
https://bugs.lede-project.org/index.php?do=details&task_id=368

I'm curious to your thoughts!

[-- Attachment #2: Type: text/html, Size: 2302 bytes --]