[Cake] [Battlemesh] Wifi Memory limits in small platforms

[Toke Høiland-Jørgensen]

Dave Taht <dave.taht at gmail.com> writes:

> On Thu, Aug 22, 2019 at 11:23 AM Toke Høiland-Jørgensen <toke at redhat.com> wrote:
>>
>> Sebastian Gottschall <s.gottschall at newmedia-net.de> writes:
>>
>> > Am 22.08.2019 um 19:03 schrieb Dave Taht:
>> >> Sebastian Gottschall <s.gottschall at newmedia-net.de> writes:
>> >>
>> >>> Am 22.08.2019 um 15:15 schrieb Dave Taht:
>> >>>> It's very good to know how much folk have been struggling to keep
>> >>>> things from OOMing on 32MB platforms. I'd like to hope that the
>> >>>> unified memory management in cake (vs a collection of QoS qdiscs) and
>> >>>> the new fq_codel for wifi stuff (cutting it down to 1 alloc from four)
>> >>>> help, massively on this issue, but until today I was unaware of how
>> >>>> much the field may have been patching things out.
>> >>>>
>> >>>> The default 32MB memory limits in fq_codel comes from the stressing
>> >>>> about 10GigE networking from google. 4MB is limit in openwrt,
>> >>>> which is suitable for ~1Gbit, and is sort of there  due to 802.11ac's
>> >>>> maximum (impossible to hit) of a txop that large.
>> >> I did kind of conflate "qos + fq_codel" vs wifi in this message. It
>> >> looks like yer staying with me.
>> >>
>> >>>> Something as small as 256K is essentially about 128 full size packets
>> >>>> (and often, acks from an ethernet device's rx ring eat 2k).
>> >>> what i miss in mac80211 is the following option "fq_codel = off"
>> >>> its essential and i will definitly work on a patch to deal with this
>> >>> way for low memory 802.11n platforms.
>> >> Well, it would be my hope that turning it off would A) not help that
>> >> much on memory or cpu and B) show such a dramatic reduction in
>> >> multi-station performance that you'd immediately turn it on again.
>> > isnt it better to have a working platform with less performance than a
>> > crashing platform with no performance?
>> > i mean i can user older mac80211 versions without that issue on a
>> > typical nanostation 2/5 which is often used just as CPE device
>>
>> So before the queueing patches to mac80211, the maximum packet queue
>> size for ath9k was 3MB in total, or 2.2MB if only a single AC was used
>> on the WiFi link (that's 128 packets in the driver + 1000 in the
>> pfifo_fast qdisc * 2074 bytes for the truesize of a full-size packet).
>> Whereas now the default is 4MB for a non-vht device. So it's not
>> actually that big of a difference, and as you've already discovered the
>> defaults can be changed.
>>
>> Would it be helpful to add support for setting the memory limit in
>> hostapd (to avoid having to patch the kernel default)?
>
> hmm. I guess exposing that via netlink, etc is a good idea. Me I just
> write the sys/kernel/debug/*/*/aqm files.

It already is, and you can set it through iw (as I pointed out
up-thread):

iw phy phy0 set txq memory_limit 2097152

But it's not supported in hostapd, so you have to do that manually as it
is now.

> btw:
>
> qos_map in my mind, for APs at this point, should default to the best
> effort queue only. Not sure how to set that in openwrt (I just patched
> it out of the kernel).

Think it's possible to set this in hostapd config; haven't tried it...

-Toke