From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 8C8D53CB36 for ; Thu, 22 Aug 2019 14:23:43 -0400 (EDT) Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7D58B859FE for ; Thu, 22 Aug 2019 18:23:42 +0000 (UTC) Received: by mail-ed1-f69.google.com with SMTP id y15so3827529edu.19 for ; Thu, 22 Aug 2019 11:23:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=8TcLCtS0Gsyqd0Yr5CDr5VZeGHpcZflQq2av39aT9u0=; b=G6lZTi1gmjfUJViD3wyqJSich2EBjOPDymNH/vZ8/jJe0n42TNK3f6Ec41BmOUeNx3 PZL6H082tnnIXE3CgYmLFKpofTk3LoC1fYYmE5GnloCxlPU1reIuLUeB0Ny5sEPJyqGj AqQSQLXL11Rr6cAkyKLLs1ZLuwebW7sOaCt4JKG8M/1NUj+s5xlY/ecp9hABpvWuN/HS fKZhsqybtHKHZQCK2YXguyYjwAbeT5XMWeoJ12wUdwOTHgmDapRI+OiDZXR8FOoB+jap xfUupBlBnQSekqdfz0itO2QJ2bTGMaM58U6OpFisa62+85+sbVfyXiXNOInmMAPJadWp qt9Q== X-Gm-Message-State: APjAAAXfJQbtpiNTEZPjU2+s1Fx2ZV6pgsp4iZvlxwC/jKvNSd344L30 sK1rRyOdNRCbVUSyHJ2WH9RdsBQ7iYFfzZTKbbZycwmgUvfWyZcHgcAp4XavZxAKMCG+8hH9bIj GOyK+u3M6PnmBs0qrQwv50w== X-Received: by 2002:a50:9eab:: with SMTP id a40mr268103edf.20.1566498221263; Thu, 22 Aug 2019 11:23:41 -0700 (PDT) X-Google-Smtp-Source: APXvYqwEvBj9F7SW1R0ZacFYeAqyvnlYEMhqeiIONIVFWlhPUwDUQUlImGdulBr0BhaGTLMj/SAs7A== X-Received: by 2002:a50:9eab:: with SMTP id a40mr268081edf.20.1566498221095; Thu, 22 Aug 2019 11:23:41 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk (borgediget.toke.dk. [85.204.121.218]) by smtp.gmail.com with ESMTPSA id g3sm34625ejj.69.2019.08.22.11.23.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2019 11:23:40 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id BFAA0181CEF; Thu, 22 Aug 2019 20:23:39 +0200 (CEST) From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= To: Sebastian Gottschall , Dave Taht Cc: Dave Taht , Cake List , Battle of the Mesh Mailing List , make-wifi-fast@lists.bufferbloat.net In-Reply-To: References: <87tvabsp99.fsf@toke.dk> <74bccc2b-b805-255f-b6a7-83ade9af6765@newmedia-net.de> <87r25fsn70.fsf@toke.dk> <54438C64-C613-438E-9CB9-6C6D0C5EAFA0@gmail.com> <87sgpvflo4.fsf@taht.net> <87wof6rf7t.fsf@toke.dk> <7656FCDE-C590-4B0C-B191-B9FAC928A762@gmail.com> <5eb4c395-c718-2d28-65a7-9762cf8d5bea@newmedia-net.de> <47AD5102-B66F-44A5-AADE-D167ECB94A61@gmx.de> <1d772664-b6cc-a528-9725-96a431032875@newmedia-net.de> <87v9uqea3x.fsf@taht.net> <87tvaap57q.fsf@toke.dk> <5bbd2b81-9846-3a7a-130c-0f59e04fd2d1@newmedia-net.de> <87ftltdter.fsf@taht.net> X-Clacks-Overhead: GNU Terry Pratchett Date: Thu, 22 Aug 2019 20:23:39 +0200 Message-ID: <87pnkxnjo4.fsf@toke.dk> MIME-Version: 1.0 Content-Type: text/plain Subject: Re: [Cake] Wifi Memory limits in small platforms X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Aug 2019 18:23:43 -0000 Sebastian Gottschall writes: > Am 22.08.2019 um 19:03 schrieb Dave Taht: >> Sebastian Gottschall writes: >> >>> Am 22.08.2019 um 15:15 schrieb Dave Taht: >>>> It's very good to know how much folk have been struggling to keep >>>> things from OOMing on 32MB platforms. I'd like to hope that the >>>> unified memory management in cake (vs a collection of QoS qdiscs) and >>>> the new fq_codel for wifi stuff (cutting it down to 1 alloc from four) >>>> help, massively on this issue, but until today I was unaware of how >>>> much the field may have been patching things out. >>>> >>>> The default 32MB memory limits in fq_codel comes from the stressing >>>> about 10GigE networking from google. 4MB is limit in openwrt, >>>> which is suitable for ~1Gbit, and is sort of there due to 802.11ac's >>>> maximum (impossible to hit) of a txop that large. >> I did kind of conflate "qos + fq_codel" vs wifi in this message. It >> looks like yer staying with me. >> >>>> Something as small as 256K is essentially about 128 full size packets >>>> (and often, acks from an ethernet device's rx ring eat 2k). >>> what i miss in mac80211 is the following option "fq_codel = off" >>> its essential and i will definitly work on a patch to deal with this >>> way for low memory 802.11n platforms. >> Well, it would be my hope that turning it off would A) not help that >> much on memory or cpu and B) show such a dramatic reduction in >> multi-station performance that you'd immediately turn it on again. > isnt it better to have a working platform with less performance than a > crashing platform with no performance? > i mean i can user older mac80211 versions without that issue on a > typical nanostation 2/5 which is often used just as CPE device So before the queueing patches to mac80211, the maximum packet queue size for ath9k was 3MB in total, or 2.2MB if only a single AC was used on the WiFi link (that's 128 packets in the driver + 1000 in the pfifo_fast qdisc * 2074 bytes for the truesize of a full-size packet). Whereas now the default is 4MB for a non-vht device. So it's not actually that big of a difference, and as you've already discovered the defaults can be changed. Would it be helpful to add support for setting the memory limit in hostapd (to avoid having to patch the kernel default)? > but with current mac80211 versions (current means last 2-3 years). they > are just unstable and running out of memory after a while > the only thing which helped was cutting of the memory limit of fq_codel > inside mac80211 > i also have another fancy testunit which is a linksys wrt400 with 32 mb > ram and 2 ath9k based wifi chipsets. no hope here fonr running stable > for only 5 minutes even with a single connection under load (my crashing > test is running a hdtv iptv stream converted to unicast using a > stateless eoip tunnel) > >> I try to encourage folk to run the rtt_fair tests in flent when >> twiddling with wifi. Those really shows how bad things are when you >> don't have ATF + FQ + Per station aggregation and lots of >> clients. Single threaded tests are misleading. > i know but even single threaded tests arent working good on such > devices. so there is no need to talk about the benefits of atf,fq_codel etc. > but there is need to talk about configurable use of it which also allows > to disable it if required. Disabling the fq part won't actually gain you much in terms of memory usage, though, as most of it is packet memory which is already configurable. The one exception to this is the static overhead of 'struct fq_flow', of which mac80211 currently allocates 4k. That's 300k of memory which is currently not configurable. But that could be fixed :) -Toke