From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 25F4B3B29E for ; Thu, 22 Aug 2019 09:22:57 -0400 (EDT) Received: by mail-io1-xd30.google.com with SMTP id 18so11708577ioe.10 for ; Thu, 22 Aug 2019 06:22:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :content-transfer-encoding; bh=oUREiXxdZHefA3g2vaBfT0ZVzbir0QuusaQ9S1XxnOg=; b=dMSpPPhOLVt/3wmX/UYoMwKuH5Zb53zjjgNAWcPOv9sCHPHkEudMRNj/1QqxKl44oc NbIhMK9gErZ+3yGktc6Y7vKbk516ndF2DmwPt96ycrTzdCy98jtps/o9EHqcL3x/VOaJ ODpI4F6xqxI5fPRmUMlrchYFweRExd51eacsiYZcRSwt4TsqLjbIDGmOBdTRnGrSh6w9 iQ0lGKia1VC8XUmPDJGLSzWtYkt2GbooUfpw4SooXCQQJPyj729uy4jLgFqVtf9Mfe6z sqHhDhyY99E/tslMTheuJqwlqxN4EfwUGKeQsXYCdmeyrX/DuF05jz6UUY/lzenyo2TT 8kKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:content-transfer-encoding; bh=oUREiXxdZHefA3g2vaBfT0ZVzbir0QuusaQ9S1XxnOg=; b=e8zGttUXLMtpONAQvKN7PHhvT/ZKgzILoBQdAwpu4J5HuxMD1OkR6se4rmMdQEVqnt N1CVxloivXg7V0GyJpAGxz/mZKOUvzegRETWP4emInZUlHpzJwG9wxLqHTBPZvoAiyr4 v/BMMGW6dUQC6oTswIzVGv9g7WuBTZ6WozMgvLMbq6fjlIXM8Z6RUfpkkCSoFkoUvphs bX+NmgBv+mGGQ5sQ7u5GhsPzruVAbmO1CQxLbUqIWUse7p67VbYJewuEXdW/fbPAzAZz U0Q8cxkr0fV328p0V83teP58+aJx2d9RbZ0pcinHzkAibXVfxYW6ZBMarfDVrUyxtRpo Yl4g== X-Gm-Message-State: APjAAAXfvtgZyHRg/Goy0gT2s3C+a3XcxDEEPU2b3kV+utrPr/K0cLEd J5JbwBQfnlFpIwStHVWz7YKEn0SG4t1ASJqsNxzw3w== X-Google-Smtp-Source: APXvYqxc6kjVYq/0+RHGHMMal/vBzV+VzixEc/Vsrzj/zOAF6LogJMOn5RVFecFPh4yqfZ+QB+MrkayyGk+2M/ysDTI= X-Received: by 2002:a05:6638:45:: with SMTP id a5mr15586759jap.61.1566480176269; Thu, 22 Aug 2019 06:22:56 -0700 (PDT) MIME-Version: 1.0 References: <384866b4-4c91-cf2c-c267-ee4036e5fbf7@newmedia-net.de> <87wof7sriw.fsf@toke.dk> <6782ec15-30eb-63b0-f54f-376c5e6b840b@newmedia-net.de> <87tvabsp99.fsf@toke.dk> <74bccc2b-b805-255f-b6a7-83ade9af6765@newmedia-net.de> <87r25fsn70.fsf@toke.dk> <54438C64-C613-438E-9CB9-6C6D0C5EAFA0@gmail.com> <87sgpvflo4.fsf@taht.net> <87wof6rf7t.fsf@toke.dk> <7656FCDE-C590-4B0C-B191-B9FAC928A762@gmail.com> <5eb4c395-c718-2d28-65a7-9762cf8d5bea@newmedia-net.de> <47AD5102-B66F-44A5-AADE-D167ECB94A61@gmx.de> <1d772664-b6cc-a528-9725-96a431032875@newmedia-net.de> <87v9uqea3x.fsf@taht.net> <87tvaap57q.fsf@toke.dk> <5bbd2b81-9846-3a7a-130c-0f59e04fd2d1@newmedia-net.de> In-Reply-To: From: Dave Taht Date: Thu, 22 Aug 2019 06:22:45 -0700 Message-ID: To: Make-Wifi-fast Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: [Make-wifi-fast] Fwd: Wifi Memory limits in small platforms X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Aug 2019 13:22:57 -0000 ---------- Forwarded message --------- From: Dave Taht Date: Thu, Aug 22, 2019 at 6:15 AM Subject: Wifi Memory limits in small platforms To: Sebastian Gottschall Cc: Toke H=C3=B8iland-J=C3=B8rgensen , Dave Taht , Cake List , Battle of the Mesh Mailing List It's very good to know how much folk have been struggling to keep things from OOMing on 32MB platforms. I'd like to hope that the unified memory management in cake (vs a collection of QoS qdiscs) and the new fq_codel for wifi stuff (cutting it down to 1 alloc from four) help, massively on this issue, but until today I was unaware of how much the field may have been patching things out. The default 32MB memory limits in fq_codel comes from the stressing about 10GigE networking from google. 4MB is limit in openwrt, which is suitable for ~1Gbit, and is sort of there due to 802.11ac's maximum (impossible to hit) of a txop that large. Something as small as 256K is essentially about 128 full size packets (and often, acks from an ethernet device's rx ring eat 2k). The structure of the new fq_codel for wifi subsystem is "one in the hardware, one ready to go, and the rest accumulating". I typically see about 13-20 packets in an aggregate. 256k strikes me as a bit small. I haven't checked, but does this patch still exist in openwrt/dd-wrt? It had helped a lot when under memory pressure from a lot of small packets. https://github.com/dtaht/cerowrt-3.10/blob/master/target/linux/generic/patc= hes-3.10/657-qdisc_reduce_truesize.patch Arguably this could be made more aggressive, but it massively reduced memory burdens at the time I did it when flooding the device, or having lots of acks, and while it cost cpu it saved on ooming. There's two other dubious things in the fq_codel for wifi stack presently. Right now the codel target is set too high for p2p use (20ms, where 6ms seems more right), and it also flips up to a really high target and interval AND turns off ecn when there's more than a few stations available (rather than "active") - it's an overly conservative figure we used back when we had major issues with powersave and multicast that I'd hoped we could cut back to normal after we got another round of research funding and feedback from the field (which didn't happen, and we never got around to making it configurable, and being 25x better than it was before seemed "enough") I was puzzled at battlemesh as to why I had dropping at about 50ms delay rather than ecn, and thought it was something else, and this morning I'm thinking that folk have been reducing the memlimit to 256k rather.... --=20 Dave T=C3=A4ht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740