From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from webmail.newmedia-net.de (smtps.newmedia-net.de [IPv6:2a05:a1c0:0:de::167]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id CC23F3B29E; Fri, 23 Aug 2019 02:26:49 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=newmedia-net.de; s=mikd; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To:Subject; bh=dmnQ06yCUfAdFV3c0Eg8HE4tfgOvKkgnS7g7l2VNwsc=; b=nCt9CH3X/cy/p+Xa4i/Q/zc0FsdZhaCGUZxIT2uo+q5YkzqA99Fi+TT6F1EqdtTTteCYah4eK+LZEjbTrhoUvO9aYaMet9tkzZGvUS7UHzRLfdxE65e4WkMCsrGcl+nRKMFZge4ZZhtXCWQYix5EYWElxQ8xVoLXyvl+qX3stiU=; To: Dave Taht Cc: Dave Taht , =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vu?= =?UTF-8?Q?sen?= , Cake List , Battle of the Mesh Mailing List , Make-Wifi-fast References: <87sgpvflo4.fsf@taht.net> <87wof6rf7t.fsf@toke.dk> <7656FCDE-C590-4B0C-B191-B9FAC928A762@gmail.com> <5eb4c395-c718-2d28-65a7-9762cf8d5bea@newmedia-net.de> <47AD5102-B66F-44A5-AADE-D167ECB94A61@gmx.de> <1d772664-b6cc-a528-9725-96a431032875@newmedia-net.de> <87v9uqea3x.fsf@taht.net> <87tvaap57q.fsf@toke.dk> <5bbd2b81-9846-3a7a-130c-0f59e04fd2d1@newmedia-net.de> <87ftltdter.fsf@taht.net> <87pnkxnjo4.fsf@toke.dk> <981dd67a-7fb8-1e6a-3e50-6f63a414f1a1@newmedia-net.de> <877e74epnn.fsf@taht.net> From: Sebastian Gottschall Message-ID: <3949f7fd-16d9-4a04-7957-df2972c6d3cb@newmedia-net.de> Date: Fri, 23 Aug 2019 08:25:43 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <877e74epnn.fsf@taht.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Received: from [2003:c9:3f25:1300:6963:8b9e:cb86:ab52] by webmail.newmedia-net.de with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.72) (envelope-from ) id 1i132F-0000Yb-K5; Fri, 23 Aug 2019 08:26:55 +0200 Subject: Re: [Make-wifi-fast] Wifi Memory limits in small platforms X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Aug 2019 06:26:50 -0000 Am 23.08.2019 um 01:39 schrieb Dave Taht: > Sebastian Gottschall writes: > >>>>> but with current mac80211 versions (current means last 2-3 years). they >>>>> are just unstable and running out of memory after a while >>>>> the only thing which helped was cutting of the memory limit of fq_codel >>>>> inside mac80211 >>>>> i also have another fancy testunit which is a linksys wrt400 with 32 mb >>>>> ram and 2 ath9k based wifi chipsets. no hope here fonr running stable >>>>> for only 5 minutes even with a single connection under load (my crashing >>>>> test is running a hdtv iptv stream converted to unicast using a >>>>> stateless eoip tunnel) >>>>> >>>>>> I try to encourage folk to run the rtt_fair tests in flent when >>>>>> twiddling with wifi. Those really shows how bad things are when you >>>>>> don't have ATF + FQ + Per station aggregation and lots of >>>>>> clients. Single threaded tests are misleading. >>>>> i know but even single threaded tests arent working good on such >>>>> devices. so there is no need to talk about the benefits of atf,fq_codel etc. >>>>> but there is need to talk about configurable use of it which also allows >>>>> to disable it if required. >>> I 110% agree that a system that can stay up for years is much better >>> than one that is fast for 5 minutes! >>> >>> However I'd like a chance, in collaborating with you and your upcoming >>> patches - to try and narrow >>> down crash bugs to various subsystems and be able to get some >>> benchmarks done that I simply >>> couldn't do anymore at the financial conclusion of the make-wifi-fast >>> and cake projects. >>> >>> I think I have a lot of gear that is dd-wrt compatible - apu2, >>> wndr3700s, 3800s.... >> if its v4, these are having 128 mb (i have them too). > These are from the cerowrt era, so, 32 or 64MB of ram. > >> and apu2 has 2 >> gb. so its getting real interesting >> if you choose such a bad one with 32 mb ram which are still commonly >> used by "freifunk" > One thing we can start doing more 'round here is to boot the x86 boxes > with mem=32MB or something similar (40% larger due to 64 bits? no idea, > maybe look at free mem on a similar config) to see what shows up. > > For example, one of my APU2s has dual ath9/ath10k cards which is a > a reasonable sim of one of your configs. since x64 have alot more differen configurations the kernels are much bigger (drivers, drivers, drivers) . i'm sure it will not work with just 32 mb > >>> The reduce truesize patch had helped a lot at the time (2012). There >>> were all kinds of flaky bugs that disappeared. >> i tested and it helped to make ethernet unavailable. it worked for > thx for making me chortle in sad empathy. > >> wifi interfaces. but the eth0 and eth1 on my ipq8064 based >> testboard did not work anymore. no dhcp lease, no ping. but i was able >> to capture inbound packets. (qos was not even enabled while testing, >> so no cake, fq_code letc. just standard sfq scheduler) >> so i reverted and all worked again > OK. Thx for trying. there have been so many bugs in gso/gro and hardware > offloads that I figure that that's why the patch was dropped over time. > > is cake's gso-splitting working on that same hardware? I'm not sure > to what extent that reduces packet size or not these days. cake works yes, but i have not checked explicit for gso-splitting. it just worked > > I'll try that again on x86, maybe it needed to pullskb.... can be hw specific. but who knows. > >>> Pico: >>> >>> root@pool2:~# free >>> total used free shared buffers >>> Mem: 28480 23796 4684 92 1868 >>> -/+ buffers: 21928 6552 >>> Swap: 0 0 0 >>> >>> root@pool2:~# uptime >>> 11:38:09 up 43 days, 21:37, load average: 0.04, 0.03, 0.04 >>> >>> Same workload over here, on a wndr3800, almost exactly the same config >>> >>> root@couch:~# free >>> total used free shared buffers cached >>> Mem: 60320 22872 37448 68 1960 6120 >>> -/+ buffers/cache: 14792 45528 >>> Swap: 0 0 0 >> NS2 >> >> root@TRO1:~# free >> >>               total        used        free      shared buff/cache >> available >> Mem:          29124       19228        3552           0 6344        7752 >> Swap:             0           0           0 > It looks like you are running even less stuff than I am. And this > machine is running with 256k bufs? yes. but it may also depend what you're running. i mean openwrt and dd-wrt are very different. the webserver etc. in dd-wrt might be more lightweight. i do not use lua or any other slow component its all written in C including the web code. thats my process list   PID USER       VSZ STAT COMMAND     1 root      1172 S    /sbin/init     2 root         0 SW   [kthreadd]     3 root         0 SW   [ksoftirqd/0]     4 root         0 SW   [kworker/0:0]     5 root         0 SW<  [kworker/0:0H]     6 root         0 SW   [kworker/u2:0]     7 root         0 SW<  [khelper]     8 root         0 SW<  [writeback]    10 root         0 SW<  [crypto]    12 root         0 SW<  [bioset]    64 root         0 SW<  [kblockd]    65 root         0 SW   [kswapd0]    66 root         0 SW   [kworker/0:1]   108 root         0 SW   [fsnotify_mark]   120 root         0 SW<  [deferwq]   121 root         0 SW   [kworker/u2:2]   503 root      1176 S    /sbin/hotplug2 --set-rules-file /etc/hotplug2.rules --persistent   524 root      1856 S    watchdog   553 root         0 SW<  [cfg80211]   574 root      1792 S    /sbin/wlanled -l generic_0:-94 -l generic_1:-80 -l generic_11:-73 -l generic_7:-65   777 root      3780 S    hostapd -B -P /var/run/ath0_hostapd.pid /tmp/ath0_hostap.conf   951 root      1812 S    wland  1025 root       904 S    cron  1083 root      1544 S    resetbutton  1086 root      1856 S    process_monitor  1217 root      1376 S    syslogd -Z -L -R 192.168.0.117  1224 root      1376 S    klogd  1341 root      1112 S    mactelnetd  1456 root      1224 S    dropbear -b /tmp/loginprompt -r /tmp/root/.ssh/ssh_host_rsa_key -p 22 -a  4449 root      3692 S    httpd -p 80 10770 root      1172 S    dnsmasq -u root -g root --conf-file=/tmp/dnsmasq.conf 29786 root      1292 R    dropbear -b /tmp/loginprompt -r /tmp/root/.ssh/ssh_host_rsa_key -p 22 -a 29787 root      1376 S    -sh 29799 root      1376 R    ps > >> wndr3700v4 >> >> root@DD-WRT:~# free >>               total        used        free      shared buff/cache >> available >> Mem:         125884       23048       92940           0 9896       99824 >> Swap:             0           0           0 >> root@DD-WRT:~# >>