moeller0 at gmx.de
Fri May 25 14:25:34 EDT 2012
On May 25, 2012, at 4:11 AM, Robert Bradley wrote:
> On 25 May 2012 07:41, Sebastian Moeller <moeller0 at gmx.de> wrote:
>> Hi Robert,
>> since I see the same log file on my router as Jim, I just want to report
>> my observations below.
>> On May 24, 2012, at 11:58 AM, Robert Bradley wrote:
> (re. guest interfaces on wireless)
>>> Are these disabled on your routers at the moment? I suppose in the
>>> worst case you could try setting an explicit channel for both of the
>>> non-mesh guest interfaces and see if the logs clear up (or somehow pass "-L
>>> /dev/null" to babeld).
>> After setting the 2.4GHz channel to 1 instead of auto
>> /tmp/babeld.log still grows with the same entries. And on a WNDR3700v2 there
>> are 30840 KB of tmpfs on /tmp so the babeld.log size of 256KB should not by
>> itself cause the router to crash. That said, while testing this hypothesis
>> by filling most of /tmp (dd if=/dev/zero of=/tmp/delete_me bs=1024
>> count=30000, so that around 340KB stayed free) the router reliably went
>> first into OOM and the rebooted itself. Might it be that the size of the
>> /tmp filesystem is too large if actually used? If I naively add the VSZs of
>> most processes I end up at around 90% of available memory, so worst case
>> there actually only seems to be room for a much smaller /tmp than 30MB. .
>> Maybe restricting /tmp to 6000 KB might make this problem go away (or
>> hooking up a swap device). Does this reasoning sound sane? Once I figure out
>> how to reduce the size of /tmp I will test this.
> Using "mount -o remount -o size=6000k /tmp" should apparently work for
> that. The reasoning sounds good to me, too.
I will go and test that.
> That said, unless we can
> find an obvious reason for /tmp overfilling, I'm not sure we should do
> that, since it will cause problems upgrading.
But if I create a file of 30000 1KB blocks in /tmp (so that around 400 KB stay available), the router goes into OOM, so I do not think that upgrading would work well if it really needs so much memory? I have a hunch that the openwork base under cerowrt does not assume something as big and demanding as the 11MB bind9 named process running :)
> There's also the issue
> that in bug #379, only wireless traffic caused problems. I think that
> even if excessive logs are the problem, the real issue must be
> somewhere within the wireless driver, but I could well be wrong…
Oh I agree the /tmp issue is a tangent, but it does not seem healthy that the router spirals into reboot once /tmp fills up (BTW if I remove my 30000KB file from /tmp while the first OOM is in process the router recovers) My hunch is that the falmost fully instantiated tmpfs takes to o much memory from the system for it to handle its usual business.
On top of that are the wireless issues, say what about a kernel memory leak caused by ath wireless that grows and grows until the problematic /tmp size is in the single digit MBs that starts the spiral to reboot?
> I'm thinking that maybe flooding wireless->wired with UDP traffic for
> 5-10 minutes is the right approach, and then vice-versa (restarting
> the router inbetween?). If there are problems like infinite retries
> or packet memory leaks, that might show them up quickly.
That sounds like the right way to process, except I am no expert at setting netsurf up so that might take a while until I get around to actually test that hypothesis. (Do you by any chance know a publicly available net server process running in the internets to which I could point a local netperf, and do you have any recommendations how to create the UDP flood with netperf ?)
> Robert Bradley
More information about the Cerowrt-devel