From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ig0-f172.google.com (mail-ig0-f172.google.com [209.85.213.172]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id A4FAB21F181 for ; Mon, 27 Jan 2014 13:07:16 -0800 (PST) Received: by mail-ig0-f172.google.com with SMTP id k19so10143775igc.5 for ; Mon, 27 Jan 2014 13:07:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=tVzmzrwHH2bb7MYlH6o/xlgk1aYGVAONDZTouUKe0p0=; b=CPUR+yFgVTtV0haqfIvI8W4PSDurYuH83w7jYH7UtjtnyFU8xZddLymEWQJ7NHfa/z +kM4QW/p0hgbnCcy2V1nNN3cYMJqQL/q1ynql3H3C1X9262Gz3iXGqOvsyOmv9ytECb0 6iBSdQK1QCPXz2CTXts/5bpPKPSIGUjp4xoHKPg3nGrHe395KC2BpRb1rxX9ckUqveGp MIrnrwturBjbJy3bLw8WMDCvJ1JdMF+OvsiyqwUb+PfG1QzLPwCmtzBqehhfYxolricC jWijqhv9lZ7p3Ou5N71OEDACC1d0bD0WZ8T8wMSm5Y7PQT5UxDsSgjfai2IF6eyQ4Yfv 0kJQ== X-Gm-Message-State: ALoCoQmJ2kjAxXDRYcMC6w2yD9BMjPasb9oSFm897BY39Mt98p6vS43Yf1GepvD0Al97F2vwb78J X-Received: by 10.50.176.201 with SMTP id ck9mr19458028igc.46.1390856835256; Mon, 27 Jan 2014 13:07:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.50.60.33 with HTTP; Mon, 27 Jan 2014 13:06:55 -0800 (PST) X-Originating-IP: [162.205.176.51] In-Reply-To: References: From: Steve Jenson Date: Mon, 27 Jan 2014 13:06:55 -0800 Message-ID: To: Dave Taht Content-Type: multipart/alternative; boundary=089e0111e27099733804f0fa179d Cc: "cerowrt-devel@lists.bufferbloat.net" Subject: Re: [Cerowrt-devel] cerowrt issues (3.10.24-8) X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Jan 2014 21:07:16 -0000 --089e0111e27099733804f0fa179d Content-Type: text/plain; charset=ISO-8859-1 On Fri, Jan 24, 2014 at 3:23 PM, Dave Taht wrote: > On Fri, Jan 24, 2014 at 6:08 PM, Steve Jenson > wrote: > > Hi everybody, > > > > I've been using cerowrt as a secondary wifi network (just a single AP for > > now) for a few weeks now. Recently, my wndr3800 got stuck in a bad state > and > > eventually rebooted. I've had this happen a few times now and am looking > for > > ways to debug the issue. I'm new to cerowrt and openwrt so any advice is > > appreciated. > > > > Since I use it as a secondary network, this is no way critical. > > Yea! I appreciate caution before putting alpha software on your gw. > > > I'm not > > looking for free tech support but I couldn't find anything on the wiki > about > > troubleshooting. I'd love to start a page and write some shell scripts to > > diagnose and report issues. I know that a cerowrt router is meant to be a > > research project rather a consumer device but these things seem helpful > > regardless. > > Sure, let me know your wiki account. I have been lax about granting > access of late as the signup process is overrun by spammers. My username is stevej on the wiki. Thanks! > > Please let me know if you'd prefer I not email the list with these > issues or > > if you'd rather I used trac or a different forum. > > > > The list is where most stuff happens. Also in the irc channel. > > If it gets to where it needs to be tracked we have a bugtracker at > > http://www.bufferbloat.net/projects/cerowrt/issues > > The first question I have is: Are you on comcast? Cerowrt > had a dhcpv6-pd implementation that "just worked" from feburary through > december. Regrettably they changed the RA announcement interval > to a really low number around then... and this triggers a firewall reload > every minute on everything prior to the release I point to below. > > If there is a memory leak somewhere that would have triggered it. I am on AT&T ADSL2+ with a Motorola NVG510 modem. > > In this state, I can connect to the cerowrt base station via wifi but am > > unable to route packets to the internet. I can connect to :81 and see the > > login page but logging in results in a lua error at `/cgi-bin/luci` > > > > > > /usr/lib/lua/luci/dispatcher.lua:448: Failed to execute function > > dispatcher target for entry '/'. > > The called action terminated with an exception: > > /usr/lib/lua/luci/sauth.lua:87: Session data invalid! > > stack traceback: > > [C]: in function 'assert' > > /usr/lib/lua/luci/dispatcher.lua:448: in function 'dispatch' > > /usr/lib/lua/luci/dispatcher.lua:195: in function > > > > > > I can ssh into the device and cat various log files until the router > hangs > > and reboots. here's a few relevant lines from my terminal history before > the > > device rebooted (I'm assuming a watchdog kicked in and rebooted it). > > > > root@buffy2-1:~# ping google.com > > ping: bad address 'google.com' > > root@buffy2-1:~# free > > total used free shared buffers > > Mem: 126336 110332 16004 0 5616 > > -/+ buffers: 104716 21620 > > Swap: 0 0 0 > > root@buffy2-1:~# uptime > > 02:08:54 up 2 days, 1:26, load average: 0.10, 0.21, 0.17 > > root@buffy2-1:~# dmesg > > [ 0.000000] Linux version 3.10.24 (cero2@snapon) (gcc version 4.6.4 > > (OpenWrt/Linaro GCC 4.6-2013.05 r38226) ) #1 Tue Dec 24 > > 10:50:15 PST 2013 > > [skipping some lines] > > > > [ 13.156250] Error: Driver 'gpio-keys-polled' is already registered, > > aborting... > > [ 19.414062] IPv6: ADDRCONF(NETDEV_UP): ge00: link is not ready > > [ 19.421875] ar71xx: pll_reg 0xb8050010: 0x11110000 > > [ 19.429687] se00: link up (1000Mbps/Full duplex) > > [ 22.140625] IPv6: ADDRCONF(NETDEV_UP): sw00: link is not ready > > [ 23.351562] IPv6: ADDRCONF(NETDEV_CHANGE): sw00: link becomes ready > > [ 23.757812] ar71xx: pll_reg 0xb8050014: 0x11110000 > > [ 23.757812] ge00: link up (1000Mbps/Full duplex) > > [ 23.773437] IPv6: ADDRCONF(NETDEV_CHANGE): ge00: link becomes ready > > > > root@buffy2-1:~# ifconfig > > ge00 Link encap:Ethernet HWaddr 2C:B0:5D:A0:C5:B1 > > inet addr:192.168.1.138 Bcast:192.168.1.255 > Mask:255.255.255.0 > > inet6 addr: fe80::2eb0:5dff:fea0:c5b1/64 Scope:Link > > inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64 > Scope:Global > > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > > RX packets:1469670 errors:0 dropped:8 overruns:0 frame:0 > > TX packets:547733 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:1000 > > RX bytes:229243410 (218.6 MiB) TX bytes:57304808 (54.6 MiB) > > Interrupt:5 > > > > lo Link encap:Local Loopback > > inet addr:127.0.0.1 Mask:255.0.0.0 > > inet6 addr: ::1/128 Scope:Host > > UP LOOPBACK RUNNING MTU:65536 Metric:1 > > RX packets:23689 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:23689 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:0 > > RX bytes:2612713 (2.4 MiB) TX bytes:2612713 (2.4 MiB) > > > > pimreg Link encap:UNSPEC HWaddr > > 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 > > UP RUNNING NOARP MTU:1472 Metric:1 > > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:0 > > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > > > > se00 Link encap:Ethernet HWaddr 2E:B0:5D:A0:C5:B0 > > inet addr:172.30.42.1 Bcast:172.30.42.31 Mask:255.255.255.224 > > inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64 > Scope:Global > > How are you assigning your ipv6 addresses? > It's been a while since I messed with this but I think IPv6 is assigned thanks to 6relayd? My modem has IPv6 enabled but no DHCPv6 options that I can find. Here's how cerowrt is configured. root@buffy2-1:/overlay/etc/config# cat 6relayd config server 'default' option fallback_relay 'rd dhcpv6 ndp' list network 'ge00' list network 'ge01' list network 'gw00' list network 'gw01' list network 'gw10' list network 'gw11' list network 'se00' list network 'sw00' list network 'sw10' option rd 'relay' option dhcpv6 'relay' option ndp 'relay' option master 'ge00' > > inet6 addr: fe80::2cb0:5dff:fea0:c5b0/64 Scope:Link > > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:191740 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:1000 > > RX bytes:0 (0.0 B) TX bytes:42184988 (40.2 MiB) > > Interrupt:4 > > > > sw00 Link encap:Ethernet HWaddr 2C:B0:5D:A0:C5:B0 > > inet addr:172.30.42.65 Bcast:172.30.42.95 > Mask:255.255.255.224 > > inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64 > Scope:Global > > inet6 addr: fe80::2eb0:5dff:fea0:c5b0/64 Scope:Link > > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > > RX packets:70239 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:286967 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:1000 > > RX bytes:15590189 (14.8 MiB) TX bytes:127357293 (121.4 MiB) > > > > root@buffy2-1:~# less /var/log/babeld.log > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > send: Cannot assign requested address > > send: Cannot assign requested address > > send: Cannot assign requested address > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > netlink_read: recvmsg(): No buffer space available > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > Couldn't determine channel of interface sw00: Invalid argument. > > This is a problem in babel detecting the channel on a "normal" > rather than a mesh interface. It's bugged me a long while, but > haven't got around to finding what triggers it. Might "fix" it by > acquiring the channel at babel start time from /etc/config/wireless. > > It messes up the diversity routing calculation, grump. > > There is a possibility a logfile got really big, but this one > generally doesn't, but I should turn off logging in some > future release... I believe I've tracked down part of what's going on. It looks like my tmpfs is filling up 100% and then the device enters a bad state: After 24 hours, with tmpfs at 50%, babeld.log is the largest file by far in tmpfs and the only file that appears to be growing (based on `du`). It takes about 48 hours from reboot to fill up tmpfs on my device. # sort babeld.log | uniq -c |sort -rn |head 503236 Couldn't determine channel of interface sw00: Invalid argument. 1376 netlink_read: recvmsg(): No buffer space available 3 send: Cannot assign requested address # wc -l babeld.log 504617 babeld.log I sped up system failure by using `dd` to fill up tmpfs and the system became immediately unusable. This also explains the luci session store errors as sessions are stored in tmpfs. The other buffer issues may or may not be related to this. Best, Steve --089e0111e27099733804f0fa179d Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On F= ri, Jan 24, 2014 at 3:23 PM, Dave Taht <dave.taht@gmail.com> wrote:
On Fri, Jan 24, 2014 at 6:08 PM, Steve Jenson <stevej@fruitless.org= > wrote:
> Hi everybody,
>
> I've been using cerowrt as a secondary wifi network (just a single= AP for
> now) for a few weeks now. Recently, my wndr3800 got stuck in a bad sta= te and
> eventually rebooted. I've had this happen a few times now and am l= ooking for
> ways to debug the issue. I'm new to cerowrt and openwrt so any adv= ice is
> appreciated.
>
> Since I use it as a secondary network, this is no way critical.

Yea! I appreciate caution before putting alpha software on your gw.

> I'm not
> looking for free tech support but I couldn't find anything on the = wiki about
> troubleshooting. I'd love to start a page and write some shell scr= ipts to
> diagnose and report issues. I know that a cerowrt router is meant to b= e a
> research project rather a consumer device but these things seem helpfu= l
> regardless.

Sure, let me know your wiki account. I have been lax about granting access of late as the signup process is overrun by spammers.

My username is stevej on the wiki. Thanks!=A0
<= br>
=A0
> Please let me know if you'd prefer I not email the list with these= issues or
> if you'd rather I used trac or a different forum.
>

The list is where most stuff happens. Also in the irc channel.

If it gets to where it needs to be tracked we have a bugtracker at

http://www.bufferbloat.net/projects/cerowrt/issues

The first question I have is: Are you on comcast? Cerowrt
had a dhcpv6-pd implementation that "just worked" from feburary t= hrough
december. Regrettably they changed the RA announcement interval
to a really low number around then... and this triggers a firewall reload every minute on everything prior to the release I point to below.

If there is a memory leak somewhere that would have triggered it.

I am on AT&T ADSL2+ with a Motorola NVG510 modem= .=A0

=A0
> In this state, I can connect to the cerowrt base station via= wifi but am
> unable to route packets to the internet. I can connect to :81 and see = the
> login page but logging in results in a lua error at `/cgi-bin/luci` >
>
> =A0 /usr/lib/lua/luci/dispatcher.lua:448: Failed to execute function > dispatcher target for entry '/'.
> =A0 The called action terminated with an exception:
> =A0 /usr/lib/lua/luci/sauth.lua:87: Session data invalid!
> =A0 stack traceback:
> =A0 [C]: in function 'assert'
> =A0 /usr/lib/lua/luci/dispatcher.lua:448: in function 'dispatch= 9;
> =A0 /usr/lib/lua/luci/dispatcher.lua:195: in function
> </usr/lib/lua/luci/dispatcher.lua:194>
>
> I can ssh into the device and cat various log files until the router h= angs
> and reboots. here's a few relevant lines from my terminal history = before the
> device rebooted (I'm assuming a watchdog kicked in and rebooted it= ).
>
> root@buffy2-1:~# ping = google.com
> ping: bad address 'google.com'
> root@buffy2-1:~# free
> =A0 =A0 =A0 =A0 =A0 =A0 =A0total =A0 =A0 =A0 =A0 used =A0 =A0 =A0 =A0 = free =A0 =A0 =A0 shared =A0 =A0 =A0buffers
> Mem: =A0 =A0 =A0 =A0126336 =A0 =A0 =A0 110332 =A0 =A0 =A0 =A016004 =A0= =A0 =A0 =A0 =A0 =A00 =A0 =A0 =A0 =A0 5616
> -/+ buffers: =A0 =A0 =A0 =A0 =A0 =A0 104716 =A0 =A0 =A0 =A021620
> Swap: =A0 =A0 =A0 =A0 =A0 =A00 =A0 =A0 =A0 =A0 =A0 =A00 =A0 =A0 =A0 = =A0 =A0 =A00
> root@buffy2-1:~# uptime
> =A002:08:54 up 2 days, =A01:26, =A0load average: 0.10, 0.21, 0.17
> root@buffy2-1:~# dmesg
> [ =A0 =A00.000000] Linux version 3.10.24 (cero2@snapon) (gcc version 4= .6.4
> (OpenWrt/Linaro GCC 4.6-2013.05 r38226) ) #1 Tue Dec 24
> 10:50:15 PST 2013
> [skipping some lines]
>
> [ =A0 13.156250] Error: Driver 'gpio-keys-polled' is already r= egistered,
> aborting...
> [ =A0 19.414062] IPv6: ADDRCONF(NETDEV_UP): ge00: link is not ready > [ =A0 19.421875] ar71xx: pll_reg 0xb8050010: 0x11110000
> [ =A0 19.429687] se00: link up (1000Mbps/Full duplex)
> [ =A0 22.140625] IPv6: ADDRCONF(NETDEV_UP): sw00: link is not ready > [ =A0 23.351562] IPv6: ADDRCONF(NETDEV_CHANGE): sw00: link becomes rea= dy
> [ =A0 23.757812] ar71xx: pll_reg 0xb8050014: 0x11110000
> [ =A0 23.757812] ge00: link up (1000Mbps/Full duplex)
> [ =A0 23.773437] IPv6: ADDRCONF(NETDEV_CHANGE): ge00: link becomes rea= dy
>
> root@buffy2-1:~# ifconfig
> ge00 =A0 =A0 =A0Link encap:Ethernet =A0HWaddr 2C:B0:5D:A0:C5:B1
> =A0 =A0 =A0 =A0 =A0 inet addr:192.168.1.138 =A0Bcast:192.168.1.255 =A0= Mask:255.255.255.0
> =A0 =A0 =A0 =A0 =A0 inet6 addr: fe80::2eb0:5dff:fea0:c5b1/64 Scope:Lin= k
> =A0 =A0 =A0 =A0 =A0 inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/= 64 Scope:Global
> =A0 =A0 =A0 =A0 =A0 UP BROADCAST RUNNING MULTICAST =A0MTU:1500 =A0Metr= ic:1
> =A0 =A0 =A0 =A0 =A0 RX packets:1469670 errors:0 dropped:8 overruns:0 f= rame:0
> =A0 =A0 =A0 =A0 =A0 TX packets:547733 errors:0 dropped:0 overruns:0 ca= rrier:0
> =A0 =A0 =A0 =A0 =A0 collisions:0 txqueuelen:1000
> =A0 =A0 =A0 =A0 =A0 RX bytes:229243410 (218.6 MiB) =A0TX bytes:5730480= 8 (54.6 MiB)
> =A0 =A0 =A0 =A0 =A0 Interrupt:5
>
> lo =A0 =A0 =A0 =A0Link encap:Local Loopback
> =A0 =A0 =A0 =A0 =A0 inet addr:127.0.0.1 =A0Mask:255.0.0.0
> =A0 =A0 =A0 =A0 =A0 inet6 addr: ::1/128 Scope:Host
> =A0 =A0 =A0 =A0 =A0 UP LOOPBACK RUNNING =A0MTU:65536 =A0Metric:1
> =A0 =A0 =A0 =A0 =A0 RX packets:23689 errors:0 dropped:0 overruns:0 fra= me:0
> =A0 =A0 =A0 =A0 =A0 TX packets:23689 errors:0 dropped:0 overruns:0 car= rier:0
> =A0 =A0 =A0 =A0 =A0 collisions:0 txqueuelen:0
> =A0 =A0 =A0 =A0 =A0 RX bytes:2612713 (2.4 MiB) =A0TX bytes:2612713 (2.= 4 MiB)
>
> pimreg =A0 =A0Link encap:UNSPEC =A0HWaddr
> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
> =A0 =A0 =A0 =A0 =A0 UP RUNNING NOARP =A0MTU:1472 =A0Metric:1
> =A0 =A0 =A0 =A0 =A0 RX packets:0 errors:0 dropped:0 overruns:0 frame:0=
> =A0 =A0 =A0 =A0 =A0 TX packets:0 errors:0 dropped:0 overruns:0 carrier= :0
> =A0 =A0 =A0 =A0 =A0 collisions:0 txqueuelen:0
> =A0 =A0 =A0 =A0 =A0 RX bytes:0 (0.0 B) =A0TX bytes:0 (0.0 B)
>
> se00 =A0 =A0 =A0Link encap:Ethernet =A0HWaddr 2E:B0:5D:A0:C5:B0
> =A0 =A0 =A0 =A0 =A0 inet addr:172.30.42.1 =A0Bcast:172.30.42.31 =A0Mas= k:255.255.255.224
> =A0 =A0 =A0 =A0 =A0 inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/= 64 Scope:Global

How are you assigning your ipv6 addresses?

It's been a while since I messed with this but I think = IPv6 is assigned thanks to 6relayd? My modem has IPv6 enabled but no DHCPv6= options that I can find. Here's how cerowrt is configured.

root@buffy2-1:/overlay/etc/config# cat 6relayd
conf= ig server 'default'
=A0 option fallback_relay 'rd dhcpv6 ndp= '
=A0 list network 'ge00'
=A0 list network 'ge01'=
=A0 list network 'gw00'
=A0 list network 'gw01'
=A0 l= ist network 'gw10'
=A0 list network 'gw11'
=A0 list n= etwork 'se00'
=A0 list network 'sw00'
=A0 list networ= k 'sw10'
=A0 option rd 'relay'
=A0 option dhcpv6 'relay'
=A0 o= ption ndp 'relay'
=A0 option master 'ge00'


> =A0 =A0 =A0 =A0 =A0 inet6 addr: fe80::2cb0:5dff:fea0:c5b0/64 Scope:Lin= k
> =A0 =A0 =A0 =A0 =A0 UP BROADCAST RUNNING MULTICAST =A0MTU:1500 =A0Metr= ic:1
> =A0 =A0 =A0 =A0 =A0 RX packets:0 errors:0 dropped:0 overruns:0 frame:0=
> =A0 =A0 =A0 =A0 =A0 TX packets:191740 errors:0 dropped:0 overruns:0 ca= rrier:0
> =A0 =A0 =A0 =A0 =A0 collisions:0 txqueuelen:1000
> =A0 =A0 =A0 =A0 =A0 RX bytes:0 (0.0 B) =A0TX bytes:42184988 (40.2 MiB)=
> =A0 =A0 =A0 =A0 =A0 Interrupt:4
>
> sw00 =A0 =A0 =A0Link encap:Ethernet =A0HWaddr 2C:B0:5D:A0:C5:B0
> =A0 =A0 =A0 =A0 =A0 inet addr:172.30.42.65 =A0Bcast:172.30.42.95 =A0Ma= sk:255.255.255.224
> =A0 =A0 =A0 =A0 =A0 inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/= 64 Scope:Global
> =A0 =A0 =A0 =A0 =A0 inet6 addr: fe80::2eb0:5dff:fea0:c5b0/64 Scope:Lin= k
> =A0 =A0 =A0 =A0 =A0 UP BROADCAST RUNNING MULTICAST =A0MTU:1500 =A0Metr= ic:1
> =A0 =A0 =A0 =A0 =A0 RX packets:70239 errors:0 dropped:0 overruns:0 fra= me:0
> =A0 =A0 =A0 =A0 =A0 TX packets:286967 errors:0 dropped:0 overruns:0 ca= rrier:0
> =A0 =A0 =A0 =A0 =A0 collisions:0 txqueuelen:1000
> =A0 =A0 =A0 =A0 =A0 RX bytes:15590189 (14.8 MiB) =A0TX bytes:127357293= (121.4 MiB)
>
> root@buffy2-1:~# less /var/log/babeld.log
> Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > send: Cannot assign requested address
> send: Cannot assign requested address
> send: Cannot assign requested address
> Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > netlink_read: recvmsg(): No buffer space available
> Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument. > Couldn't determine channel of interface sw00: Invalid argument.
This is a problem in babel detecting the channel on a "nor= mal"
rather than a mesh interface. It's bugged me a long while, but
haven't got around to finding what triggers it. Might "fix" i= t by
acquiring the channel at babel start time from /etc/config/wireless.

It messes up the diversity routing calculation, grump.

There is a possibility a logfile got really big, but this one
generally doesn't, but I should turn off logging in some
future release...

I believe I've tracke= d down part of what's going on. It looks like my tmpfs is filling up 10= 0% and then the device enters a bad state:

After 2= 4 hours, with tmpfs at 50%, babeld.log is the largest file by far in tmpfs = and the only file that appears to be growing (based on `du`). It takes abou= t 48 hours from reboot to fill up tmpfs on my device.

#=A0 sort babeld.log | uniq -c |sort -rn |head

<= div>

=A0503236 Couldn't determine channel of interface sw00: I= nvalid argument.

=A0=A0 1376 netlink_read: recvmsg(): No buffer space availabl= e

=A0 =A0 =A0 3 send: Cannot assign requested address

=

# wc -l babeld.log

504617 babeld.l= og

I sped up system failure by using `dd` to fill up tm= pfs and the system became immediately unusable.

This also explains the luci session store errors as ses= sions are stored in tmpfs.

The other buffer issues= may or may not be related to this.

Best,
Steve
--089e0111e27099733804f0fa179d--