[Cerowrt-devel] cerowrt issues (3.10.24-8)

[Cerowrt-devel] cerowrt issues (3.10.24-8)

[Steve Jenson]

[-- Attachment #1: Type: text/plain, Size: 13280 bytes --]

Hi everybody,

I've been using cerowrt as a secondary wifi network (just a single AP for
now) for a few weeks now. Recently, my wndr3800 got stuck in a bad state
and eventually rebooted. I've had this happen a few times now and am
looking for ways to debug the issue. I'm new to cerowrt and openwrt so any
advice is appreciated.

Since I use it as a secondary network, this is no way critical. I'm not
looking for free tech support but I couldn't find anything on the wiki
about troubleshooting. I'd love to start a page and write some shell
scripts to diagnose and report issues. I know that a cerowrt router is
meant to be a research project rather a consumer device but these things
seem helpful regardless.

Please let me know if you'd prefer I not email the list with these issues
or if you'd rather I used trac or a different forum.


--

In this state, I can connect to the cerowrt base station via wifi but am
unable to route packets to the internet. I can connect to :81 and see the
login page but logging in results in a lua error at `/cgi-bin/luci`


  /usr/lib/lua/luci/dispatcher.lua:448: Failed to execute function
dispatcher target for entry '/'.
  The called action terminated with an exception:
  /usr/lib/lua/luci/sauth.lua:87: Session data invalid!
  stack traceback:
  [C]: in function 'assert'
  /usr/lib/lua/luci/dispatcher.lua:448: in function 'dispatch'
  /usr/lib/lua/luci/dispatcher.lua:195: in function
</usr/lib/lua/luci/dispatcher.lua:194>

I can ssh into the device and cat various log files until the router hangs
and reboots. here's a few relevant lines from my terminal history before
the device rebooted (I'm assuming a watchdog kicked in and rebooted it).

root@buffy2-1:~# ping google.com
ping: bad address 'google.com'
root@buffy2-1:~# free
             total         used         free       shared      buffers
Mem:        126336       110332        16004            0         5616
-/+ buffers:             104716        21620
Swap:            0            0            0
root@buffy2-1:~# uptime
 02:08:54 up 2 days,  1:26,  load average: 0.10, 0.21, 0.17
root@buffy2-1:~# dmesg
[    0.000000] Linux version 3.10.24 (cero2@snapon) (gcc version 4.6.4
(OpenWrt/Linaro GCC 4.6-2013.05 r38226) ) #1 Tue Dec 24
10:50:15 PST 2013
[skipping some lines]

[   13.156250] Error: Driver 'gpio-keys-polled' is already registered,
aborting...
[   19.414062] IPv6: ADDRCONF(NETDEV_UP): ge00: link is not ready
[   19.421875] ar71xx: pll_reg 0xb8050010: 0x11110000
[   19.429687] se00: link up (1000Mbps/Full duplex)
[   22.140625] IPv6: ADDRCONF(NETDEV_UP): sw00: link is not ready
[   23.351562] IPv6: ADDRCONF(NETDEV_CHANGE): sw00: link becomes ready
[   23.757812] ar71xx: pll_reg 0xb8050014: 0x11110000
[   23.757812] ge00: link up (1000Mbps/Full duplex)
[   23.773437] IPv6: ADDRCONF(NETDEV_CHANGE): ge00: link becomes ready

root@buffy2-1:~# ifconfig
ge00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B1
          inet addr:192.168.1.138  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::2eb0:5dff:fea0:c5b1/64 Scope:Link
          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1469670 errors:0 dropped:8 overruns:0 frame:0
          TX packets:547733 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:229243410 (218.6 MiB)  TX bytes:57304808 (54.6 MiB)
          Interrupt:5

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:23689 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23689 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2612713 (2.4 MiB)  TX bytes:2612713 (2.4 MiB)

pimreg    Link encap:UNSPEC  HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          UP RUNNING NOARP  MTU:1472  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

se00      Link encap:Ethernet  HWaddr 2E:B0:5D:A0:C5:B0
          inet addr:172.30.42.1  Bcast:172.30.42.31  Mask:255.255.255.224
          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64 Scope:Global
          inet6 addr: fe80::2cb0:5dff:fea0:c5b0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:191740 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:42184988 (40.2 MiB)
          Interrupt:4

sw00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B0
          inet addr:172.30.42.65  Bcast:172.30.42.95  Mask:255.255.255.224
          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64 Scope:Global
          inet6 addr: fe80::2eb0:5dff:fea0:c5b0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:70239 errors:0 dropped:0 overruns:0 frame:0
          TX packets:286967 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:15590189 (14.8 MiB)  TX bytes:127357293 (121.4 MiB)

root@buffy2-1:~# less /var/log/babeld.log
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
send: Cannot assign requested address
send: Cannot assign requested address
send: Cannot assign requested address
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
netlink_read: recvmsg(): No buffer space available
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
Couldn't determine channel of interface sw00: Invalid argument.
</quote>

and then the system hang. Note: the first time I ran 'less
/var/log/babeld.log', it was Killed. I would assume that's an OOM
killer?

I'm running cerowrt 3.10.24-8 on a wndr3800. I have a spare wndr3800
that I'm getting ready to flash with the same version and I can see if I get
the same issue. What's the best way to clone a configuration?

---

Here are some log lines from a separate instance of the same problem, I
recovered these log lines from the web server (I was able to login to
the web interface for another hour until I got the same lua exception
as in the beginning of this email)

This time I was completely unable to connect to wifi from my iPhone
and you can see where hostapd disassociates the iPhone soon after
it connects.

the kernel log:

[158995.550781] icmp6_send: no reply to icmp error

[160930.527343] ath: phy0: Failed to stop TX DMA, queues=0x004!

[160930.542968] ath: phy0: DMA failed to stop in 10 ms AR_CR=0x00000024
AR_DIAG_SW=0x42000020 DMADBG_7=0x000084c0

[160930.554687] ath: phy0: Could not stop RX, we could be confusing the DMA
engine when we start RX up

and in the system log:

Fri Jan 24 15:51:15 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0):
No buffer space available

Fri Jan 24 15:51:15 2014 daemon.err avahi-daemon[1273]: netlink.c:
recvmsg() failed: No buffer space available

Fri Jan 24 15:52:18 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0):
No buffer space available


root@buffy2-1:/tmp/log# free

             total         used         free       shared      buffers

Mem:        126336       110128        16208            0         5616

-/+ buffers:             104512        21824

Swap:            0            0            0


Fri Jan 24 17:14:53 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0):
No buffer space available

Fri Jan 24 17:15:53 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0):
No buffer space available

Fri Jan 24 17:16:18 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
IEEE 802.11: authenticated

Fri Jan 24 17:16:18 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
IEEE 802.11: associated (aid 1)

Fri Jan 24 17:16:18 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
RADIUS: starting accounting session 52DC70A7-00000010

Fri Jan 24 17:16:18 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
WPA: pairwise key handshake completed (RSN)

Fri Jan 24 17:16:18 2014 daemon.info dnsmasq-dhcp[4999]: DHCPREQUEST(sw00)
192.168.1.114 90:72:40:e9:9c:b6

Fri Jan 24 17:16:18 2014 daemon.info dnsmasq-dhcp[4999]: DHCPNAK(sw00)
192.168.1.114 90:72:40:e9:9c:b6 wrong network

Fri Jan 24 17:16:21 2014 daemon.info dnsmasq-dhcp[4999]: DHCPDISCOVER(sw00)
90:72:40:e9:9c:b6

Fri Jan 24 17:16:21 2014 daemon.info dnsmasq-dhcp[4999]: DHCPOFFER(sw00)
172.30.42.82 90:72:40:e9:9c:b6

Fri Jan 24 17:16:21 2014 daemon.info dnsmasq-dhcp[4999]: DHCPDISCOVER(sw00)
90:72:40:e9:9c:b6

Fri Jan 24 17:16:21 2014 daemon.info dnsmasq-dhcp[4999]: DHCPOFFER(sw00)
172.30.42.82 90:72:40:e9:9c:b6

Fri Jan 24 17:16:22 2014 daemon.info dnsmasq-dhcp[4999]: DHCPREQUEST(sw00)
172.30.42.82 90:72:40:e9:9c:b6

Fri Jan 24 17:16:22 2014 daemon.info dnsmasq-dhcp[4999]: DHCPACK(sw00)
172.30.42.82 90:72:40:e9:9c:b6 stevejs-iPhone

Fri Jan 24 17:16:35 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
IEEE 802.11: disassociated

Fri Jan 24 17:16:36 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)

Fri Jan 24 17:16:57 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0):
No buffer space available

Fri Jan 24 17:17:59 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0):
No buffer space available

Fri Jan 24 17:19:04 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0):
No buffer space available

Fri Jan 24 17:19:04 2014 daemon.err avahi-daemon[1273]: netlink.c:
recvmsg() failed: No buffer space available

Fri Jan 24 17:19:45 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
IEEE 802.11: authenticated

Fri Jan 24 17:19:45 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
IEEE 802.11: associated (aid 1)

Fri Jan 24 17:19:45 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
RADIUS: starting accounting session 52DC70A7-00000011

Fri Jan 24 17:19:45 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
WPA: pairwise key handshake completed (RSN)

Fri Jan 24 17:19:45 2014 daemon.info dnsmasq-dhcp[4999]: DHCPDISCOVER(sw00)
90:72:40:e9:9c:b6

Fri Jan 24 17:19:45 2014 daemon.info dnsmasq-dhcp[4999]: DHCPOFFER(sw00)
172.30.42.82 90:72:40:e9:9c:b6

Fri Jan 24 17:19:47 2014 daemon.info dnsmasq-dhcp[4999]: DHCPREQUEST(sw00)
172.30.42.82 90:72:40:e9:9c:b6

Fri Jan 24 17:19:47 2014 daemon.info dnsmasq-dhcp[4999]: DHCPACK(sw00)
172.30.42.82 90:72:40:e9:9c:b6 stevejs-iPhone

Fri Jan 24 17:20:01 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
IEEE 802.11: disassociated

Fri Jan 24 17:20:02 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)

Fri Jan 24 17:20:09 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0):
No buffer space available

Fri Jan 24 17:21:10 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0):
No buffer space available

and then it hung and rebooted.

I bought this particular wndr3800 used so it's possible I have a bad piece
of hardware. Are there any hardware testing scripts or anything like
memtest86 for these devices?  Are there entries in /sys that I should be
looking at? I've noticed about 9k unaligned_instructions on boot but that
never grows.

Thanks!
Steve Jenson
@stevej

[-- Attachment #2: Type: text/html, Size: 16203 bytes --]

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[Dave Taht]

On Fri, Jan 24, 2014 at 6:08 PM, Steve Jenson <stevej@fruitless.org> wrote:
> Hi everybody,
>
> I've been using cerowrt as a secondary wifi network (just a single AP for
> now) for a few weeks now. Recently, my wndr3800 got stuck in a bad state and
> eventually rebooted. I've had this happen a few times now and am looking for
> ways to debug the issue. I'm new to cerowrt and openwrt so any advice is
> appreciated.
>
> Since I use it as a secondary network, this is no way critical.

Yea! I appreciate caution before putting alpha software on your gw.

> I'm not
> looking for free tech support but I couldn't find anything on the wiki about
> troubleshooting. I'd love to start a page and write some shell scripts to
> diagnose and report issues. I know that a cerowrt router is meant to be a
> research project rather a consumer device but these things seem helpful
> regardless.

Sure, let me know your wiki account. I have been lax about granting
access of late as the signup process is overrun by spammers.

> Please let me know if you'd prefer I not email the list with these issues or
> if you'd rather I used trac or a different forum.
>

The list is where most stuff happens. Also in the irc channel.

If it gets to where it needs to be tracked we have a bugtracker at

http://www.bufferbloat.net/projects/cerowrt/issues

The first question I have is: Are you on comcast? Cerowrt
had a dhcpv6-pd implementation that "just worked" from feburary through
december. Regrettably they changed the RA announcement interval
to a really low number around then... and this triggers a firewall reload
every minute on everything prior to the release I point to below.

If there is a memory leak somewhere that would have triggered it.

> --
>
> In this state, I can connect to the cerowrt base station via wifi but am
> unable to route packets to the internet. I can connect to :81 and see the
> login page but logging in results in a lua error at `/cgi-bin/luci`
>
>
>   /usr/lib/lua/luci/dispatcher.lua:448: Failed to execute function
> dispatcher target for entry '/'.
>   The called action terminated with an exception:
>   /usr/lib/lua/luci/sauth.lua:87: Session data invalid!
>   stack traceback:
>   [C]: in function 'assert'
>   /usr/lib/lua/luci/dispatcher.lua:448: in function 'dispatch'
>   /usr/lib/lua/luci/dispatcher.lua:195: in function
> </usr/lib/lua/luci/dispatcher.lua:194>
>
> I can ssh into the device and cat various log files until the router hangs
> and reboots. here's a few relevant lines from my terminal history before the
> device rebooted (I'm assuming a watchdog kicked in and rebooted it).
>
> root@buffy2-1:~# ping google.com
> ping: bad address 'google.com'
> root@buffy2-1:~# free
>              total         used         free       shared      buffers
> Mem:        126336       110332        16004            0         5616
> -/+ buffers:             104716        21620
> Swap:            0            0            0
> root@buffy2-1:~# uptime
>  02:08:54 up 2 days,  1:26,  load average: 0.10, 0.21, 0.17
> root@buffy2-1:~# dmesg
> [    0.000000] Linux version 3.10.24 (cero2@snapon) (gcc version 4.6.4
> (OpenWrt/Linaro GCC 4.6-2013.05 r38226) ) #1 Tue Dec 24
> 10:50:15 PST 2013
> [skipping some lines]
>
> [   13.156250] Error: Driver 'gpio-keys-polled' is already registered,
> aborting...
> [   19.414062] IPv6: ADDRCONF(NETDEV_UP): ge00: link is not ready
> [   19.421875] ar71xx: pll_reg 0xb8050010: 0x11110000
> [   19.429687] se00: link up (1000Mbps/Full duplex)
> [   22.140625] IPv6: ADDRCONF(NETDEV_UP): sw00: link is not ready
> [   23.351562] IPv6: ADDRCONF(NETDEV_CHANGE): sw00: link becomes ready
> [   23.757812] ar71xx: pll_reg 0xb8050014: 0x11110000
> [   23.757812] ge00: link up (1000Mbps/Full duplex)
> [   23.773437] IPv6: ADDRCONF(NETDEV_CHANGE): ge00: link becomes ready
>
> root@buffy2-1:~# ifconfig
> ge00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B1
>           inet addr:192.168.1.138  Bcast:192.168.1.255  Mask:255.255.255.0
>           inet6 addr: fe80::2eb0:5dff:fea0:c5b1/64 Scope:Link
>           inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64 Scope:Global
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:1469670 errors:0 dropped:8 overruns:0 frame:0
>           TX packets:547733 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000
>           RX bytes:229243410 (218.6 MiB)  TX bytes:57304808 (54.6 MiB)
>           Interrupt:5
>
> lo        Link encap:Local Loopback
>           inet addr:127.0.0.1  Mask:255.0.0.0
>           inet6 addr: ::1/128 Scope:Host
>           UP LOOPBACK RUNNING  MTU:65536  Metric:1
>           RX packets:23689 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:23689 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0
>           RX bytes:2612713 (2.4 MiB)  TX bytes:2612713 (2.4 MiB)
>
> pimreg    Link encap:UNSPEC  HWaddr
> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
>           UP RUNNING NOARP  MTU:1472  Metric:1
>           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0
>           RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>
> se00      Link encap:Ethernet  HWaddr 2E:B0:5D:A0:C5:B0
>           inet addr:172.30.42.1  Bcast:172.30.42.31  Mask:255.255.255.224
>           inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64 Scope:Global

How are you assigning your ipv6 addresses?

>           inet6 addr: fe80::2cb0:5dff:fea0:c5b0/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:191740 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000
>           RX bytes:0 (0.0 B)  TX bytes:42184988 (40.2 MiB)
>           Interrupt:4
>
> sw00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B0
>           inet addr:172.30.42.65  Bcast:172.30.42.95  Mask:255.255.255.224
>           inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64 Scope:Global
>           inet6 addr: fe80::2eb0:5dff:fea0:c5b0/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:70239 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:286967 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000
>           RX bytes:15590189 (14.8 MiB)  TX bytes:127357293 (121.4 MiB)
>
> root@buffy2-1:~# less /var/log/babeld.log
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> send: Cannot assign requested address
> send: Cannot assign requested address
> send: Cannot assign requested address
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> netlink_read: recvmsg(): No buffer space available
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.
> Couldn't determine channel of interface sw00: Invalid argument.

This is a problem in babel detecting the channel on a "normal"
rather than a mesh interface. It's bugged me a long while, but
haven't got around to finding what triggers it. Might "fix" it by
acquiring the channel at babel start time from /etc/config/wireless.

It messes up the diversity routing calculation, grump.

There is a possibility a logfile got really big, but this one
generally doesn't, but I should turn off logging in some
future release...

> </quote>
>
> and then the system hang. Note: the first time I ran 'less
> /var/log/babeld.log', it was Killed. I would assume that's an OOM
> killer?

Yes it really sounds like you were out of memory. Your free does not
indicate that however.

> I'm running cerowrt 3.10.24-8 on a wndr3800. I have a spare wndr3800
> that I'm getting ready to flash with the same version and I can see if I get
> the same issue. What's the best way to clone a configuration?

scp -r therouter:/overlay somewheresafe

then reflash it clean

>
> ---
>
> Here are some log lines from a separate instance of the same problem, I
> recovered these log lines from the web server (I was able to login to
> the web interface for another hour until I got the same lua exception
> as in the beginning of this email)

You can setup remote syslog logging if you lke.

> This time I was completely unable to connect to wifi from my iPhone
> and you can see where hostapd disassociates the iPhone soon after
> it connects.
>
> the kernel log:
>
> [158995.550781] icmp6_send: no reply to icmp error
>
> [160930.527343] ath: phy0: Failed to stop TX DMA, queues=0x004!

We see this all the time for 3 years now.

> [160930.542968] ath: phy0: DMA failed to stop in 10 ms AR_CR=0x00000024
> AR_DIAG_SW=0x42000020 DMADBG_7=0x000084c0

that is kind of new.

>
> [160930.554687] ath: phy0: Could not stop RX, we could be confusing the DMA
> engine when we start RX up

So is that.

>
> and in the system log:
>
> Fri Jan 24 15:51:15 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0): No
> buffer space available
>
> Fri Jan 24 15:51:15 2014 daemon.err avahi-daemon[1273]: netlink.c: recvmsg()
> failed: No buffer space available
>
> Fri Jan 24 15:52:18 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0): No
> buffer space available
>
>
> root@buffy2-1:/tmp/log# free
>
>              total         used         free       shared      buffers
>
> Mem:        126336       110128        16208            0         5616
>
> -/+ buffers:             104512        21824
>
> Swap:            0            0            0
>
>
> Fri Jan 24 17:14:53 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0): No
> buffer space available
>
> Fri Jan 24 17:15:53 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0): No
> buffer space available
>
> Fri Jan 24 17:16:18 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
> IEEE 802.11: authenticated
>
> Fri Jan 24 17:16:18 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
> IEEE 802.11: associated (aid 1)
>
> Fri Jan 24 17:16:18 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
> RADIUS: starting accounting session 52DC70A7-00000010
>
> Fri Jan 24 17:16:18 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
> WPA: pairwise key handshake completed (RSN)
>
> Fri Jan 24 17:16:18 2014 daemon.info dnsmasq-dhcp[4999]: DHCPREQUEST(sw00)
> 192.168.1.114 90:72:40:e9:9c:b6
>
> Fri Jan 24 17:16:18 2014 daemon.info dnsmasq-dhcp[4999]: DHCPNAK(sw00)
> 192.168.1.114 90:72:40:e9:9c:b6 wrong network
>
> Fri Jan 24 17:16:21 2014 daemon.info dnsmasq-dhcp[4999]: DHCPDISCOVER(sw00)
> 90:72:40:e9:9c:b6
>
> Fri Jan 24 17:16:21 2014 daemon.info dnsmasq-dhcp[4999]: DHCPOFFER(sw00)
> 172.30.42.82 90:72:40:e9:9c:b6
>
> Fri Jan 24 17:16:21 2014 daemon.info dnsmasq-dhcp[4999]: DHCPDISCOVER(sw00)
> 90:72:40:e9:9c:b6
>
> Fri Jan 24 17:16:21 2014 daemon.info dnsmasq-dhcp[4999]: DHCPOFFER(sw00)
> 172.30.42.82 90:72:40:e9:9c:b6
>
> Fri Jan 24 17:16:22 2014 daemon.info dnsmasq-dhcp[4999]: DHCPREQUEST(sw00)
> 172.30.42.82 90:72:40:e9:9c:b6
>
> Fri Jan 24 17:16:22 2014 daemon.info dnsmasq-dhcp[4999]: DHCPACK(sw00)
> 172.30.42.82 90:72:40:e9:9c:b6 stevejs-iPhone
>
> Fri Jan 24 17:16:35 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
> IEEE 802.11: disassociated
>
> Fri Jan 24 17:16:36 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
> IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)
>
> Fri Jan 24 17:16:57 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0): No
> buffer space available
>
> Fri Jan 24 17:17:59 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0): No
> buffer space available
>
> Fri Jan 24 17:19:04 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0): No
> buffer space available
>
> Fri Jan 24 17:19:04 2014 daemon.err avahi-daemon[1273]: netlink.c: recvmsg()
> failed: No buffer space available
>
> Fri Jan 24 17:19:45 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
> IEEE 802.11: authenticated
>
> Fri Jan 24 17:19:45 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
> IEEE 802.11: associated (aid 1)
>
> Fri Jan 24 17:19:45 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
> RADIUS: starting accounting session 52DC70A7-00000011
>
> Fri Jan 24 17:19:45 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
> WPA: pairwise key handshake completed (RSN)
>
> Fri Jan 24 17:19:45 2014 daemon.info dnsmasq-dhcp[4999]: DHCPDISCOVER(sw00)
> 90:72:40:e9:9c:b6
>
> Fri Jan 24 17:19:45 2014 daemon.info dnsmasq-dhcp[4999]: DHCPOFFER(sw00)
> 172.30.42.82 90:72:40:e9:9c:b6
>
> Fri Jan 24 17:19:47 2014 daemon.info dnsmasq-dhcp[4999]: DHCPREQUEST(sw00)
> 172.30.42.82 90:72:40:e9:9c:b6
>
> Fri Jan 24 17:19:47 2014 daemon.info dnsmasq-dhcp[4999]: DHCPACK(sw00)
> 172.30.42.82 90:72:40:e9:9c:b6 stevejs-iPhone
>
> Fri Jan 24 17:20:01 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
> IEEE 802.11: disassociated
>
> Fri Jan 24 17:20:02 2014 daemon.info hostapd: sw00: STA 90:72:40:e9:9c:b6
> IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)
>
> Fri Jan 24 17:20:09 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0): No
> buffer space available
>
> Fri Jan 24 17:21:10 2014 daemon.err minissdpd[4933]: recvmsg(s, &hdr, 0): No
> buffer space available
>
> and then it hung and rebooted.

You are running out of ram somewhere unobservable, and yes, I'd be
a little nervous that it isn't actually a memory or hw problem.

>
> I bought this particular wndr3800 used so it's possible I have a bad piece
> of hardware. Are there any hardware testing scripts or anything like
> memtest86 for these devices?  Are there entries in /sys that I should be
> looking at? I've noticed about 9k unaligned_instructions on boot but that
> never grows.

I have only observed 4 in the last 24 hours on this build:
https://lists.bufferbloat.net/pipermail/cerowrt-devel/2014-January/002093.html

>
> Thanks!
> Steve Jenson
> @stevej
>
>
>
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[Steve Jenson]

[-- Attachment #1: Type: text/plain, Size: 11366 bytes --]

On Fri, Jan 24, 2014 at 3:23 PM, Dave Taht <dave.taht@gmail.com> wrote:

> On Fri, Jan 24, 2014 at 6:08 PM, Steve Jenson <stevej@fruitless.org>
> wrote:
> > Hi everybody,
> >
> > I've been using cerowrt as a secondary wifi network (just a single AP for
> > now) for a few weeks now. Recently, my wndr3800 got stuck in a bad state
> and
> > eventually rebooted. I've had this happen a few times now and am looking
> for
> > ways to debug the issue. I'm new to cerowrt and openwrt so any advice is
> > appreciated.
> >
> > Since I use it as a secondary network, this is no way critical.
>
> Yea! I appreciate caution before putting alpha software on your gw.
>
> > I'm not
> > looking for free tech support but I couldn't find anything on the wiki
> about
> > troubleshooting. I'd love to start a page and write some shell scripts to
> > diagnose and report issues. I know that a cerowrt router is meant to be a
> > research project rather a consumer device but these things seem helpful
> > regardless.
>
> Sure, let me know your wiki account. I have been lax about granting
> access of late as the signup process is overrun by spammers.


My username is stevej on the wiki. Thanks!



>  > Please let me know if you'd prefer I not email the list with these
> issues or
> > if you'd rather I used trac or a different forum.
> >
>
> The list is where most stuff happens. Also in the irc channel.
>
> If it gets to where it needs to be tracked we have a bugtracker at
>
> http://www.bufferbloat.net/projects/cerowrt/issues
>
> The first question I have is: Are you on comcast? Cerowrt
> had a dhcpv6-pd implementation that "just worked" from feburary through
> december. Regrettably they changed the RA announcement interval
> to a really low number around then... and this triggers a firewall reload
> every minute on everything prior to the release I point to below.
>
> If there is a memory leak somewhere that would have triggered it.


I am on AT&T ADSL2+ with a Motorola NVG510 modem.



> > In this state, I can connect to the cerowrt base station via wifi but am
> > unable to route packets to the internet. I can connect to :81 and see the
> > login page but logging in results in a lua error at `/cgi-bin/luci`
> >
> >
> >   /usr/lib/lua/luci/dispatcher.lua:448: Failed to execute function
> > dispatcher target for entry '/'.
> >   The called action terminated with an exception:
> >   /usr/lib/lua/luci/sauth.lua:87: Session data invalid!
> >   stack traceback:
> >   [C]: in function 'assert'
> >   /usr/lib/lua/luci/dispatcher.lua:448: in function 'dispatch'
> >   /usr/lib/lua/luci/dispatcher.lua:195: in function
> > </usr/lib/lua/luci/dispatcher.lua:194>
> >
> > I can ssh into the device and cat various log files until the router
> hangs
> > and reboots. here's a few relevant lines from my terminal history before
> the
> > device rebooted (I'm assuming a watchdog kicked in and rebooted it).
> >
> > root@buffy2-1:~# ping google.com
> > ping: bad address 'google.com'
> > root@buffy2-1:~# free
> >              total         used         free       shared      buffers
> > Mem:        126336       110332        16004            0         5616
> > -/+ buffers:             104716        21620
> > Swap:            0            0            0
> > root@buffy2-1:~# uptime
> >  02:08:54 up 2 days,  1:26,  load average: 0.10, 0.21, 0.17
> > root@buffy2-1:~# dmesg
> > [    0.000000] Linux version 3.10.24 (cero2@snapon) (gcc version 4.6.4
> > (OpenWrt/Linaro GCC 4.6-2013.05 r38226) ) #1 Tue Dec 24
> > 10:50:15 PST 2013
> > [skipping some lines]
> >
> > [   13.156250] Error: Driver 'gpio-keys-polled' is already registered,
> > aborting...
> > [   19.414062] IPv6: ADDRCONF(NETDEV_UP): ge00: link is not ready
> > [   19.421875] ar71xx: pll_reg 0xb8050010: 0x11110000
> > [   19.429687] se00: link up (1000Mbps/Full duplex)
> > [   22.140625] IPv6: ADDRCONF(NETDEV_UP): sw00: link is not ready
> > [   23.351562] IPv6: ADDRCONF(NETDEV_CHANGE): sw00: link becomes ready
> > [   23.757812] ar71xx: pll_reg 0xb8050014: 0x11110000
> > [   23.757812] ge00: link up (1000Mbps/Full duplex)
> > [   23.773437] IPv6: ADDRCONF(NETDEV_CHANGE): ge00: link becomes ready
> >
> > root@buffy2-1:~# ifconfig
> > ge00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B1
> >           inet addr:192.168.1.138  Bcast:192.168.1.255
>  Mask:255.255.255.0
> >           inet6 addr: fe80::2eb0:5dff:fea0:c5b1/64 Scope:Link
> >           inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
> Scope:Global
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:1469670 errors:0 dropped:8 overruns:0 frame:0
> >           TX packets:547733 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:1000
> >           RX bytes:229243410 (218.6 MiB)  TX bytes:57304808 (54.6 MiB)
> >           Interrupt:5
> >
> > lo        Link encap:Local Loopback
> >           inet addr:127.0.0.1  Mask:255.0.0.0
> >           inet6 addr: ::1/128 Scope:Host
> >           UP LOOPBACK RUNNING  MTU:65536  Metric:1
> >           RX packets:23689 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:23689 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:0
> >           RX bytes:2612713 (2.4 MiB)  TX bytes:2612713 (2.4 MiB)
> >
> > pimreg    Link encap:UNSPEC  HWaddr
> > 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
> >           UP RUNNING NOARP  MTU:1472  Metric:1
> >           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:0
> >           RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
> >
> > se00      Link encap:Ethernet  HWaddr 2E:B0:5D:A0:C5:B0
> >           inet addr:172.30.42.1  Bcast:172.30.42.31  Mask:255.255.255.224
> >           inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
> Scope:Global
>
> How are you assigning your ipv6 addresses?
>

It's been a while since I messed with this but I think IPv6 is assigned
thanks to 6relayd? My modem has IPv6 enabled but no DHCPv6 options that I
can find. Here's how cerowrt is configured.

root@buffy2-1:/overlay/etc/config# cat 6relayd
config server 'default'
  option fallback_relay 'rd dhcpv6 ndp'
  list network 'ge00'
  list network 'ge01'
  list network 'gw00'
  list network 'gw01'
  list network 'gw10'
  list network 'gw11'
  list network 'se00'
  list network 'sw00'
  list network 'sw10'
  option rd 'relay'
  option dhcpv6 'relay'
  option ndp 'relay'
  option master 'ge00'


> >           inet6 addr: fe80::2cb0:5dff:fea0:c5b0/64 Scope:Link
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:191740 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:1000
> >           RX bytes:0 (0.0 B)  TX bytes:42184988 (40.2 MiB)
> >           Interrupt:4
> >
> > sw00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B0
> >           inet addr:172.30.42.65  Bcast:172.30.42.95
>  Mask:255.255.255.224
> >           inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
> Scope:Global
> >           inet6 addr: fe80::2eb0:5dff:fea0:c5b0/64 Scope:Link
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:70239 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:286967 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:1000
> >           RX bytes:15590189 (14.8 MiB)  TX bytes:127357293 (121.4 MiB)
> >
> > root@buffy2-1:~# less /var/log/babeld.log
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > send: Cannot assign requested address
> > send: Cannot assign requested address
> > send: Cannot assign requested address
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > netlink_read: recvmsg(): No buffer space available
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
> > Couldn't determine channel of interface sw00: Invalid argument.
>
> This is a problem in babel detecting the channel on a "normal"
> rather than a mesh interface. It's bugged me a long while, but
> haven't got around to finding what triggers it. Might "fix" it by
> acquiring the channel at babel start time from /etc/config/wireless.
>
> It messes up the diversity routing calculation, grump.
>
> There is a possibility a logfile got really big, but this one
> generally doesn't, but I should turn off logging in some
> future release...


I believe I've tracked down part of what's going on. It looks like my tmpfs
is filling up 100% and then the device enters a bad state:

After 24 hours, with tmpfs at 50%, babeld.log is the largest file by far in
tmpfs and the only file that appears to be growing (based on `du`). It
takes about 48 hours from reboot to fill up tmpfs on my device.

#  sort babeld.log | uniq -c |sort -rn |head

 503236 Couldn't determine channel of interface sw00: Invalid argument.

   1376 netlink_read: recvmsg(): No buffer space available

      3 send: Cannot assign requested address

# wc -l babeld.log

504617 babeld.log
I sped up system failure by using `dd` to fill up tmpfs and the system
became immediately unusable.

This also explains the luci session store errors as sessions are stored in
tmpfs.

The other buffer issues may or may not be related to this.

Best,
Steve

[-- Attachment #2: Type: text/html, Size: 14513 bytes --]

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[Steve Jenson]

[-- Attachment #1: Type: text/plain, Size: 11966 bytes --]

Looking more, the buffer errors are showing up in syslog well before tmpfs
fills up. Is the memtester openwrt package available for cerowrt? I don't
see it under `Available packages`.

Thanks,
Steve


On Mon, Jan 27, 2014 at 1:06 PM, Steve Jenson <stevej@fruitless.org> wrote:

> On Fri, Jan 24, 2014 at 3:23 PM, Dave Taht <dave.taht@gmail.com> wrote:
>
>> On Fri, Jan 24, 2014 at 6:08 PM, Steve Jenson <stevej@fruitless.org>
>> wrote:
>> > Hi everybody,
>> >
>> > I've been using cerowrt as a secondary wifi network (just a single AP
>> for
>> > now) for a few weeks now. Recently, my wndr3800 got stuck in a bad
>> state and
>> > eventually rebooted. I've had this happen a few times now and am
>> looking for
>> > ways to debug the issue. I'm new to cerowrt and openwrt so any advice is
>> > appreciated.
>> >
>> > Since I use it as a secondary network, this is no way critical.
>>
>> Yea! I appreciate caution before putting alpha software on your gw.
>>
>> > I'm not
>> > looking for free tech support but I couldn't find anything on the wiki
>> about
>> > troubleshooting. I'd love to start a page and write some shell scripts
>> to
>> > diagnose and report issues. I know that a cerowrt router is meant to be
>> a
>> > research project rather a consumer device but these things seem helpful
>> > regardless.
>>
>> Sure, let me know your wiki account. I have been lax about granting
>> access of late as the signup process is overrun by spammers.
>
>
> My username is stevej on the wiki. Thanks!
>
>
>
>>  > Please let me know if you'd prefer I not email the list with these
>> issues or
>> > if you'd rather I used trac or a different forum.
>> >
>>
>> The list is where most stuff happens. Also in the irc channel.
>>
>> If it gets to where it needs to be tracked we have a bugtracker at
>>
>> http://www.bufferbloat.net/projects/cerowrt/issues
>>
>> The first question I have is: Are you on comcast? Cerowrt
>> had a dhcpv6-pd implementation that "just worked" from feburary through
>> december. Regrettably they changed the RA announcement interval
>> to a really low number around then... and this triggers a firewall reload
>> every minute on everything prior to the release I point to below.
>>
>> If there is a memory leak somewhere that would have triggered it.
>
>
> I am on AT&T ADSL2+ with a Motorola NVG510 modem.
>
>
>
>> > In this state, I can connect to the cerowrt base station via wifi but am
>> > unable to route packets to the internet. I can connect to :81 and see
>> the
>> > login page but logging in results in a lua error at `/cgi-bin/luci`
>> >
>> >
>> >   /usr/lib/lua/luci/dispatcher.lua:448: Failed to execute function
>> > dispatcher target for entry '/'.
>> >   The called action terminated with an exception:
>> >   /usr/lib/lua/luci/sauth.lua:87: Session data invalid!
>> >   stack traceback:
>> >   [C]: in function 'assert'
>> >   /usr/lib/lua/luci/dispatcher.lua:448: in function 'dispatch'
>> >   /usr/lib/lua/luci/dispatcher.lua:195: in function
>> > </usr/lib/lua/luci/dispatcher.lua:194>
>> >
>> > I can ssh into the device and cat various log files until the router
>> hangs
>> > and reboots. here's a few relevant lines from my terminal history
>> before the
>> > device rebooted (I'm assuming a watchdog kicked in and rebooted it).
>> >
>> > root@buffy2-1:~# ping google.com
>> > ping: bad address 'google.com'
>> > root@buffy2-1:~# free
>> >              total         used         free       shared      buffers
>> > Mem:        126336       110332        16004            0         5616
>> > -/+ buffers:             104716        21620
>> > Swap:            0            0            0
>> > root@buffy2-1:~# uptime
>> >  02:08:54 up 2 days,  1:26,  load average: 0.10, 0.21, 0.17
>> > root@buffy2-1:~# dmesg
>> > [    0.000000] Linux version 3.10.24 (cero2@snapon) (gcc version 4.6.4
>> > (OpenWrt/Linaro GCC 4.6-2013.05 r38226) ) #1 Tue Dec 24
>> > 10:50:15 PST 2013
>> > [skipping some lines]
>> >
>> > [   13.156250] Error: Driver 'gpio-keys-polled' is already registered,
>> > aborting...
>> > [   19.414062] IPv6: ADDRCONF(NETDEV_UP): ge00: link is not ready
>> > [   19.421875] ar71xx: pll_reg 0xb8050010: 0x11110000
>> > [   19.429687] se00: link up (1000Mbps/Full duplex)
>> > [   22.140625] IPv6: ADDRCONF(NETDEV_UP): sw00: link is not ready
>> > [   23.351562] IPv6: ADDRCONF(NETDEV_CHANGE): sw00: link becomes ready
>> > [   23.757812] ar71xx: pll_reg 0xb8050014: 0x11110000
>> > [   23.757812] ge00: link up (1000Mbps/Full duplex)
>> > [   23.773437] IPv6: ADDRCONF(NETDEV_CHANGE): ge00: link becomes ready
>> >
>> > root@buffy2-1:~# ifconfig
>> > ge00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B1
>> >           inet addr:192.168.1.138  Bcast:192.168.1.255
>>  Mask:255.255.255.0
>> >           inet6 addr: fe80::2eb0:5dff:fea0:c5b1/64 Scope:Link
>> >           inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>> Scope:Global
>> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>> >           RX packets:1469670 errors:0 dropped:8 overruns:0 frame:0
>> >           TX packets:547733 errors:0 dropped:0 overruns:0 carrier:0
>> >           collisions:0 txqueuelen:1000
>> >           RX bytes:229243410 (218.6 MiB)  TX bytes:57304808 (54.6 MiB)
>> >           Interrupt:5
>> >
>> > lo        Link encap:Local Loopback
>> >           inet addr:127.0.0.1  Mask:255.0.0.0
>> >           inet6 addr: ::1/128 Scope:Host
>> >           UP LOOPBACK RUNNING  MTU:65536  Metric:1
>> >           RX packets:23689 errors:0 dropped:0 overruns:0 frame:0
>> >           TX packets:23689 errors:0 dropped:0 overruns:0 carrier:0
>> >           collisions:0 txqueuelen:0
>> >           RX bytes:2612713 (2.4 MiB)  TX bytes:2612713 (2.4 MiB)
>> >
>> > pimreg    Link encap:UNSPEC  HWaddr
>> > 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
>> >           UP RUNNING NOARP  MTU:1472  Metric:1
>> >           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>> >           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>> >           collisions:0 txqueuelen:0
>> >           RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>> >
>> > se00      Link encap:Ethernet  HWaddr 2E:B0:5D:A0:C5:B0
>> >           inet addr:172.30.42.1  Bcast:172.30.42.31
>>  Mask:255.255.255.224
>> >           inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>> Scope:Global
>>
>> How are you assigning your ipv6 addresses?
>>
>
> It's been a while since I messed with this but I think IPv6 is assigned
> thanks to 6relayd? My modem has IPv6 enabled but no DHCPv6 options that I
> can find. Here's how cerowrt is configured.
>
> root@buffy2-1:/overlay/etc/config# cat 6relayd
> config server 'default'
>   option fallback_relay 'rd dhcpv6 ndp'
>   list network 'ge00'
>   list network 'ge01'
>   list network 'gw00'
>   list network 'gw01'
>   list network 'gw10'
>   list network 'gw11'
>   list network 'se00'
>   list network 'sw00'
>   list network 'sw10'
>   option rd 'relay'
>   option dhcpv6 'relay'
>   option ndp 'relay'
>   option master 'ge00'
>
>
>> >           inet6 addr: fe80::2cb0:5dff:fea0:c5b0/64 Scope:Link
>> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>> >           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>> >           TX packets:191740 errors:0 dropped:0 overruns:0 carrier:0
>> >           collisions:0 txqueuelen:1000
>> >           RX bytes:0 (0.0 B)  TX bytes:42184988 (40.2 MiB)
>> >           Interrupt:4
>> >
>> > sw00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B0
>> >           inet addr:172.30.42.65  Bcast:172.30.42.95
>>  Mask:255.255.255.224
>> >           inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>> Scope:Global
>> >           inet6 addr: fe80::2eb0:5dff:fea0:c5b0/64 Scope:Link
>> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>> >           RX packets:70239 errors:0 dropped:0 overruns:0 frame:0
>> >           TX packets:286967 errors:0 dropped:0 overruns:0 carrier:0
>> >           collisions:0 txqueuelen:1000
>> >           RX bytes:15590189 (14.8 MiB)  TX bytes:127357293 (121.4 MiB)
>> >
>> > root@buffy2-1:~# less /var/log/babeld.log
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > send: Cannot assign requested address
>> > send: Cannot assign requested address
>> > send: Cannot assign requested address
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > netlink_read: recvmsg(): No buffer space available
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>> > Couldn't determine channel of interface sw00: Invalid argument.
>>
>> This is a problem in babel detecting the channel on a "normal"
>> rather than a mesh interface. It's bugged me a long while, but
>> haven't got around to finding what triggers it. Might "fix" it by
>> acquiring the channel at babel start time from /etc/config/wireless.
>>
>> It messes up the diversity routing calculation, grump.
>>
>> There is a possibility a logfile got really big, but this one
>> generally doesn't, but I should turn off logging in some
>> future release...
>
>
> I believe I've tracked down part of what's going on. It looks like my
> tmpfs is filling up 100% and then the device enters a bad state:
>
> After 24 hours, with tmpfs at 50%, babeld.log is the largest file by far
> in tmpfs and the only file that appears to be growing (based on `du`). It
> takes about 48 hours from reboot to fill up tmpfs on my device.
>
> #  sort babeld.log | uniq -c |sort -rn |head
>
>  503236 Couldn't determine channel of interface sw00: Invalid argument.
>
>    1376 netlink_read: recvmsg(): No buffer space available
>
>       3 send: Cannot assign requested address
>
> # wc -l babeld.log
>
> 504617 babeld.log
> I sped up system failure by using `dd` to fill up tmpfs and the system
> became immediately unusable.
>
> This also explains the luci session store errors as sessions are stored in
> tmpfs.
>
> The other buffer issues may or may not be related to this.
>
> Best,
> Steve
>

[-- Attachment #2: Type: text/html, Size: 15217 bytes --]

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[Dave Taht]

certainly turn off the babeld log! I will leave it off in the next release.

On Mon, Jan 27, 2014 at 4:10 PM, Steve Jenson <stevej@fruitless.org> wrote:
> Looking more, the buffer errors are showing up in syslog well before tmpfs
> fills up. Is the memtester openwrt package available for cerowrt? I don't
> see it under `Available packages`.
>
> Thanks,
> Steve
>
>
> On Mon, Jan 27, 2014 at 1:06 PM, Steve Jenson <stevej@fruitless.org> wrote:
>>
>> On Fri, Jan 24, 2014 at 3:23 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>>
>>> On Fri, Jan 24, 2014 at 6:08 PM, Steve Jenson <stevej@fruitless.org>
>>> wrote:
>>> > Hi everybody,
>>> >
>>> > I've been using cerowrt as a secondary wifi network (just a single AP
>>> > for
>>> > now) for a few weeks now. Recently, my wndr3800 got stuck in a bad
>>> > state and
>>> > eventually rebooted. I've had this happen a few times now and am
>>> > looking for
>>> > ways to debug the issue. I'm new to cerowrt and openwrt so any advice
>>> > is
>>> > appreciated.
>>> >
>>> > Since I use it as a secondary network, this is no way critical.
>>>
>>> Yea! I appreciate caution before putting alpha software on your gw.
>>>
>>> > I'm not
>>> > looking for free tech support but I couldn't find anything on the wiki
>>> > about
>>> > troubleshooting. I'd love to start a page and write some shell scripts
>>> > to
>>> > diagnose and report issues. I know that a cerowrt router is meant to be
>>> > a
>>> > research project rather a consumer device but these things seem helpful
>>> > regardless.
>>>
>>> Sure, let me know your wiki account. I have been lax about granting
>>> access of late as the signup process is overrun by spammers.
>>
>>
>> My username is stevej on the wiki. Thanks!
>>
>>
>>>
>>> > Please let me know if you'd prefer I not email the list with these
>>> > issues or
>>> > if you'd rather I used trac or a different forum.
>>> >
>>>
>>> The list is where most stuff happens. Also in the irc channel.
>>>
>>> If it gets to where it needs to be tracked we have a bugtracker at
>>>
>>> http://www.bufferbloat.net/projects/cerowrt/issues
>>>
>>> The first question I have is: Are you on comcast? Cerowrt
>>> had a dhcpv6-pd implementation that "just worked" from feburary through
>>> december. Regrettably they changed the RA announcement interval
>>> to a really low number around then... and this triggers a firewall reload
>>> every minute on everything prior to the release I point to below.
>>>
>>> If there is a memory leak somewhere that would have triggered it.
>>
>>
>> I am on AT&T ADSL2+ with a Motorola NVG510 modem.
>>
>>
>>>
>>> > In this state, I can connect to the cerowrt base station via wifi but
>>> > am
>>> > unable to route packets to the internet. I can connect to :81 and see
>>> > the
>>> > login page but logging in results in a lua error at `/cgi-bin/luci`
>>> >
>>> >
>>> >   /usr/lib/lua/luci/dispatcher.lua:448: Failed to execute function
>>> > dispatcher target for entry '/'.
>>> >   The called action terminated with an exception:
>>> >   /usr/lib/lua/luci/sauth.lua:87: Session data invalid!
>>> >   stack traceback:
>>> >   [C]: in function 'assert'
>>> >   /usr/lib/lua/luci/dispatcher.lua:448: in function 'dispatch'
>>> >   /usr/lib/lua/luci/dispatcher.lua:195: in function
>>> > </usr/lib/lua/luci/dispatcher.lua:194>
>>> >
>>> > I can ssh into the device and cat various log files until the router
>>> > hangs
>>> > and reboots. here's a few relevant lines from my terminal history
>>> > before the
>>> > device rebooted (I'm assuming a watchdog kicked in and rebooted it).
>>> >
>>> > root@buffy2-1:~# ping google.com
>>> > ping: bad address 'google.com'
>>> > root@buffy2-1:~# free
>>> >              total         used         free       shared      buffers
>>> > Mem:        126336       110332        16004            0         5616
>>> > -/+ buffers:             104716        21620
>>> > Swap:            0            0            0
>>> > root@buffy2-1:~# uptime
>>> >  02:08:54 up 2 days,  1:26,  load average: 0.10, 0.21, 0.17
>>> > root@buffy2-1:~# dmesg
>>> > [    0.000000] Linux version 3.10.24 (cero2@snapon) (gcc version 4.6.4
>>> > (OpenWrt/Linaro GCC 4.6-2013.05 r38226) ) #1 Tue Dec 24
>>> > 10:50:15 PST 2013
>>> > [skipping some lines]
>>> >
>>> > [   13.156250] Error: Driver 'gpio-keys-polled' is already registered,
>>> > aborting...
>>> > [   19.414062] IPv6: ADDRCONF(NETDEV_UP): ge00: link is not ready
>>> > [   19.421875] ar71xx: pll_reg 0xb8050010: 0x11110000
>>> > [   19.429687] se00: link up (1000Mbps/Full duplex)
>>> > [   22.140625] IPv6: ADDRCONF(NETDEV_UP): sw00: link is not ready
>>> > [   23.351562] IPv6: ADDRCONF(NETDEV_CHANGE): sw00: link becomes ready
>>> > [   23.757812] ar71xx: pll_reg 0xb8050014: 0x11110000
>>> > [   23.757812] ge00: link up (1000Mbps/Full duplex)
>>> > [   23.773437] IPv6: ADDRCONF(NETDEV_CHANGE): ge00: link becomes ready
>>> >
>>> > root@buffy2-1:~# ifconfig
>>> > ge00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B1
>>> >           inet addr:192.168.1.138  Bcast:192.168.1.255
>>> > Mask:255.255.255.0
>>> >           inet6 addr: fe80::2eb0:5dff:fea0:c5b1/64 Scope:Link
>>> >           inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>> > Scope:Global
>>> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>> >           RX packets:1469670 errors:0 dropped:8 overruns:0 frame:0
>>> >           TX packets:547733 errors:0 dropped:0 overruns:0 carrier:0
>>> >           collisions:0 txqueuelen:1000
>>> >           RX bytes:229243410 (218.6 MiB)  TX bytes:57304808 (54.6 MiB)
>>> >           Interrupt:5
>>> >
>>> > lo        Link encap:Local Loopback
>>> >           inet addr:127.0.0.1  Mask:255.0.0.0
>>> >           inet6 addr: ::1/128 Scope:Host
>>> >           UP LOOPBACK RUNNING  MTU:65536  Metric:1
>>> >           RX packets:23689 errors:0 dropped:0 overruns:0 frame:0
>>> >           TX packets:23689 errors:0 dropped:0 overruns:0 carrier:0
>>> >           collisions:0 txqueuelen:0
>>> >           RX bytes:2612713 (2.4 MiB)  TX bytes:2612713 (2.4 MiB)
>>> >
>>> > pimreg    Link encap:UNSPEC  HWaddr
>>> > 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
>>> >           UP RUNNING NOARP  MTU:1472  Metric:1
>>> >           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>> >           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>> >           collisions:0 txqueuelen:0
>>> >           RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>>> >
>>> > se00      Link encap:Ethernet  HWaddr 2E:B0:5D:A0:C5:B0
>>> >           inet addr:172.30.42.1  Bcast:172.30.42.31
>>> > Mask:255.255.255.224
>>> >           inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>> > Scope:Global
>>>
>>> How are you assigning your ipv6 addresses?
>>
>>
>> It's been a while since I messed with this but I think IPv6 is assigned
>> thanks to 6relayd? My modem has IPv6 enabled but no DHCPv6 options that I
>> can find. Here's how cerowrt is configured.
>>
>> root@buffy2-1:/overlay/etc/config# cat 6relayd
>> config server 'default'
>>   option fallback_relay 'rd dhcpv6 ndp'
>>   list network 'ge00'
>>   list network 'ge01'
>>   list network 'gw00'
>>   list network 'gw01'
>>   list network 'gw10'
>>   list network 'gw11'
>>   list network 'se00'
>>   list network 'sw00'
>>   list network 'sw10'
>>   option rd 'relay'
>>   option dhcpv6 'relay'
>>   option ndp 'relay'
>>   option master 'ge00'
>>
>>>
>>> >           inet6 addr: fe80::2cb0:5dff:fea0:c5b0/64 Scope:Link
>>> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>> >           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>> >           TX packets:191740 errors:0 dropped:0 overruns:0 carrier:0
>>> >           collisions:0 txqueuelen:1000
>>> >           RX bytes:0 (0.0 B)  TX bytes:42184988 (40.2 MiB)
>>> >           Interrupt:4
>>> >
>>> > sw00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B0
>>> >           inet addr:172.30.42.65  Bcast:172.30.42.95
>>> > Mask:255.255.255.224
>>> >           inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>> > Scope:Global
>>> >           inet6 addr: fe80::2eb0:5dff:fea0:c5b0/64 Scope:Link
>>> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>> >           RX packets:70239 errors:0 dropped:0 overruns:0 frame:0
>>> >           TX packets:286967 errors:0 dropped:0 overruns:0 carrier:0
>>> >           collisions:0 txqueuelen:1000
>>> >           RX bytes:15590189 (14.8 MiB)  TX bytes:127357293 (121.4 MiB)
>>> >
>>> > root@buffy2-1:~# less /var/log/babeld.log
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > send: Cannot assign requested address
>>> > send: Cannot assign requested address
>>> > send: Cannot assign requested address
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > netlink_read: recvmsg(): No buffer space available
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>> > Couldn't determine channel of interface sw00: Invalid argument.
>>>
>>> This is a problem in babel detecting the channel on a "normal"
>>> rather than a mesh interface. It's bugged me a long while, but
>>> haven't got around to finding what triggers it. Might "fix" it by
>>> acquiring the channel at babel start time from /etc/config/wireless.
>>>
>>> It messes up the diversity routing calculation, grump.
>>>
>>> There is a possibility a logfile got really big, but this one
>>> generally doesn't, but I should turn off logging in some
>>> future release...
>>
>>
>> I believe I've tracked down part of what's going on. It looks like my
>> tmpfs is filling up 100% and then the device enters a bad state:
>>
>> After 24 hours, with tmpfs at 50%, babeld.log is the largest file by far
>> in tmpfs and the only file that appears to be growing (based on `du`). It
>> takes about 48 hours from reboot to fill up tmpfs on my device.
>>
>> #  sort babeld.log | uniq -c |sort -rn |head
>>
>>  503236 Couldn't determine channel of interface sw00: Invalid argument.
>>
>>    1376 netlink_read: recvmsg(): No buffer space available
>>
>>       3 send: Cannot assign requested address
>>
>> # wc -l babeld.log
>>
>> 504617 babeld.log
>>
>> I sped up system failure by using `dd` to fill up tmpfs and the system
>> became immediately unusable.
>>
>> This also explains the luci session store errors as sessions are stored in
>> tmpfs.
>>
>> The other buffer issues may or may not be related to this.
>>
>> Best,
>> Steve
>
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[Sebastian Moeller]

Hi Dave,

quick question, how does one turn of logging for babeld? It seems that if daemonized it defaults to logging to /var/log/babeld.log (or similar). Is setting the log file to /dev/null really the answer?
	(Since I have no the IPv6 issue not yet resolved, I assume babeld is unhappy) I resorted to stopping babeld completely, but that feels like a crutch…

Best
	Sebastian



On Jan 27, 2014, at 22:14 , Dave Taht <dave.taht@gmail.com> wrote:

> certainly turn off the babeld log! I will leave it off in the next release.
> 
> On Mon, Jan 27, 2014 at 4:10 PM, Steve Jenson <stevej@fruitless.org> wrote:
>> Looking more, the buffer errors are showing up in syslog well before tmpfs
>> fills up. Is the memtester openwrt package available for cerowrt? I don't
>> see it under `Available packages`.
>> 
>> Thanks,
>> Steve
>> 
>> 
>> On Mon, Jan 27, 2014 at 1:06 PM, Steve Jenson <stevej@fruitless.org> wrote:
>>> 
>>> On Fri, Jan 24, 2014 at 3:23 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>>> 
>>>> On Fri, Jan 24, 2014 at 6:08 PM, Steve Jenson <stevej@fruitless.org>
>>>> wrote:
>>>>> Hi everybody,
>>>>> 
>>>>> I've been using cerowrt as a secondary wifi network (just a single AP
>>>>> for
>>>>> now) for a few weeks now. Recently, my wndr3800 got stuck in a bad
>>>>> state and
>>>>> eventually rebooted. I've had this happen a few times now and am
>>>>> looking for
>>>>> ways to debug the issue. I'm new to cerowrt and openwrt so any advice
>>>>> is
>>>>> appreciated.
>>>>> 
>>>>> Since I use it as a secondary network, this is no way critical.
>>>> 
>>>> Yea! I appreciate caution before putting alpha software on your gw.
>>>> 
>>>>> I'm not
>>>>> looking for free tech support but I couldn't find anything on the wiki
>>>>> about
>>>>> troubleshooting. I'd love to start a page and write some shell scripts
>>>>> to
>>>>> diagnose and report issues. I know that a cerowrt router is meant to be
>>>>> a
>>>>> research project rather a consumer device but these things seem helpful
>>>>> regardless.
>>>> 
>>>> Sure, let me know your wiki account. I have been lax about granting
>>>> access of late as the signup process is overrun by spammers.
>>> 
>>> 
>>> My username is stevej on the wiki. Thanks!
>>> 
>>> 
>>>> 
>>>>> Please let me know if you'd prefer I not email the list with these
>>>>> issues or
>>>>> if you'd rather I used trac or a different forum.
>>>>> 
>>>> 
>>>> The list is where most stuff happens. Also in the irc channel.
>>>> 
>>>> If it gets to where it needs to be tracked we have a bugtracker at
>>>> 
>>>> http://www.bufferbloat.net/projects/cerowrt/issues
>>>> 
>>>> The first question I have is: Are you on comcast? Cerowrt
>>>> had a dhcpv6-pd implementation that "just worked" from feburary through
>>>> december. Regrettably they changed the RA announcement interval
>>>> to a really low number around then... and this triggers a firewall reload
>>>> every minute on everything prior to the release I point to below.
>>>> 
>>>> If there is a memory leak somewhere that would have triggered it.
>>> 
>>> 
>>> I am on AT&T ADSL2+ with a Motorola NVG510 modem.
>>> 
>>> 
>>>> 
>>>>> In this state, I can connect to the cerowrt base station via wifi but
>>>>> am
>>>>> unable to route packets to the internet. I can connect to :81 and see
>>>>> the
>>>>> login page but logging in results in a lua error at `/cgi-bin/luci`
>>>>> 
>>>>> 
>>>>>  /usr/lib/lua/luci/dispatcher.lua:448: Failed to execute function
>>>>> dispatcher target for entry '/'.
>>>>>  The called action terminated with an exception:
>>>>>  /usr/lib/lua/luci/sauth.lua:87: Session data invalid!
>>>>>  stack traceback:
>>>>>  [C]: in function 'assert'
>>>>>  /usr/lib/lua/luci/dispatcher.lua:448: in function 'dispatch'
>>>>>  /usr/lib/lua/luci/dispatcher.lua:195: in function
>>>>> </usr/lib/lua/luci/dispatcher.lua:194>
>>>>> 
>>>>> I can ssh into the device and cat various log files until the router
>>>>> hangs
>>>>> and reboots. here's a few relevant lines from my terminal history
>>>>> before the
>>>>> device rebooted (I'm assuming a watchdog kicked in and rebooted it).
>>>>> 
>>>>> root@buffy2-1:~# ping google.com
>>>>> ping: bad address 'google.com'
>>>>> root@buffy2-1:~# free
>>>>>             total         used         free       shared      buffers
>>>>> Mem:        126336       110332        16004            0         5616
>>>>> -/+ buffers:             104716        21620
>>>>> Swap:            0            0            0
>>>>> root@buffy2-1:~# uptime
>>>>> 02:08:54 up 2 days,  1:26,  load average: 0.10, 0.21, 0.17
>>>>> root@buffy2-1:~# dmesg
>>>>> [    0.000000] Linux version 3.10.24 (cero2@snapon) (gcc version 4.6.4
>>>>> (OpenWrt/Linaro GCC 4.6-2013.05 r38226) ) #1 Tue Dec 24
>>>>> 10:50:15 PST 2013
>>>>> [skipping some lines]
>>>>> 
>>>>> [   13.156250] Error: Driver 'gpio-keys-polled' is already registered,
>>>>> aborting...
>>>>> [   19.414062] IPv6: ADDRCONF(NETDEV_UP): ge00: link is not ready
>>>>> [   19.421875] ar71xx: pll_reg 0xb8050010: 0x11110000
>>>>> [   19.429687] se00: link up (1000Mbps/Full duplex)
>>>>> [   22.140625] IPv6: ADDRCONF(NETDEV_UP): sw00: link is not ready
>>>>> [   23.351562] IPv6: ADDRCONF(NETDEV_CHANGE): sw00: link becomes ready
>>>>> [   23.757812] ar71xx: pll_reg 0xb8050014: 0x11110000
>>>>> [   23.757812] ge00: link up (1000Mbps/Full duplex)
>>>>> [   23.773437] IPv6: ADDRCONF(NETDEV_CHANGE): ge00: link becomes ready
>>>>> 
>>>>> root@buffy2-1:~# ifconfig
>>>>> ge00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B1
>>>>>          inet addr:192.168.1.138  Bcast:192.168.1.255
>>>>> Mask:255.255.255.0
>>>>>          inet6 addr: fe80::2eb0:5dff:fea0:c5b1/64 Scope:Link
>>>>>          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>> Scope:Global
>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>          RX packets:1469670 errors:0 dropped:8 overruns:0 frame:0
>>>>>          TX packets:547733 errors:0 dropped:0 overruns:0 carrier:0
>>>>>          collisions:0 txqueuelen:1000
>>>>>          RX bytes:229243410 (218.6 MiB)  TX bytes:57304808 (54.6 MiB)
>>>>>          Interrupt:5
>>>>> 
>>>>> lo        Link encap:Local Loopback
>>>>>          inet addr:127.0.0.1  Mask:255.0.0.0
>>>>>          inet6 addr: ::1/128 Scope:Host
>>>>>          UP LOOPBACK RUNNING  MTU:65536  Metric:1
>>>>>          RX packets:23689 errors:0 dropped:0 overruns:0 frame:0
>>>>>          TX packets:23689 errors:0 dropped:0 overruns:0 carrier:0
>>>>>          collisions:0 txqueuelen:0
>>>>>          RX bytes:2612713 (2.4 MiB)  TX bytes:2612713 (2.4 MiB)
>>>>> 
>>>>> pimreg    Link encap:UNSPEC  HWaddr
>>>>> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
>>>>>          UP RUNNING NOARP  MTU:1472  Metric:1
>>>>>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>>          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>>>          collisions:0 txqueuelen:0
>>>>>          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>>>>> 
>>>>> se00      Link encap:Ethernet  HWaddr 2E:B0:5D:A0:C5:B0
>>>>>          inet addr:172.30.42.1  Bcast:172.30.42.31
>>>>> Mask:255.255.255.224
>>>>>          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>> Scope:Global
>>>> 
>>>> How are you assigning your ipv6 addresses?
>>> 
>>> 
>>> It's been a while since I messed with this but I think IPv6 is assigned
>>> thanks to 6relayd? My modem has IPv6 enabled but no DHCPv6 options that I
>>> can find. Here's how cerowrt is configured.
>>> 
>>> root@buffy2-1:/overlay/etc/config# cat 6relayd
>>> config server 'default'
>>>  option fallback_relay 'rd dhcpv6 ndp'
>>>  list network 'ge00'
>>>  list network 'ge01'
>>>  list network 'gw00'
>>>  list network 'gw01'
>>>  list network 'gw10'
>>>  list network 'gw11'
>>>  list network 'se00'
>>>  list network 'sw00'
>>>  list network 'sw10'
>>>  option rd 'relay'
>>>  option dhcpv6 'relay'
>>>  option ndp 'relay'
>>>  option master 'ge00'
>>> 
>>>> 
>>>>>          inet6 addr: fe80::2cb0:5dff:fea0:c5b0/64 Scope:Link
>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>>          TX packets:191740 errors:0 dropped:0 overruns:0 carrier:0
>>>>>          collisions:0 txqueuelen:1000
>>>>>          RX bytes:0 (0.0 B)  TX bytes:42184988 (40.2 MiB)
>>>>>          Interrupt:4
>>>>> 
>>>>> sw00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B0
>>>>>          inet addr:172.30.42.65  Bcast:172.30.42.95
>>>>> Mask:255.255.255.224
>>>>>          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>> Scope:Global
>>>>>          inet6 addr: fe80::2eb0:5dff:fea0:c5b0/64 Scope:Link
>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>          RX packets:70239 errors:0 dropped:0 overruns:0 frame:0
>>>>>          TX packets:286967 errors:0 dropped:0 overruns:0 carrier:0
>>>>>          collisions:0 txqueuelen:1000
>>>>>          RX bytes:15590189 (14.8 MiB)  TX bytes:127357293 (121.4 MiB)
>>>>> 
>>>>> root@buffy2-1:~# less /var/log/babeld.log
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> send: Cannot assign requested address
>>>>> send: Cannot assign requested address
>>>>> send: Cannot assign requested address
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> netlink_read: recvmsg(): No buffer space available
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>> 
>>>> This is a problem in babel detecting the channel on a "normal"
>>>> rather than a mesh interface. It's bugged me a long while, but
>>>> haven't got around to finding what triggers it. Might "fix" it by
>>>> acquiring the channel at babel start time from /etc/config/wireless.
>>>> 
>>>> It messes up the diversity routing calculation, grump.
>>>> 
>>>> There is a possibility a logfile got really big, but this one
>>>> generally doesn't, but I should turn off logging in some
>>>> future release...
>>> 
>>> 
>>> I believe I've tracked down part of what's going on. It looks like my
>>> tmpfs is filling up 100% and then the device enters a bad state:
>>> 
>>> After 24 hours, with tmpfs at 50%, babeld.log is the largest file by far
>>> in tmpfs and the only file that appears to be growing (based on `du`). It
>>> takes about 48 hours from reboot to fill up tmpfs on my device.
>>> 
>>> #  sort babeld.log | uniq -c |sort -rn |head
>>> 
>>> 503236 Couldn't determine channel of interface sw00: Invalid argument.
>>> 
>>>   1376 netlink_read: recvmsg(): No buffer space available
>>> 
>>>      3 send: Cannot assign requested address
>>> 
>>> # wc -l babeld.log
>>> 
>>> 504617 babeld.log
>>> 
>>> I sped up system failure by using `dd` to fill up tmpfs and the system
>>> became immediately unusable.
>>> 
>>> This also explains the luci session store errors as sessions are stored in
>>> tmpfs.
>>> 
>>> The other buffer issues may or may not be related to this.
>>> 
>>> Best,
>>> Steve
>> 
>> 
> 
> 
> 
> -- 
> Dave Täht
> 
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[Dave Taht]

On Wed, Jan 29, 2014 at 4:45 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
> Hi Dave,
>
> quick question, how does one turn of logging for babeld? It seems that if daemonized it defaults to logging to /var/log/babeld.log (or similar). Is setting the log file to /dev/null really the answer?

seems so.

>         (Since I have no the IPv6 issue not yet resolved, I assume babeld is unhappy) I resorted to stopping babeld completely, but that feels like a crutch...

no daemon in an embedded system should ever write to flash in an
uncontrollable manner.

I will also argue that not being able to find the channel is a bug
that messes with diversity routing
in particular.

>
> Best
>         Sebastian
>
>
>
> On Jan 27, 2014, at 22:14 , Dave Taht <dave.taht@gmail.com> wrote:
>
>> certainly turn off the babeld log! I will leave it off in the next release.
>>
>> On Mon, Jan 27, 2014 at 4:10 PM, Steve Jenson <stevej@fruitless.org> wrote:
>>> Looking more, the buffer errors are showing up in syslog well before tmpfs
>>> fills up. Is the memtester openwrt package available for cerowrt? I don't
>>> see it under `Available packages`.
>>>
>>> Thanks,
>>> Steve
>>>
>>>
>>> On Mon, Jan 27, 2014 at 1:06 PM, Steve Jenson <stevej@fruitless.org> wrote:
>>>>
>>>> On Fri, Jan 24, 2014 at 3:23 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>>>>
>>>>> On Fri, Jan 24, 2014 at 6:08 PM, Steve Jenson <stevej@fruitless.org>
>>>>> wrote:
>>>>>> Hi everybody,
>>>>>>
>>>>>> I've been using cerowrt as a secondary wifi network (just a single AP
>>>>>> for
>>>>>> now) for a few weeks now. Recently, my wndr3800 got stuck in a bad
>>>>>> state and
>>>>>> eventually rebooted. I've had this happen a few times now and am
>>>>>> looking for
>>>>>> ways to debug the issue. I'm new to cerowrt and openwrt so any advice
>>>>>> is
>>>>>> appreciated.
>>>>>>
>>>>>> Since I use it as a secondary network, this is no way critical.
>>>>>
>>>>> Yea! I appreciate caution before putting alpha software on your gw.
>>>>>
>>>>>> I'm not
>>>>>> looking for free tech support but I couldn't find anything on the wiki
>>>>>> about
>>>>>> troubleshooting. I'd love to start a page and write some shell scripts
>>>>>> to
>>>>>> diagnose and report issues. I know that a cerowrt router is meant to be
>>>>>> a
>>>>>> research project rather a consumer device but these things seem helpful
>>>>>> regardless.
>>>>>
>>>>> Sure, let me know your wiki account. I have been lax about granting
>>>>> access of late as the signup process is overrun by spammers.
>>>>
>>>>
>>>> My username is stevej on the wiki. Thanks!
>>>>
>>>>
>>>>>
>>>>>> Please let me know if you'd prefer I not email the list with these
>>>>>> issues or
>>>>>> if you'd rather I used trac or a different forum.
>>>>>>
>>>>>
>>>>> The list is where most stuff happens. Also in the irc channel.
>>>>>
>>>>> If it gets to where it needs to be tracked we have a bugtracker at
>>>>>
>>>>> http://www.bufferbloat.net/projects/cerowrt/issues
>>>>>
>>>>> The first question I have is: Are you on comcast? Cerowrt
>>>>> had a dhcpv6-pd implementation that "just worked" from feburary through
>>>>> december. Regrettably they changed the RA announcement interval
>>>>> to a really low number around then... and this triggers a firewall reload
>>>>> every minute on everything prior to the release I point to below.
>>>>>
>>>>> If there is a memory leak somewhere that would have triggered it.
>>>>
>>>>
>>>> I am on AT&T ADSL2+ with a Motorola NVG510 modem.
>>>>
>>>>
>>>>>
>>>>>> In this state, I can connect to the cerowrt base station via wifi but
>>>>>> am
>>>>>> unable to route packets to the internet. I can connect to :81 and see
>>>>>> the
>>>>>> login page but logging in results in a lua error at `/cgi-bin/luci`
>>>>>>
>>>>>>
>>>>>>  /usr/lib/lua/luci/dispatcher.lua:448: Failed to execute function
>>>>>> dispatcher target for entry '/'.
>>>>>>  The called action terminated with an exception:
>>>>>>  /usr/lib/lua/luci/sauth.lua:87: Session data invalid!
>>>>>>  stack traceback:
>>>>>>  [C]: in function 'assert'
>>>>>>  /usr/lib/lua/luci/dispatcher.lua:448: in function 'dispatch'
>>>>>>  /usr/lib/lua/luci/dispatcher.lua:195: in function
>>>>>> </usr/lib/lua/luci/dispatcher.lua:194>
>>>>>>
>>>>>> I can ssh into the device and cat various log files until the router
>>>>>> hangs
>>>>>> and reboots. here's a few relevant lines from my terminal history
>>>>>> before the
>>>>>> device rebooted (I'm assuming a watchdog kicked in and rebooted it).
>>>>>>
>>>>>> root@buffy2-1:~# ping google.com
>>>>>> ping: bad address 'google.com'
>>>>>> root@buffy2-1:~# free
>>>>>>             total         used         free       shared      buffers
>>>>>> Mem:        126336       110332        16004            0         5616
>>>>>> -/+ buffers:             104716        21620
>>>>>> Swap:            0            0            0
>>>>>> root@buffy2-1:~# uptime
>>>>>> 02:08:54 up 2 days,  1:26,  load average: 0.10, 0.21, 0.17
>>>>>> root@buffy2-1:~# dmesg
>>>>>> [    0.000000] Linux version 3.10.24 (cero2@snapon) (gcc version 4.6.4
>>>>>> (OpenWrt/Linaro GCC 4.6-2013.05 r38226) ) #1 Tue Dec 24
>>>>>> 10:50:15 PST 2013
>>>>>> [skipping some lines]
>>>>>>
>>>>>> [   13.156250] Error: Driver 'gpio-keys-polled' is already registered,
>>>>>> aborting...
>>>>>> [   19.414062] IPv6: ADDRCONF(NETDEV_UP): ge00: link is not ready
>>>>>> [   19.421875] ar71xx: pll_reg 0xb8050010: 0x11110000
>>>>>> [   19.429687] se00: link up (1000Mbps/Full duplex)
>>>>>> [   22.140625] IPv6: ADDRCONF(NETDEV_UP): sw00: link is not ready
>>>>>> [   23.351562] IPv6: ADDRCONF(NETDEV_CHANGE): sw00: link becomes ready
>>>>>> [   23.757812] ar71xx: pll_reg 0xb8050014: 0x11110000
>>>>>> [   23.757812] ge00: link up (1000Mbps/Full duplex)
>>>>>> [   23.773437] IPv6: ADDRCONF(NETDEV_CHANGE): ge00: link becomes ready
>>>>>>
>>>>>> root@buffy2-1:~# ifconfig
>>>>>> ge00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B1
>>>>>>          inet addr:192.168.1.138  Bcast:192.168.1.255
>>>>>> Mask:255.255.255.0
>>>>>>          inet6 addr: fe80::2eb0:5dff:fea0:c5b1/64 Scope:Link
>>>>>>          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>>> Scope:Global
>>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>          RX packets:1469670 errors:0 dropped:8 overruns:0 frame:0
>>>>>>          TX packets:547733 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>          collisions:0 txqueuelen:1000
>>>>>>          RX bytes:229243410 (218.6 MiB)  TX bytes:57304808 (54.6 MiB)
>>>>>>          Interrupt:5
>>>>>>
>>>>>> lo        Link encap:Local Loopback
>>>>>>          inet addr:127.0.0.1  Mask:255.0.0.0
>>>>>>          inet6 addr: ::1/128 Scope:Host
>>>>>>          UP LOOPBACK RUNNING  MTU:65536  Metric:1
>>>>>>          RX packets:23689 errors:0 dropped:0 overruns:0 frame:0
>>>>>>          TX packets:23689 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>          collisions:0 txqueuelen:0
>>>>>>          RX bytes:2612713 (2.4 MiB)  TX bytes:2612713 (2.4 MiB)
>>>>>>
>>>>>> pimreg    Link encap:UNSPEC  HWaddr
>>>>>> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
>>>>>>          UP RUNNING NOARP  MTU:1472  Metric:1
>>>>>>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>>>          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>          collisions:0 txqueuelen:0
>>>>>>          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>>>>>>
>>>>>> se00      Link encap:Ethernet  HWaddr 2E:B0:5D:A0:C5:B0
>>>>>>          inet addr:172.30.42.1  Bcast:172.30.42.31
>>>>>> Mask:255.255.255.224
>>>>>>          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>>> Scope:Global
>>>>>
>>>>> How are you assigning your ipv6 addresses?
>>>>
>>>>
>>>> It's been a while since I messed with this but I think IPv6 is assigned
>>>> thanks to 6relayd? My modem has IPv6 enabled but no DHCPv6 options that I
>>>> can find. Here's how cerowrt is configured.
>>>>
>>>> root@buffy2-1:/overlay/etc/config# cat 6relayd
>>>> config server 'default'
>>>>  option fallback_relay 'rd dhcpv6 ndp'
>>>>  list network 'ge00'
>>>>  list network 'ge01'
>>>>  list network 'gw00'
>>>>  list network 'gw01'
>>>>  list network 'gw10'
>>>>  list network 'gw11'
>>>>  list network 'se00'
>>>>  list network 'sw00'
>>>>  list network 'sw10'
>>>>  option rd 'relay'
>>>>  option dhcpv6 'relay'
>>>>  option ndp 'relay'
>>>>  option master 'ge00'
>>>>
>>>>>
>>>>>>          inet6 addr: fe80::2cb0:5dff:fea0:c5b0/64 Scope:Link
>>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>>>          TX packets:191740 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>          collisions:0 txqueuelen:1000
>>>>>>          RX bytes:0 (0.0 B)  TX bytes:42184988 (40.2 MiB)
>>>>>>          Interrupt:4
>>>>>>
>>>>>> sw00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B0
>>>>>>          inet addr:172.30.42.65  Bcast:172.30.42.95
>>>>>> Mask:255.255.255.224
>>>>>>          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>>> Scope:Global
>>>>>>          inet6 addr: fe80::2eb0:5dff:fea0:c5b0/64 Scope:Link
>>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>          RX packets:70239 errors:0 dropped:0 overruns:0 frame:0
>>>>>>          TX packets:286967 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>          collisions:0 txqueuelen:1000
>>>>>>          RX bytes:15590189 (14.8 MiB)  TX bytes:127357293 (121.4 MiB)
>>>>>>
>>>>>> root@buffy2-1:~# less /var/log/babeld.log
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> send: Cannot assign requested address
>>>>>> send: Cannot assign requested address
>>>>>> send: Cannot assign requested address
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> netlink_read: recvmsg(): No buffer space available
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>
>>>>> This is a problem in babel detecting the channel on a "normal"
>>>>> rather than a mesh interface. It's bugged me a long while, but
>>>>> haven't got around to finding what triggers it. Might "fix" it by
>>>>> acquiring the channel at babel start time from /etc/config/wireless.
>>>>>
>>>>> It messes up the diversity routing calculation, grump.
>>>>>
>>>>> There is a possibility a logfile got really big, but this one
>>>>> generally doesn't, but I should turn off logging in some
>>>>> future release...
>>>>
>>>>
>>>> I believe I've tracked down part of what's going on. It looks like my
>>>> tmpfs is filling up 100% and then the device enters a bad state:
>>>>
>>>> After 24 hours, with tmpfs at 50%, babeld.log is the largest file by far
>>>> in tmpfs and the only file that appears to be growing (based on `du`). It
>>>> takes about 48 hours from reboot to fill up tmpfs on my device.
>>>>
>>>> #  sort babeld.log | uniq -c |sort -rn |head
>>>>
>>>> 503236 Couldn't determine channel of interface sw00: Invalid argument.
>>>>
>>>>   1376 netlink_read: recvmsg(): No buffer space available
>>>>
>>>>      3 send: Cannot assign requested address
>>>>
>>>> # wc -l babeld.log
>>>>
>>>> 504617 babeld.log
>>>>
>>>> I sped up system failure by using `dd` to fill up tmpfs and the system
>>>> became immediately unusable.
>>>>
>>>> This also explains the luci session store errors as sessions are stored in
>>>> tmpfs.
>>>>
>>>> The other buffer issues may or may not be related to this.
>>>>
>>>> Best,
>>>> Steve
>>>
>>>
>>
>>
>>
>> --
>> Dave Täht
>>
>> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[Sebastian Moeller]

On January 29, 2014 5:10:18 PM CET, Dave Taht <dave.taht@gmail.com> wrote:
>On Wed, Jan 29, 2014 at 4:45 AM, Sebastian Moeller <moeller0@gmx.de>
>wrote:
>> Hi Dave,
>>
>> quick question, how does one turn of logging for babeld? It seems
>that if daemonized it defaults to logging to /var/log/babeld.log (or
>similar). Is setting the log file to /dev/null really the answer?
>
>seems so.     

      Okay, I guess I will try that then...


>
>>         (Since I have no the IPv6 issue not yet resolved, I assume
>babeld is unhappy) I resorted to stopping babeld completely, but that
>feels like a crutch...
>
>no daemon in an embedded system should ever write to flash in an
>uncontrollable manner.

     What bugs me is that it basically keeps repeating the same error over and over again. If it would rate limit and:or push messages to the system log it would be nicer.


>
>I will also argue that not being able to find the channel is a bug
>that messes with diversity routing
>in particular.

      I have actually not yet understood what it wants to tell me ;), since I got your attention, is there an easy way to run a babel client under macosx?


Best Regards
       Sebastian


>
>>
>> Best
>>         Sebastian
>>
>>
>>
>> On Jan 27, 2014, at 22:14 , Dave Taht <dave.taht@gmail.com> wrote:
>>
>>> certainly turn off the babeld log! I will leave it off in the next
>release.
>>>
>>> On Mon, Jan 27, 2014 at 4:10 PM, Steve Jenson <stevej@fruitless.org>
>wrote:
>>>> Looking more, the buffer errors are showing up in syslog well
>before tmpfs
>>>> fills up. Is the memtester openwrt package available for cerowrt? I
>don't
>>>> see it under `Available packages`.
>>>>
>>>> Thanks,
>>>> Steve
>>>>
>>>>
>>>> On Mon, Jan 27, 2014 at 1:06 PM, Steve Jenson
><stevej@fruitless.org> wrote:
>>>>>
>>>>> On Fri, Jan 24, 2014 at 3:23 PM, Dave Taht <dave.taht@gmail.com>
>wrote:
>>>>>>
>>>>>> On Fri, Jan 24, 2014 at 6:08 PM, Steve Jenson
><stevej@fruitless.org>
>>>>>> wrote:
>>>>>>> Hi everybody,
>>>>>>>
>>>>>>> I've been using cerowrt as a secondary wifi network (just a
>single AP
>>>>>>> for
>>>>>>> now) for a few weeks now. Recently, my wndr3800 got stuck in a
>bad
>>>>>>> state and
>>>>>>> eventually rebooted. I've had this happen a few times now and am
>>>>>>> looking for
>>>>>>> ways to debug the issue. I'm new to cerowrt and openwrt so any
>advice
>>>>>>> is
>>>>>>> appreciated.
>>>>>>>
>>>>>>> Since I use it as a secondary network, this is no way critical.
>>>>>>
>>>>>> Yea! I appreciate caution before putting alpha software on your
>gw.
>>>>>>
>>>>>>> I'm not
>>>>>>> looking for free tech support but I couldn't find anything on
>the wiki
>>>>>>> about
>>>>>>> troubleshooting. I'd love to start a page and write some shell
>scripts
>>>>>>> to
>>>>>>> diagnose and report issues. I know that a cerowrt router is
>meant to be
>>>>>>> a
>>>>>>> research project rather a consumer device but these things seem
>helpful
>>>>>>> regardless.
>>>>>>
>>>>>> Sure, let me know your wiki account. I have been lax about
>granting
>>>>>> access of late as the signup process is overrun by spammers.
>>>>>
>>>>>
>>>>> My username is stevej on the wiki. Thanks!
>>>>>
>>>>>
>>>>>>
>>>>>>> Please let me know if you'd prefer I not email the list with
>these
>>>>>>> issues or
>>>>>>> if you'd rather I used trac or a different forum.
>>>>>>>
>>>>>>
>>>>>> The list is where most stuff happens. Also in the irc channel.
>>>>>>
>>>>>> If it gets to where it needs to be tracked we have a bugtracker
>at
>>>>>>
>>>>>> http://www.bufferbloat.net/projects/cerowrt/issues
>>>>>>
>>>>>> The first question I have is: Are you on comcast? Cerowrt
>>>>>> had a dhcpv6-pd implementation that "just worked" from feburary
>through
>>>>>> december. Regrettably they changed the RA announcement interval
>>>>>> to a really low number around then... and this triggers a
>firewall reload
>>>>>> every minute on everything prior to the release I point to below.
>>>>>>
>>>>>> If there is a memory leak somewhere that would have triggered it.
>>>>>
>>>>>
>>>>> I am on AT&T ADSL2+ with a Motorola NVG510 modem.
>>>>>
>>>>>
>>>>>>
>>>>>>> In this state, I can connect to the cerowrt base station via
>wifi but
>>>>>>> am
>>>>>>> unable to route packets to the internet. I can connect to :81
>and see
>>>>>>> the
>>>>>>> login page but logging in results in a lua error at
>`/cgi-bin/luci`
>>>>>>>
>>>>>>>
>>>>>>>  /usr/lib/lua/luci/dispatcher.lua:448: Failed to execute
>function
>>>>>>> dispatcher target for entry '/'.
>>>>>>>  The called action terminated with an exception:
>>>>>>>  /usr/lib/lua/luci/sauth.lua:87: Session data invalid!
>>>>>>>  stack traceback:
>>>>>>>  [C]: in function 'assert'
>>>>>>>  /usr/lib/lua/luci/dispatcher.lua:448: in function 'dispatch'
>>>>>>>  /usr/lib/lua/luci/dispatcher.lua:195: in function
>>>>>>> </usr/lib/lua/luci/dispatcher.lua:194>
>>>>>>>
>>>>>>> I can ssh into the device and cat various log files until the
>router
>>>>>>> hangs
>>>>>>> and reboots. here's a few relevant lines from my terminal
>history
>>>>>>> before the
>>>>>>> device rebooted (I'm assuming a watchdog kicked in and rebooted
>it).
>>>>>>>
>>>>>>> root@buffy2-1:~# ping google.com
>>>>>>> ping: bad address 'google.com'
>>>>>>> root@buffy2-1:~# free
>>>>>>>             total         used         free       shared     
>buffers
>>>>>>> Mem:        126336       110332        16004            0       
> 5616
>>>>>>> -/+ buffers:             104716        21620
>>>>>>> Swap:            0            0            0
>>>>>>> root@buffy2-1:~# uptime
>>>>>>> 02:08:54 up 2 days,  1:26,  load average: 0.10, 0.21, 0.17
>>>>>>> root@buffy2-1:~# dmesg
>>>>>>> [    0.000000] Linux version 3.10.24 (cero2@snapon) (gcc version
>4.6.4
>>>>>>> (OpenWrt/Linaro GCC 4.6-2013.05 r38226) ) #1 Tue Dec 24
>>>>>>> 10:50:15 PST 2013
>>>>>>> [skipping some lines]
>>>>>>>
>>>>>>> [   13.156250] Error: Driver 'gpio-keys-polled' is already
>registered,
>>>>>>> aborting...
>>>>>>> [   19.414062] IPv6: ADDRCONF(NETDEV_UP): ge00: link is not
>ready
>>>>>>> [   19.421875] ar71xx: pll_reg 0xb8050010: 0x11110000
>>>>>>> [   19.429687] se00: link up (1000Mbps/Full duplex)
>>>>>>> [   22.140625] IPv6: ADDRCONF(NETDEV_UP): sw00: link is not
>ready
>>>>>>> [   23.351562] IPv6: ADDRCONF(NETDEV_CHANGE): sw00: link becomes
>ready
>>>>>>> [   23.757812] ar71xx: pll_reg 0xb8050014: 0x11110000
>>>>>>> [   23.757812] ge00: link up (1000Mbps/Full duplex)
>>>>>>> [   23.773437] IPv6: ADDRCONF(NETDEV_CHANGE): ge00: link becomes
>ready
>>>>>>>
>>>>>>> root@buffy2-1:~# ifconfig
>>>>>>> ge00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B1
>>>>>>>          inet addr:192.168.1.138  Bcast:192.168.1.255
>>>>>>> Mask:255.255.255.0
>>>>>>>          inet6 addr: fe80::2eb0:5dff:fea0:c5b1/64 Scope:Link
>>>>>>>          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>>>> Scope:Global
>>>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>>          RX packets:1469670 errors:0 dropped:8 overruns:0
>frame:0
>>>>>>>          TX packets:547733 errors:0 dropped:0 overruns:0
>carrier:0
>>>>>>>          collisions:0 txqueuelen:1000
>>>>>>>          RX bytes:229243410 (218.6 MiB)  TX bytes:57304808 (54.6
>MiB)
>>>>>>>          Interrupt:5
>>>>>>>
>>>>>>> lo        Link encap:Local Loopback
>>>>>>>          inet addr:127.0.0.1  Mask:255.0.0.0
>>>>>>>          inet6 addr: ::1/128 Scope:Host
>>>>>>>          UP LOOPBACK RUNNING  MTU:65536  Metric:1
>>>>>>>          RX packets:23689 errors:0 dropped:0 overruns:0 frame:0
>>>>>>>          TX packets:23689 errors:0 dropped:0 overruns:0
>carrier:0
>>>>>>>          collisions:0 txqueuelen:0
>>>>>>>          RX bytes:2612713 (2.4 MiB)  TX bytes:2612713 (2.4 MiB)
>>>>>>>
>>>>>>> pimreg    Link encap:UNSPEC  HWaddr
>>>>>>> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
>>>>>>>          UP RUNNING NOARP  MTU:1472  Metric:1
>>>>>>>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>>>>          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>>          collisions:0 txqueuelen:0
>>>>>>>          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>>>>>>>
>>>>>>> se00      Link encap:Ethernet  HWaddr 2E:B0:5D:A0:C5:B0
>>>>>>>          inet addr:172.30.42.1  Bcast:172.30.42.31
>>>>>>> Mask:255.255.255.224
>>>>>>>          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>>>> Scope:Global
>>>>>>
>>>>>> How are you assigning your ipv6 addresses?
>>>>>
>>>>>
>>>>> It's been a while since I messed with this but I think IPv6 is
>assigned
>>>>> thanks to 6relayd? My modem has IPv6 enabled but no DHCPv6 options
>that I
>>>>> can find. Here's how cerowrt is configured.
>>>>>
>>>>> root@buffy2-1:/overlay/etc/config# cat 6relayd
>>>>> config server 'default'
>>>>>  option fallback_relay 'rd dhcpv6 ndp'
>>>>>  list network 'ge00'
>>>>>  list network 'ge01'
>>>>>  list network 'gw00'
>>>>>  list network 'gw01'
>>>>>  list network 'gw10'
>>>>>  list network 'gw11'
>>>>>  list network 'se00'
>>>>>  list network 'sw00'
>>>>>  list network 'sw10'
>>>>>  option rd 'relay'
>>>>>  option dhcpv6 'relay'
>>>>>  option ndp 'relay'
>>>>>  option master 'ge00'
>>>>>
>>>>>>
>>>>>>>          inet6 addr: fe80::2cb0:5dff:fea0:c5b0/64 Scope:Link
>>>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>>>>          TX packets:191740 errors:0 dropped:0 overruns:0
>carrier:0
>>>>>>>          collisions:0 txqueuelen:1000
>>>>>>>          RX bytes:0 (0.0 B)  TX bytes:42184988 (40.2 MiB)
>>>>>>>          Interrupt:4
>>>>>>>
>>>>>>> sw00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B0
>>>>>>>          inet addr:172.30.42.65  Bcast:172.30.42.95
>>>>>>> Mask:255.255.255.224
>>>>>>>          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>>>> Scope:Global
>>>>>>>          inet6 addr: fe80::2eb0:5dff:fea0:c5b0/64 Scope:Link
>>>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>>          RX packets:70239 errors:0 dropped:0 overruns:0 frame:0
>>>>>>>          TX packets:286967 errors:0 dropped:0 overruns:0
>carrier:0
>>>>>>>          collisions:0 txqueuelen:1000
>>>>>>>          RX bytes:15590189 (14.8 MiB)  TX bytes:127357293 (121.4
>MiB)
>>>>>>>
>>>>>>> root@buffy2-1:~# less /var/log/babeld.log
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> send: Cannot assign requested address
>>>>>>> send: Cannot assign requested address
>>>>>>> send: Cannot assign requested address
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> netlink_read: recvmsg(): No buffer space available
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>
>>>>>> This is a problem in babel detecting the channel on a "normal"
>>>>>> rather than a mesh interface. It's bugged me a long while, but
>>>>>> haven't got around to finding what triggers it. Might "fix" it by
>>>>>> acquiring the channel at babel start time from
>/etc/config/wireless.
>>>>>>
>>>>>> It messes up the diversity routing calculation, grump.
>>>>>>
>>>>>> There is a possibility a logfile got really big, but this one
>>>>>> generally doesn't, but I should turn off logging in some
>>>>>> future release...
>>>>>
>>>>>
>>>>> I believe I've tracked down part of what's going on. It looks like
>my
>>>>> tmpfs is filling up 100% and then the device enters a bad state:
>>>>>
>>>>> After 24 hours, with tmpfs at 50%, babeld.log is the largest file
>by far
>>>>> in tmpfs and the only file that appears to be growing (based on
>`du`). It
>>>>> takes about 48 hours from reboot to fill up tmpfs on my device.
>>>>>
>>>>> #  sort babeld.log | uniq -c |sort -rn |head
>>>>>
>>>>> 503236 Couldn't determine channel of interface sw00: Invalid
>argument.
>>>>>
>>>>>   1376 netlink_read: recvmsg(): No buffer space available
>>>>>
>>>>>      3 send: Cannot assign requested address
>>>>>
>>>>> # wc -l babeld.log
>>>>>
>>>>> 504617 babeld.log
>>>>>
>>>>> I sped up system failure by using `dd` to fill up tmpfs and the
>system
>>>>> became immediately unusable.
>>>>>
>>>>> This also explains the luci session store errors as sessions are
>stored in
>>>>> tmpfs.
>>>>>
>>>>> The other buffer issues may or may not be related to this.
>>>>>
>>>>> Best,
>>>>> Steve
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Dave Täht
>>>
>>> Fixing bufferbloat with cerowrt:
>http://www.teklibre.com/cerowrt/subscribe.html
>>> _______________________________________________
>>> Cerowrt-devel mailing list
>>> Cerowrt-devel@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>

Hi Dave,
-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[Dave Taht]

On Wed, Jan 29, 2014 at 9:44 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
> On January 29, 2014 5:10:18 PM CET, Dave Taht <dave.taht@gmail.com> wrote:
>>On Wed, Jan 29, 2014 at 4:45 AM, Sebastian Moeller <moeller0@gmx.de>
>>wrote:
>>> Hi Dave,
>>>
>>> quick question, how does one turn of logging for babeld? It seems
>>that if daemonized it defaults to logging to /var/log/babeld.log (or
>>similar). Is setting the log file to /dev/null really the answer?
>>
>>seems so.
>
>       Okay, I guess I will try that then...
>
>
>>
>>>         (Since I have no the IPv6 issue not yet resolved, I assume
>>babeld is unhappy) I resorted to stopping babeld completely, but that
>>feels like a crutch...
>>
>>no daemon in an embedded system should ever write to flash in an
>>uncontrollable manner.
>
>      What bugs me is that it basically keeps repeating the same error over and over again. If it would rate limit and:or push messages to the system log it would be nicer.
>
>
>>
>>I will also argue that not being able to find the channel is a bug
>>that messes with diversity routing
>>in particular.
>
>       I have actually not yet understood what it wants to tell me ;), since I got your attention, is there an easy way to run a babel client under macosx?

for coping with the mac I use "macports" to get a compiler and support
for open source software.

I haven't ever tried to run babeld on the mac I have, I will put it on
my list...

>
>
> Best Regards
>        Sebastian
>
>
>>
>>>
>>> Best
>>>         Sebastian
>>>
>>>
>>>
>>> On Jan 27, 2014, at 22:14 , Dave Taht <dave.taht@gmail.com> wrote:
>>>
>>>> certainly turn off the babeld log! I will leave it off in the next
>>release.
>>>>
>>>> On Mon, Jan 27, 2014 at 4:10 PM, Steve Jenson <stevej@fruitless.org>
>>wrote:
>>>>> Looking more, the buffer errors are showing up in syslog well
>>before tmpfs
>>>>> fills up. Is the memtester openwrt package available for cerowrt? I
>>don't
>>>>> see it under `Available packages`.
>>>>>
>>>>> Thanks,
>>>>> Steve
>>>>>
>>>>>
>>>>> On Mon, Jan 27, 2014 at 1:06 PM, Steve Jenson
>><stevej@fruitless.org> wrote:
>>>>>>
>>>>>> On Fri, Jan 24, 2014 at 3:23 PM, Dave Taht <dave.taht@gmail.com>
>>wrote:
>>>>>>>
>>>>>>> On Fri, Jan 24, 2014 at 6:08 PM, Steve Jenson
>><stevej@fruitless.org>
>>>>>>> wrote:
>>>>>>>> Hi everybody,
>>>>>>>>
>>>>>>>> I've been using cerowrt as a secondary wifi network (just a
>>single AP
>>>>>>>> for
>>>>>>>> now) for a few weeks now. Recently, my wndr3800 got stuck in a
>>bad
>>>>>>>> state and
>>>>>>>> eventually rebooted. I've had this happen a few times now and am
>>>>>>>> looking for
>>>>>>>> ways to debug the issue. I'm new to cerowrt and openwrt so any
>>advice
>>>>>>>> is
>>>>>>>> appreciated.
>>>>>>>>
>>>>>>>> Since I use it as a secondary network, this is no way critical.
>>>>>>>
>>>>>>> Yea! I appreciate caution before putting alpha software on your
>>gw.
>>>>>>>
>>>>>>>> I'm not
>>>>>>>> looking for free tech support but I couldn't find anything on
>>the wiki
>>>>>>>> about
>>>>>>>> troubleshooting. I'd love to start a page and write some shell
>>scripts
>>>>>>>> to
>>>>>>>> diagnose and report issues. I know that a cerowrt router is
>>meant to be
>>>>>>>> a
>>>>>>>> research project rather a consumer device but these things seem
>>helpful
>>>>>>>> regardless.
>>>>>>>
>>>>>>> Sure, let me know your wiki account. I have been lax about
>>granting
>>>>>>> access of late as the signup process is overrun by spammers.
>>>>>>
>>>>>>
>>>>>> My username is stevej on the wiki. Thanks!
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>> Please let me know if you'd prefer I not email the list with
>>these
>>>>>>>> issues or
>>>>>>>> if you'd rather I used trac or a different forum.
>>>>>>>>
>>>>>>>
>>>>>>> The list is where most stuff happens. Also in the irc channel.
>>>>>>>
>>>>>>> If it gets to where it needs to be tracked we have a bugtracker
>>at
>>>>>>>
>>>>>>> http://www.bufferbloat.net/projects/cerowrt/issues
>>>>>>>
>>>>>>> The first question I have is: Are you on comcast? Cerowrt
>>>>>>> had a dhcpv6-pd implementation that "just worked" from feburary
>>through
>>>>>>> december. Regrettably they changed the RA announcement interval
>>>>>>> to a really low number around then... and this triggers a
>>firewall reload
>>>>>>> every minute on everything prior to the release I point to below.
>>>>>>>
>>>>>>> If there is a memory leak somewhere that would have triggered it.
>>>>>>
>>>>>>
>>>>>> I am on AT&T ADSL2+ with a Motorola NVG510 modem.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>> In this state, I can connect to the cerowrt base station via
>>wifi but
>>>>>>>> am
>>>>>>>> unable to route packets to the internet. I can connect to :81
>>and see
>>>>>>>> the
>>>>>>>> login page but logging in results in a lua error at
>>`/cgi-bin/luci`
>>>>>>>>
>>>>>>>>
>>>>>>>>  /usr/lib/lua/luci/dispatcher.lua:448: Failed to execute
>>function
>>>>>>>> dispatcher target for entry '/'.
>>>>>>>>  The called action terminated with an exception:
>>>>>>>>  /usr/lib/lua/luci/sauth.lua:87: Session data invalid!
>>>>>>>>  stack traceback:
>>>>>>>>  [C]: in function 'assert'
>>>>>>>>  /usr/lib/lua/luci/dispatcher.lua:448: in function 'dispatch'
>>>>>>>>  /usr/lib/lua/luci/dispatcher.lua:195: in function
>>>>>>>> </usr/lib/lua/luci/dispatcher.lua:194>
>>>>>>>>
>>>>>>>> I can ssh into the device and cat various log files until the
>>router
>>>>>>>> hangs
>>>>>>>> and reboots. here's a few relevant lines from my terminal
>>history
>>>>>>>> before the
>>>>>>>> device rebooted (I'm assuming a watchdog kicked in and rebooted
>>it).
>>>>>>>>
>>>>>>>> root@buffy2-1:~# ping google.com
>>>>>>>> ping: bad address 'google.com'
>>>>>>>> root@buffy2-1:~# free
>>>>>>>>             total         used         free       shared
>>buffers
>>>>>>>> Mem:        126336       110332        16004            0
>> 5616
>>>>>>>> -/+ buffers:             104716        21620
>>>>>>>> Swap:            0            0            0
>>>>>>>> root@buffy2-1:~# uptime
>>>>>>>> 02:08:54 up 2 days,  1:26,  load average: 0.10, 0.21, 0.17
>>>>>>>> root@buffy2-1:~# dmesg
>>>>>>>> [    0.000000] Linux version 3.10.24 (cero2@snapon) (gcc version
>>4.6.4
>>>>>>>> (OpenWrt/Linaro GCC 4.6-2013.05 r38226) ) #1 Tue Dec 24
>>>>>>>> 10:50:15 PST 2013
>>>>>>>> [skipping some lines]
>>>>>>>>
>>>>>>>> [   13.156250] Error: Driver 'gpio-keys-polled' is already
>>registered,
>>>>>>>> aborting...
>>>>>>>> [   19.414062] IPv6: ADDRCONF(NETDEV_UP): ge00: link is not
>>ready
>>>>>>>> [   19.421875] ar71xx: pll_reg 0xb8050010: 0x11110000
>>>>>>>> [   19.429687] se00: link up (1000Mbps/Full duplex)
>>>>>>>> [   22.140625] IPv6: ADDRCONF(NETDEV_UP): sw00: link is not
>>ready
>>>>>>>> [   23.351562] IPv6: ADDRCONF(NETDEV_CHANGE): sw00: link becomes
>>ready
>>>>>>>> [   23.757812] ar71xx: pll_reg 0xb8050014: 0x11110000
>>>>>>>> [   23.757812] ge00: link up (1000Mbps/Full duplex)
>>>>>>>> [   23.773437] IPv6: ADDRCONF(NETDEV_CHANGE): ge00: link becomes
>>ready
>>>>>>>>
>>>>>>>> root@buffy2-1:~# ifconfig
>>>>>>>> ge00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B1
>>>>>>>>          inet addr:192.168.1.138  Bcast:192.168.1.255
>>>>>>>> Mask:255.255.255.0
>>>>>>>>          inet6 addr: fe80::2eb0:5dff:fea0:c5b1/64 Scope:Link
>>>>>>>>          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>>>>> Scope:Global
>>>>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>>>          RX packets:1469670 errors:0 dropped:8 overruns:0
>>frame:0
>>>>>>>>          TX packets:547733 errors:0 dropped:0 overruns:0
>>carrier:0
>>>>>>>>          collisions:0 txqueuelen:1000
>>>>>>>>          RX bytes:229243410 (218.6 MiB)  TX bytes:57304808 (54.6
>>MiB)
>>>>>>>>          Interrupt:5
>>>>>>>>
>>>>>>>> lo        Link encap:Local Loopback
>>>>>>>>          inet addr:127.0.0.1  Mask:255.0.0.0
>>>>>>>>          inet6 addr: ::1/128 Scope:Host
>>>>>>>>          UP LOOPBACK RUNNING  MTU:65536  Metric:1
>>>>>>>>          RX packets:23689 errors:0 dropped:0 overruns:0 frame:0
>>>>>>>>          TX packets:23689 errors:0 dropped:0 overruns:0
>>carrier:0
>>>>>>>>          collisions:0 txqueuelen:0
>>>>>>>>          RX bytes:2612713 (2.4 MiB)  TX bytes:2612713 (2.4 MiB)
>>>>>>>>
>>>>>>>> pimreg    Link encap:UNSPEC  HWaddr
>>>>>>>> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
>>>>>>>>          UP RUNNING NOARP  MTU:1472  Metric:1
>>>>>>>>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>>>>>          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>>>          collisions:0 txqueuelen:0
>>>>>>>>          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>>>>>>>>
>>>>>>>> se00      Link encap:Ethernet  HWaddr 2E:B0:5D:A0:C5:B0
>>>>>>>>          inet addr:172.30.42.1  Bcast:172.30.42.31
>>>>>>>> Mask:255.255.255.224
>>>>>>>>          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>>>>> Scope:Global
>>>>>>>
>>>>>>> How are you assigning your ipv6 addresses?
>>>>>>
>>>>>>
>>>>>> It's been a while since I messed with this but I think IPv6 is
>>assigned
>>>>>> thanks to 6relayd? My modem has IPv6 enabled but no DHCPv6 options
>>that I
>>>>>> can find. Here's how cerowrt is configured.
>>>>>>
>>>>>> root@buffy2-1:/overlay/etc/config# cat 6relayd
>>>>>> config server 'default'
>>>>>>  option fallback_relay 'rd dhcpv6 ndp'
>>>>>>  list network 'ge00'
>>>>>>  list network 'ge01'
>>>>>>  list network 'gw00'
>>>>>>  list network 'gw01'
>>>>>>  list network 'gw10'
>>>>>>  list network 'gw11'
>>>>>>  list network 'se00'
>>>>>>  list network 'sw00'
>>>>>>  list network 'sw10'
>>>>>>  option rd 'relay'
>>>>>>  option dhcpv6 'relay'
>>>>>>  option ndp 'relay'
>>>>>>  option master 'ge00'
>>>>>>
>>>>>>>
>>>>>>>>          inet6 addr: fe80::2cb0:5dff:fea0:c5b0/64 Scope:Link
>>>>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>>>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>>>>>          TX packets:191740 errors:0 dropped:0 overruns:0
>>carrier:0
>>>>>>>>          collisions:0 txqueuelen:1000
>>>>>>>>          RX bytes:0 (0.0 B)  TX bytes:42184988 (40.2 MiB)
>>>>>>>>          Interrupt:4
>>>>>>>>
>>>>>>>> sw00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B0
>>>>>>>>          inet addr:172.30.42.65  Bcast:172.30.42.95
>>>>>>>> Mask:255.255.255.224
>>>>>>>>          inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>>>>> Scope:Global
>>>>>>>>          inet6 addr: fe80::2eb0:5dff:fea0:c5b0/64 Scope:Link
>>>>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>>>          RX packets:70239 errors:0 dropped:0 overruns:0 frame:0
>>>>>>>>          TX packets:286967 errors:0 dropped:0 overruns:0
>>carrier:0
>>>>>>>>          collisions:0 txqueuelen:1000
>>>>>>>>          RX bytes:15590189 (14.8 MiB)  TX bytes:127357293 (121.4
>>MiB)
>>>>>>>>
>>>>>>>> root@buffy2-1:~# less /var/log/babeld.log
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> send: Cannot assign requested address
>>>>>>>> send: Cannot assign requested address
>>>>>>>> send: Cannot assign requested address
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> netlink_read: recvmsg(): No buffer space available
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>
>>>>>>> This is a problem in babel detecting the channel on a "normal"
>>>>>>> rather than a mesh interface. It's bugged me a long while, but
>>>>>>> haven't got around to finding what triggers it. Might "fix" it by
>>>>>>> acquiring the channel at babel start time from
>>/etc/config/wireless.
>>>>>>>
>>>>>>> It messes up the diversity routing calculation, grump.
>>>>>>>
>>>>>>> There is a possibility a logfile got really big, but this one
>>>>>>> generally doesn't, but I should turn off logging in some
>>>>>>> future release...
>>>>>>
>>>>>>
>>>>>> I believe I've tracked down part of what's going on. It looks like
>>my
>>>>>> tmpfs is filling up 100% and then the device enters a bad state:
>>>>>>
>>>>>> After 24 hours, with tmpfs at 50%, babeld.log is the largest file
>>by far
>>>>>> in tmpfs and the only file that appears to be growing (based on
>>`du`). It
>>>>>> takes about 48 hours from reboot to fill up tmpfs on my device.
>>>>>>
>>>>>> #  sort babeld.log | uniq -c |sort -rn |head
>>>>>>
>>>>>> 503236 Couldn't determine channel of interface sw00: Invalid
>>argument.
>>>>>>
>>>>>>   1376 netlink_read: recvmsg(): No buffer space available
>>>>>>
>>>>>>      3 send: Cannot assign requested address
>>>>>>
>>>>>> # wc -l babeld.log
>>>>>>
>>>>>> 504617 babeld.log
>>>>>>
>>>>>> I sped up system failure by using `dd` to fill up tmpfs and the
>>system
>>>>>> became immediately unusable.
>>>>>>
>>>>>> This also explains the luci session store errors as sessions are
>>stored in
>>>>>> tmpfs.
>>>>>>
>>>>>> The other buffer issues may or may not be related to this.
>>>>>>
>>>>>> Best,
>>>>>> Steve
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Dave Täht
>>>>
>>>> Fixing bufferbloat with cerowrt:
>>http://www.teklibre.com/cerowrt/subscribe.html
>>>> _______________________________________________
>>>> Cerowrt-devel mailing list
>>>> Cerowrt-devel@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>>
>
> Hi Dave,
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[Sebastian Moeller]

Hi Dave,

On Jan 29, 2014, at 19:21 , Dave Taht <dave.taht@gmail.com> wrote:

> On Wed, Jan 29, 2014 at 9:44 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
>> On January 29, 2014 5:10:18 PM CET, Dave Taht <dave.taht@gmail.com> wrote:
>>> On Wed, Jan 29, 2014 at 4:45 AM, Sebastian Moeller <moeller0@gmx.de>
>>> wrote:
>>>> Hi Dave,
>>>> 
>>>> quick question, how does one turn of logging for babeld? It seems
>>> that if daemonized it defaults to logging to /var/log/babeld.log (or
>>> similar). Is setting the log file to /dev/null really the answer?
>>> 
>>> seems so.
>> 
>>      Okay, I guess I will try that then...
>> 
>> 
>>> 
>>>>        (Since I have no the IPv6 issue not yet resolved, I assume
>>> babeld is unhappy) I resorted to stopping babeld completely, but that
>>> feels like a crutch...
>>> 
>>> no daemon in an embedded system should ever write to flash in an
>>> uncontrollable manner.
>> 
>>     What bugs me is that it basically keeps repeating the same error over and over again. If it would rate limit and:or push messages to the system log it would be nicer.
>> 
>> 
>>> 
>>> I will also argue that not being able to find the channel is a bug
>>> that messes with diversity routing
>>> in particular.
>> 
>>      I have actually not yet understood what it wants to tell me ;), since I got your attention, is there an easy way to run a babel client under macosx?
> 
> for coping with the mac I use "macports" to get a compiler and support
> for open source software.

	Ah, same here (even though I would have thought you a homebrew user, no idea why) ;)
Alas, "port search babel" does not find anything babeld related...

> 
> I haven't ever tried to run babeld on the mac I have, I will put it on
> my list…

	I just thought it would be nice to finally get into the "stay connected while switching between wired and wire-less fun", but this is in no way essential for me. 

Best Regards
	Sebastian

> 
>> 
>> 
>> Best Regards
>>       Sebastian
>> 
>> 
>>> 
>>>> 
>>>> Best
>>>>        Sebastian
>>>> 
>>>> 
>>>> 
>>>> On Jan 27, 2014, at 22:14 , Dave Taht <dave.taht@gmail.com> wrote:
>>>> 
>>>>> certainly turn off the babeld log! I will leave it off in the next
>>> release.
>>>>> 
>>>>> On Mon, Jan 27, 2014 at 4:10 PM, Steve Jenson <stevej@fruitless.org>
>>> wrote:
>>>>>> Looking more, the buffer errors are showing up in syslog well
>>> before tmpfs
>>>>>> fills up. Is the memtester openwrt package available for cerowrt? I
>>> don't
>>>>>> see it under `Available packages`.
>>>>>> 
>>>>>> Thanks,
>>>>>> Steve
>>>>>> 
>>>>>> 
>>>>>> On Mon, Jan 27, 2014 at 1:06 PM, Steve Jenson
>>> <stevej@fruitless.org> wrote:
>>>>>>> 
>>>>>>> On Fri, Jan 24, 2014 at 3:23 PM, Dave Taht <dave.taht@gmail.com>
>>> wrote:
>>>>>>>> 
>>>>>>>> On Fri, Jan 24, 2014 at 6:08 PM, Steve Jenson
>>> <stevej@fruitless.org>
>>>>>>>> wrote:
>>>>>>>>> Hi everybody,
>>>>>>>>> 
>>>>>>>>> I've been using cerowrt as a secondary wifi network (just a
>>> single AP
>>>>>>>>> for
>>>>>>>>> now) for a few weeks now. Recently, my wndr3800 got stuck in a
>>> bad
>>>>>>>>> state and
>>>>>>>>> eventually rebooted. I've had this happen a few times now and am
>>>>>>>>> looking for
>>>>>>>>> ways to debug the issue. I'm new to cerowrt and openwrt so any
>>> advice
>>>>>>>>> is
>>>>>>>>> appreciated.
>>>>>>>>> 
>>>>>>>>> Since I use it as a secondary network, this is no way critical.
>>>>>>>> 
>>>>>>>> Yea! I appreciate caution before putting alpha software on your
>>> gw.
>>>>>>>> 
>>>>>>>>> I'm not
>>>>>>>>> looking for free tech support but I couldn't find anything on
>>> the wiki
>>>>>>>>> about
>>>>>>>>> troubleshooting. I'd love to start a page and write some shell
>>> scripts
>>>>>>>>> to
>>>>>>>>> diagnose and report issues. I know that a cerowrt router is
>>> meant to be
>>>>>>>>> a
>>>>>>>>> research project rather a consumer device but these things seem
>>> helpful
>>>>>>>>> regardless.
>>>>>>>> 
>>>>>>>> Sure, let me know your wiki account. I have been lax about
>>> granting
>>>>>>>> access of late as the signup process is overrun by spammers.
>>>>>>> 
>>>>>>> 
>>>>>>> My username is stevej on the wiki. Thanks!
>>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>>> Please let me know if you'd prefer I not email the list with
>>> these
>>>>>>>>> issues or
>>>>>>>>> if you'd rather I used trac or a different forum.
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> The list is where most stuff happens. Also in the irc channel.
>>>>>>>> 
>>>>>>>> If it gets to where it needs to be tracked we have a bugtracker
>>> at
>>>>>>>> 
>>>>>>>> http://www.bufferbloat.net/projects/cerowrt/issues
>>>>>>>> 
>>>>>>>> The first question I have is: Are you on comcast? Cerowrt
>>>>>>>> had a dhcpv6-pd implementation that "just worked" from feburary
>>> through
>>>>>>>> december. Regrettably they changed the RA announcement interval
>>>>>>>> to a really low number around then... and this triggers a
>>> firewall reload
>>>>>>>> every minute on everything prior to the release I point to below.
>>>>>>>> 
>>>>>>>> If there is a memory leak somewhere that would have triggered it.
>>>>>>> 
>>>>>>> 
>>>>>>> I am on AT&T ADSL2+ with a Motorola NVG510 modem.
>>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>>> In this state, I can connect to the cerowrt base station via
>>> wifi but
>>>>>>>>> am
>>>>>>>>> unable to route packets to the internet. I can connect to :81
>>> and see
>>>>>>>>> the
>>>>>>>>> login page but logging in results in a lua error at
>>> `/cgi-bin/luci`
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> /usr/lib/lua/luci/dispatcher.lua:448: Failed to execute
>>> function
>>>>>>>>> dispatcher target for entry '/'.
>>>>>>>>> The called action terminated with an exception:
>>>>>>>>> /usr/lib/lua/luci/sauth.lua:87: Session data invalid!
>>>>>>>>> stack traceback:
>>>>>>>>> [C]: in function 'assert'
>>>>>>>>> /usr/lib/lua/luci/dispatcher.lua:448: in function 'dispatch'
>>>>>>>>> /usr/lib/lua/luci/dispatcher.lua:195: in function
>>>>>>>>> </usr/lib/lua/luci/dispatcher.lua:194>
>>>>>>>>> 
>>>>>>>>> I can ssh into the device and cat various log files until the
>>> router
>>>>>>>>> hangs
>>>>>>>>> and reboots. here's a few relevant lines from my terminal
>>> history
>>>>>>>>> before the
>>>>>>>>> device rebooted (I'm assuming a watchdog kicked in and rebooted
>>> it).
>>>>>>>>> 
>>>>>>>>> root@buffy2-1:~# ping google.com
>>>>>>>>> ping: bad address 'google.com'
>>>>>>>>> root@buffy2-1:~# free
>>>>>>>>>            total         used         free       shared
>>> buffers
>>>>>>>>> Mem:        126336       110332        16004            0
>>> 5616
>>>>>>>>> -/+ buffers:             104716        21620
>>>>>>>>> Swap:            0            0            0
>>>>>>>>> root@buffy2-1:~# uptime
>>>>>>>>> 02:08:54 up 2 days,  1:26,  load average: 0.10, 0.21, 0.17
>>>>>>>>> root@buffy2-1:~# dmesg
>>>>>>>>> [    0.000000] Linux version 3.10.24 (cero2@snapon) (gcc version
>>> 4.6.4
>>>>>>>>> (OpenWrt/Linaro GCC 4.6-2013.05 r38226) ) #1 Tue Dec 24
>>>>>>>>> 10:50:15 PST 2013
>>>>>>>>> [skipping some lines]
>>>>>>>>> 
>>>>>>>>> [   13.156250] Error: Driver 'gpio-keys-polled' is already
>>> registered,
>>>>>>>>> aborting...
>>>>>>>>> [   19.414062] IPv6: ADDRCONF(NETDEV_UP): ge00: link is not
>>> ready
>>>>>>>>> [   19.421875] ar71xx: pll_reg 0xb8050010: 0x11110000
>>>>>>>>> [   19.429687] se00: link up (1000Mbps/Full duplex)
>>>>>>>>> [   22.140625] IPv6: ADDRCONF(NETDEV_UP): sw00: link is not
>>> ready
>>>>>>>>> [   23.351562] IPv6: ADDRCONF(NETDEV_CHANGE): sw00: link becomes
>>> ready
>>>>>>>>> [   23.757812] ar71xx: pll_reg 0xb8050014: 0x11110000
>>>>>>>>> [   23.757812] ge00: link up (1000Mbps/Full duplex)
>>>>>>>>> [   23.773437] IPv6: ADDRCONF(NETDEV_CHANGE): ge00: link becomes
>>> ready
>>>>>>>>> 
>>>>>>>>> root@buffy2-1:~# ifconfig
>>>>>>>>> ge00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B1
>>>>>>>>>         inet addr:192.168.1.138  Bcast:192.168.1.255
>>>>>>>>> Mask:255.255.255.0
>>>>>>>>>         inet6 addr: fe80::2eb0:5dff:fea0:c5b1/64 Scope:Link
>>>>>>>>>         inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>>>>>> Scope:Global
>>>>>>>>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>>>>         RX packets:1469670 errors:0 dropped:8 overruns:0
>>> frame:0
>>>>>>>>>         TX packets:547733 errors:0 dropped:0 overruns:0
>>> carrier:0
>>>>>>>>>         collisions:0 txqueuelen:1000
>>>>>>>>>         RX bytes:229243410 (218.6 MiB)  TX bytes:57304808 (54.6
>>> MiB)
>>>>>>>>>         Interrupt:5
>>>>>>>>> 
>>>>>>>>> lo        Link encap:Local Loopback
>>>>>>>>>         inet addr:127.0.0.1  Mask:255.0.0.0
>>>>>>>>>         inet6 addr: ::1/128 Scope:Host
>>>>>>>>>         UP LOOPBACK RUNNING  MTU:65536  Metric:1
>>>>>>>>>         RX packets:23689 errors:0 dropped:0 overruns:0 frame:0
>>>>>>>>>         TX packets:23689 errors:0 dropped:0 overruns:0
>>> carrier:0
>>>>>>>>>         collisions:0 txqueuelen:0
>>>>>>>>>         RX bytes:2612713 (2.4 MiB)  TX bytes:2612713 (2.4 MiB)
>>>>>>>>> 
>>>>>>>>> pimreg    Link encap:UNSPEC  HWaddr
>>>>>>>>> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
>>>>>>>>>         UP RUNNING NOARP  MTU:1472  Metric:1
>>>>>>>>>         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>>>>>>         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>>>>         collisions:0 txqueuelen:0
>>>>>>>>>         RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>>>>>>>>> 
>>>>>>>>> se00      Link encap:Ethernet  HWaddr 2E:B0:5D:A0:C5:B0
>>>>>>>>>         inet addr:172.30.42.1  Bcast:172.30.42.31
>>>>>>>>> Mask:255.255.255.224
>>>>>>>>>         inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>>>>>> Scope:Global
>>>>>>>> 
>>>>>>>> How are you assigning your ipv6 addresses?
>>>>>>> 
>>>>>>> 
>>>>>>> It's been a while since I messed with this but I think IPv6 is
>>> assigned
>>>>>>> thanks to 6relayd? My modem has IPv6 enabled but no DHCPv6 options
>>> that I
>>>>>>> can find. Here's how cerowrt is configured.
>>>>>>> 
>>>>>>> root@buffy2-1:/overlay/etc/config# cat 6relayd
>>>>>>> config server 'default'
>>>>>>> option fallback_relay 'rd dhcpv6 ndp'
>>>>>>> list network 'ge00'
>>>>>>> list network 'ge01'
>>>>>>> list network 'gw00'
>>>>>>> list network 'gw01'
>>>>>>> list network 'gw10'
>>>>>>> list network 'gw11'
>>>>>>> list network 'se00'
>>>>>>> list network 'sw00'
>>>>>>> list network 'sw10'
>>>>>>> option rd 'relay'
>>>>>>> option dhcpv6 'relay'
>>>>>>> option ndp 'relay'
>>>>>>> option master 'ge00'
>>>>>>> 
>>>>>>>> 
>>>>>>>>>         inet6 addr: fe80::2cb0:5dff:fea0:c5b0/64 Scope:Link
>>>>>>>>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>>>>         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>>>>>>         TX packets:191740 errors:0 dropped:0 overruns:0
>>> carrier:0
>>>>>>>>>         collisions:0 txqueuelen:1000
>>>>>>>>>         RX bytes:0 (0.0 B)  TX bytes:42184988 (40.2 MiB)
>>>>>>>>>         Interrupt:4
>>>>>>>>> 
>>>>>>>>> sw00      Link encap:Ethernet  HWaddr 2C:B0:5D:A0:C5:B0
>>>>>>>>>         inet addr:172.30.42.65  Bcast:172.30.42.95
>>>>>>>>> Mask:255.255.255.224
>>>>>>>>>         inet6 addr: 2602:30a:2cdb:330:2eb0:5dff:fea0:c5b1/64
>>>>>>>>> Scope:Global
>>>>>>>>>         inet6 addr: fe80::2eb0:5dff:fea0:c5b0/64 Scope:Link
>>>>>>>>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>>>>         RX packets:70239 errors:0 dropped:0 overruns:0 frame:0
>>>>>>>>>         TX packets:286967 errors:0 dropped:0 overruns:0
>>> carrier:0
>>>>>>>>>         collisions:0 txqueuelen:1000
>>>>>>>>>         RX bytes:15590189 (14.8 MiB)  TX bytes:127357293 (121.4
>>> MiB)
>>>>>>>>> 
>>>>>>>>> root@buffy2-1:~# less /var/log/babeld.log
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> send: Cannot assign requested address
>>>>>>>>> send: Cannot assign requested address
>>>>>>>>> send: Cannot assign requested address
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> netlink_read: recvmsg(): No buffer space available
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>>> Couldn't determine channel of interface sw00: Invalid argument.
>>>>>>>> 
>>>>>>>> This is a problem in babel detecting the channel on a "normal"
>>>>>>>> rather than a mesh interface. It's bugged me a long while, but
>>>>>>>> haven't got around to finding what triggers it. Might "fix" it by
>>>>>>>> acquiring the channel at babel start time from
>>> /etc/config/wireless.
>>>>>>>> 
>>>>>>>> It messes up the diversity routing calculation, grump.
>>>>>>>> 
>>>>>>>> There is a possibility a logfile got really big, but this one
>>>>>>>> generally doesn't, but I should turn off logging in some
>>>>>>>> future release...
>>>>>>> 
>>>>>>> 
>>>>>>> I believe I've tracked down part of what's going on. It looks like
>>> my
>>>>>>> tmpfs is filling up 100% and then the device enters a bad state:
>>>>>>> 
>>>>>>> After 24 hours, with tmpfs at 50%, babeld.log is the largest file
>>> by far
>>>>>>> in tmpfs and the only file that appears to be growing (based on
>>> `du`). It
>>>>>>> takes about 48 hours from reboot to fill up tmpfs on my device.
>>>>>>> 
>>>>>>> #  sort babeld.log | uniq -c |sort -rn |head
>>>>>>> 
>>>>>>> 503236 Couldn't determine channel of interface sw00: Invalid
>>> argument.
>>>>>>> 
>>>>>>>  1376 netlink_read: recvmsg(): No buffer space available
>>>>>>> 
>>>>>>>     3 send: Cannot assign requested address
>>>>>>> 
>>>>>>> # wc -l babeld.log
>>>>>>> 
>>>>>>> 504617 babeld.log
>>>>>>> 
>>>>>>> I sped up system failure by using `dd` to fill up tmpfs and the
>>> system
>>>>>>> became immediately unusable.
>>>>>>> 
>>>>>>> This also explains the luci session store errors as sessions are
>>> stored in
>>>>>>> tmpfs.
>>>>>>> 
>>>>>>> The other buffer issues may or may not be related to this.
>>>>>>> 
>>>>>>> Best,
>>>>>>> Steve
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Dave Täht
>>>>> 
>>>>> Fixing bufferbloat with cerowrt:
>>> http://www.teklibre.com/cerowrt/subscribe.html
>>>>> _______________________________________________
>>>>> Cerowrt-devel mailing list
>>>>> Cerowrt-devel@lists.bufferbloat.net
>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>>> 
>> 
>> Hi Dave,
>> --
>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
> 
> 
> 
> -- 
> Dave Täht
> 
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[David Personette]

[-- Attachment #1: Type: text/plain, Size: 951 bytes --]

On Wed, Jan 29, 2014 at 2:51 PM, Sebastian Moeller <moeller0@gmx.de> wrote:

> >>      I have actually not yet understood what it wants to tell me ;),
> since I got your attention, is there an easy way to run a babel client
> under macosx?
> >
> > for coping with the mac I use "macports" to get a compiler and support
> > for open source software.
>
>         Ah, same here (even though I would have thought you a homebrew
> user, no idea why) ;)
> Alas, "port search babel" does not find anything babeld related...
>
> >
> > I haven't ever tried to run babeld on the mac I have, I will put it on
> > my list…
>
>         I just thought it would be nice to finally get into the "stay
> connected while switching between wired and wire-less fun", but this is in
> no way essential for me.



FYI, it does look like homebrew has babel.

dperson@argos2$ brew search babel
babeld      gpsbabel    open-babel

-- 
David P.

[-- Attachment #2: Type: text/html, Size: 1477 bytes --]

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[Sebastian Moeller]

Hi David,

On Jan 29, 2014, at 20:56 , David Personette <dperson@gmail.com> wrote:

> 
> On Wed, Jan 29, 2014 at 2:51 PM, Sebastian Moeller <moeller0@gmx.de> wrote:
> >>      I have actually not yet understood what it wants to tell me ;), since I got your attention, is there an easy way to run a babel client under macosx?
> >
> > for coping with the mac I use "macports" to get a compiler and support
> > for open source software.
> 
>         Ah, same here (even though I would have thought you a homebrew user, no idea why) ;)
> Alas, "port search babel" does not find anything babeld related...
> 
> >
> > I haven't ever tried to run babeld on the mac I have, I will put it on
> > my list…
> 
>         I just thought it would be nice to finally get into the "stay connected while switching between wired and wire-less fun", but this is in no way essential for me.
> 
> 
> FYI, it does look like homebrew has babel.
> 
> dperson@argos2$ brew search babel
> babeld      gpsbabel    open-babel

	Thanks. The first one seems to be the real McCoy (the others are also in macports, for what it is worth). I am very reluctant to switch to homebrew though since I got macports working well and am weary of the learning urge involved in getting the same proficiency in home-brew….

	A different question, did you manage figure out your del link's overhead? If not I would be happy to help (especially if octave/mtlab availability is an issue), just send me the ping log file (best via a file drop).


Best Regards
	Sebastian


> 
> -- 
> David P.

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[Steve Jenson]

[-- Attachment #1: Type: text/plain, Size: 688 bytes --]

On Wed, Jan 29, 2014 at 9:44 AM, Sebastian Moeller <moeller0@gmx.de> wrote:

> On January 29, 2014 5:10:18 PM CET, Dave Taht <dave.taht@gmail.com> wrote:
> >On Wed, Jan 29, 2014 at 4:45 AM, Sebastian Moeller <moeller0@gmx.de>
> >wrote:
> >> Hi Dave,
> >>
> >> quick question, how does one turn of logging for babeld? It seems
> >that if daemonized it defaults to logging to /var/log/babeld.log (or
> >similar). Is setting the log file to /dev/null really the answer?
> >
> >seems so.
>
>       Okay, I guess I will try that then...
>

Here's the directive I'm using in /etc/babeld.conf

log-file /dev/null

and then you can restart either via the web gui or `/etc/rc.d/S70babeld
restart`

[-- Attachment #2: Type: text/html, Size: 1325 bytes --]

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[Sebastian Moeller]

Hi Steve,


On Jan 29, 2014, at 19:24 , Steve Jenson <stevej@fruitless.org> wrote:

> On Wed, Jan 29, 2014 at 9:44 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
> On January 29, 2014 5:10:18 PM CET, Dave Taht <dave.taht@gmail.com> wrote:
> >On Wed, Jan 29, 2014 at 4:45 AM, Sebastian Moeller <moeller0@gmx.de>
> >wrote:
> >> Hi Dave,
> >>
> >> quick question, how does one turn of logging for babeld? It seems
> >that if daemonized it defaults to logging to /var/log/babeld.log (or
> >similar). Is setting the log file to /dev/null really the answer?
> >
> >seems so.
> 
>       Okay, I guess I will try that then...
> 
> Here's the directive I'm using in /etc/babeld.conf
> 
> log-file /dev/null
> 
> and then you can restart either via the web gui or `/etc/rc.d/S70babeld restart`

	Ah, thanks. Since I am on 3.10.28-1 this was /etc/init.d/babeld restart. And I opted for putting:
option 'log-file' '/dev/null'
into /etc/config/babeld, since that seemed the more openwork way of doing things; I wonder whether it really is wise to carry both files…
	Babeld runs again, and no /var/log/babeld.log appeared, but whether it works I do not know (and I doubt it given that babeld.log was growing due to nasty repeating error messages...)


Best Regards
	Sebastian

> 

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[Dave Taht]

On Wed, Jan 29, 2014 at 11:55 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
> Hi Steve,
>
>
> On Jan 29, 2014, at 19:24 , Steve Jenson <stevej@fruitless.org> wrote:
>
>> On Wed, Jan 29, 2014 at 9:44 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
>> On January 29, 2014 5:10:18 PM CET, Dave Taht <dave.taht@gmail.com> wrote:
>> >On Wed, Jan 29, 2014 at 4:45 AM, Sebastian Moeller <moeller0@gmx.de>
>> >wrote:
>> >> Hi Dave,
>> >>
>> >> quick question, how does one turn of logging for babeld? It seems
>> >that if daemonized it defaults to logging to /var/log/babeld.log (or
>> >similar). Is setting the log file to /dev/null really the answer?
>> >
>> >seems so.
>>
>>       Okay, I guess I will try that then...
>>
>> Here's the directive I'm using in /etc/babeld.conf
>>
>> log-file /dev/null
>>
>> and then you can restart either via the web gui or `/etc/rc.d/S70babeld restart`
>
>         Ah, thanks. Since I am on 3.10.28-1 this was /etc/init.d/babeld restart. And I opted for putting:
> option 'log-file' '/dev/null'
> into /etc/config/babeld, since that seemed the more openwork way of doing things; I wonder whether it really is wise to carry both files...


I stuck it in /etc/config/babeld.

>         Babeld runs again, and no /var/log/babeld.log appeared, but whether it works I do not know (and I doubt it given that babeld.log was growing due to nasty repeating error messages...)

It's working. It is just not making an optimal routing decision
between AP-managed networks and meshy ones.

The feature is called diversity routing, and it is key to making
wireless networks scale better. There are (now), quite a few papers on
it, but I like Juliusz's best...

http://www.pps.univ-paris-diderot.fr/~jch/software/babel/wbmv4.pdf

Notably this feature is also in batman, but it's called something else
that I forget.

In an example with two radios on a cerowrt AP:

If you have a packet come in from channel 36, it's best that it goes
out via ethernet if possible, channel 11 if not, and not channel 36.
Even if the number of hops seems less, don't go back out 36, if at all
possible, use a different route.

So right now babel is incorrectly distinquishing between the AP
managed SSIDs (sw00, sw10, gw10, gw00),
so the routing decisions there are sub-optimal. As in most cases you
are going to go out ethernet or one of
the more meshy interfaces, or you have no choice but to send stuff
along on one SSID... it's not very sub-optimal.

still, annoying. rule 22 in embedded design is "never write infinitely
long files as the probability of running out of memory or flash always
hits 100%"



>
> Best Regards
>         Sebastian
>
>>
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] cerowrt issues (3.10.24-8)

[Sebastian Moeller]

Hi Dave, hi list,

On Jan 30, 2014, at 17:21 , Dave Taht <dave.taht@gmail.com> wrote:

> On Wed, Jan 29, 2014 at 11:55 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
>> Hi Steve,
>> 
>> 
>> On Jan 29, 2014, at 19:24 , Steve Jenson <stevej@fruitless.org> wrote:
>> 
>>> On Wed, Jan 29, 2014 at 9:44 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
>>> On January 29, 2014 5:10:18 PM CET, Dave Taht <dave.taht@gmail.com> wrote:
>>>> On Wed, Jan 29, 2014 at 4:45 AM, Sebastian Moeller <moeller0@gmx.de>
>>>> wrote:
>>>>> Hi Dave,
>>>>> 
>>>>> quick question, how does one turn of logging for babeld? It seems
>>>> that if daemonized it defaults to logging to /var/log/babeld.log (or
>>>> similar). Is setting the log file to /dev/null really the answer?
>>>> 
>>>> seems so.
>>> 
>>>      Okay, I guess I will try that then...
>>> 
>>> Here's the directive I'm using in /etc/babeld.conf
>>> 
>>> log-file /dev/null
>>> 
>>> and then you can restart either via the web gui or `/etc/rc.d/S70babeld restart`
>> 
>>        Ah, thanks. Since I am on 3.10.28-1 this was /etc/init.d/babeld restart. And I opted for putting:
>> option 'log-file' '/dev/null'
>> into /etc/config/babeld, since that seemed the more openwork way of doing things; I wonder whether it really is wise to carry both files...
> 
> 
> I stuck it in /etc/config/babeld.
> 
>>        Babeld runs again, and no /var/log/babeld.log appeared, but whether it works I do not know (and I doubt it given that babeld.log was growing due to nasty repeating error messages...)
> 
> It's working. It is just not making an optimal routing decision
> between AP-managed networks and meshy ones.

	Ah, if babel can work around this issue than there is no good reason to spam the log with repeats (at least not at the frequency it currently does, once per day might be more reasonable...)

> 
> The feature is called diversity routing, and it is key to making
> wireless networks scale better. There are (now), quite a few papers on
> it, but I like Juliusz's best...
> 
> http://www.pps.univ-paris-diderot.fr/~jch/software/babel/wbmv4.pdf

	Interesting, thanks for the pointer.

> 
> Notably this feature is also in batman, but it's called something else
> that I forget.
> 
> In an example with two radios on a cerowrt AP:
> 
> If you have a packet come in from channel 36, it's best that it goes
> out via ethernet if possible, channel 11 if not, and not channel 36.
> Even if the number of hops seems less, don't go back out 36, if at all
> possible, use a different route.
> 
> So right now babel is incorrectly distinquishing between the AP
> managed SSIDs (sw00, sw10, gw10, gw00),
> so the routing decisions there are sub-optimal.

	Is it really hard to get the radio and frequency for each interface from linux?

> As in most cases you
> are going to go out ethernet or one of
> the more meshy interfaces, or you have no choice but to send stuff
> along on one SSID... it's not very sub-optimal.

	Okay, that does not justify the log spam ;)


> 
> still, annoying. rule 22 in embedded design is "never write infinitely
> long files as the probability of running out of memory or flash always
> hits 100%"

	I agree (and then I would be happier if openwrt would have a preemptive mechanism to avoid this situation, be it log rotation and deletion or similar methods.)

Best Regards & mangy thanks for the explanation.
	Sebastian

> 
> 
> 
>> 
>> Best Regards
>>        Sebastian
>> 
>>> 
>> 
> 
> 
> 
> -- 
> Dave Täht
> 
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html