[Cerowrt-devel] development snapshot of cerowrt-3.3.8-21 released

[Cerowrt-devel] development snapshot of cerowrt-3.3.8-21 released

[Dave Taht]

I spent the last two weeks hunting down memory related
issues and trying to fold in some development work I'd had
going already.

As best as I can tell, the core memory issues are killed dead; whether
this is due to freeing up tons of memory or by the various other stuff
remains to be determined.

But:

Under *no circumstances* install this release on your default router.

I'm putting this out primarily because I'm seeing an odd behavior
on the iwl card I have, but not on the ath9ks (yet!, still testing).
If there is someone with a 3rd type of wireless card is out there,
anything non-iwl or non-linux,  please beat this up.

Radical changes:

+ bind replaced with dnsmasq (bind available as an option)
+ support for AAAA naming and RA announcements in dnsmasq
+ Implementation of experimental codel code from kathie's ns2 work
+ fq_codel engages codel sooner
+ fq_codel has a CS1 deprioritization hack
+ codel and fq_codel shrink skbs under overload
+ debloat has reduced defaults for packet limits
+ debloat uses qlens of 2,4,12,12 (up from 2,3,3,3)
+ qos-scripts has reduced defaults
+ strongswan available again

- I haven't looked at the hurricane ipv6 issue
- dlna, upnp, either
- didn't fix ath9k to use smaller allocations
- no tcp small queues

Big bugs remain.

1) htb does weird things at all bandwidths, and with all qdiscs,
not just codel/fq_codel.

It may well have been doing this for a while (like, months), which would explain
a lot. hfsc is also being weird. (hfsc is used by qos-scripts, htb by
simple_qos)


2) wifi vs the x86 iwl card.

This is the error that I get in /var/log/messages, and the only way to get
connectivity back and clear it is to reboot the *x86* box.

[67046.216150] iwlwifi 0000:03:00.0: fail to flush all tx fifo queues
[67048.224185] iwlwifi 0000:03:00.0: fail to flush all tx fifo queues
[67056.868185] iwlwifi 0000:03:00.0: fail to flush all tx fifo queues

This is how I get the error, inside of about 30 seconds. (where the ip
is the router's ip)

netperf -Y CS5,CS5 -l 120 -H 172.20.42.65 -t TCP_STREAM &
netperf -Y EF,EF -l 120 -H 172.20.42.65 -t TCP_STREAM &
netperf -Y CS1,CS1 -l 120 -H 172.20.42.65 -t TCP_STREAM &
netperf -Y CS0,CS0 -l 120 -H 172.20.42.65 -t TCP_STREAM &

The above saturates the EF and VI queues, and for some reason
starves the BE, BK queues (on iwl)...

Packet traces indicate strongly that it's the iwl that's hosed, in
this tcpdump, it
is receiving packets from cero, but no longer able to transmit them.

15:26:15.889147 ARP, Request who-has ida.home.lan tell 172.20.11.97, length 28
15:26:15.889185 ARP, Reply ida.home.lan is-at 00:26:c6:42:76:e2 (oui
Unknown), length 28

(and I'm not running codel/fq_codel on the x86 box, either, on this test)

I will refine this bug report more over time and get it to the
linux-wireless mailing list.
I just need to setup more boxes.

The same codel related patch set for linux-3.6-rc3 x86 is now up as
"codel2-ns2",
where htb, the codel patches, etc is *just fine*, over ethernet. htb
on x86/that version
is also just fine. I'm pretty happy with this patch set, it feels like
an improvement
(at least on x86) over codel and fq_codel from before.

http://snapon.lab.bufferbloat.net/~cero1/deb/

But on cero, htb has got extra-ordinary delays that shouldn't be there.

and 3.3.8-21 is at (have I given you enough warning yet?)

http://snapon.lab.bufferbloat.net/~cero1/3.3/3.3.8-21/

Up next for me is backing off to a way earlier version of cero, and
incrementally
adding back in stuff. But first up is a dip in the pool, and beating my
head against a tree. Or vice versa.

And if I'm lucky some x86 boxes will arrive soon and I can go build those
instead of going nutso on this.

-- 
Dave Täht
http://www.bufferbloat.net/projects/cerowrt/wiki - "3.3.8-17 is out
with fq_codel!"

Re: [Cerowrt-devel] development snapshot of cerowrt-3.3.8-21 released

[Richard Brown]

[-- Attachment #1: Type: text/plain, Size: 1329 bytes --]

Dave,

Dave Täht wrote:

I spent the last two weeks hunting down memory related
issues and trying to fold in some development work I'd had
going already.

As best as I can tell, the core memory issues are killed dead; whether
this is due to freeing up tons of memory or by the various other stuff
remains to be determined.

Great work and excellent news!

But:

Under *no circumstances* install this release on your default router.

Yeah. A few short comments on 3.3.8-21:

1)  I cannot connect to gw.home.lan via DNS. Dig gives no answers. I can log in just fine on 172.30.42.1, etc. and config the router, etc.

2) More on the Hurricane Electric tunnel. After adding these lines to the config script on the wiki page:

uci set network.henet.adv_subnet=1        # added for 3.3.8-17
uci set network.henet.adv_interface=se00
uci set network.henet.adv_interface=sw10

I still cannot connect/get an IPv6 address on se00, *but* I can get addresses on all four wireless channels. (In retrospect, I believe this was true for 3.3.8-17 as well.)

3) Is it possible that DNS come back earlier in the boot sequence? It seemed that CeroWrt began responding within a minute (or so) to DNS queries, instead of the 2-5 minutes that I've seen with earlier builds.

Thanks again.

Rich Brown
Hanover, NH USA

[-- Attachment #2: Type: text/html, Size: 3144 bytes --]

[Cerowrt-devel] development snapshot of cerowrt-3.3.8-21 released

[Robert Bradley]

On 29 August 2012 11:57, Robert Bradley <robert.bradley1@gmail.com> wrote:
> On 29 August 2012 02:28, Richard Brown <richard.e.brown@dartware.com> wrote:
>> I still cannot connect/get an IPv6 address on se00, *but* I can get
>> addresses on all four wireless channels. (In retrospect, I believe this was
>> true for 3.3.8-17 as well.)
>
> Now I finally have a WNDR3800 to play with, I set up my own tunnel on
> 3.3.8-17 a couple of weeks back.  I'm using www.broker.ipv6.ac.uk
> instead of Hurricane Electric, but the script still worked for me with
> minor changes to IP addresses and the like.

One thing I remembered is that you're presumably using OS X for this,
as opposed to the Windows 7 and Ubuntu clients I'm using.

After seeing https://isc.sans.edu/diary.html?storyid=13978&rss by
chance, I decided to start logging packets.  In CeroWRT's case, we
have the "managed" flag unset, but the "other" flag set.  We also
appear to lack a DHCPv6 server.  Maybe setting "option
AdvOtherConfigFlag '0'" for all the interfaces and restarting radvd
would help?

Alternatively, you might have to set up a DHCPv6 server.  CeroWRT has
dnsmasq-dhcpv6 already, as far as I can tell, but you will probably
need to follow the instructions at
http://wiki.openwrt.org/doc/howto/ipv6#dnsmasq-dhcpv6 to get that
working (including removing dnsmasq and reinstalling dnsmasq-dhcpv6).

-- 
Robert Bradley

Re: [Cerowrt-devel] development snapshot of cerowrt-3.3.8-21 released

[Robert Bradley]

On 29 August 2012 13:18, Robert Bradley <robert.bradley1@gmail.com> wrote:
> Alternatively, you might have to set up a DHCPv6 server.  CeroWRT has
> dnsmasq-dhcpv6 already, as far as I can tell, but you will probably
> need to follow the instructions at
> http://wiki.openwrt.org/doc/howto/ipv6#dnsmasq-dhcpv6 to get that
> working (including removing dnsmasq and reinstalling dnsmasq-dhcpv6).

The following instructions for CeroWRT worked for me with Windows 7
clients.  I set it so that addresses are assigned via SLAAC with radvd
doing announcements, and DNS servers etc. are announced over DHCPv6.

# opkg update
# opkg remove dnsmasq
# opkg install --force-reinstall dnsmasq-dhcpv6

Insert several lines into /etc/dnsmasq.conf describing the dhcp ranges:

dhcp-range=se00,</64 goes here>, ra-stateless, ra-names
etc.

# /etc/init.d/dnsmasq restart

At this point, you'll probably get the router's local IPv6 addresses
as DNS servers.  This is fine except that named is not configured to
accept IPv6 traffic.   This should not be relevant for 3.3.8-21 since
that uses dnsmasq only, but for 3.3.8-17 and earlier, edit
/etc/chroot/named/etc/bind/conf/acls.local.conf and add in your IPv6
prefix, then "killall -HUP named" to update BIND.

We could replace radvd entirely at this point and use "enable-ra" in
dnsmasq.conf, but I think radvd gives us more options.  For
DHCPv6-only address assignment, you can remove the "ra-stateless,
ra-names" parts in our new dhcp-range lines, and set "AdvManagedFlag
on" in radvd.  I can't imagine OS X requiring that level of attention
though to get an IPv6 address; merely using "AdvOtherConfigFlag off"
in radvd.conf ought to do it.
-- 
Robert Bradley