[Bloat] Cerowrt 3.3.8-10 is released

[Dave Taht]

Get it at : http://huchra.bufferbloat.net/~cero1/3.3/3.3.8-10

This will be the last release of cerowrt for a while. I think this is
even stabler than 3.3.8-6 was, but we'll (you'll) see.

I will be traveling for most of the next month and unable to do much
bloat-related stuff.

Everything I deeply care about has been pushed into openwrt, anyway.

Cerowrt-3.3.8-10 is stable but forward-looking. It has an outline
towards what a more wifi-bloat-free future would look like. Maybe.

While the code remains experimental (as always) I did spend the last 2
weeks doing a test deployment of 12 (3800, pico 2HP, nano-m5) radios
at a campground, with what is basically in 3.3.8-10. Uptimes are good.
Performance is excellent. Latency is remarkably low....

Did I mention you can get it from :
http://huchra.bufferbloat.net/~cero1/3.3/3.3.8-10

But:

First up are the minuses in this release

- ntp keeps getting restarted due to badly parsing ntpc (#113 strikes again)
I keep being annoyed by this and then getting intimidated by #113
again and failing.

- simple_qos still isn't done, and is ever less simple

Despite much fiddling with various models, with ECN dropping, with buffering,
etc, nothing I would consider worthy of replacing the openwrt QoS
system got done.
Certain things are good in simple_qos - ipv6 and diffserv support - others
are not (gui, flexibility, actual performance)

- dlna
- upnpd
Neither compiled out of the box and I lacked time or tools that use
these to look at them. I had multiple requests for them but I didn't
know they were borked to start with. Apologies to the requestors.

- ECN dropping - after several high level conversations with many
people smarter than me, I decided that dropping ECN packets at a
certain point made sense. So did everyone else. The "certain point"
remains puzzling to all, and
rather than continue to waste time on it in cero I decided to play
with models instead, and frankly, hope that someone else comes up with
some sane way to combine ECN and codel sojourn time.

I note that as a side effect of worrying about ECN (and the cause of
much controversy on the babel list), I arbitrarily marked babel
packets as CS6+ECN, as one means of exploring explosive but
non-dropping behavior in fq_codel + ECN.

Now, on to the plusses in Cerowrt-3.3.8-10

+ fresh openwrt merge
+ gpsd 3.7
+ switch to quagga (thx denis and Juliusz)
+ babelm available as an option - smoother convergence algo from julius
+ diffserv support (mostly to classify "ants" into the VI queue) (me)
+ hw queue length patches from Felix Feitkau (now in openwrt mainline)

Re: openwrt merge - openwrt still hasn't frozen but it looks close

Re: gpsd - I hope to finally work on the cosmic background bufferbloat
detector some, now that I have some geography to play with.

Re: Quagga

Most of my own excitement this past month has been in seeing quagga
become a routing platform that was not only usable for babel (with
authentication!) but also to interoperate with other protocols like
ra, bgp, ospf, etc. I am delighted to finally make the switch to
quagga-babeld as cerowrt's default routing daemon.

An ipv6 default routing bug may remain in this...

Re: babelm - Features a new, smoother converging babel algorithm.

Work on the original babel continues, but this algo arrived too late
and in the wrong source-base to play with much. It's in ceropackages
and should build for any version of openwrt.

Re: diffserv work

Unlike current Linux wifi, cerowrt wifi obeys the most rational set of
rules for things like EF, CS6, CS7 and ant-like packets I could come
up with. Basically everything except EF got moved out of the VO queue,
and many other markings ended up in the VI queue...

Re: hw queue reduction

Probably the most interesting of all these changes is the ath9k
hardware qlen support, which gives us a knob to play with deep in the
ath9k wireless driver to control it's native buffering. It defaults to
128 buffers per hardware queue.

I cut that down to 2 for VO, and 3 for VI, BE, BK. These are
front-ended by fq_codel running at mildly higher than it's default 5ms
target.

I get remarkably low latency results at all (even marginal) transmit
rates, at the expense of a LOT of raw bandwidth in more lab-like
conditions.

I'm in the process of running real-life benchmarks out of the Yurtlab.
I'm not prepared to publish what I've collected thus far, hopefully by
IETF I'll have something pulled together. I am very interested in
seeing how fq_codel reacts to sudden bandwith changes in wifi outside
of the lab and simulations.

I would encourage those doing their own benchmarks to PLEASE do them
at reasonable distances under difficult (NOT LAB!) conditions, and I
also note things like youtube streaming are good indicators of actual
usability.

However: the original pre-3.3.8-10 behavior can be restored by editing
/usr/sbin/debloat and changing the qlen_whatever variables to 128,
from their current 2,3,3,3.

We are painfully aware of how hard it will be to get good aggregation
AND low latency back into wireless-n, and have begun to document a way
forward here: http://www.bufferbloat.net/projects/cerowrt/wiki/Fq_Codel_on_Wireless

Anyway: Install cerowrt-3.3.8-10 and enjoy.

PS: I will be traveling extensively over the next 60 days.

In Paris July 15-27, then Vancouver, then Seattle, aug 3-5, Linux
plumbers aug 28-31, NJ sept 7-12. Perhaps I will see some of you in
one of those places?

PPS:

Multiple people have thought I was kidding when I said I was living in
a yurt. I'm not kidding.

http://snapon.lab.bufferbloat.net/~d/lupin/yurtlab.jpg

It's not just a yurt, it's a regular high-tech hut of baba yaga! It's
pleasantly located midway between Santa Cruz and San Jose, and I have
110 acres of mostly-wifi-free space to play in. And it's got a 24/7
pool, with the most advanced wifi on the planet now run to it. It's an
inexpensive place to call a temporary home, better than a shipping
container by far.

In August I mostly plan to do more analysis, and develop more tests
and benchmarks,  utilizing the acreage and radios I've emplaced here
(and having a bit of fun), and to continue attempting to fix the
ongoing funding issues, than further develop cerowrt. That's the plan,
as I write, anyway.


-- 
Dave Täht
http://www.bufferbloat.net/projects/cerowrt/wiki - "3.3.8-6 is out
with fq_codel!"