[Cerowrt-devel] [Babel-users] switching cerowrt to quagga-babeld issues

Mon Jul 2 11:36:05 EDT 2012

On Mon, Jul 2, 2012 at 7:52 AM, Denis Ovsienko <infrastation at yandex.ru> wrote:
> 01.07.2012, 03:53, "Dave Taht" <dave.taht at gmail.com>:
>> On Sat, Jun 30, 2012 at 7:44 PM, Juliusz Chroboczek <jch at pps.jussieu.fr> wrote:
>>
>>>>>>    babel rxcost 100
>>>  Implemented, committed, tested, pushed, and documented.
>>
>> Packaged, compiled, pushed to ceropackages-3-3 on github, and built in
>> the (as yet untested) development-only build of cerowrt, here:
>>
>> http://snapon.lab.bufferbloat.net/~cero1/3.3/3.3.8-8/
>
> There are 7 boxes in my temporary possession: one WNDR3700v2 and six WNDR3800. For the next couple of weeks these won't be used for anything but CeroWrt build tests (no idea what happens after that). I have just flashed all of them with 3.3.8-8 and passed through a custom config script, which besides other things currently configures Babel exchanges to be authenticated. The only major outstanding issue seems to be the default route one (which doesn't reproduce in today's topology, but I know it exists).

How goeth ipv6?

>
> The project may get more options, if we drive the prototype towards a finished deliverable.

I am very enthusiastic about babel's new authenticated mesh routing!

It is also my opinion that without a decent drop and packet mixing
strategy that mesh networks will perform badly under load. I'm hoping
that fq_codel does well, although it seems very likely that an
aggregation aware and fq_codel-like strategy needs to move into the
mac80211 layer, which is perhaps years worth of work.

What would a deliverable look like? What would interest people enough
to get some good data, papers written, progress made, more
users/developers and cash in the door?

...

I presently have 4 Nanostation M5, 3 picostation HP, and 5 wndr3700s.
I've found a good site to work with 2.4ghz and 5.x ghz radios (a 110
acre campground that has given me permission to play here) and test
fq_codel in various forms of cerowrt in, for as long as I like.

One of the things we've really struggled with was in today's saturated
2.4ghz environment there is no way to benchmark both radios. We've
been reduced to suggesting we 'flee to the mountains' in order to get
good results. OK, I just did that.

My own primary purpose *right now* is to be repeating a swath of tests
we ran on predecessors to fq_codel, (sfq, sfqred) to get a baseline on
what happens with the current OS and then to be able to compare them
against fq_codel. BEFORE linux 3.5 goes stable in august.

Some of this can happen in bloatlab #1 and on the x86 hw, but as for
wireless, that needs something of a greenfield environment to do
right.

This was the first post-BQL result I'd got back in november, 2011 -
the first one that showed that there was hope we could, indeed, fix
bufferbloat, on ethernet

http://www.teklibre.com/~d/bloat/pfifo_fast_vs_sfq_qfq_log.png

We'd finally got (under ethernet only) enough control of the stack to
reduce latency under load on linux by nearly 2 orders of magnitude.
Now, QFQ was heavyweight, so a string of mods to sfq went into linux
3.2 and 3.3, ultimately culminating in sfqred, which was very nice,
although
we had to rip out a feature (head of queue) as unstable that had made
it very competitive with QFQ...

http://www.teklibre.com/~d/bloat/hoqvssfqred.ps (april 15, 2012)

(the pfifo_fast results are so bad as to be NOT worth showing)

but the red component of sfqred proved hard to tune, although it did
manage to hold latency under wireless load to a relatively sane
60-90ms "band".

http://www.teklibre.com/~d/bloat/ping_log.ps

...

and then

...

codel

http://www.bufferbloat.net/projects/codel/wiki

arrived from van jacobson and kathie nichols. A short time later eric
dumazet wrote fq_codel... which basically ripped out the red component
of sfqred AND found a middle path towards having new stream entry be
less under-optimized than in sfq...

and all the results are looking amazing...

but I simply haven't had time/energy/money to repeat these three tests
to see how much/if we won or not.

Anyway, I'm planning on repeating these tests on current
cerowrt/openwrt software and a variety of hardware.  And I/someone
need to make the test and statistics collection more robust, and
writing that code over the next few months in the yurt here seems to
be a relaxing prospect.

(I will be in Paris July 15-29 however not enough spare cash to make
ietf in vancouver. :( )

Also, as in the current cerowrt codebase are also patches to be able
to dynamically reduce queuing in the ath9k drivers, my secondary
purpose is to gather some data on wireless's behavior w and w/o
fq_codel and sane amounts of buffering at that layer. I really don't
expect to win enough for it to be useful but that's why we experiment.

In any case, I find babel very useful in letting me setup (and keep
up, and then crash) large numbers of nodes, and as it's a layer 3
protocol it's a lot easier to spread it across ethernet and wifi and
also observe what's going vs a vs other mesh protocols.

1) I could start publishing the images/source for a babel-mesh-enabled
nanostation m5 and picostation hp which are cerowrt derived but
seriously cut down in size.

2) Trying to have a project with a definable short and long term goal
seems a good idea. I outlined my own main ones above.

3) Given all the hoopla about crowdfunding, I was thinking of setting
up an idea like this as a kickstarter project.

The amount of time/money I've spent on the bufferbloat projects long
ago exceeded my own resources. Yet: I just turned down a lucrative day
job because I'm uncomfortable with fq_codel being undertested - and
what I'd like to be doing far exceeds my own resources.

Ideas towards a project definition/deliverable that would be useful,
interesting, & sustainable
would be useful.

> --
>     Denis Ovsienko
>
> _______________________________________________
> Babel-users mailing list
> Babel-users at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users

-- 
Dave Täht
http://www.bufferbloat.net/projects/cerowrt/wiki - "3.3.8-6 is out
with fq_codel!"