Re: [Babel-users] switching cerowrt to quagga-babeld issues

Re: [Babel-users] switching cerowrt to quagga-babeld issues

[Dave Taht]

On Mon, Jul 2, 2012 at 7:52 AM, Denis Ovsienko <infrastation@yandex.ru> wrote:
> 01.07.2012, 03:53, "Dave Taht" <dave.taht@gmail.com>:
>> On Sat, Jun 30, 2012 at 7:44 PM, Juliusz Chroboczek <jch@pps.jussieu.fr> wrote:
>>
>>>>>>    babel rxcost 100
>>>  Implemented, committed, tested, pushed, and documented.
>>
>> Packaged, compiled, pushed to ceropackages-3-3 on github, and built in
>> the (as yet untested) development-only build of cerowrt, here:
>>
>> http://snapon.lab.bufferbloat.net/~cero1/3.3/3.3.8-8/
>
> There are 7 boxes in my temporary possession: one WNDR3700v2 and six WNDR3800. For the next couple of weeks these won't be used for anything but CeroWrt build tests (no idea what happens after that). I have just flashed all of them with 3.3.8-8 and passed through a custom config script, which besides other things currently configures Babel exchanges to be authenticated. The only major outstanding issue seems to be the default route one (which doesn't reproduce in today's topology, but I know it exists).

How goeth ipv6?

>
> The project may get more options, if we drive the prototype towards a finished deliverable.

I am very enthusiastic about babel's new authenticated mesh routing!

It is also my opinion that without a decent drop and packet mixing
strategy that mesh networks will perform badly under load. I'm hoping
that fq_codel does well, although it seems very likely that an
aggregation aware and fq_codel-like strategy needs to move into the
mac80211 layer, which is perhaps years worth of work.

What would a deliverable look like? What would interest people enough
to get some good data, papers written, progress made, more
users/developers and cash in the door?

...

I presently have 4 Nanostation M5, 3 picostation HP, and 5 wndr3700s.
I've found a good site to work with 2.4ghz and 5.x ghz radios (a 110
acre campground that has given me permission to play here) and test
fq_codel in various forms of cerowrt in, for as long as I like.

One of the things we've really struggled with was in today's saturated
2.4ghz environment there is no way to benchmark both radios. We've
been reduced to suggesting we 'flee to the mountains' in order to get
good results. OK, I just did that.

My own primary purpose *right now* is to be repeating a swath of tests
we ran on predecessors to fq_codel, (sfq, sfqred) to get a baseline on
what happens with the current OS and then to be able to compare them
against fq_codel. BEFORE linux 3.5 goes stable in august.

Some of this can happen in bloatlab #1 and on the x86 hw, but as for
wireless, that needs something of a greenfield environment to do
right.

This was the first post-BQL result I'd got back in november, 2011 -
the first one that showed that there was hope we could, indeed, fix
bufferbloat, on ethernet

http://www.teklibre.com/~d/bloat/pfifo_fast_vs_sfq_qfq_log.png

We'd finally got (under ethernet only) enough control of the stack to
reduce latency under load on linux by nearly 2 orders of magnitude.
Now, QFQ was heavyweight, so a string of mods to sfq went into linux
3.2 and 3.3, ultimately culminating in sfqred, which was very nice,
although
we had to rip out a feature (head of queue) as unstable that had made
it very competitive with QFQ...

http://www.teklibre.com/~d/bloat/hoqvssfqred.ps (april 15, 2012)

(the pfifo_fast results are so bad as to be NOT worth showing)

but the red component of sfqred proved hard to tune, although it did
manage to hold latency under wireless load to a relatively sane
60-90ms "band".

http://www.teklibre.com/~d/bloat/ping_log.ps

...

and then


...

codel

http://www.bufferbloat.net/projects/codel/wiki

arrived from van jacobson and kathie nichols. A short time later eric
dumazet wrote fq_codel... which basically ripped out the red component
of sfqred AND found a middle path towards having new stream entry be
less under-optimized than in sfq...

and all the results are looking amazing...

but I simply haven't had time/energy/money to repeat these three tests
to see how much/if we won or not.

Anyway, I'm planning on repeating these tests on current
cerowrt/openwrt software and a variety of hardware.  And I/someone
need to make the test and statistics collection more robust, and
writing that code over the next few months in the yurt here seems to
be a relaxing prospect.

(I will be in Paris July 15-29 however not enough spare cash to make
ietf in vancouver. :( )

Also, as in the current cerowrt codebase are also patches to be able
to dynamically reduce queuing in the ath9k drivers, my secondary
purpose is to gather some data on wireless's behavior w and w/o
fq_codel and sane amounts of buffering at that layer. I really don't
expect to win enough for it to be useful but that's why we experiment.

In any case, I find babel very useful in letting me setup (and keep
up, and then crash) large numbers of nodes, and as it's a layer 3
protocol it's a lot easier to spread it across ethernet and wifi and
also observe what's going vs a vs other mesh protocols.

1) I could start publishing the images/source for a babel-mesh-enabled
nanostation m5 and picostation hp which are cerowrt derived but
seriously cut down in size.

2) Trying to have a project with a definable short and long term goal
seems a good idea. I outlined my own main ones above.

3) Given all the hoopla about crowdfunding, I was thinking of setting
up an idea like this as a kickstarter project.

The amount of time/money I've spent on the bufferbloat projects long
ago exceeded my own resources. Yet: I just turned down a lucrative day
job because I'm uncomfortable with fq_codel being undertested - and
what I'd like to be doing far exceeds my own resources.

Ideas towards a project definition/deliverable that would be useful,
interesting, & sustainable
would be useful.

> --
>     Denis Ovsienko
>
> _______________________________________________
> Babel-users mailing list
> Babel-users@lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users



-- 
Dave Täht
http://www.bufferbloat.net/projects/cerowrt/wiki - "3.3.8-6 is out
with fq_codel!"

Re: [Babel-users] switching cerowrt to quagga-babeld issues

[L. Aaron Kaplan]

[-- Attachment #1: Type: text/plain, Size: 2156 bytes --]


On Jul 2, 2012, at 5:36 PM, Dave Taht wrote:

> How goeth ipv6?
> 
>> 
>> The project may get more options, if we drive the prototype towards a finished deliverable.
> 
> I am very enthusiastic about babel's new authenticated mesh routing!
> 
> It is also my opinion that without a decent drop and packet mixing
> strategy that mesh networks will perform badly under load. I'm hoping
> that fq_codel does well, although it seems very likely that an
> aggregation aware and fq_codel-like strategy needs to move into the
> mac80211 layer, which is perhaps years worth of work.
Or have better cross-layer communication - see below....
> 
> What would a deliverable look like? What would interest people enough
> to get some good data, papers written, progress made, more
> users/developers and cash in the door?


I would like to point you towards a nice feature that Henning is currently implementing: DLEP - a (mostly routing protocol independent) framework for communicating settings and metrics between a radio and a routing daemon.
See https://datatracker.ietf.org/doc/draft-ietf-manet-dlep/

Abstract


   When routing devices rely on modems to effect communications over 
   wireless links, they need timely and accurate knowledge of the 
   characteristics of the link (speed, state, etc.) in order to make 
   forwarding decisions. In mobile or other environments where these 
   characteristics change frequently, manual configurations or the  
   inference of state through routing or transport protocols does not 
   allow the router to make the best decisions. A bidirectional, event-
   driven communication channel between the router and the modem is 
   necessary.



More info and code samples on the olsr.org git repo. I'd like to also encourage Babel coders to take a look at it and collectively improve this.
It will be - as Dave already mentioned - anyway a hard problem to have this working for many radios in a standardized way, so it would really be helpful if different mesh developers for once could work *together* on some general feature which improves everybody's metrics.


Aaron.


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 203 bytes --]

Re: [Babel-users] switching cerowrt to quagga-babeld issues

[Dave Taht]

On Mon, Jul 2, 2012 at 12:16 PM, L. Aaron Kaplan <aaron@lo-res.org> wrote:
>
> On Jul 2, 2012, at 5:36 PM, Dave Taht wrote:
>
>> How goeth ipv6?
>>
>>>
>>> The project may get more options, if we drive the prototype towards a finished deliverable.
>>
>> I am very enthusiastic about babel's new authenticated mesh routing!
>>
>> It is also my opinion that without a decent drop and packet mixing
>> strategy that mesh networks will perform badly under load. I'm hoping
>> that fq_codel does well, although it seems very likely that an
>> aggregation aware and fq_codel-like strategy needs to move into the
>> mac80211 layer, which is perhaps years worth of work.
> Or have better cross-layer communication - see below....
>>
>> What would a deliverable look like? What would interest people enough
>> to get some good data, papers written, progress made, more
>> users/developers and cash in the door?
>
>
> I would like to point you towards a nice feature that Henning is currently implementing: DLEP - a (mostly routing protocol independent) framework for communicating settings and metrics between a radio and a routing daemon.
> See https://datatracker.ietf.org/doc/draft-ietf-manet-dlep/
>
> Abstract
>
>
>    When routing devices rely on modems to effect communications over
>    wireless links, they need timely and accurate knowledge of the
>    characteristics of the link (speed, state, etc.) in order to make
>    forwarding decisions. In mobile or other environments where these
>    characteristics change frequently, manual configurations or the
>    inference of state through routing or transport protocols does not
>    allow the router to make the best decisions. A bidirectional, event-
>    driven communication channel between the router and the modem is
>    necessary.
>
>
>
> More info and code samples on the olsr.org git repo. I'd like to also encourage Babel coders to take a look at it and collectively improve this.

I just did. I like it.

I note that minstrel also supplies currently unreachable-via-api
information as to rates and retries that would be useful to be
exposed, but has a higher complexity than the above spec.
(note I only just read it, soo...). Andrew mcgregor has a
too-big-for-the-MPU paper on how minstrel works he's been distributing
privately.

Also I was fiddling with supplying GPS data and sensor data (over
babel, but it's not the best choice) both for more optimal wifi access
and for a prototype of an earthquake early warning system... and I'd
argue that GPS (and GPS time) data is generally useful and gps's are
now everywhere, a good way to distribute it could be added to the
above.

> It will be - as Dave already mentioned - anyway a hard problem to have this working for many radios in a standardized way, so it would really be helpful if different mesh developers for once could work *together* on some general feature which improves everybody's metrics.

YES! Please, can we mesh people find a way to work together for a change?

my own work on bufferbloat to a large extent is driven by a (5 years
worth of) quest to solve the "dense mesh" problem that OLPCs had. The
fq_codel work at least theoretically (but currently at the wrong
layer) is a means to improve that for all wired/wireless networks and
any given mesh protocol in particular would benefit. I have been
tracking batman-adv as well, but not olsr to any extent. Where does
"working code" lie?

I note that fq_codel throws an interesting congestion-related stat
away right now, which I'd like to find a way to include in a route
metric.

I also note that I am intensely dissatisfied with babel's current
metric, which is basically "ping" over multicast, but I'm happy as
babel's metric is pluggable and can be changed.

>
> Aaron.
>



-- 
Dave Täht
http://www.bufferbloat.net/projects/cerowrt/wiki - "3.3.8-6 is out
with fq_codel!"

Re: [Babel-users] switching cerowrt to quagga-babeld issues

[L. Aaron Kaplan]

[-- Attachment #1: Type: text/plain, Size: 4906 bytes --]


On Jul 2, 2012, at 6:44 PM, Dave Taht wrote:

> On Mon, Jul 2, 2012 at 12:16 PM, L. Aaron Kaplan <aaron@lo-res.org> wrote:
>> 
>> On Jul 2, 2012, at 5:36 PM, Dave Taht wrote:
>> 
>>> How goeth ipv6?
>>> 
>>>> 
>>>> The project may get more options, if we drive the prototype towards a finished deliverable.
>>> 
>>> I am very enthusiastic about babel's new authenticated mesh routing!
>>> 
>>> It is also my opinion that without a decent drop and packet mixing
>>> strategy that mesh networks will perform badly under load. I'm hoping
>>> that fq_codel does well, although it seems very likely that an
>>> aggregation aware and fq_codel-like strategy needs to move into the
>>> mac80211 layer, which is perhaps years worth of work.
>> Or have better cross-layer communication - see below....
>>> 
>>> What would a deliverable look like? What would interest people enough
>>> to get some good data, papers written, progress made, more
>>> users/developers and cash in the door?
>> 
>> 
>> I would like to point you towards a nice feature that Henning is currently implementing: DLEP - a (mostly routing protocol independent) framework for communicating settings and metrics between a radio and a routing daemon.
>> See https://datatracker.ietf.org/doc/draft-ietf-manet-dlep/
>> 
>> Abstract
>> 
>> 
>>   When routing devices rely on modems to effect communications over
>>   wireless links, they need timely and accurate knowledge of the
>>   characteristics of the link (speed, state, etc.) in order to make
>>   forwarding decisions. In mobile or other environments where these
>>   characteristics change frequently, manual configurations or the
>>   inference of state through routing or transport protocols does not
>>   allow the router to make the best decisions. A bidirectional, event-
>>   driven communication channel between the router and the modem is
>>   necessary.
>> 
>> 
>> 
>> More info and code samples on the olsr.org git repo. I'd like to also encourage Babel coders to take a look at it and collectively improve this.
> 
> I just did. I like it.
> 

Nice :)

> I note that minstrel also supplies currently unreachable-via-api
> information as to rates and retries that would be useful to be
> exposed, but has a higher complexity than the above spec.
> (note I only just read it, soo...). Andrew mcgregor has a
> too-big-for-the-MPU paper on how minstrel works he's been distributing
> privately.
> 
> Also I was fiddling with supplying GPS data and sensor data (over
> babel, but it's not the best choice) both for more optimal wifi access
> and for a prototype of an earthquake early warning system... and I'd
> argue that GPS (and GPS time) data is generally useful and gps's are
> now everywhere, a good way to distribute it could be added to the
> above.
> 
>> It will be - as Dave already mentioned - anyway a hard problem to have this working for many radios in a standardized way, so it would really be helpful if different mesh developers for once could work *together* on some general feature which improves everybody's metrics.
> 
> YES! Please, can we mesh people find a way to work together for a change?

Absolutely!

> 
> my own work on bufferbloat to a large extent is driven by a (5 years
> worth of) quest to solve the "dense mesh" problem that OLPCs had. The

Oh!... I remember being at One Kendall Sq. and discussing this with Michailis LOL...
I know exactly what you mean :) 30 XOs in close proximity was a recipe for disaster :)

> fq_codel work at least theoretically (but currently at the wrong
> layer) is a means to improve that for all wired/wireless networks and
> any given mesh protocol in particular would benefit. I have been
> tracking batman-adv as well, but not olsr to any extent. Where does
> "working code" lie?

olsr.org/git/

What you are looking for in particular is:
http://www.olsr.org/git/?p=framework.git;a=summary
and DLEP:
http://www.olsr.org/git/?p=dlep_app.git;a=summary

> 
> I note that fq_codel throws an interesting congestion-related stat
> away right now, which I'd like to find a way to include in a route
> metric.
> 
> I also note that I am intensely dissatisfied with babel's current
> metric, which is basically "ping" over multicast, but I'm happy as

;-) the dirty little secret is : all of the current mesh protos have unsatisfactory metrics :)

> babel's metric is pluggable and can be changed.

Same for OLSR. Good design choice.


Okay, Dave . I recommend that you get a grip on the DELP draft and get in touch with Henning.
I shifted more towards the "making things happen" side of mesh R&D than to the implementation side. So you need to talk with Henning in this respect. I would love to see a common framework - well tested and hardened by multiple developers which is usable for all of us.

Aaron.



[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 203 bytes --]

DLEP [was: switching cerowrt to quagga-babeld issues]

[Juliusz Chroboczek]

> I would like to point you towards a nice feature that Henning is
> currently implementing: DLEP - a (mostly routing protocol independent)
> framework for communicating settings and metrics between a radio and
> a routing daemon.

I reviewed an early draft of that.

As I understand DLEP, it is a standardised way to implement the "thin
access point" model that Cisco and friends are promoting -- a complex
(and expensive) controller associated with a number of dumb APs.  This
is a good way to enable mobility between APs with unmodified stations,
it's a good way to make network administrators happy (centralised
administration), and also a good way to make Cisco happy (since the
controller can be sold for much, much more than an AP).

Standardising the protocol that the controller uses to communicate with
the APs is a worthy goal, but pretty much orthogonal to mesh networking
research.

-- Juliusz

Re: [Babel-users] DLEP [was: switching cerowrt to quagga-babeld issues]

[Henning Rogge]

On Mon, Jul 2, 2012 at 7:13 PM, Juliusz Chroboczek <jch@pps.jussieu.fr> wrote:
>> I would like to point you towards a nice feature that Henning is
>> currently implementing: DLEP - a (mostly routing protocol independent)
>> framework for communicating settings and metrics between a radio and
>> a routing daemon.
>
> I reviewed an early draft of that.

The current draft is not really in a good shape, still a lot of
"proprietary" stuff and complexity that has the get out.

> As I understand DLEP, it is a standardised way to implement the "thin
> access point" model that Cisco and friends are promoting -- a complex
> (and expensive) controller associated with a number of dumb APs.  This
> is a good way to enable mobility between APs with unmodified stations,
> it's a good way to make network administrators happy (centralised
> administration), and also a good way to make Cisco happy (since the
> controller can be sold for much, much more than an AP).

It also will allow us with an easy way to access link layer metrics,
even from radios we can only reach over ethernet.

> Standardising the protocol that the controller uses to communicate with
> the APs is a worthy goal, but pretty much orthogonal to mesh networking
> research.

Yes, it is... it might become a good raw data source for the metric
calculation of any kind of mesh routing protocol.

Henning Rogge

-- 
Steven Hawkings about cosmic inflation: "An increase of billions of
billions of percent in a tiny fraction of a second. Of course, that
was before the present government."

Re: DLEP [was: switching cerowrt to quagga-babeld issues]

[L. Aaron Kaplan]

[-- Attachment #1: Type: text/plain, Size: 1158 bytes --]

Juliusz, remember that there are two approaches right now concernig DLEP:  CISCO and the rest of the world ;-)
nuff said ;)))


On Jul 2, 2012, at 7:13 PM, Juliusz Chroboczek wrote:

>> I would like to point you towards a nice feature that Henning is
>> currently implementing: DLEP - a (mostly routing protocol independent)
>> framework for communicating settings and metrics between a radio and
>> a routing daemon.
> 
> I reviewed an early draft of that.
> 
> As I understand DLEP, it is a standardised way to implement the "thin
> access point" model that Cisco and friends are promoting -- a complex
> (and expensive) controller associated with a number of dumb APs.  This
> is a good way to enable mobility between APs with unmodified stations,
> it's a good way to make network administrators happy (centralised
> administration), and also a good way to make Cisco happy (since the
> controller can be sold for much, much more than an AP).
> 
> Standardising the protocol that the controller uses to communicate with
> the APs is a worthy goal, but pretty much orthogonal to mesh networking
> research.
> 
> -- Juliusz
> 


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 203 bytes --]