[Cerowrt-devel] An interesting application of tunneling and ipv6 mesh networking

Wed Jan 23 14:23:58 EST 2013

On Wed, Jan 23, 2013 at 1:34 PM, <dpreed at reed.com> wrote:

> Cool!   Let's make sure the OpenVPN links on it don't have bufferbloat in
> them - VPNs often do, and perhaps need a layer of fq_codel on top of each
> link (also underneath, of course, where the routers can be fixed).
>
>
I have seen extremely interesting behavior with openvpn. vtun will drop
packets like crazy under load.

>
>
> More interestingly - is this an openly growing network?  Or is it intended
> to be an island?
>
>
>

The author of the original post, cc'd.

> For those stuck in IPv4 land (which includes folks who live inside
> corporate Intranets, sadly, maybe this is a good step.
>
>
>

I note that I've been told that osx for the "backtomymac" feature uses a
private ipv6 backbone. It's probably one of the largest in the world.

> -----Original Message-----
> From: "Dave Taht" <dave.taht at gmail.com>
> Sent: Wednesday, January 23, 2013 12:11pm
> To: cerowrt-devel at lists.bufferbloat.net
> Subject: [Cerowrt-devel] An interesting application of tunneling and ipv6
> mesh networking
>
>
>
> ---------- Forwarded message ----------
> From:  <jp at nexedi.com>
> Date: Wed, Jan 23, 2013 at 6:30 AM
> Subject: [Babel-users] A happy babel user: re6st
> To: babel-users at lists.alioth.debian.org
>
>
> Hi,
>
> Very often, people complain on mailing lists. Today, I would like to say
> thank you.
>
> Last summer, we have implemented a wired mesh network system based on
> babel which can provide stable IPv6 to all nodes of a decentralized cloud
> operation system. It works great.
>
> Thank you babel.
>
> If you are in a hurry, here is the code:
> http://git.erp5.org/gitweb/re6stnet.git
> What you can do with that code: provide reliable IPv6 to the world
>
> If you think re6st is useful, please feel free to add it to the list of
> babel links.
>
> Details bellow.
>
> Regards,
>
> JP Smets.
> Nexedi CEO
> +33 629 02 44 25
> ---
>
> 1- The problem to solve
>
> We implemented a couple of years ago (http://bit.ly/SWVQlx) a Cloud
> system called SlapOS (http://www.slapos.org) which relies on servers
> located in people's home and now also in offices, data centers or even your
> smartphone, tablet or TV. SlapOS is now used by some large corporations.
> One of its main applications is to create a disaster recovery cloud which
> can resist any force majeur event (ex. war, terrorism, political
> instability, software bug) which does affect traditional clouds from time
> to time (http://iwgcr.org). It is also much cheaper and environmental
> friendly.
>
> SlapOS relies on IPv6 in order to interconnect all nodes. Each node is
> allocated usually 100 global IPv6 addresses or more.
>
> This is where our problem started: all IPv6 providers we tried were unable
> to provide reliable connectivity. We tried providers in France, Germany,
> Japan, Norway. For example, in France among 200 IPv6 adresses provided by a
> Freebox (Free), 3 becomes unreachable from time to time, during a couple of
> minutes or hours. OVH routers sometimes no longer route packers to Free,
> but only for IPv6, during a couple of hours. Telia routers somtimes "eat" a
> few bytes during the initialization of a session.
>
> Overall, the use of native IPv6 of ISPs lead to a service availability of
> 99% or worse. We we searching for a solution.
>
> We also had had the experience that from time to time, IPv4 transit
> between ISPs can be cut for a while - a couple of hours -although less
> often. Our ideal solution should also solve that.
>
> 2- The solution: re6st + babel
>
> Step 1: create a wired mesh
>
> We coded a litlle deaemon called re6st which is able to find 10 IPv4
> neighbours randomly and create a tunnel to each neighbour. re6st can be
> placed behind a NAT. It is able to capture public IPv4 address of your
> router through UPnP. After some time, all nodes which run re6st form a
> global mesh.
>
> Step 2: start babel
>
> Once tunnels are created, babel is used for routing. Babel then finds the
> best route to interconnect all re6st nodes.
>
> 3- Results
>
> After a couple of month of using re6st + babel we can say that it works
> quite well. SlapOS no longer experiences the connectivity problems of
> native IPv6. We can safely host websites with SlapOS over re6st+babel.
>
> 4- Next steps
>
> A report will be published.
>
> 5- Remaining problems to solve
>
> The problems which remain to be solved are the following:
>
> a- How can we prevent one babel participant to act against other
> participants by providing wrong information to other participants ? Imagine
> for example that a bad organization joins re6st + babel network and starts
> capturing all routes in order to analyze traffic or even block it.
>
> b- How can we create a hierarchical addressing system ? The idea here is
> to group participants dynamically and assign them a "big" IPv6 address
> range. Each participant connects to another participant through another
> participant by first connecting randomly to one participant in a dynamic
> group and next connect to other participant in the same group. With this
> grouping approach, there is no need to create a hierachical network with a
> bakbone. It also solves the problem of scalability.
>
> c- How can we implement more policies (ex. latency) ?
>
> d- How could we implement accounting and billing in a way or another ?
> (open question, but quite important for example to solve the problem of
> FTTH participants with upload limited to 3GB / day as in Japan)
>
> 6- Credits
>
> Most of the coding of re6st was done by Julien Muchembled (Nexedi),
>  Ulysse Beaugnon (ENS) and Guillaume Bury (ENS).
>
> 7- Alternatives
>
> We could have used other routing protocols (ex. OLSR). But we felt that
> Babel pluggable policy system was a key design difference which could be
> used to later customize it to different needs of Cloud applications (ex.
> low latency). We would also feel ashamed to use a protocol which babel's
> creator proved that it was flawed.
>
> We could have used tinc. But tinc creates a fully connected mesh. There is
> also a difference between what it claims to do and what it actually does.
> Last, mixing tunneling and routing is a bad idea as we were suggested by
> Juliusz C.
>
> We could have used gre instead of OpenVPN for tunnels. But that does work
> behind an IPv4 NAT. Yet, nothing prevents use from later using gre.
> _______________________________________________
> Babel-users mailing list
> Babel-users at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users
>
>
>
> --
> Dave Täht
>
> Fixing bufferbloat with cerowrt:
> http://www.teklibre.com/cerowrt/subscribe.html
>

-- 
Dave Täht

Fixing bufferbloat with cerowrt:
http://www.teklibre.com/cerowrt/subscribe.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/cerowrt-devel/attachments/20130123/2f89df27/attachment-0002.html>