Re: Source-specific routes in Linux [was: atomic updates...]

Re: Source-specific routes in Linux [was: atomic updates...]

[Dave Taht]

On Sun, May 5, 2013 at 1:05 PM, Juliusz Chroboczek
<jch@pps.univ-paris-diderot.fr> wrote:
>> It's not ECMP for ipv6 I'm looking to see solved for multiple routing
>> protocols, but the multiple ipv6 prefixes/multiple exits source
>> routing problem.
>
> As you well know, Dave, we've been working on that for the last
> fortnight or so.
>
> We've run into a limitation of the kernel:
>
>   http://www.spinics.net/lists/netdev/msg235316.html

Hmm. Given that this code path has only barely been exercised before
and so many changes to the routing paths have happened in the last 2+
years, I'm not surprised its broken.

>
> Unfortunately, I've had no reply to my question:
>
>   http://www.spinics.net/lists/netdev/msg235346.html
>
> So it looks like we're going to have to dynamically generate rule
> table entries... sigh.

Well, given the work going into mptcp in particular, a better API for
src specific routing for ipv6 than the ip rule system seems like a
good idea.

 Widening the distribution a little bit...

>> The homenet folk demonstrated this form of ipv6 multihoming working
>> (using a version of OSPF and BIRD, I believe) at the last ietf.
>
> That is interesting.  Would you happen to know how they're interfacing
> with the kernel?  Or perhaps have a link to their code?

I don't know where the ietf 86 bird work on multihoming went. (?)
Patterson's original work on ospf is here:

https://github.com/paterben/bird-homenet

and the ietf drafts are here

http://tools.ietf.org/wg/homenet/

>
> -- Juliusz



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: Source-specific routes in Linux [was: atomic updates...]

[Dave Taht]

On Sun, May 5, 2013 at 2:38 PM, Dave Taht <dave.taht@gmail.com> wrote:
> On Sun, May 5, 2013 at 1:05 PM, Juliusz Chroboczek
> <jch@pps.univ-paris-diderot.fr> wrote:
>>> It's not ECMP for ipv6 I'm looking to see solved for multiple routing
>>> protocols, but the multiple ipv6 prefixes/multiple exits source
>>> routing problem.
>>
>> As you well know, Dave, we've been working on that for the last
>> fortnight or so.
>>
>> We've run into a limitation of the kernel:
>>
>>   http://www.spinics.net/lists/netdev/msg235316.html
>
> Hmm. Given that this code path has only barely been exercised before
> and so many changes to the routing paths have happened in the last 2+
> years, I'm not surprised its broken.
>
>>
>> Unfortunately, I've had no reply to my question:
>>
>>   http://www.spinics.net/lists/netdev/msg235346.html
>>
>> So it looks like we're going to have to dynamically generate rule
>> table entries... sigh.
>
> Well, given the work going into mptcp in particular,

This was a very good overview of where that work stands.

http://lwn.net/Articles/544399/



> a better API for
> src specific routing for ipv6 than the ip rule system seems like a
> good idea.
>
>  Widening the distribution a little bit...
>
>>> The homenet folk demonstrated this form of ipv6 multihoming working
>>> (using a version of OSPF and BIRD, I believe) at the last ietf.
>>
>> That is interesting.  Would you happen to know how they're interfacing
>> with the kernel?  Or perhaps have a link to their code?
>
> I don't know where the ietf 86 bird work on multihoming went. (?)
> Patterson's original work on ospf is here:
>
> https://github.com/paterben/bird-homenet
>
> and the ietf drafts are here
>
> http://tools.ietf.org/wg/homenet/
>
>>
>> -- Juliusz
>
>
>
> --
> Dave Täht
>
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [homenet] Source-specific routes in Linux [was: atomic updates...]

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 2072 bytes --]

On Tue, May 7, 2013 at 5:03 AM, Juliusz Chroboczek <
jch@pps.univ-paris-diderot.fr> wrote:

> > > If you're actually doing source-specific routing, please show me how
> > > you speak to the kernel.
>
> > Most of that code is in Lua, and while I could have included
> > C extension module that did similar things as ip -6 does, I didn't
> > bother.
>
> We're doing pretty the same in our prototype code (saying
> system("ip...") rather than speaking the netlink dialect of the day).
> Could you point us at the relevant code, please?
>
> > So I'm just using ip -6 {route,rule} to set up source-specific rules
> > that map to destination tables.
>
> Ah, okay.  So you're using rules, just as we found out (the hard way)
> that we need to do.
>
>   http://www.spinics.net/lists/netdev/msg235316.html
>   http://www.spinics.net/lists/netdev/msg235346.html
>

One thing that bugs me about hacks and workarounds like this is that Linux
(as well as openwrt) are intensely mutable systems, and it's totally
possible to improve linux rather than limp around in userspace.

I have long disliked the ip rule system in its primary use prior to now
(vpns), as buggy, arbitrary, and subject to race conditions, so if a better
api and methods for injecting/managing source address dependent routing
information could be designed I'm pretty sure there would be much
enthusiasm across the vpn, mptcp/sctp, and routing worlds for getting it
into linux itself.



>
> > One (destination-based) routing table is maintained by routing
> > protocol, and the rest by Lua code which figures external defaults
> > based on routes within routing protocol implementation.
>
> Nice hack.
>

Except that probably it interacts badly with existing vpn manipulation of
these tables.


>
> -- Juliusz
> _______________________________________________
> homenet mailing list
> homenet@ietf.org
> https://www.ietf.org/mailman/listinfo/homenet
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt:
http://www.teklibre.com/cerowrt/subscribe.html

[-- Attachment #2: Type: text/html, Size: 3271 bytes --]

Re: [homenet] Source-specific routes in Linux [was: atomic updates...]

[Steven Barth]

On 08.05.2013 10:51, Dave Taht wrote:
> One thing that bugs me about hacks and workarounds like this is that
> Linux (as well as openwrt) are intensely mutable systems, and it's
> totally possible to improve linux rather than limp around in userspace.
>
> I have long disliked the ip rule system in its primary use prior to now
> (vpns), as buggy, arbitrary, and subject to race conditions, so if a
> better api and methods for injecting/managing source address dependent
> routing information could be designed I'm pretty sure there would be
> much enthusiasm across the vpn, mptcp/sctp, and routing worlds for
> getting it into linux itself.

Hmm I must admit I'm wondering about this for the OpenWrt stack as well.

We have switched to RA-Handling in userspace for similar reasons already 
so I guess it's only the next logical step to create separate routing 
tables for each upstream interface to do source-based routing and filter 
out ULA-traffic on this layer instead of through iptables.

Having one central userspace management daemon for routing and address / 
prefix delegation in general might not be the best or cleanest solution 
in the end but I guess there is no better way right now.

Re: [homenet] Source-specific routes in Linux [was: atomic updates...]

[Ole Troan]

[...]

> We have switched to RA-Handling in userspace for similar reasons already so I guess it's only the next logical step to create separate routing tables for each upstream interface to do source-based routing and filter out ULA-traffic on this layer instead of through iptables.

don't do it per upstream interface, that wouldn't work. per next-hop might. the draft suggests a single table with source constrained routers and backtracking.

> Having one central userspace management daemon for routing and address / prefix delegation in general might not be the best or cleanest solution in the end but I guess there is no better way right now.

cheers,
Ole

_______________________________________________
homenet mailing list
homenet@ietf.org
https://www.ietf.org/mailman/listinfo/homenet

Re: [homenet] Source-specific routes in Linux [was: atomic updates...]

[Steven Barth]

Ole Troan <otroan@employees.org> wrote:

>[...]
>
>> We have switched to RA-Handling in userspace for similar reasons
>already so I guess it's only the next logical step to create separate
>routing tables for each upstream interface to do source-based routing
>and filter out ULA-traffic on this layer instead of through iptables.
>
>don't do it per upstream interface, that wouldn't work. per next-hop
>might. the draft suggests a single table with source constrained
>routers and backtracking.
>

Ah yes thanks for the hint. Please correct me if I got this wrong: I guess per interface would be problematic if there are multiple routers on the upstream link offering different prefixes. However in case of prefix delegation via DHCPV6-pd like on usual home ISP connections would it not be problematic to attribute the prefix to any specific router? - if there would be multiple routers which I guess is unlikely in that situation. One could maybe attribute the prefix to the source address of the DHCPv6 server but that sounds problematic to me aswell. Hmm did I miss something or am I completely on the wrong track now?

Thanks,
Steven

Re: [homenet] Source-specific routes in Linux [was: atomic updates...]

[Ole Troan]

Steven,

>>> We have switched to RA-Handling in userspace for similar reasons
>> already so I guess it's only the next logical step to create separate
>> routing tables for each upstream interface to do source-based routing
>> and filter out ULA-traffic on this layer instead of through iptables.
>> 
>> don't do it per upstream interface, that wouldn't work. per next-hop
>> might. the draft suggests a single table with source constrained
>> routers and backtracking.
>> 
> 
> Ah yes thanks for the hint. Please correct me if I got this wrong: I guess per interface would be problematic if there are multiple routers on the upstream link offering different prefixes. However in case of prefix delegation via DHCPV6-pd like on usual home ISP connections would it not be problematic to attribute the prefix to any specific router? - if there would be multiple routers which I guess is unlikely in that situation. One could maybe attribute the prefix to the source address of the DHCPv6 server but that sounds problematic to me aswell. Hmm did I miss something or am I completely on the wrong track now?

at least we're on the same track (and I think the correct one). ;-)

on the border router this is quite simple. if a border router uses PD and it discovers a default router on the same interface,
that will result in a SADR route (S, D) -> interface, next-hop. where S is PD prefix, D is ::/0, interface is the interface the PD was received
on and next-hop is whatever router discovery came back with.

the issue is with internal routers, where you may have an internal router connected to two exits on the same link, or behind another
IR that is connected to both..., i.e. arbitrary topology. 

cheers,
Ole

Re: [homenet] Source-specific routes in Linux [was: atomic updates...]

[Steven Barth]

On 08.05.2013 12:58, Ole Troan wrote:
> at least we're on the same track (and I think the correct one). ;-)
>
> on the border router this is quite simple. if a border router uses PD and it discovers a default router on the same interface,
> that will result in a SADR route (S, D) -> interface, next-hop. where S is PD prefix, D is ::/0, interface is the interface the PD was received
> on and next-hop is whatever router discovery came back with.
>
> the issue is with internal routers, where you may have an internal router connected to two exits on the same link, or behind another
> IR that is connected to both..., i.e. arbitrary topology.
OK, thanks again for the explanation. I think I will add an additional 
routing-table-ID parameter to the network daemon. This way any routing 
daemon that is going to be properly integrated into OpenWrt can select a 
target table for each route, prefix, etc.


Cheers,

Steven

Re: [homenet] Source-specific routes in Linux [was: atomic updates...]

[Lorenzo Colitti]

[-- Attachment #1: Type: text/plain, Size: 709 bytes --]

On Wed, May 8, 2013 at 7:51 PM, Steven Barth <cyrus@openwrt.org> wrote:

> if there would be multiple routers which I guess is unlikely in that
> situation. One could maybe attribute the prefix to the source address of
> the DHCPv6 server but that sounds problematic to me aswell.
>

So you're talking about the case where you have ISP1 and ISP2 "plugged into
a switch together with your router"?

In that case I can't see any better way than attempting to match the
link-local IPv6 address that you got the RA from to the link-local address
of the DHCPv6 server you got the PD from.

I can't think of a situation why these would be different, but perhaps
there is. Can the DHCPv6 server be a global address?

[-- Attachment #2: Type: text/html, Size: 1185 bytes --]

Re: [homenet] Source-specific routes in Linux [was: atomic updates...]

[Juliusz Chroboczek]

>       http://www.spinics.net/lists/netdev/msg235316.html
>       http://www.spinics.net/lists/netdev/msg235346.html

> One thing that bugs me about hacks and workarounds like this is that Linux (as
> well as openwrt) are intensely mutable systems, and it's totally possible to
> improve linux rather than limp around in userspace.

I can only agree with you.

Now, as mentioned in my second message linked above, source-specific
routing for IPv6 in a single table appears to be supposed to work with
CONFIG_IPV6_SUBTREES.  My tests indicate it doesn't, at least in
recent kernels.

I've tried to look at the relevant code (in ipv6/ipv6_fib.c and
ipv6/route.c), and this code is way beyond my grokking abilities.
Should a friendly kernel hacker choose to fix this, we can switch our
implementation to using that simpler API with little effort.
(Acutally deleting a bunch of code -- Matthieu is going to hate me.)

> I have long disliked the ip rule system in its primary use prior to now
> (vpns), as buggy, arbitrary, and subject to race conditions, so if a better
> api and methods for injecting/managing source address dependent routing

My preference goes towards dumping all routes into a single routing
table.  This applies both to source-specific routes, to
input-interface specific routes (which is what Steven Barth was
referring to), as well as to any future extensions to the model
(TOS-specific routes, flow-id-specific routes, evil-bit-specific
routes, etc.)

The semantics of such an API is ambiguous, but userspace can work
around that by injecting extra routes into the FIB.  The number of
such routes is at worst proportional to r^s, where r is the number of
"real" routes and s is the number of selection mechanisms, but (1)
this worst case is unlikely to happen in practice, and (2) this number
can be greatly reduced by having reasonable default disambiguation
rules in the kernel (use the destination as the primary key, the
source as the secondary key, and any other selectors as the tertiary key).

-- Juliusz