[Cerowrt-devel] 6relayd

Dave Taht dave.taht at gmail.com
Sat Jan 18 11:22:47 EST 2014


On Sat, Jan 18, 2014 at 9:46 AM, Steven Barth <cyrus at openwrt.org> wrote:
> That firewall reloading is due to comcast unnecessarily spamming ras every 3
> seconds. We already filter it down to one reload per minute. I prepared
> another filter yesterday which will filter out updates that dont change
> anything but adress / route timers. So expect some solution for this reload
> spam in the coming days.

Wow, policy routing has really sprouted wings. No visible default route...


root at cerowrt:~# ip -6 route
2601:mynet:c30::/64 dev gw00  proto kernel  metric 256  expires 300345sec
2601:mynet:c31::/64 dev gw10  proto kernel  metric 256  expires 300345sec
2601:mynet:c32::/64 dev se00  proto kernel  metric 256
2601:mynet:c33::/64 dev sw00  proto kernel  metric 256  expires 300345sec
2601:mynet:c34::/64 dev sw10  proto kernel  metric 256  expires 300345sec
unreachable 2601:9:8580:c30::/60 dev lo  proto static  metric
2147483647  error -128
(the above is trick to reject stuff going to subnets you have but have
not delegated)

and the actual routing is all done via the rule table.

Can this already co-exist with 6in4 or 6rd running at the same time?

root at cerowrt:~# ip -6 rule
0:    from all lookup local
32766:    from all lookup main
80000:    from 2001:558:mywanaddr lookup 1004
90000:    from 2001:558:mywanaddr lookup 1004
90000:    from 2601:mynet:c32::1/64 iif se00 lookup 1004
90000:    from 2601:mynet:c33::1/64 iif sw00 lookup 1004
90000:    from 2601:mynet:c30::1/64 iif gw00 lookup 1004

but I'm confused, how do I get from mynet:c30 to mynet:c32

90000:    from 2601:mynet:c34::1/64 iif sw10 lookup 1004
90000:    from 2601:mynet:c31::1/64 iif gw10 lookup 1004
90001:    from all iif lo lookup 1001
90002:    from all iif lo lookup 1002
90003:    from all iif lo lookup 1003
90003:    from all iif lo lookup 1004
90013:    from all iif lo lookup 1010
90014:    from all iif lo lookup 1009
90015:    from all iif lo lookup 1006
90016:    from all iif lo lookup 1005
90017:    from all iif lo lookup 1007
90018:    from all iif lo lookup 1008
4200000000:    from 2601:mynet:c32::1/64 iif se00 unreachable
4200000000:    from 2601:mynet:c33::1/64 iif sw00 unreachable
4200000000:    from 2601:mynet:c30::1/64 iif gw00 unreachable
4200000000:    from 2601:mynet:c34::1/64 iif sw10 unreachable
4200000000:    from 2601:mynet:c31::1/64 iif gw10 unreachable
4200000001:    from all iif lo failed_policy
4200000002:    from all iif se00 failed_policy
4200000003:    from all iif ge00 failed_policy
4200000003:    from all iif ge00 failed_policy
4200000013:    from all iif gw11 failed_policy
4200000014:    from all iif gw01 failed_policy
4200000015:    from all iif sw10 failed_policy
4200000016:    from all iif sw00 failed_policy
4200000017:    from all iif gw00 failed_policy
4200000018:    from all iif gw10 failed_policy
root at cerowrt:~# ip -6 route show table 1004
default via fe80::201:5cff:fe62:4e46 dev ge00  proto static  metric 1024

>
>
> Dave Taht <dave.taht at gmail.com> schrieb:
>>
>> I just filed bug http://www.bufferbloat.net/issues/438 on this issue
>> after working with matt until the wee hours.
>>
>> I have to take a couple packet captures next.
>>
>> To copy from the bug report:
>>
>> On the plus side:
>>
>> comcast ipv6 had been working fine between august and december on
>> cerowrt 3.10.7 (?)
>>
>> we do get an external IPv6 address AND /60 dhcpv6-pd delegation from
>> comcast, and distribute the /64s to each of the subnets on cero. The
>> resulting native ipv6 connection works for getting into the router
>> itself and stays up all night...
>>
>> On the minus side(s)
>>
>> 1) The AAAA record on the wan interface (ge00) is withdrawn and
>> renewed every minute or two. This triggers reloading the firewall,
>> which really isn't something you want happening every minute or two.
>> The delegation seems to persist longer than that,
>> but...
>>
>> 2) We do not get dnsmasq distributing that /64 on any interface.
>> Interestingly if you manually add a new IPv6 address from that range
>> (say, whatever::2/64) dnsmasq picks it up and starts serving ipv6
>> addresses. (theory: we don't have that ipv6 delegation long enough for
>> dnsmasq to see it before they are withdrawn)
>>
>> 3) We get plenty of instruction traps IF you delegate to the wireless
>> and use it.
>> (there may be other factors on the instruction traps so don't take the
>> above as canon), but Running all night with just the ::2 manually
>> inserted on ethernet results in no instruction traps (but there was no
>> traffic either). running with with the manual ::2/64 inserted does
>> result in routable, working, ipv6 subnet addresses that dnsmasq sees
>> and distributes from.
>>
>> 4) tweak: ge01 needs to be added to the firewall rules for wan. maybe.
>>
>> The net result is unusable native ipv6 on comcast
>>  . (comcast6.net is
>> also reporting unusable ipv6 on wireless on the xbox 1, and I don't
>> know if that's related)
>>
>> Working theories: A) is we have an endianess problem on parsing
>> dhcpv6-pd from comcast for the timeout, B) comcast has an endianess
>> problem C) we are not keeping properly track of the ipv6 address
>> assignment and/or lease length. D) Comcast isn't assigning ipv6
>> external addresses and subnets for more than a minute. E) we have some
>> problem on the wireless side in particular (but that seems independent
>> of the problem)
>>
>> We have all generally been running fine with ipv6 tunneled through
>> hurricane, so
>> my assumption is that this is something specific to the directly connected
>> ge00
>> interface, in negotiating something with the upstream dhcpv6 and
>> dhcpv6-pd stuff.
>>
>> So here's one of the symptoms. I have some packet captures and straces to
>> do:
>>
>> Sat Jan 18 1
>>  3:18:55
>> 2014 user.notice firewall: Reloading firewall due
>> to ifupdate of ge01 ()
>> Sat Jan 18 13:19:57 2014 user.notice firewall: Reloading firewall due
>> to ifupdate of ge01 ()
>> Sat Jan 18 13:21:01 2014 user.notice firewall: Reloading firewall due
>> to ifupdate of ge01 ()
>> Sat Jan 18 13:22:02 2014 user.notice firewall: Reloading firewall due
>> to ifupdate of ge01 ()
>> Sat Jan 18 13:23:02 2014 user.notice firewall: Reloading firewall due
>> to ifupdate of ge01 ()
>> Sat Jan 18 13:24:04 2014 user.notice firewall: Reloading firewall due
>> to ifupdate of ge01 ()
>> Sat Jan 18 13:25:04 2014 user.notice firewall: Reloading firewall due
>> to ifupdate of ge01 ()
>> Sat Jan 18 13:25:45 2014 daemon.info dnsmasq-dhcp3318:
>> RTR-ADVERT 2601:9:8580:c32::
>> Sat Jan 18 13:26:07 2014 user.notice firewall: Reloading firewall due
>> to ifupdate of ge01 ()
>> Sat Jan 18 13:27:09 2014 user.notice firewall: Reloading fi
>>  rewall
>> due
>> to ifupdate of ge01 ()
>> Sat Jan 18 13:28:11 2014 user.notice firewall: Reloading firewall due
>> to ifupdate of ge01 ()
>>
>>
>> On Sat, Jan 18, 2014 at 9:23 AM, Steven Barth <cyrus at openwrt.org> wrote:
>>>
>>> Fyi as stated earlier i made the switch to odhcpd yesterday. With that i
>>> also switched routing from individual tables to source-constrained routes
>>> in
>>> the maintable.
>>>
>>> Cheers,
>>> Steven
>>>
>>>
>>>
>>>
>>> Dave Taht <dave.taht at gmail.com> schrieb:
>>>
>>>> On Fri, Jan 17, 2014 at 1:52 AM, Matt Mathis <mattmathis at google.com>
>>>> wrote:
>>>>
>>>>> I'm final
>>>>>  ly
>>>>> getting back to this.
>>>>>
>>>>>> Hmm. if you uncomment everything in /etc/dnsmasq.conf and restart
>>>>>> dnsmasq what happens? If you have got /64s you would end up doing
>>>>>> slaac and ra announcements via dnsmasq in this case.
>>>>>>
>>>>>> That was on by default before (and what was tested in feburary). Later
>>>>>> on 6relayd started having a race with it and seemed to be "the
>>>>>> future", so I disabled the dnsmasq version, thinking that 6relayd was
>>>>>> the answer. It's entirely possible that's
>>>>>> merely configured wrong.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Now I get global /64's on my LAN interfaces, but I am still not
>>>>> answering
>>>>> dh
>>>>> cp6 for
>>>>> attached hosts.  I retried both version of the 6relayd init
>>>>> script....
>>>>>
>>>>> dnsmasq.conf contains:
>>>>> enable-ra
>>>>> dhcp-range=::1,::400,constructor:se00,ra-names,ra-stateless
>>>>> dhcp-range=::1,::400,constructor:sw00,ra-names,ra-stateless
>>>>> dhcp-range=::1,::400,constructor:gw00,ra-names,ra-stateless
>>>>> dhcp-range=::1,::400,constructor:sw10,ra-names,ra-stateless
>>>>> dhcp-range=::1,::400,constructor:gw10,ra-names,ra-stateless
>>>>>
>>>>>
>>>>> I am running: Linux cerowrt 3.10.24 #1 Tue Dec 24 10:50:15 PST
>>>>> 2013.....
>>>>> which might be just a bit too fresh....  Would you suggest another?
>>>>
>>>>
>>>>
>>>> You are not getting slaac either?
>>>>
>>>> An ifconfig on an interface and a packet dump of ipv6 packets would be
>>>> helpful.
>>>>
>>>>> I have a spare 3700, so I think I will try some alternate vintages.
>>>>>
>>>>> Thanks,
>>>>> --MM--
>>>>> The
>>>>> best way to predict the future is to create it.  - Alan Kay
>>>>>
>>>>>
>>>>> Privacy matters!  We know from recent events that people are using our
>>>>> services to speak in
>>>>> defiance of unjust governments.   We treat privacy
>>>>> and
>>>>> security as matters of life and death, because for some users, they
>>>>> are.
>>>>>
>>>>>
>>>>> On Sun, Jan 5, 2014 at 7:48 PM, Dave Taht <dave.taht at gmail.com> wrote:
>>>>>
>>>>>> On Sat, Jan 4, 2014 at 1:30 AM, Steven Barth <cyrus at openwrt.org>
>>>>>> wrote:
>>>>>>
>>>>>>> On 03.01.2014 19:43, Dave Taht wrote:
>>>>>>>
>>>>>>>
>>>>>>>> I was also experiencing a race condition with dnsmasq, while I had
>>>>>>>> it
>>>>>>>> enabling
>>>>>>>> ra
>>>>>>>> and
>>>>>>>> dhcpv6 via dnsmasq. At the moment that's turned off by default,
>>>>>>>> but
>>>>>>>> I did rather prefer having dns names for my ipv6
>>>>>>>> addresses...
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Well 6relayd and odhcpd collect hostnames of clients acquired via
>>>>>>> stateful
>>>>>>> DHCPv6 and export them to dnsmasq in an additional hostfiles. At
>>>>>>> least
>>>>>>> that
>>>>>>> seemed to work when I last tried it a few months ago. The only
>>>>>>> disadvantage
>>>>>>> is that there is no "ra-names" feature there.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Getting to names from dhcpv4 to slaac was a neat hack and a potential
>>>>>> RFC. So i figure spending the time to add the same functionality into
>>>>>> into something other than dnsmasq would be useful towards writing that
>>>>>> rfc.
>>>>>>
>>>>>>
>>>>>>>> is there a good way for 6re
>>>>>>>> layd
>>>>>>>> and dnsmasq-dhcpv6 to co-exist?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Ideally they could coexist in a way that you c
>>>>>>>  ould
>>>>>>> select dnsmasq and /
>>>>>>> or
>>>>>>> odhcpd for different interfaces on the same machine. odhcpd supports
>>>>>>> that
>>>>>>> but dnsmasq the last time I've looked seemed to use a single socket
>>>>>>> binding
>>>>>>> to all interfaces for DHCP/v6 which prevents coexistance from working
>>>>>>> correctly because odhcpd / 6relayd can't bind the socket after
>>>>>>> dnsmasq
>>>>>>> did
>>>>>>> and vice versa.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>> Feel free to provide me with some debugging information of the
>>>>>>>>> system
>>>>>>>>> while
>>>>>>>>> PD fails for you so I can have a look at the probable cause:
>>>>>>>>>
>>>>>>>>> * "ifstatus ge00" (replace ge00 with your IPv6 upstream interface)
>>>>>>>>> * "ip addr list dev
>>>>>>>>> ge01"
>>>>>>>>> (replace ge01 with the interface your
>>>>>>>>> downstream
>>>>>>>>> router is connected)
>>>>>>>>> * "ps
>>>>>>>>>  | grep
>>>>>>>>> 6relayd"
>>>>>>>>>
>>>>>>>>> Anyway I will migrate all the stuff to odhcpd soon (it's successor
>>>>>>>>> which
>>>>>>>>> shares a good part of the codebase but is a bit better integrated
>>>>>>>>> with
>>>>>>>>> the
>>>>>>>>> rest of the environment).
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> same question re dnsmasq.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Yeah as pointed out coexistence is a matter of binding sockets.
>>>>>>> odhcpd
>>>>>>> will
>>>>>>> bring the functionality of dynamically enabling / disabling DHCPv4/v6
>>>>>>> on
>>>>>>> interfaces without restarting the daemon and loosing state. This is
>>>>>>> one
>>>>>>> of
>>>>>>> the main reasons for the change and very much eases things for
>>>>>>> high-level
>>>>>>> protocols that do dynamic wan/lan detection.
>>>>>>>
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Steven
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>> Regard
>>>>>>>>>  s,
>>>>>>>>>
>>>>>>>>> Steven
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 03.01.2014 18:31, Dave Taht wrote:
>>>>>>>>>
>>>>>>>>>> On Fri, Jan 3, 2014 at 11:50 AM, cb.list6 <cb.list6 at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Fri, Jan 3, 2014 at 8:40 AM, Dave Taht <dave.taht at gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> At one level I am happy to figure out this is a recently
>>>>>>>>>>>> introduced
>>>>>>>>>>>> bug.
>>>>>>>>>>>>
>>>>>>>>>>>> On the other hand I am not sure if it is 6relayd.
>>>>>>>>>>>>
>>>>>>>>>>>> What version of cero was working for you?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I am not entirely sure, but i think it was from September.
>>>>>>>>>>>
>>>>>>>>>>> CB
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> At the moment I lack the ability to d
>>>>>>>>>>  ebug
>>>>>>>>>> the breakage in ipv6
>>>>>>>>>> dhcp-pd
>>>>>>>>>> (which is odhcpd) (I am travelling).
>>>>>>>>>>
>>>>>>>>>> I will on my next stop next week (tuesday) setup a dhcpv6pd server
>>>>>>>>>> and
>>>>>>>>>> see what I can see.
>>>>>>>>>>
>>>>>>>>>>>> On Jan 3, 2014 12:21 AM, "cb.list6" <cb.list6 at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have been using CeroWRT on Comcast with a 3800 for about 6
>>>>>>>>>>>>> month.
>>>>>>>>>>>>> The
>>>>>>>>>>>>> DHCP-PD config has always been a little unstable for me, but
>>>>>>>>>>>>> working.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I recently upgraded to:
>>>>>>>>>>>>>
>>>>>>>>>>>>> root at cerowrt:/etc/config# uname -a
>>>>>>>>>>>>> Linux cerowrt 3.10.24 #1 Tue Dec 24 1
>>>>>>>>>>>>> 0:50:15
>>>>>>>>>>>>> PST 2013 mips
>>>>>>>>>>>>> GNU/Linux
>>>>>>>>>>>>>
>>>>>>>>>>>>> My WAN
>>>>>>>>>>>>>   gets a
>>>>>>>>>>>>> /128, but i cannot get DHCP-PD to work to get
>>>>>>>>>>>>> addresses
>>>>>>>>>>>>> on
>>>>>>>>>>>>> the rest of my interfaces.  The router does seem to have good
>>>>>>>>>>>>> IPv6
>>>>>>>>>>>>> access.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I fiddled with the 6relayd config and came up with this, but it
>>>>>>>>>>>>> does
>>>>>>>>>>>>> not
>>>>>>>>>>>>> work.  Any pointers on how to get this back on track?  The
>>>>>>>>>>>>> result
>>>>>>>>>>>>> of
>>>>>>>>>>>>> the
>>>>>>>>>>>>> below config is that the /128 from the WAN interfaces is now
>>>>>>>>>>>>> present
>>>>>>>>>>>>> on
>>>>>>>>>>>>> all
>>>>>>>>>>>>> the interfaces but my attached computers get no addresses.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> config server 'default'
>>>>>>>>>>>>> option rd 'server'
>>>>>>>>>>>>> option dhcpv6 'server'
>>>>>>>>>>>>> option management_level '1'
>>>>>>>>>>>>> list network 'ge01'
>>>>>>>>>>>>> list network 'gw00'
>>>>>>>>>>>>> list network 'gw01'
>>>>>>>>>>>>> list network 'gw10'
>>>>>>>>>>>>> list network 'gw11'
>>>>>>>>>>>>> list network 'se00'
>>>>>>>>>>>>> list network 'sw00'
>>>>>>>>>>>>> list network 'sw10'
>>>>>>>>>>>>> option fallback_relay 'rd dhcpv6 ndp'
>>>>>>>>>>>>> option master 'ge00'
>>>>>>>>>>>>>
>>>>>>>>>>>>> root at cerowrt:/etc/config# un
>>>>>>>>>>>>> ame
>>>>>>>>>>>>> -a
>>>>>>>>>>>>>
>>>>>>>>>>>>> ________________________________
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cerowrt-devel mailing list
>>>>>>>>>>>>> Cerowrt-devel at lists.bufferbloat.net
>>>>>>>>>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>> --
>>>>>> Dave Täht
>>>>>>
>>>>>> Fixing bufferbloat with cerowrt:
>>>>>> http://www.teklibre.com/cerowrt/subscribe.html
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> --
>>>> Dave Täht
>>>>
>>>> Fixing bufferbloat with cerowrt:
>>>> http://www.teklibre.com/cerowrt/subscribe.html
>>>
>>>
>>>
>>>
>> --
>> Dave Täht
>>
>> Fixing bufferbloat with cerowrt:
>> http://www.teklibre.com/cerowrt/subscribe.html



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html



More information about the Cerowrt-devel mailing list