[Cerowrt-devel] testing 3.3rc6-3 out on jim (good, but tons of ipv6 issues)

[Dave Taht]

Yesterday we got jim up and running on the latest build.

https://plus.google.com/u/0/110299325941327120246/posts/DD5taJG6ixV

I did some testing of the aqm scripts still under development, after
some iterations, the results were rather good, but some more
comprehensive work is needed. In particular, I had not been working on
uplink speeds greater than about 4Mbit, nor thinking very hard about
the effects of everything on very long RTT paths (of late! In part,
why I went to paris was to be able to analyze those in the real world,
rather than in emulation). More news on this as it happens. I'd like
very much to have a few more dedicated routers and test boxes spread
around the world (NZ/australia and europe, etc) sometime soon.
Volunteers?

We did run into some problems along the way.

0) we ran into a WEIRD bug with one of jim's printers.

Although this printer is visible, has an ipv4 address, a distinct mac,
attempts to access it via http would get re-routed to the *routers*
web server for some reason. As this does not happen on ANY of the
other devices on the lan, I'm inclined to think it's a problem with
the printer's firmware, rather than some problem with icmp
redirection, or a routing bug. That said, it's very weird.

1) Ipv6 issues

A) Notably, 6to4 for some reason isn't spreading it's acquired ipv6
addresses across all the devices anymore. I have no idea why this is.
It used to work.

B) And relevant to that, the default firewall rules I have for ipv6 in
this case do not appear to be restrictive enough. While it was cool to
actually be able to do a smbclient -L //jims_ipv6_addresses/ and get
responses all the way from california, I don't regard the cifs
(windows filesharing) protocol as secure enough to actually run on the
open internet. So I added to /etc/firewall.user stanzas to block the
relevant ports incoming from the 6to4-ge01 device.

I note that I have not kept up with enhancements to the samba protocol
suite and for all I know samba 3.6.3 can actually be configured to be
'secure enough'. ?

I WOULD like to keep open the possibility of running reasonably secure
protocols over ipv6. In part, that's the whole point of restoring E2E
connectivity to the internet with ipv6! ssh, https, imaps, kerberos,
etc, all seem desirable to allow by default. Also protocols that work
best in an E2E fashion (sip/rtp, snmp, dns, etc), and also newfangled
protocols such as hip, shim6, etc. Also simple, useful file
distribution protocols like torrent and rsync.

Finding the right compromise between openness, research and security
is going to be a PITA, and I welcome suggestions. I frankly would like
to avoid a solution so like our existing ipv4 natted universe as to
offer none of the benefits of ipv6.

C) Propagating dynamically assigned ipv6 addresses requires fiddling
with configurations of various
other daemons.

C1) /etc/xinetd.conf needs the local ipv6 address range permitted
C2) in /etc/chroot/bin/etc/named/conf, the local ipv6 address range
needs to be added to the acl. Also, it's helpful to actually run dns
over ipv6 via the local ipv6 forwarder (comcast has those),
C2A) reverse dns for ipv6 is a REAL PITA.
C3) /etc/config/polipo needs the local ipv6 address range permitted
C4) the mesh needs to pick up one delegation and distribute it via a
/128 ip on all relevant interfaces
C5) the mrd daemon appears to not work at all, and eats tons of cpu. I
merely disabled it in this case
/etc/init.d/mrd6 disable.

Despite the size of the list, fixing by hand most of these takes only
few minutes.

2) Mesh issues

I'd mildly misconfigured the mesh interfaces by default - they need ip
addresses assigned in the server case, statically, to 'just work'.
Clients, just work, however, once a server is set up. It's kind of
cool to just bring up a new cero box and see it automatically connect
and route...

3) DNSSEC and ntp can take minutes to sort out on boot

bug 113 MUST DIE.

4) Renumbering is a pita

The sed script method is not really what we want. Renumbering needs to
be fast, easy, and just work, if we ever are going to escape rfc1918
hell.

5) Renamig is a pita.

See 4. Notably it's helpful to edit /etc/avahi/*.conf and change the
name of the router to something unique, mdns proxying 'just worked',
even with two devices proxying, it worries me.

Similarly, I like having multiple dns servers if you are going to have
multiple routers, and setting those up to mirror one another is a
PITA.

6) I need to document how to set up an interior router better

(especially as this is how I test and encourage others to test)

-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
http://www.bufferbloat.net