Re: [Cerowrt-devel] upgrading from CeroWRT --- seeking advice on rule testing

[Dave Taht <dave@taht.net>]

Michael Richardson <mcr@sandelman.ca> writes:

> For the past 9 months I've been trying to replace my 3800 running
> CeroWRT
> with an 18.06 openwrt build running on another 3800.  Thank god for
> serial consoles....
>
> It's proving not so trivial.  No complaints against you Dave: you did
> an
> awesome job, but openwrt wasn't ready for many of your ideas.  I drank
> all your koolaid and added more flavour.  Routed wifi, 172.30.42.x.
> CeroWRT replaced a power-hungry NetBSD 1U system, and at a similar
> time,
> I also replaced multiple 16-port unmanaged switches with a single
> 24-port GE
> managed switch.  So I used multiple VLAN in/out of the 3800 for
> routing
> between my 3+ subnets. ("trusted", "service", "voip/media", 4 wifi. I
> also
> have a VLAN for NFS traffic, which the 3800 does not see).
>
> My uplink is PPPoE over VDSL2 (external SmartRG in modem mode), and I
> have native IPv6, and a static IPv4/28 routed as /32s to systems that
> need
> it.  Most systems are IPv6 only with a Jool box providing NAT64, and
> other
> systems having DualStack with NAT44.

I'm under the impression various ipv6 -> ipv4 nat tools are working much
better now. I can't bring myself to care much about ipv6 until I too can
get a static IPv6 allocation. I'm so fed up with the deployment that
I've been working on adding ips to ipv4....

> In the fall I moved the wifi off the 3800 to a gen-one Turris machine
> that
> got located in the kitchen, closer to the wifi users.
>
> I have numerous netifd issues (the 32-bit int indexed by ifindex bug
> bites
> me), and DHCPv4 and RA has just stopped working on one network. No
> understanding why... something deeper than the ifindex issue.  And I
> know
> that there are known vulnerabilities in some of the system components.
> Thus my strong desire to upgrade.
>
> A backup/restore didn't work.... and so since network is critical to
> my
> day-to-day work, and my family gets pissed if I break Netflix and
> youtube,
> I can only test for short periods of times when the family is out, and
> I'm
> exceptionally lucid.

I know that feeling.

> The naming "se00" vs "ethXX" gets in the way.  I have weird problems
> where
> machines behind the gateway can ping 8.8.8.8, but I can't ping it from
> the
> gateway.  The details don't matter. I'm mostly writing this for future
> people
> googling. I spent another two hours today trying to debug (the first
> time, I
> had no working uplink, and I was missing tcpdump on the new unit. I
> was
> convinced my ISP had dropped my static routes)... 
>
> So I will be starting again from scratch (total factory reset), get it
> going, and then add my custom configuration.

I generally prototype by having a second router entirely take over the
functions of the network. Much like you added a pure wifi router, in
your case I'd have got another router entirely, flashed openwrt, and
tried to get each feature you needed working that way.

I do wish cerowrt's stateless firewall idea had been adopted by openwrt,
it leads to much less complicated rules to just pattern match for s+,
g+, etc.

>
> I particularly find the per-port vs 802.1q VLAN stuff difficult to
> sort out,
> as both come in to the eth0 interface in some kind of tagging, and I'm
> totally unclear if I can have the four LAN switch ports come in as
> seperate
> networks, and *also* have stuff coming in as 802.1q tagged on those
> ports.
> The UI gets it right, but it's hard to use the UI if you've toasted
> the
> network, and are reduced to serial console.  
>
> Aside from any further advice on the switch/vlan issue in the 3800,
> I'm wondering if there are any recent innovations in firewall
> configuration
> testing.  What I'd like (and I've done this before in the distant
> past, but
> always manually) is to have a script that I run from an untrusted
> cloud
> location, that basically just does a series of TCP and UDP (v4 and v6)
> connections to verify that I've got everything configured sanely.

nmap and metasploit are my frameworks.

> That is, it should verify that my mail server answers port 25, but
> nothing else does, that my DNS server answers authoritatively, but not
> recursively, and that my web servers answer with all the right virtual
> hosts.  Unit and regression testing for firewalls.
>
> I used to do this with a hand-craft shell script that used
> nc/telnet/wget/dig.
> I'm hoping that the state of the art has progressed.... maybe there is
> a
> service out there for this?

Not that I'm aware of. I just hit things from the cloud. I worry a lot
about ipv6 holes in general, but haven't pursued it very hard.