[Cerowrt-devel] cerowrt 3.3.8-17: nice latency improvements, some issues with bind

Fri Aug 17 12:52:53 PDT 2012

On Fri, Aug 17, 2012 at 12:05 PM, Török Edwin
<edwin+ml-cerowrt at etorok.net> wrote:
> On 08/17/2012 09:05 PM, Dave Taht wrote:
>> I'm widening the distribution of this email a little bit in light of
>> the benchmark results (somewhat too far) below, and some of the other
>> issues raised.
>
> [ok, will write a separate reply for the benchmark numbers]
>
>>
>> On Fri, Aug 17, 2012 at 1:52 AM, Török Edwin
>> <edwin+ml-cerowrt at etorok.net> wrote:
>>> However I've encountered some issues with bind. After powering on the router this morning DNS wasn't working,
>>> and logread showed a lot of errors from bind about a broken trust chain on every domain name.
>>> Any idea what could've caused this behaviour?
>>
>> This is http://www.bufferbloat.net/issues/113 (relevant bugs have also
>> been filed in the dnssec and ntp databases)
>>
>> a long standing circular problem between getting accurate time via ntp
>> and dns, so that dnssec can be enabled.
>
> I was using unbound on openwrt for dnssec before and I haven't noticed this problem.

How is that on memory and configurability?

> However I had some .ro time servers configured, and apparently they use quite a wide range
> for their RRSIG, so maybe I was just lucky not to hit a situation where both .ro and .org would fail to validate.
> RRSIG   NS 5 2 7200 20120819122953 20120720122953....
> RRSIG   NSEC 8 1 86400 20120824000000 20120816230000 ...
>
> While the .org RRSIG has quite a recent timestamp:
> org.                    900     IN      RRSIG   SOA 7 1 900 20120907184119 20120817174119
>
> Added the .ro timeservers to cerowrt now, and will see if the problem occurs again.

You were lucky, and it will. openwrt/cerowrt can periodically write
the current time to flash, but not often enough for dnssec on a fresh
boot, and more often would be mildly bad on flash wear.

I wasn't aware however that some timeservers were available that

>>> Another minor issue is that p910nd and luci-app-p910nd were not available via opkg install, but I found them on openwrt.org, so that works now.
>>
>> I don't know what they are but I can enable them in the next build.
>
> Thanks, they are a simple way to share your USB printer across the network without running CUPS on the router itself.

Useful. OK. Added to next build (as modules)

>
>>
>>> DHCPv6-PD had to be configured manually of course, same as with openwrt, the difference is that I only get IPv6 on wired interfaces now,
>>> and not on wireless.
>>> That seems to be by design because the interfaces are not bridged anymore and I get only a /64 from my ISP (slan_len 0), so can't really create
>>> more sub-networks from it.
>>
>> As multiple providers seem to think that a single /64 "is all you
>> need", despite the prevalence of guest and other sorts of secondary
>> networks on ipv4. This is a HUGE problem on the current native ipv6
>> deployments.
>>
>> Note that it's not exactly fair to blame the providers, most of the
>> home CPE gear they are dealing with can barely handle ipv6 in the
>> first place, being based on ancient kernels and specifications. That
>> gear is improving, all too slowly, with things like openwrt/cerowrt in
>> the lead.... (apple seems to be doing fairly well, too)
>>
>> Having only a single /64 delegated makes ipv6 unusable IMHO.
>
> Still usable on my main machine, a pain to make it work with VMs though.
> (only way I could make it work was to bridge them to host's ethernet).

one of the many use cases where a single /64 is not enough.

>>
>> I (or rather juliusz) solved the single /64-only problem years ago by
>> switching to using babel and ahcp, which pushes out ipv6 /128 ips.
>> This method has the added benefit of making switching between multiple
>> wired and wireless APs utterly transparent, even for long held TCP
>> connections.
>
> Thanks, will have to try that out.

http://www.bufferbloat.net/projects/cerowrt/wiki/Mesh has some info on
that. Although it makes it look overly complex. On linux with network
manager off, it's 3-5 lines of script...

There is a #babel channel on irc and #bufferbloat to help get that going.

>
>>
>> I run my own networks this way whenever possible, as it's *really
>> nice* to be able to unplug and not lose 20 ssh connections, and plug
>> back in, to get bandwidth, and have babel figure out the right way to
>> go automagically.
>>
>> However fixing both the APs and the hosts (via adding ahcp and babel)
>> is kind of fixing a global infrastructure issue that is hard to get
>> the rest of the world to agree to, and things like network manager
>> don't think this way, either... But I'm glad to see progress being
>> made in homenet towards having a flooding prefix distribution protocol
>> based on something like ahcp, this will cut down on NAT usage in ipv4
>> and lead to a more flexible network in the future. - and I'm sure more
>> and more native deployments will delegate /60s or better in the
>> future.
>>
>> Using dhcpv6 it is also possible to do allocations of /80s but this
>> breaks the 95% of all devices that only can do SLAAC.
>>
>> It is best to get at least a /60 delegation from the ISP.
>
> They probably won't listen. For the amount I pay them I'm happy I get IPv6 at all :)

Enough noise and it'll happen. There is enough demand from small
business and so on for it to happen. Eventually.
>>
>> My way of coping with the half-arsed single /64 delegation ipv6 native
>> deployments I've dealt with thus far has been 6to4 and 6in4, which do
>> /48s.
>
> I get slightly higher throughput in IPv6 though (75 Mbps vs 45 Mbps).
> Apparenly ISP forgot to shape my IPv6 traffic the same way it does with my IPv4.
>
> Although... if I have QoS enabled in cerowrt and I set download speed limit to 47000 kbit/s
> I'm still able to do 62000kbit/s with wget.
> If I set the limit to 27000 then wget -4 speed drops to 24000 kbit/s, but -6 speed stays the same.
>
> Doesn't QoS apply to IPv6 as well?

Openwrt's QoS system has a dependency on conntrack, so it does NOT
handle native ipv6, and does not shape ipv6 at all. This is a bad
thing,
and I've not found a way to fix it given the current structure and
arcane-ness (awk??) of the existing openwrt qos script. I'm not happy
with how it does prioritization of smaller packets, either....

The need to treat ipv6 traffic as well as ipv4 traffic in a
shaper/limiter is in part why the simple_qos.sh script exists. It does
ipv6 and diffserv...

I have been trying to come up with a ipv6 and ipv4 enabled alternative
to std openwrt qos for a while, but that one is buggy (on restart),
and not gui enabled, and has issues at high and low bandwidths in htb
setup as to quantum size. Also evolving something towards the
available complexity/utility of the openwrt is hard.

So I've been building ever simpler models here in my own quest for the
"one true bandwidth limiter/shaper", but working out bugs in the
underlying fq_codel system(s) has been taking priority. I THINK a 2
tier model will work almost as well as a 3 or 4 tier one, and it helps
on queue length, or using perhaps using qfq-esq weighting rather than
htb tiers will be better... and then there's wifi to fix which has 4
tiers... and the issues are too long to go into here....

PLEASE feel free to hack on either openwrt qos or simple qos or any of
the many alternatives available (dan siemon had a good start at one),
gargoyle is doing interesting stuff, etc.

Most of the time these days, I think the ultimate goal will be to do a
tbf version of fq_codel. htb and hfsc add interesting sorts of
overhead, and fq_codel handles multiple queues well already, so...

>> IF you'd like to have more bandwidth back, you can jiggle the qlen_*
>> variables in the debloat script up, but remember that tcp's reaction
>> time is quadratic as to the amount of buffering. I'd be interested in
>> you repeating your benchmark after doing that? The difference between
>> 3 buffers and 8 is pretty dramatic...
>
> Will do, and report back (to both ML).

Yes, that last message was rather overlong and unfocused for the bloat
list. Thx.

>
> Thanks for the detailed reply.
>
> --Edwin

-- 
Dave Täht
http://www.bufferbloat.net/projects/cerowrt/wiki - "3.3.8-17 is out
with fq_codel!"