From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-we0-f171.google.com (mail-we0-f171.google.com [74.125.82.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 458E5208AB0 for ; Fri, 17 Aug 2012 12:52:55 -0700 (PDT) Received: by weyx43 with SMTP id x43so5910501wey.16 for ; Fri, 17 Aug 2012 12:52:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=xVzQHZCKxTp4toJMNXV4Nd4Jkl6VvRoNDDEUHxbNWhQ=; b=f0pb7Ku6RxfwvbzvzWMa7dVrNj18HhnCvEKZfmvIkEceftmGmWoMd/5a+V+hliKIb/ JW+LCiwwbVCJ15QArufDHLh+aHUw+EtL/F50Cb9D+rO/ldnVo0IGOx9K8d9neUTyoYG8 UDOSKmTjS+DzgnD/e8tIFJG+kPIqAMmqAC5L6jNHsmrjIkLGzA10blNTchsTaBtJHN4a zhIkxOWTd8aHLkQqg+F4z90wauL6vYLXpVjKwBaK6HZBo5WoYYsGHOkcaBkSE9I4RNiI g5cgAVm+SbA921dHqGeec1YbnC/ha+b7zP2lf6YMuYGR+y6FOMpr1NDt8MNSuQQXGDqT rfWw== MIME-Version: 1.0 Received: by 10.216.181.67 with SMTP id k45mr2884099wem.17.1345233173363; Fri, 17 Aug 2012 12:52:53 -0700 (PDT) Received: by 10.223.143.69 with HTTP; Fri, 17 Aug 2012 12:52:53 -0700 (PDT) In-Reply-To: <502E9609.5040800@etorok.net> References: <502E064C.50305@etorok.net> <502E9609.5040800@etorok.net> Date: Fri, 17 Aug 2012 12:52:53 -0700 Message-ID: From: Dave Taht To: =?ISO-8859-1?Q?T=F6r=F6k_Edwin?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: cerowrt-devel@lists.bufferbloat.net Subject: Re: [Cerowrt-devel] cerowrt 3.3.8-17: nice latency improvements, some issues with bind X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Aug 2012 19:52:56 -0000 On Fri, Aug 17, 2012 at 12:05 PM, T=F6r=F6k Edwin wrote: > On 08/17/2012 09:05 PM, Dave Taht wrote: >> I'm widening the distribution of this email a little bit in light of >> the benchmark results (somewhat too far) below, and some of the other >> issues raised. > > [ok, will write a separate reply for the benchmark numbers] > >> >> On Fri, Aug 17, 2012 at 1:52 AM, T=F6r=F6k Edwin >> wrote: >>> However I've encountered some issues with bind. After powering on the r= outer this morning DNS wasn't working, >>> and logread showed a lot of errors from bind about a broken trust chain= on every domain name. >>> Any idea what could've caused this behaviour? >> >> This is http://www.bufferbloat.net/issues/113 (relevant bugs have also >> been filed in the dnssec and ntp databases) >> >> a long standing circular problem between getting accurate time via ntp >> and dns, so that dnssec can be enabled. > > I was using unbound on openwrt for dnssec before and I haven't noticed th= is problem. How is that on memory and configurability? > However I had some .ro time servers configured, and apparently they use q= uite a wide range > for their RRSIG, so maybe I was just lucky not to hit a situation where b= oth .ro and .org would fail to validate. > RRSIG NS 5 2 7200 20120819122953 20120720122953.... > RRSIG NSEC 8 1 86400 20120824000000 20120816230000 ... > > While the .org RRSIG has quite a recent timestamp: > org. 900 IN RRSIG SOA 7 1 900 2012090718411= 9 20120817174119 > > Added the .ro timeservers to cerowrt now, and will see if the problem occ= urs again. You were lucky, and it will. openwrt/cerowrt can periodically write the current time to flash, but not often enough for dnssec on a fresh boot, and more often would be mildly bad on flash wear. I wasn't aware however that some timeservers were available that >>> Another minor issue is that p910nd and luci-app-p910nd were not availab= le via opkg install, but I found them on openwrt.org, so that works now. >> >> I don't know what they are but I can enable them in the next build. > > Thanks, they are a simple way to share your USB printer across the networ= k without running CUPS on the router itself. Useful. OK. Added to next build (as modules) > >> >>> DHCPv6-PD had to be configured manually of course, same as with openwrt= , the difference is that I only get IPv6 on wired interfaces now, >>> and not on wireless. >>> That seems to be by design because the interfaces are not bridged anymo= re and I get only a /64 from my ISP (slan_len 0), so can't really create >>> more sub-networks from it. >> >> As multiple providers seem to think that a single /64 "is all you >> need", despite the prevalence of guest and other sorts of secondary >> networks on ipv4. This is a HUGE problem on the current native ipv6 >> deployments. >> >> Note that it's not exactly fair to blame the providers, most of the >> home CPE gear they are dealing with can barely handle ipv6 in the >> first place, being based on ancient kernels and specifications. That >> gear is improving, all too slowly, with things like openwrt/cerowrt in >> the lead.... (apple seems to be doing fairly well, too) >> >> Having only a single /64 delegated makes ipv6 unusable IMHO. > > Still usable on my main machine, a pain to make it work with VMs though. > (only way I could make it work was to bridge them to host's ethernet). one of the many use cases where a single /64 is not enough. >> >> I (or rather juliusz) solved the single /64-only problem years ago by >> switching to using babel and ahcp, which pushes out ipv6 /128 ips. >> This method has the added benefit of making switching between multiple >> wired and wireless APs utterly transparent, even for long held TCP >> connections. > > Thanks, will have to try that out. http://www.bufferbloat.net/projects/cerowrt/wiki/Mesh has some info on that. Although it makes it look overly complex. On linux with network manager off, it's 3-5 lines of script... There is a #babel channel on irc and #bufferbloat to help get that going. > >> >> I run my own networks this way whenever possible, as it's *really >> nice* to be able to unplug and not lose 20 ssh connections, and plug >> back in, to get bandwidth, and have babel figure out the right way to >> go automagically. >> >> However fixing both the APs and the hosts (via adding ahcp and babel) >> is kind of fixing a global infrastructure issue that is hard to get >> the rest of the world to agree to, and things like network manager >> don't think this way, either... But I'm glad to see progress being >> made in homenet towards having a flooding prefix distribution protocol >> based on something like ahcp, this will cut down on NAT usage in ipv4 >> and lead to a more flexible network in the future. - and I'm sure more >> and more native deployments will delegate /60s or better in the >> future. >> >> Using dhcpv6 it is also possible to do allocations of /80s but this >> breaks the 95% of all devices that only can do SLAAC. >> >> It is best to get at least a /60 delegation from the ISP. > > They probably won't listen. For the amount I pay them I'm happy I get IPv= 6 at all :) Enough noise and it'll happen. There is enough demand from small business and so on for it to happen. Eventually. >> >> My way of coping with the half-arsed single /64 delegation ipv6 native >> deployments I've dealt with thus far has been 6to4 and 6in4, which do >> /48s. > > I get slightly higher throughput in IPv6 though (75 Mbps vs 45 Mbps). > Apparenly ISP forgot to shape my IPv6 traffic the same way it does with m= y IPv4. > > Although... if I have QoS enabled in cerowrt and I set download speed lim= it to 47000 kbit/s > I'm still able to do 62000kbit/s with wget. > If I set the limit to 27000 then wget -4 speed drops to 24000 kbit/s, but= -6 speed stays the same. > > Doesn't QoS apply to IPv6 as well? Openwrt's QoS system has a dependency on conntrack, so it does NOT handle native ipv6, and does not shape ipv6 at all. This is a bad thing, and I've not found a way to fix it given the current structure and arcane-ness (awk??) of the existing openwrt qos script. I'm not happy with how it does prioritization of smaller packets, either.... The need to treat ipv6 traffic as well as ipv4 traffic in a shaper/limiter is in part why the simple_qos.sh script exists. It does ipv6 and diffserv... I have been trying to come up with a ipv6 and ipv4 enabled alternative to std openwrt qos for a while, but that one is buggy (on restart), and not gui enabled, and has issues at high and low bandwidths in htb setup as to quantum size. Also evolving something towards the available complexity/utility of the openwrt is hard. So I've been building ever simpler models here in my own quest for the "one true bandwidth limiter/shaper", but working out bugs in the underlying fq_codel system(s) has been taking priority. I THINK a 2 tier model will work almost as well as a 3 or 4 tier one, and it helps on queue length, or using perhaps using qfq-esq weighting rather than htb tiers will be better... and then there's wifi to fix which has 4 tiers... and the issues are too long to go into here.... PLEASE feel free to hack on either openwrt qos or simple qos or any of the many alternatives available (dan siemon had a good start at one), gargoyle is doing interesting stuff, etc. Most of the time these days, I think the ultimate goal will be to do a tbf version of fq_codel. htb and hfsc add interesting sorts of overhead, and fq_codel handles multiple queues well already, so... >> IF you'd like to have more bandwidth back, you can jiggle the qlen_* >> variables in the debloat script up, but remember that tcp's reaction >> time is quadratic as to the amount of buffering. I'd be interested in >> you repeating your benchmark after doing that? The difference between >> 3 buffers and 8 is pretty dramatic... > > Will do, and report back (to both ML). Yes, that last message was rather overlong and unfocused for the bloat list. Thx. > > Thanks for the detailed reply. > > --Edwin --=20 Dave T=E4ht http://www.bufferbloat.net/projects/cerowrt/wiki - "3.3.8-17 is out with fq_codel!"