From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-iw0-f171.google.com (mail-iw0-f171.google.com [209.85.214.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 518E32000DD for ; Fri, 1 Jul 2011 10:28:22 -0700 (PDT) Received: by iwn34 with SMTP id 34so4862635iwn.16 for ; Fri, 01 Jul 2011 10:59:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=O9ewbSI+C85ER5HmlOWKZ6jA67vvE2gblf0FV9cbB40=; b=pig/LssOPa9qu4ah/UZUe60q4dWXFvTa3E9QrfmH8fqEiX13YxJWfXkQxLww161lTb CygZdAKzRoEfOgpJ7AKbVp4z0FZaWqVJuxyqIv+gh6/wVoJY15uzi6Gg0DcGtOn9gM+r 0TRBFYdi46SHi/PI7VKptVo0D1CQGLSmEPqUc= MIME-Version: 1.0 Received: by 10.231.63.67 with SMTP id a3mr3152486ibi.94.1309543183582; Fri, 01 Jul 2011 10:59:43 -0700 (PDT) Received: by 10.231.32.137 with HTTP; Fri, 1 Jul 2011 10:59:43 -0700 (PDT) Date: Fri, 1 Jul 2011 11:59:43 -0600 Message-ID: Subject: some notes regarding cerowrt From: Dave Taht To: bismark-devel@projectbismark.net, bloat-devel Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-BeenThere: bloat-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Developers working on AQM, device drivers, and networking stacks" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2011 17:28:22 -0000 I had hoped to have an alpha release of cerowrt 'Ocean City' out today. Due to multiple conflicts on my schedule, and several packages that have not landed (notably ahcp, netperf 2.5, iproute-2.6.39, debloat with blacklists, and cerowrt-chrome) I'm slipping this to next friday and taking off for the holiday. A little history on this - cerowrt came from the wisp6 and iscwrt projects and started more or less formally back in January, when I realized the scope and extent of bufferbloat was a huge blocker to future development of those, and indeed all - millions of - wireless-n based network designs. By May, cerowrt and iscwrt were looking pretty good and stable, cerowrt had been through extensive QA, I had multiple happy (although geeky) users in the field, good uptimes everywhere, and I was busy working on getting iscwrt ready for world IPv6 day... ... and the projectbismark's 'get 15 routers to capetown for testing in 3 weeks' thing came up. It seemed a good idea to take cerowrt, add on the 3 (or so I thought) extra packages, and get it out to more users for feedback. I thought I could do that fast, and cheap, as I'd spent a week at georgia tech the previous month training 3 people on how to do packaging and builds, and thus thought coming in for the last two weeks of their effort would be a win for everybody. As it turned out, it was mostly successful, with only a few problems, and with both cerowrt and bismark I fear we may end up with a success disaster on our hands. It took a week longer than expected as fewer resources were available, but still, getting a distro done in 4 weeks is something that not a lot of people could pull off.... The problems we ran into with bismark in the final (extra) week forced me to cancel iscwrt's release, and defer it for 3 months to a year, which I deeply regret. After catching up on sleep, I have been folding the 'baked' features from iscwrt back into cerowrt as a result. I don't know if iscwrt will continue as a separate project at this point. Anyway... 1) The most useful thing for me that happened from that exercise was that the QoS scripts had a knee in them that cut raw bandwidth in the third world by an unacceptable 30% at those lower bandwidths. I note that this problem is not limited to openwrt, but probably exists everywhere people are trying to do QoS, with the possible exception of Gargoyle. Getting to where the internet with or without QoS 'just works' everywhere in the world - is a major goal. 2) Nobody non-geeky liked the xWrt gui. Additionally everyone found it hard to adjust to using something other than 192.168.1.1 as an address. 3) the need for wireless networking to 'just work' in cases where someone is out of range of the main access point was repeatedly re-proven, and the fact that nobody cared (until they hit the real world) that I deliver that feature, bugged me, because I'd known better. 4) Bismark worked. The darn things seem to be staying up. I care mostly about that. Not only that but the previous generation of cerowrt has routers that have now been up for several months. I'm even happier about that. Many suggestions from both the bismark and cerowrt efforts and results have made it into the openwrt mainline code, multiple bugs found in the Linux kernel and tools have been fixed and pushed out, and I'm utterly delighted about that and deeply grateful to the openwrt and Linux and tool developers that have all been making that happen. So, to address some of the feedback we got from bismark in the cerowrt thin= g 1) QoS is still enabled by default in the gui, but the bandwidths are set to 16Mbit down/4Mbit up, which is about the norm for most cable systems in the US. I hope that with the additional traffic shapers included, notably SFB, it will become possible to not have to be as bandwidth aware as at present, but that will require more work and research. Secondly, there is now the ability to simulate delays using the netem emulator, which should have interesting results in the ability to simulate other parts of the world as well as normal traffic. 2) The entire gui has been swapped from xWRt to luci. This cost me way more time that I'd have liked, but the feedback from the pre-alpha users is all good, and I cannot go back to xwrt. The negative aspect is that I do not 'know' the language luci is written in, 'lua', which makes it very difficult and slow for me to do any actual work on the gui, at present. 3) Mesh networking is closer to 'just working'. Actually mesh *routing* does 'just work', it's the address distribution scheme (AHCP) that is busted. The problem has been found and multiple approaches discussed as to how to fix it. 4) network numbering is such a pain that after discussing the problems with the broken edge with jg, I pulled bind-latest (bind 9.8-P2) from iscwrt into the cerowrt build. The user visible changes are that a vendor provided name server is no longer needed (NXDOMAIN WORKS!), that DNS 'just works' even without the internet, and you can get to 'gw.home.lan' by default rather than having to type a network address. Under the hood, DNSsec is enable by default (at present) and seems to 'just work' and ipv6 autonumbering will usually result in a AAAA address. It is also now possible (though not easy at present) to finally rejoin your own network with that of your domain name, example - fl.taht.net, using bind9. Additionally: Update to kernel 2.6.39.2 (from 2.6.37.6) iptables 1.4.11 netem, sfb, drr packet schedulers lighttpd rsync netperf wndr3700v2, x86 kvm, nano-m5 support now some decent but still in progress vpn stuff many other fixes... Lastly, I am deeply grateful to the openwrt people for their contributions towards beating the bloat. Jow in particular, just extensively revised and VASTLY improved the firewall code, and has contributed fixes to the switch and network code that reduce the bloat and make routing better, nbd continues to hammer at wireless-n, and a cast of hundreds more continue to push the state of the art forward. Deeply grateful also to nanog, the members of these lists, the linux, netfilter, & networking folk, and project bismark for making all of this happen. I hope that cerowrt will help serve as a base for future experimentation and network research for many people. More information about all these projects is available at http://www.bufferbloat.net/projects/uberwrt More news as it happens... Have a good July 4! --=20 Dave T=E4ht SKYPE: davetaht US Tel: 1-239-829-5608 http://the-edge.blogspot.com