From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wg0-f47.google.com (mail-wg0-f47.google.com [74.125.82.47]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 4C4152012AD for ; Mon, 17 Sep 2012 10:24:41 -0700 (PDT) Received: by wgbfa7 with SMTP id fa7so4129459wgb.28 for ; Mon, 17 Sep 2012 10:24:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=Y3lBdnUxO21RcFg+1qr6ZM5NXm7kITRfOSuo/ilAXfQ=; b=D45W5gv+ahsp7lZU0t2NM5KfGgMNo7eYFpUHvspKasQRGMeY3tvk5x3/HdK0xCRavj yVr5pe3U+U6J+IN0F9YYfjD7u3WGY/CIweA2UHn9oG6qByBzslarvW66oTJzL8wfIc8w hpOqxg/LvUMZ77lyXd9wByrzE6nO/YVNz2jQ1cSPZhzKlXBPpQz5Bl68YJRoANLDARg8 q2EeuX8gNeJEPeec8h9Vvrx5pH0Lxsd+u15N2ZJ9aRHJLP5lgsdyI7HY+OC/JpnJnVZM I6LI+jw3IBkcLhIGBKixYFkVr/zRrvZ+TKNnOZw3hHQd5K93ekHnswcngFcDXSL2XNZP xodw== MIME-Version: 1.0 Received: by 10.216.237.161 with SMTP id y33mr6208218weq.62.1347902678762; Mon, 17 Sep 2012 10:24:38 -0700 (PDT) Received: by 10.223.159.134 with HTTP; Mon, 17 Sep 2012 10:24:38 -0700 (PDT) In-Reply-To: References: Date: Mon, 17 Sep 2012 10:24:38 -0700 Message-ID: From: Dave Taht To: cerowrt-devel@lists.bufferbloat.net Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Cerowrt-devel] next cerowrt release plan (of sorts) X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Sep 2012 17:24:41 -0000 I continue to hack primarily on getting a bunch of codels primed for this release, on x86, working with htb and a few other things in conjunction. Preliminary notes: ns2_codel smokes straight codel under heavy load. In my 150 bidirectional stream test, ns2_codel can reduce a backlog far more than codel. It's difficult to tell the difference between fq_codel (using the original codel) and nfq_codel, but it seems to be a win. Have tons of data that I'd need to plot. Using a smaller quantum than 1514 in the general case seems to be a good idea, it pushes big data to the back of the queue and allows interactive traffic to work mildly better. I completed "efq_codel" which attempts to remove all single fifo dependencies from the codel algo. I think I succeeded in doing that. It's kind of ugly though. Performance is comparable to nfq_codel, and perhaps, with some more thought as to how to grow the window, there's some room to manoeuvre ultimately that might drain the horizontal queue better, but I'll settle for "equivalent" right now. bfq_codel - which is my first attempt at handling the bittorrent problem - currently crashes. If I can't get it to work before the end of today, I'm going to bail on it for this release. I'm not happy with the idea anyway... Working with htb at speeds below 4Mbit is a real disappointment. Given that a big packet at 1Mbit is 13ms I'm tempted to set codel's target at above that when fiddling at these low rates to see what happens. Adding hfsc + fq_codel to the mix of tests as I write (or rather, as I compile new kernels) I think having flows > 1024 is a small win. also: I will try to get the daap stuff done tonight also: Steven walker has fixed package signing in the Cerowrt-3.3 repo. I'm still throwing ENOTIME errors, and he's included some interesting, fragmentary documentation on it, which could use a third party to try... One thought of mine would be that it would be nice to allow for multiple signing keys on the keyring, so that folk that build more of cero (or one day openwrt) and are trusted, can also make their binary repos available. Anyway, he laid out the following instructions. If there is anyone out there building cero, they might want to try this.... Add your key: mkdir -p /tmp/etc/opkg gpg --export | /staging_dir/host/bin/opkg-key -o /tmp add = - Push /tmp/etc/opkg/{secring,trustdb,trusted}.gpg to cerofiles-3.3/files/etc/opkg/ The signing script would: Download bin/ar71xx/packages/Packages, sign it with https://github.com/dtaht/Cerowrt-3.3/blob/master/package/Makefile#L64 and upload Packages.sig Download the images in bin/ar71xx/, sign them with https://github.com/dtaht/Cerowrt-3.3/blob/master/include/image.mk#L157 and upload *.asc Enable signature checking on an opkg-gpg enabled build: echo "option check_signature 1" >> /etc/opkg.conf Packages.sig should be verified upon an opkg update On Sun, Sep 16, 2012 at 7:02 AM, Dave Taht wrote: > I took a few days off to get back to the east coast, take care of a > few family matters, and visit culturecon 2012 in Philly and Boston, > where I learned that "agile" had established a theoretical framework > around few things I was already doing (continuous integration, > engagement with testers and users, short development cycles, small > feature improvements, etc), that might guide some of my future > behavior around this project. > > I liked very much the concept of just dealing with a small backlog of > items for a given 2 week (or less) scrum. It makes the overwhelming > backlog of 150 bugs in the cerowrt db along with coping with the > prospect of having to fix 2,000,000,000 machines in the long run a > little less overwhelming! > > I have a backlog of related reading to do... > > While I didn't drink all the kool-aid at the conferences, (I was > attracted to the kanban concept, and need to think about the > sociocratic principles as outlined in: > http://www.amazon.com/We-People-Consenting-Deeper-Democracy/dp/0979282705 > ) I did meet a lot of interesting people and got a chance to exercise > the non-coding portion of my brain a bit. > > So while I did that and tried to also deal with the funding floor > problem, I did get some bits of cerowrt done > > + updated quagga and babeld to git head > + Merge with openwrt, which contains a major upgrade to > wireless-compat and the usual ream of bug fixes > + Fixed yet another CVE in bind > + Added aiccu and ipset back into the build. I don't know when ipset > vanished, sorry about that. > > > - Along the way I managed to break opkg support. I note that a very > important and out-of-openwrt tree feature has been in cerowrt for a > while - that of the ability to sign packages, added by stephen walker > a while back. Due to the need for manual intervention to actually sign > packages, I've never actually used it, and now both opkg and opkg > with-signing are broken as I write in the main trees. Apologies. > > My own focus for this release is to be able to A/B the current linux > codel and the ns2 implementation I put in the last cerowrt release in > the same firmware, to determine (after lots of testing!) which is > better. I've also burned a lot of time trying to come up with a > wfq_codel, that uses a codel implementation with a few single fifio > assumptions removed and some saner support for a background queue for > things like bittorrent. There will also need to be patches to > iproute2. Getting those kernel modules and iproute2 patches done is my > personal blocker for getting a new release out. Simultaneously I plan > on getting an x86 version of the kernel done on 3.6-rcX.... > > I'd like very much to get aqm-scripts/simple_qos.sh actually pulling > data from the luci web page and conf file, too, but find my own gui > and uci skills lacking. anyone? I find the qos-script's usage of awk > and sh to be impossible to understand and would prefer to just come up > with something in lua that could do the job - particularly as I'd like > to be able to A/B/C the various variants of the underlying algos via > remote control on the testbed boxes.... but I don't care how it > interfaces, just so that (someone!) comes up with an interface. > > > Other things in my backlog are two multicast responder-like daemons - > but can't remember their names mldna? > Need to give gw.home.lan a DNS entry for dnsmasq > > > The code name for this release attempt is "sugarland" - see > http://esr.ibiblio.org/?p=3D4566 for details > > I'm shooting for wednesday. > If you have trivial feature requests, please make 'em now! > If you have code you want to check in, please get them in by tuesday! > > > -- > Dave T=E4ht --=20 Dave T=E4ht http://www.bufferbloat.net/projects/cerowrt/wiki - "3.3.8-17 is out with fq_codel!"