[Cerowrt-devel] next cerowrt release plan (of sorts)

Mon Sep 17 13:24:38 EDT 2012

I continue to hack primarily on getting a bunch of codels primed for
this release, on x86, working with htb and a few other things in
conjunction.

Preliminary notes:

ns2_codel smokes straight codel under heavy load. In my 150
bidirectional stream test, ns2_codel can reduce a backlog far more
than codel.

It's difficult to tell the difference between fq_codel (using the
original codel) and nfq_codel, but it seems to be a win. Have tons of
data that I'd need to plot.

Using a smaller quantum than 1514 in the general case seems to be a
good idea, it pushes big data to the back of the queue and allows
interactive traffic to work mildly better.

I completed "efq_codel" which attempts to remove all single fifo
dependencies from the codel algo.

I think I succeeded in doing that. It's kind of ugly though.

Performance is comparable to nfq_codel, and perhaps, with some more
thought as to how to grow the window, there's some room to manoeuvre
ultimately that might drain the horizontal queue better, but I'll
settle for "equivalent" right now.

bfq_codel - which is my first attempt at handling the bittorrent
problem - currently crashes. If I can't get it to work before the end
of today, I'm going to bail on it for this release. I'm not happy with
the idea anyway...

Working with htb at speeds below 4Mbit is a real disappointment. Given
that a big packet at 1Mbit is 13ms I'm tempted to set codel's target
at above that when fiddling at these low rates to see what happens.
Adding hfsc + fq_codel to the mix of tests as I write (or rather, as I
compile new kernels)

I think having flows > 1024 is a small win.

also:

I will try to get the daap stuff done tonight

also:

Steven walker has fixed package signing in the Cerowrt-3.3 repo. I'm
still throwing ENOTIME errors, and he's included some interesting,
fragmentary documentation on it, which could use a third party to
try...

One thought of mine would be that it would be nice to allow for
multiple signing keys on the keyring, so that folk that build more of
cero (or one day openwrt) and are trusted, can also make their binary
repos available.

Anyway, he laid out the following instructions. If there is anyone out
there building cero, they might want to try this....

Add your key:
mkdir -p /tmp/etc/opkg
gpg --export <key> | <buildroot>/staging_dir/host/bin/opkg-key -o /tmp add -
Push /tmp/etc/opkg/{secring,trustdb,trusted}.gpg to
cerofiles-3.3/files/etc/opkg/

The signing script would:
Download bin/ar71xx/packages/Packages, sign it with
https://github.com/dtaht/Cerowrt-3.3/blob/master/package/Makefile#L64
and upload Packages.sig
Download the images in bin/ar71xx/, sign them with
https://github.com/dtaht/Cerowrt-3.3/blob/master/include/image.mk#L157
and upload *.asc

Enable signature checking on an opkg-gpg enabled build:
echo "option check_signature 1" >> /etc/opkg.conf

Packages.sig should be verified upon an opkg update

On Sun, Sep 16, 2012 at 7:02 AM, Dave Taht <dave.taht at gmail.com> wrote:
> I took a few days off to get back to the east coast, take care of a
> few family matters, and visit culturecon 2012 in Philly and Boston,
> where I learned that "agile" had established a theoretical framework
> around few things I was already doing (continuous integration,
> engagement with testers and users, short development cycles, small
> feature improvements, etc), that might guide some of my future
> behavior around this project.
>
> I liked very much the concept of just dealing with a small backlog of
> items for a given 2 week (or less) scrum. It makes the overwhelming
> backlog of 150 bugs in the cerowrt db along with coping with the
> prospect of having to fix 2,000,000,000 machines in the long run a
> little less overwhelming!
>
> I have a backlog of related reading to do...
>
> While I didn't drink all the kool-aid at the conferences, (I was
> attracted to the kanban concept, and need to think about the
> sociocratic principles as outlined in:
> http://www.amazon.com/We-People-Consenting-Deeper-Democracy/dp/0979282705
> ) I did meet a lot of interesting people and got a chance to exercise
> the non-coding portion of my brain a bit.
>
> So while I did that and tried to also deal with the funding floor
> problem, I did get some bits of cerowrt done
>
> + updated quagga and babeld to git head
> + Merge with openwrt, which contains a major upgrade to
> wireless-compat and the usual ream of bug fixes
> + Fixed yet another CVE in bind
> + Added aiccu and ipset back into the build. I don't know when ipset
> vanished, sorry about that.
>
>
> - Along the way I managed to break opkg support. I note that a very
> important and out-of-openwrt tree feature has been in cerowrt for a
> while - that of the ability to sign packages, added by stephen walker
> a while back. Due to the need for manual intervention to actually sign
> packages, I've never actually used it, and now both opkg and opkg
> with-signing are broken as I write in the main trees. Apologies.
>
> My own focus for this release is to be able to A/B the current linux
> codel and the ns2 implementation I put in the last cerowrt release in
> the same firmware, to determine (after lots of testing!) which is
> better. I've also burned a lot of time trying to come up with a
> wfq_codel, that uses a codel implementation with a few single fifio
> assumptions removed and some saner support for a background queue for
> things like bittorrent. There will also need to be patches to
> iproute2. Getting those kernel modules and iproute2 patches done is my
> personal blocker for getting a new release out. Simultaneously I plan
> on getting an x86 version of the kernel done on 3.6-rcX....
>
> I'd like very much to get aqm-scripts/simple_qos.sh actually pulling
> data from the luci web page and conf file, too, but find my own gui
> and uci skills lacking. anyone? I find the qos-script's usage of awk
> and sh to be impossible to understand and would prefer to just come up
> with something in lua that could do the job - particularly as I'd like
> to be able to A/B/C the various variants of the underlying algos via
> remote control on the testbed boxes.... but I don't care how it
> interfaces, just so that (someone!) comes up with an interface.
>
>
> Other things in my backlog are two multicast responder-like daemons -
> but can't remember their names mldna?
> Need to give gw.home.lan a DNS entry for dnsmasq
>
>
> The code name for this release attempt is "sugarland" -  see
> http://esr.ibiblio.org/?p=4566 for details
>
> I'm shooting for wednesday.
> If you have trivial feature requests, please make 'em now!
> If you have code you want to check in, please get them in by tuesday!
>
>
> --
> Dave Täht

-- 
Dave Täht
http://www.bufferbloat.net/projects/cerowrt/wiki - "3.3.8-17 is out
with fq_codel!"