From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-iy0-f171.google.com (mail-iy0-f171.google.com [209.85.210.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 65B55200372 for ; Sun, 8 Jan 2012 01:47:54 -0800 (PST) Received: by iagw33 with SMTP id w33so7743422iag.16 for ; Sun, 08 Jan 2012 01:47:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=ELyDq9kd5KacoxJK+S+/+GesnY4Z3lS/SM1fR1tASIc=; b=qA4O6xm3YqJ5nVZHhEXT7ekSVz777NNBLWszULCAJJNKkDgKRNlzVpvr0cb9Dlu+NA 6FAhptauBpe23pR6k6EEASZa52kfTs29YHeW9kXc9pXZdFVtDwZ/rbrNDLrjHHIFETUf sq7ieUFEQefOZHeKQT/hzCHWwhmA/ErsMS/PE= MIME-Version: 1.0 Received: by 10.50.180.167 with SMTP id dp7mr4958394igc.26.1326016072984; Sun, 08 Jan 2012 01:47:52 -0800 (PST) Received: by 10.231.159.193 with HTTP; Sun, 8 Jan 2012 01:47:52 -0800 (PST) In-Reply-To: References: Date: Sun, 8 Jan 2012 10:47:52 +0100 Message-ID: From: Dave Taht To: Richard Brown Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "cerowrt-devel@lists.bufferbloat.net" Subject: Re: [Cerowrt-devel] keeping a better lab notebook, publishing results, etc? X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jan 2012 09:47:54 -0000 Dear Richard: You will find your permissions on the wiki suitably elevated. I'd got behind on user registrations as nearly 95% of them of late have been spam. I'm still about 15 behind... On Sat, Jan 7, 2012 at 9:30 PM, Richard Brown wrote: > Thanks for the update. > > re: A better lab notebook... I don't have any suggestions for better tool= s for a lab notebook. But I have a few thoughts for combining your internal= record-keeping with the publicly-facing pages. Please don't think I'm bein= g whiny - I have a concrete suggestion at the end of all these comments: > > - An important reason (the only reason?) to use any tool is so that you c= an be productive. You need the ability to record your important work/though= ts/plans. http://the-edge.blogspot.com/2009/08/going-retro-re-adopting-emacs.html The original conception of the web included an integrated editor. > - You also need a tool to tell others how things are progressing. This is= the public face to the project that draws interest/testers/excitement/part= icipation. > > - One reason to like Redmine is that it makes the project look profession= al and/or serious. I will confess that its appearance gave me the confidenc= e to take a few minutes to look hard at the project. The documentation is g= ood, correct, and substantially matches the current state of the code. Thank you. > - But.. =A0the Bufferbloat pages for Cerowrt today also provide lots of u= nnecessary tabs/sections that absorb your brain cycles to keep up-to-date. = I will also confess that I was a bit put off by the notes about RC7 & RC8 b= eing delayed, and the deadlines that were missed. I was too! we started going off the rails in september... I like to think we're getting back on track. > > - Another problem with Redmine was that it took me a while to figure out = which version I should consider. > - I looked at ikiwiki. Its presentation is pretty bare bones. Although it= looks as if it could reflect your activities (whether on-line or not), I'm= not sure it would be a good public face for the project. I am considering that, as perhaps my own 'local store' of the bug/wiki database. I can handle the bug database via email, IF I have a copy of it in front of me, as the email to bug database integration is good in redmine. So my own ikiwiki seems like a way to get that. Or RSS, or sorting my email better... A nice thing about ikiwiki would be the ability for me to at least draft a new page - or a revised one - in the text format that redmine uses (textile). I find textile hard to think in as it's enough different from mediawiki and markdown for me to get wrong a lot. That said I do most of my own work in emacs, and there might be some parts of what I'm doing that can be better done in org-mode, and published to html for those that dig html, and kept in git for those that like git logs or having their own personal copy. Org-mode output can be quite lovely. http://orgmode.org/worg/index.html > > - I am pleased that you've put a lot of info into github.com. It makes it= easy for us spectators to see when progress has been made. I like that we are trying to open source and crowdsource 'science', a lot. As one example, I could have sat on the QFQ work that showed it outperforming SFQ, written a paper, waited 9 months for it to be published, only to find that it was actually showing a bug in SFQ that was was fixed by a 2 line patch. I sat on the data long enough to prove to myself that what I was seeing was real, and published it, and eric fixed the problem before breakfast (at milliways). But... from a personal perspective 'being on stage' all the time is difficult. My 'notebook' tends to be full of scurrilous opinions and virtual coffee stains, and worse, incorrect or tenative conclusions. I'd really hate for stuff in it to get heavily linked to. As one (ironic) example - google for 'chair hanging sex'. In reviewing the notebook I've been keeping in git on the deBloat repo on github, I felt compelled recently to revisit and correct several notes I'd made earlier. https://lists.bufferbloat.net/pipermail/cerowrt-commits/2012-January/000030= .html So while I don't mind *ever* being wrong and being corrected - I liked the good ole days of peer review somewhat. Similarly, on the website, our principal recommendations last year, need substantial revision in light of the improvements to the AQMs. Also, preliminary data is indicating that more extensively adopting the wireless VI queue for 'ant'-like packets is maybe the wrong thing in the presence of packet aggregation and decent hybrid AQM FQ technology. We were hot on the VI queue idea last year (and it may still work), I'm not hot on it now, but we are months away from having the infrastructure required to play with wireless to the extent required to truly nail it, even if funding arrived for it. > - I would also like a way to know the fixes in progressive releases of Ce= rowrt. For example, I see that bql-15 is available. (NB It installs and ope= rates just fine.) Is there a way to see the changes/release notes? Alg=FAn dia. The problem with the bql development cycle is that I've had to rebase the tree(s) 4 times, thus far, completely swapping out major patch sets. So having a public repo for that would make people crazy. There's a reason why bql-{2,5-8,11-13} never saw the light of day. I'd like to get towards a linear release cycle where we can march towards an end goal, that's not what we have at the moment. Perhaps I can hook up the local git log hook to send mail on that. bql-15 is basically the major SFQ head of line fix backported, and the time in queue patch removed. I am currently seeing a regression in performance (since november at least) of going from ~540Mbit/sec forwarding rate (which is what we had in mid-august) to half that. Removing time in queue had a small effect. I fear that a new bug in hostapd, netlink, the network device driver(s) has appeared during the upgrade process from kernel 2.6.39 to 3.1.x (this problem is in all releases since at least mid-november - and if you aren't running at 260Mbit + you aren't going to be bothered by it) I have a problem in that Linux 3.2 contains nearly nothing of importance to the cerowrt project. 3.3 does. Porting openwrt to 3.3 will be a PITA. Waiting for 3.3 to stabilize and working on other stuff - or on 3.3 on x86 - is far more productive, which is mostly what I'm doing now. btw, a version of the bql-16 for x86_64 is now up at: http://huchra.bufferbloat.net/~d/bql/ This is basically linux-3.3 head + 1 patch: SFQRED. There's so much good stuff in this worth playing with: sfq rocks in the general case - red is better adaptive red is implemented qfq works as well Even getting people to do the simplest tests (like running sfq on their ethernet devices) would be awesome. tc qdisc add dev eth0 root sfq is all you need for some basic FQ wondefulness. in the case of wireless it's a little more complex, and kind of needs a script like my staqfq to enable it... > - Finally, I know you're not looking for extra work, and certainly not do= cumentation. A good solution has to minimize the amount of work you put int= o communicating the state of the project beyond what you would do anyway. F= or example: Well my goal here is to get more people involved in various aspects of what they can and want to do. > - Notes in github, mailing lists: Good > - Time spent tweaking words in Redmine: Less good > > And now my suggestion: > > 1) Continue to use Redmine. It's attractive, and there's a lot of good in= formation is already there. Random visitors are familiar with its format an= d capabilities and will feel right at home. We have been considering to chiliproject sometime this year, as it has several features we need. Git integration and the search engine has never worked in our deployment. We also need CVE support. https://www.chiliproject.org/ > 2) Prune the Redmine site mercilessly. Cutting something that is less than perfect is hard. Re-organising it is ha= rd. >Remove anything that's no longer your current focus, or that distracts. Af= ter all, that's why you chose to cut out all the features that weren't goin= g to happen anyway... I actually haven't done that. I *temporarily* ripped out every package that broke so I could move forward on what's important. We're certainly considering what a feature set for a viable release would look like, given available resources. I'm totally cool with not making any decisions regarding the feature set for a while. What's in play works well enough... The nice thing about not making decisions and trying to get things back on a fixed schedule is that... Fixes for nearly all the features that broke in november have landed. I just haven't taken the time to integrate them. I have a few fixes to push out, too. >For exaxmple, I would remove or minimize references to UberWRT, iSCWRT, Bi= smark, and other related projects unless you get a significant return from = those projects. Redmine can then reflect a) things that are and will always= be true about your current effort, and b) link to the info that's changing= rapidly in other resources. (See my suggestions below). the uberwrt wiki needs to be entirely moved over to the cerowrt wiki, and uberwrt killed as a name and project. as for iscwrt, not my call. we have a fabulous relationship with isc, but they've been buried of late and haven't been able to do much coding. I merged most of the core features of iscwrt into cerowrt back in march. Bismark was very active for a while, their project however has moved to being somewhat insular (it's gaining steam, tho), if they are not going to continue using us to manage their lives I'll consider giving them a dump of the wiki during a future upgrade cycle. > > 3) Continue to use github to reflect useful test cases, thoughts, etc. OK, doing that. > 4) If you need for a personal notebook, pick your favorite. But find a wa= y to get it into github from time to time so that we can read about it and = so you're protected against failures. I am really torn between git and git + org-mode. I will evolve towards > 5) You don't need to provide publicly-editable wiki-like facilities today= . You and perhaps a couple others are driving the development, and Redmine = site doesn't need to be burdened with the ability to support lots =A0of pos= ters right now. Nononononono... getting more people than what we have working on the wiki is the point. :) For example there was a huge series on hfsc that needs wikification... the sfq and sfqred work needs to be wikified... etc. etc. > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Suggestions for the Redmine site = =A0=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > Based on my understanding of your current goals, you might edit the Redmi= ne site at http://www.bufferbloat.net/projects/cerowrt to say the following= . (The information below would have answered the questions that I had about= the project, and given me a good feeling about it right away. However, ple= ase feel free to ignore/adjust any of this...) > > ---- Overview tab ----- > > Overview: > > Bufferbloat is a huge drag on Internet performance created, ironically, b= y previous attempts to make it work better. The one-sentence summary is "Bl= oated buffers lead to network-crippling latency spikes." You can read more = about this problem at http://www.bufferbloat.net/projects/bloat/wiki/Introd= uction. [NB - this is a brilliant description of the problem -reb] ESR wrote this and the general purpose guide using the traffic analogy. I was very grateful for his wordsmithing. I should credit him. > > CeroWrt is a project built using OpenWrt to resolve these endemic problem= s in home networking today, and to push the state of the art of edge networ= ks and routers forward. Projects include proper IPv6 support, tighter integ= ration with DNSSEC, wireless mesh networking, among others, notably reducin= g bufferbloat in both the wired and wireless components of the stack. > > To minimize the effects of hardware dependencies, we have chosen the Netg= ear WNDR3700v2 as the sole hardware for the experiments. The open source su= pport for it is extensive, it has 16MB of flash and 64MB of RAM, it support= s a USB flash stick, they are inexpensive (around $120). We have the 3800 working just fine now, and it's a better platform (128MB ram). That said, we've had zero help from netgear and I'd consider using any of the other 34 ar71xx-using products in the future as the default. (not that that belongs in the wiki - WNDR3700v2 and WNDR3800) > Goals of the Cerowrt Project: > > The next generation of cerowrt's feature set is currently being discussed= on the Cerowrt-devel mailing list. At this stage, we are working to create= a minimal, functioning build as a platform for experimentation. > > Sources of Information about the project: > > General list for discussing Bufferbloat: https://lists.bufferbloat.net/li= stinfo/bloat > Cerowrt-devel mailing list: https://lists.bufferbloat.net/listinfo/cerowr= t-devel > Lab Notebook in Github: https://github.com/dtaht/deBloat All good. > Installation Instructions: > > The Cerowrt Installation Guide at http://www.bufferbloat.net/projects/cer= owrt/wiki/OCEAN_CITY_INSTALLATION_GUIDE provides information for getting th= e software installed on the WNDF3700v2 and initial configuration. > > Try the Software: > > The current software is being used for experimentation on the algorithms.= A few brave souls are also using it as a second router in their homes. At = this stage, we do not recommend it as your production router. Heh. > The most recent builds are the "bql-smoketests". They are under active de= velopment. > > The rc6 build is older, but in wider use. We have not received any report= s of crashes so feel free to try in less experimental situations. > > You can retrieve either from the Files tab. > > ------- Activity tab ------- > > Not sure what you should go here. The info up to 29Dec2011 is good, but t= here is nothing about newer incremental builds. Should the ongoing narrativ= e be transitioned to github or the mailing lists? If so, you need a link on= this tab to point people there. > > ----- Roadmap tab ----- > > This needs the most pruning. Remove most (all?) of the text and replace w= ith a brief description of where you expect to take things in the next mont= h and three months (What you told me at the end of https://lists.bufferbloa= t.net/pipermail/cerowrt-devel/2012-January/000050.html might be a good star= t.) Ahhmmmmmmmmmmm I don't have a roadmap. I'd maybe move stuff to 'old roadmap' and put in something short. > --- Issues tab --- > > Probably can remain, unless it's not really correct. No the bug database is very up to date. > > --- Gantt tab --- > > Is there anything valuable here? I find the gantt charts in this implementation completely useless. I confess to missing ms-project. It is helpful to go through the existing bugs, figure out how long they will take to fix, and who can fix them (not in that order), and jg and I have only reviewed about 1/3 the outstanding ones. > > --- Calendar tab --- > > Is there anything valuable here? Not really. > --- News tab --- > > Remove anything that's more than 3 months old, or that's not relevant to = the new goals. makes some sense > > The About Cerowrt section at the bottom may have something to retain, per= haps moving it to the Overview page. But many of the features listed have n= ow been dropped from near-term consideration (IPv6, for example). um, er, ipv6 works, just not testing it > Given that things are moving so rapidly, it makes sense that this page sh= ould refer people to the mailing lists or the lab notebooks, and not strive= to be updated in parallel. Yes. > --- Documents tab --- > > Again, this should reflect the project's current state. Jim Gettys' paper= is good and should remain. I wonder if everything from the Wiki tab should= be moved here. that's not how redmine works. Documents are uploaded non-wiki-format docume= nts. > > Maybe a link to the Files tab as well. > > --- Wiki tab --- > > Maybe move all these links to the Documents page, then say "No wiki entri= es at this time. See the mailing lists and Lab Notebook". wiki is big, cross-linked etc.... > > --- Files tab --- > > I feel an urge for a single page that provides a description of "stable" = and "current" versions. It might say: Excellent idea. Go for it. > > The current builds of Cerowrt are being used for experimentation on the a= lgorithms to eliminate bufferbloat. Some brave souls are also using it as a= second router in their homes. At this stage, we do not recommend it as you= r production router. > > The most recent builds are the "bql-smoketests". They are available from = http://huchra.bufferbloat.net/~cero1/bql-smoketests/ Use the openwrt-ar71xx= -generic-wndr3700v2-squashfs-factory.img version. The Bufferbloat mailing l= ist (https://lists.bufferbloat.net/listinfo/bloat) describes the changes fo= r each version. > > The rc6 build is older, but in wider use. We have not received any report= s of crashes so feel free to try in less experimental situaitons. Get it fr= om http://huchra.bufferbloat.net/~cero1/cerowrt-wndr3700-1.0rc6/ Use the op= enwrt-ar71xx-generic-wndr3700v2-squashfs-factory.img version. > > The Cerowrt Installation Guide gives good information about installing an= d configuring these builds. > > ----- That's it! ----- > > Best regards from cloudy and chilly New Hampshire. > > Rich Brown > _______________________________________________ > Cerowrt-devel mailing list > Cerowrt-devel@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cerowrt-devel --=20 Dave T=E4ht SKYPE: davetaht US Tel: 1-239-829-5608 FR Tel: 0638645374 http://www.bufferbloat.net