From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.rs.github.com (smtp1-ext.rs.github.com [207.97.227.250]) by huchra.bufferbloat.net (Postfix) with ESMTP id B35A4200377 for ; Mon, 2 Jan 2012 04:49:34 -0800 (PST) Received: from github.com (sh1.rs.github.com [172.17.1.41]) by smtp1.rs.github.com (Postfix) with ESMTP id 152C142585 for ; Mon, 2 Jan 2012 04:49:34 -0800 (PST) Date: Mon, 2 Jan 2012 04:49:34 -0800 To: cerowrt-commits@lists.bufferbloat.net Subject: [dtaht/deBloat] ba537a: Treating git as a lab notebook Content-Type: text/plain; charset=UTF-8 Message-Id: <20120102124934.152C142585@smtp1.rs.github.com> From: noreply@github.com X-BeenThere: cerowrt-commits@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development commits for the cerowrt project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jan 2012 12:49:34 -0000 Branch: refs/heads/master Home: https://github.com/dtaht/deBloat Commit: ba537a63f80ecd3ac7a97ab10d2814ea95cab4e9 https://github.com/dtaht/deBloat/commit/ba537a63f80ecd3ac7a97ab10d2814ea95cab4e9 Author: Dave Taht Date: 2012-01-02 (Mon, 02 Jan 2012) Changed paths: M test/README.org M test/ftest.lua M test/itest Log Message: ----------- Treating git as a lab notebook I like the idea of continuing to use git as a 'lab notebook' - the entries are timestamped, I know from what machine they came from, I can get to the notebook from whatever machine I'm on, etc. (I returned to testing 10 streams at a time for wireless, as 50 streams would fail to connect up to 15 of them...) So, that said, there are several entries in my log thus far that I need to update and correct: commit: 52b1106e4e733b1e38bc0cfbb2dd2aafb94c6261 Non locking up eqfq script The theories here were incorrect. I merely had had a run of 'luck' where I couldn't crash the system for several hours. At the time I was also wrestling with other bugs and abandoned the work temporarily. (I'd had similar issues back in august/september as well) commit: 6721d1b0bf5e1113a68d128025c46e7c93fc2c6b 4 Mbit shaper plus fair queuing plus AQM that is fair to users This shaper came closest to solving all the known problems in the home routing universe. I have several thoughts regarding making it far better, but I'm just correcting the logbook now: Several comments were incorrect. 1) I believe - but have not confirmed - that the root of the 'bursty ping during single stream' problem came down to one of two factors. A) Since I started doing tests like this, back in january of last year, GSO offloads became ON by default in the Linux kernel. I was turning off TSO, but did not check to see if GSO was also off. Worse, at least on the e1000e, TSO gets disabled automatically at 100Mbit using the ethtool -s 0x008 trick. But then... GSO goes on... and then, if you try to turn it off, you still have to turn off TSO explicitly, first. So during that test series I had had GSO inadvertently on. Turning TSO and GSO OFF is embedded in my current codebase but I hadn't logged it anywhere. It really messed up several months worth of tests. B) I have also seen 'burstyness' when using BQL's default controller at 100Mbit. I have been hammering it down with a setting of 4500 (determined empirically), and will get back to testing the hard limit of 4500 vs the 'automatic' limit after things get more stable. 2) The clock interrupt theory is unproven. This was a shaper that did HTB, then QFQ, then QFQ AGAIN. What I'd wanted was HTB + QFQ (to balance the users), then something as simple as SFQ - to balance streams to a user. At the time, I was getting dramatically better results from QFQ throughout - so I used QFQ twice - which led to creating the qdiscs taking minutes - which led to me rewriting everything in C and now lua - where it's still many seconds to actually get this super-shaper to work. Now that eric's got SFQ doing well, I think HTB + QFQ + SFQ with the right filters for the QFQ stage (filtering on a perfect hash for mac address would be good - handles ipv6 and ipv4 transparently) would be good. I did run with a higher clock interrupt for the bql-3 through bql-8 series of cerowrt but any difference in results may merely have been an illusion.