[dtaht/deBloat] ba537a: Treating git as a lab notebook

[noreply at github.com]

  Branch: refs/heads/master
  Home:   https://github.com/dtaht/deBloat

  Commit: ba537a63f80ecd3ac7a97ab10d2814ea95cab4e9
      https://github.com/dtaht/deBloat/commit/ba537a63f80ecd3ac7a97ab10d2814ea95cab4e9
  Author: Dave Taht <d at taht.net>
  Date:   2012-01-02 (Mon, 02 Jan 2012)

  Changed paths:
    M test/README.org
  M test/ftest.lua
  M test/itest

  Log Message:
  -----------
  Treating git as a lab notebook

I like the idea of continuing to use git as a 'lab notebook' - the
entries are timestamped, I know from what machine they came from, I
can get to the notebook from whatever machine I'm on, etc.

(I returned to testing 10 streams at a time for wireless, as
50 streams would fail to connect up to 15 of them...)

So, that said, there are several entries in my log thus far that I
need to update and correct:

commit: 52b1106e4e733b1e38bc0cfbb2dd2aafb94c6261
Non locking up eqfq script

The theories here were incorrect. I merely had had a run of 'luck'
where I couldn't crash the system for several hours.

At the time I was also wrestling with other bugs and abandoned the
work temporarily. (I'd had similar issues back in august/september as
well)

commit: 6721d1b0bf5e1113a68d128025c46e7c93fc2c6b
4 Mbit shaper plus fair queuing plus AQM that is fair to users

This shaper came closest to solving all the known problems in the home
routing universe. I have several thoughts regarding making it far
better, but I'm just correcting the logbook now:

Several comments were incorrect.

1) I believe - but have not confirmed - that the root of the
'bursty ping during single stream' problem came down to one of two factors.

A) Since I started doing tests like this, back in january of last
year, GSO offloads became ON by default in the Linux kernel. I was
turning off TSO, but did not check to see if GSO was also off. Worse,
at least on the e1000e, TSO gets disabled automatically at 100Mbit
using the ethtool -s 0x008 trick.

But then... GSO goes on...

and then, if you try to turn it off, you still have to turn off TSO
explicitly, first.

So during that test series I had had GSO inadvertently on.

Turning TSO and GSO OFF is embedded in my current codebase but I
hadn't logged it anywhere. It really messed up several months worth of
tests.

B) I have also seen 'burstyness' when using BQL's default controller
at 100Mbit. I have been hammering it down with a setting of 4500
(determined empirically), and will get back to testing the hard limit
of 4500 vs the 'automatic' limit after things get more stable.

2) The clock interrupt theory is unproven.

This was a shaper that did HTB, then QFQ, then QFQ AGAIN. What I'd
wanted was HTB + QFQ (to balance the users), then something as simple
as SFQ - to balance streams to a user. At the time, I was getting
dramatically better results from QFQ throughout - so I used QFQ twice
- which led to creating the qdiscs taking minutes - which led to me
rewriting everything in C and now lua - where it's still many seconds
to actually get this super-shaper to work.

Now that eric's got SFQ doing well, I think HTB + QFQ + SFQ with the
right filters for the QFQ stage (filtering on a perfect hash for mac
address would be good - handles ipv6 and ipv4 transparently) would be
good.

I did run with a higher clock interrupt for the bql-3 through bql-8
series of cerowrt but any difference in results may merely have been
an illusion.