Re: [Bloat] finally... winning on wired!

[Dave Taht <dave.taht@gmail.com>]

On Mon, Jan 2, 2012 at 6:22 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le lundi 02 janvier 2012 à 01:40 +0100, Dave Taht a écrit :
>
>> SFQ has generally been quite good in many respects. SFQ also does
>> improved hashing on net-next. But: QFQ seemed very promising also, and
>> it took until now to see it clearly, with BQL turned on.
>>
>> To look in more detail at sfq vs qfq, under even heavier load:
>>
>> http://www.teklibre.com/~d/bloat/pfifo_sfq_vs_qfq_linear50.png
>>
>> It's really very rare in my life that I've seen a win vs an existing
>> system of these orders of magnitude. It's taken me a week to make sure
>> the results were real, and repeatable... I thought about sitting on
>> them for a while longer actually. I'd really like someone else to
>> repeat these tests and tell me I'm not seeing things!
>>
>> I have hopes QFQ will do even better at 10Mbit vs SFQ.
>
> Happy New Year Dave
>
> This makes no sense to me.

GOOD! I spent the holiday disbelieving the quality of the results and
writing tests to automate getting them. Instead of having a holiday.

> For most uses on a host or residential router, SFQ should perform the
> same than QFQ.

Well, I don't know. QFQ's scheduling mechanism is interesting, and the
bursty quality of streams generated from a web browser and IW10
(web sites are about 1MB in size these days, 70+ tcp streams),
is different from what we used to understand. Similarly, we get
these enormous bursts of of TSO/GSO offloads that are hard to
requeue....

But that's why we test... I have to tie the general architecture above
into HTB and try it at 4Mbit or so, and then figure out a way to
simulate web access. I kept seeing things like DNS, and tcp syn/syn acks
dramatically 'leaping forward' in the queue, in the packet captures,
in the QFQ case. QFQ kept 'winning big' for  the past 6 weeks
(between crashes), and I didn't understand why.

The other thing I like about QFQ is the ability to play with pfifo_drop_head
and red as sub-qdiscs and to be able to 'see' multi-queue
behavior with things such as ledbat, (bittorrent) which is the one thing
that keeps me awake at night:

http://tools.ietf.org/html/draft-ietf-ledbat-congestion-09

"   Further study is required to fully understand the behaviour of LEDBAT
   with non-drop-tail, non-FIFO queues. "

> QFQ is the thing you want to use on a big node, when SFQ limits are
> reached (SFQ as implemented in linux : at most 127 concurrent flows, and
> at most 127 packets in queue. This could be changed with an increase of
> memory cost [ which are really small anyway : about 4 Kbytes per SFQ
> queue ].

My primary interest in QFQ came from trying to find a solution for GigE and
above for servers and desktops, as well as something that could be
embedded on GigE (more notably 10GigE) ethernet cards themselves.

The amount of extra cpu and memory QFQ eats turned out to be trivial.

And hey, I found a bug. You solved it. A nice way to wake up this morning.

>A "nolimit" implementation could use a dynamic memory allocator
> schem, eventually consuming less memory on typical use :)

The memory use on the router with either SFQ or QFQ is minimal, as is cpu.

> Please try the patch I posted this morning to solve this SFQ bug.

Hell. I'd hoped to go do the holiday that I just missed. :)

> http://patchwork.ozlabs.org/patch/133793/

Yea, that seems like sanity. I just have to patch 2 sources trees,
recompile, update 2 laptops, reflash two routers, and can test that...

but I was seriously thinking of taking today off.

>
>



-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
FR Tel: 0638645374
http://www.bufferbloat.net