[Bloat] What is fairness, anyway? was: Re: finally... winning on wired!

[Dave Taht]

On Wed, Jan 4, 2012 at 4:25 PM, Jim Gettys <jg at freedesktop.org> wrote:
> On 01/02/2012 04:31 PM, Dave Taht wrote:
>> On Mon, Jan 2, 2012 at 9:07 AM, Dave Taht <dave.taht at gmail.com> wrote:
>> Yes, that patch brings SFQ at light workloads to being
>> indistinguishable from QFQ!
>> http://www.teklibre.com/~d/bloat/sfqnewvsqfq10iperfs.png (if you stare
>> at this image long enough you might see a pattern, but I don't) (I
>> certainly am seeing an afterimage, though)
>>>> A "nolimit" implementation could use a dynamic memory allocator
>>>> scheme, eventually consuming less memory on typical use :)
>> At what point could SFQ be considered a replacement for pfifo_fast? :)
>>
>> I have not managed to crash QFQ yet with your other new patch. I will
>> run it overnight.
>>
>>
>
> As I read this thread, there are three questions that go through my mind:

the thread moved to netdev - more sfq improvements on the way

http://www.spinics.net/lists/netdev/msg184686.html

>    1) since TCP is not "fair", particularly when given flows of
> different RTT's, how do we best deal with this issue?  Do either/both
> SFQ/QFQ deal with this problem, and how do they differ?

The reason why QFQ was outperforming SFQ here

http://www.teklibre.com/~d/bloat/pfifo_sfq_vs_qfq_linear50.png

was because SFQ was  enqueuing the first packet of a new stream at the
end of the existing streams.

After eric moved the SFQ enqueue to head, the two started performing
the same at light workloads.

(keep in mind the comparison already to pfifo_fast - log scale)

http://www.teklibre.com/~d/bloat/pfifo_fast_vs_sfq_qfq_log.png

So, the net effect of either fq mechanism is that slower flows jump
forward in the queue.
RTT  is not really relevant. [1]

The more sparse a flow is, the higher the likelyhood that it will be
higher prioritized in this way.

So we converge towards an optimum for longer RTT flows vs shorter
ones, automagically, or so I think.

But you ask a good question, the testing done thus far has been in
fixed RTT scenarios, with the exception of the wireless test I ran a
few days back.

http://www.teklibre.com/~d/bloat/qfq_vs_pfifo_fast_wireless_iwl_card_vs_cerowrt.pdf
[4]

>    2) Web browsers are doing "unfair" things at the moment
> (unless/until HTTP/1.1 pipelining and/or SPDY deploys), by opening many
> TCP connections at the same time.  So it's easy for there to be a bunch
> of flows by the same user.

> Is "fairness" better a per host property in
> the home environment, or a per TCP flow?  Particularly if we someday
> start diffserv marking traffic, I suspect per host is more "fair", at
> least for unmarked traffic.

I don't think it's an "or" question. I think it's an "and", and
diffserv of very limited use.

The proof of concept implementation I did a few months back did:

At the router:

FQ on a per host basis
and then FQ within each queue for each host. [2]

Hosts should DEFINITELY run something like SFQ rather than PFIFO_FAST.
syns, dns lookups, flows in slow start, all benefit. [3]

and on top of that, since most web flows are short, and the uplink slow,
sending 8 acks for 8 streams gives you more data faster, in aggregate,
than sending all the acks for one stream, then the next, as a short flow
expires quicker.

and of course the other nice thing of FQ is that if you lose a bunch
of acks they are
spread across multiple rather than one stream.

Determining the net effect of ALL THAT is up for more testing, but
I've been running BQL + QFQ (and now SFQ) since mid-november now
and the improvement in my web 'experience' is often amazing,
as are things like interactive ssh sessions. I can have a fully loaded
up network - totally saturated - and barely notice.

>    3) since game manufacturers have noted the diffserv marking in
> PFIFO-FAST, what do these queuing disciplines currently do?

They pay no attention to diffserv. It's possible we have a small
problem here, but I'd wager not.

If their packets are not saturating the link, and/or it's per
host FQ, they are going to win.

isochronous flows (like voice, even (especially) skype) should respond well to
this stuff, too. do, actually. skype is nicer now.


footnotes:

1: the 'slower flows gain priority' question is my gravest concern
(eg, ledbat, bittorrent). It's fixable with per-host FQ.

2: I note that identifying "a host" is harder than it was in nagle's
day - with multiple protocols in use a hash against an IP address is
not accurate, nor, by the time you hit the egress interface, do you
have a mac address to use.

I did not solve that problem in my home router gw/aqm/fq prototype.
The approximation was a hash that matched an ip or ipv6 address.

3: (and conversely, existing elephants slow down). (and when you drop
packets there is a good question, but you do end up with classic tail
drop behavior in both QFQ and SFQ on a per flow basis. You can get
head drop out of QFQ, actually)

4: especially in variable bandwidth scenarios, the depth of even the
fairest queue needs to be managed somehow.

But I can live with the nearly two orders of magnitude of basic
latency improvement we've got now, as well
as the probability this induces of doing smarter/effective queue
length management, and keep getting the bugs out of it....

>
>



-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
FR Tel: 0638645374
http://www.bufferbloat.net