From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-iy0-f171.google.com (mail-iy0-f171.google.com [209.85.210.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id CED2E200681 for ; Wed, 4 Jan 2012 08:16:45 -0800 (PST) Received: by iagw33 with SMTP id w33so47831152iag.16 for ; Wed, 04 Jan 2012 08:16:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=BwNHcoIv43ae6jH3QlWT2ZAFrLCWNSLyuJ40uDtFFHA=; b=QK5a3jUI1anAbN6oEhDmm0IOO2RUw9O3dcbyPb2yPe51hEHwfft3oDoXkhhQa3zQt1 4dGEZUeEBilspx5A071gXblto1O4XSEBVTps+87K+Qv0bWJsb3reVrmu8DNa8QdWtLO0 EFWf7hC61TCJuZJIq9LIVaPr+ysam/I+azvro= MIME-Version: 1.0 Received: by 10.42.151.68 with SMTP id d4mr59466533icw.36.1325693804082; Wed, 04 Jan 2012 08:16:44 -0800 (PST) Received: by 10.231.159.193 with HTTP; Wed, 4 Jan 2012 08:16:44 -0800 (PST) In-Reply-To: <4F046F7B.6030905@freedesktop.org> References: <1325481751.2526.23.camel@edumazet-laptop> <4F046F7B.6030905@freedesktop.org> Date: Wed, 4 Jan 2012 17:16:44 +0100 Message-ID: From: Dave Taht To: Jim Gettys Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: bloat Subject: Re: [Bloat] What is fairness, anyway? was: Re: finally... winning on wired! X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jan 2012 16:16:46 -0000 On Wed, Jan 4, 2012 at 4:25 PM, Jim Gettys wrote: > On 01/02/2012 04:31 PM, Dave Taht wrote: >> On Mon, Jan 2, 2012 at 9:07 AM, Dave Taht wrote: >> Yes, that patch brings SFQ at light workloads to being >> indistinguishable from QFQ! >> http://www.teklibre.com/~d/bloat/sfqnewvsqfq10iperfs.png (if you stare >> at this image long enough you might see a pattern, but I don't) (I >> certainly am seeing an afterimage, though) >>>> A "nolimit" implementation could use a dynamic memory allocator >>>> scheme, eventually consuming less memory on typical use :) >> At what point could SFQ be considered a replacement for pfifo_fast? :) >> >> I have not managed to crash QFQ yet with your other new patch. I will >> run it overnight. >> >> > > As I read this thread, there are three questions that go through my mind: the thread moved to netdev - more sfq improvements on the way http://www.spinics.net/lists/netdev/msg184686.html > =A0 =A01) since TCP is not "fair", particularly when given flows of > different RTT's, how do we best deal with this issue? =A0Do either/both > SFQ/QFQ deal with this problem, and how do they differ? The reason why QFQ was outperforming SFQ here http://www.teklibre.com/~d/bloat/pfifo_sfq_vs_qfq_linear50.png was because SFQ was enqueuing the first packet of a new stream at the end of the existing streams. After eric moved the SFQ enqueue to head, the two started performing the same at light workloads. (keep in mind the comparison already to pfifo_fast - log scale) http://www.teklibre.com/~d/bloat/pfifo_fast_vs_sfq_qfq_log.png So, the net effect of either fq mechanism is that slower flows jump forward in the queue. RTT is not really relevant. [1] The more sparse a flow is, the higher the likelyhood that it will be higher prioritized in this way. So we converge towards an optimum for longer RTT flows vs shorter ones, automagically, or so I think. But you ask a good question, the testing done thus far has been in fixed RTT scenarios, with the exception of the wireless test I ran a few days back. http://www.teklibre.com/~d/bloat/qfq_vs_pfifo_fast_wireless_iwl_card_vs_cer= owrt.pdf [4] > =A0 =A02) Web browsers are doing "unfair" things at the moment > (unless/until HTTP/1.1 pipelining and/or SPDY deploys), by opening many > TCP connections at the same time. =A0So it's easy for there to be a bunch > of flows by the same user. > Is "fairness" better a per host property in > the home environment, or a per TCP flow? =A0Particularly if we someday > start diffserv marking traffic, I suspect per host is more "fair", at > least for unmarked traffic. I don't think it's an "or" question. I think it's an "and", and diffserv of very limited use. The proof of concept implementation I did a few months back did: At the router: FQ on a per host basis and then FQ within each queue for each host. [2] Hosts should DEFINITELY run something like SFQ rather than PFIFO_FAST. syns, dns lookups, flows in slow start, all benefit. [3] and on top of that, since most web flows are short, and the uplink slow, sending 8 acks for 8 streams gives you more data faster, in aggregate, than sending all the acks for one stream, then the next, as a short flow expires quicker. and of course the other nice thing of FQ is that if you lose a bunch of acks they are spread across multiple rather than one stream. Determining the net effect of ALL THAT is up for more testing, but I've been running BQL + QFQ (and now SFQ) since mid-november now and the improvement in my web 'experience' is often amazing, as are things like interactive ssh sessions. I can have a fully loaded up network - totally saturated - and barely notice. > =A0 =A03) since game manufacturers have noted the diffserv marking in > PFIFO-FAST, what do these queuing disciplines currently do? They pay no attention to diffserv. It's possible we have a small problem here, but I'd wager not. If their packets are not saturating the link, and/or it's per host FQ, they are going to win. isochronous flows (like voice, even (especially) skype) should respond well= to this stuff, too. do, actually. skype is nicer now. footnotes: 1: the 'slower flows gain priority' question is my gravest concern (eg, ledbat, bittorrent). It's fixable with per-host FQ. 2: I note that identifying "a host" is harder than it was in nagle's day - with multiple protocols in use a hash against an IP address is not accurate, nor, by the time you hit the egress interface, do you have a mac address to use. I did not solve that problem in my home router gw/aqm/fq prototype. The approximation was a hash that matched an ip or ipv6 address. 3: (and conversely, existing elephants slow down). (and when you drop packets there is a good question, but you do end up with classic tail drop behavior in both QFQ and SFQ on a per flow basis. You can get head drop out of QFQ, actually) 4: especially in variable bandwidth scenarios, the depth of even the fairest queue needs to be managed somehow. But I can live with the nearly two orders of magnitude of basic latency improvement we've got now, as well as the probability this induces of doing smarter/effective queue length management, and keep getting the bugs out of it.... > > --=20 Dave T=E4ht SKYPE: davetaht US Tel: 1-239-829-5608 FR Tel: 0638645374 http://www.bufferbloat.net