From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pb0-f43.google.com (mail-pb0-f43.google.com [209.85.160.43]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 0B5592022B8; Wed, 16 May 2012 00:05:05 -0700 (PDT) Received: by pbcwz7 with SMTP id wz7so1230229pbc.16 for ; Wed, 16 May 2012 00:05:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=ANHQLeE5E1TSJQarNh2dvx2B62wGjPS2gQjjxTtVZio=; b=pBK/LfeWffyZKtVTy1dy9ElKZnm/bS+qLcoxAr3YvrN9O0sewwSXyOcwAYPul7PJus F3px0OaKotj2qHpChEfvUC7V/5CHMWi7rQQBJ7GYSwx6d8DH2oq17xvt5g5fDtDEyWWO hZL9NhKJB3i8nppezsVD3LoCUhYVK7jW841Qp2TyeZy6wFArr52eN1av5EQuAeYkjnw6 V757azP2XopAPOxGxCY04uaxPp3nAdHIDpdaybmzDYlEeiENk8eE+FWNtZO7CZtbsGXI gPsTLAK/Ce+2yzkcwaH1RJ2amtjBxlipXUIZigZyRGoXTBu9mM8kfv7bgny2CI5mVZxI 7hqw== Received: by 10.68.227.169 with SMTP id sb9mr13406259pbc.157.1337151904542; Wed, 16 May 2012 00:05:04 -0700 (PDT) Received: from ?IPv6:2001:4f8:3:203::c001? ([2001:4f8:3:203::c001]) by mx.google.com with ESMTPS id og6sm4491711pbb.42.2012.05.16.00.05.02 (version=SSLv3 cipher=OTHER); Wed, 16 May 2012 00:05:03 -0700 (PDT) Message-ID: <4FB3519D.3020809@gmail.com> Date: Wed, 16 May 2012 00:05:01 -0700 From: dave taht User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:11.0) Gecko/20120410 Thunderbird/11.0.1 MIME-Version: 1.0 To: codel@lists.bufferbloat.net, bloat@lists.bufferbloat.net References: <4FA9FDC0.9010600@superduper.net> <1337148560.8512.1123.camel@edumazet-glaptop> In-Reply-To: <1337148560.8512.1123.camel@edumazet-glaptop> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Codel] Exploring the potential of codel, fq_codel, and qfq X-BeenThere: codel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: CoDel AQM discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2012 07:05:05 -0000 On 05/15/2012 11:09 PM, Eric Dumazet wrote: > On Wed, 2012-05-16 at 08:55 +0300, Jonathan Morton wrote: >> On 9 May, 2012, at 4:04 pm, Kevin Gross wrote: >> >>> From the paper (figure 7), you can see that CoDel still leaves spikes of buffer occupancy when network conditions change. These will still be disruptive to real-time traffic. Many networks that need QoS now will still need QoS. Networks that do not have QoS will be much more usable with CoDel. >> Combining AQM with FQ certainly seems like a good idea to me. I haven't had a chance to try the implementation of fq_codel which already exists yet, but it's compiled and just waiting for me to get around to it. If it works, then it should be an excellent default. > > fq_codel has no priority concepts. It's only Fair. But what is > Fairness ? > > In the presence of elephant flows, you get more packets drops from > elephant flows than from thin ones. The overall effects of fq_codel's implementation on top of codel's idea of good queue vs bad queue has yet to be evaluated. I'd like to see people explore iw10 and TCP mice, for example on an apache benchmark emulating a typical web site, as well as typical accesses of a web site, as well as TCP_RR stats against saturating forward and reverse traffic, with and without ecn, at rates ranging from 1Mbit to 40GigE. I'm hoping to get some of that said myself, but it's a lot to do, so I hope more people jump on it. Then there's wireless. All that said, I am loving what I see of fq_codel so far. I think in general fq is superior to anything a human fiddling with priorities can do. Also I have hooked up qfq to codel in the debloat script, which might provide some useful comparisons to fq_codel, and also offers the possibility of using qfq to hash on device, and fq_codel to fq within flows to a given device, thus yielding 1/d behavior. But I am throwing ENOTIME, ENOBRAINCELLS, and ENOENERGY errors a lot personally, right now. I'd planned to be on vacation last week and really need some time in a yurt far away from anything electronic. I am looking forward very much towards published results, analysis, and benchmarks from everyone else. Certainly I intend to get to where I can do CDF plots of what I reference below over large sample periods, etc, at some point... but for your low latency pleasure: 50 netperf -l 120 -t TCP_STREAM, then 1 netperf -l 30 -t TCP_RR, ecn on, 100Mbit ethernet, 3 hops 1 machine has codel, the other does not (so many kernels, so little time) I note that QFQ is WAY more cpu/memory overhead and I haven't captured or analysed the streams. TCP_RR PING PFIFO_FAST on both, no load: 2440.42 .26 to .59 PFIFO_FAST to PFIFO_FAST: 8.13 121ms CODEL to PFIFO_FAST: 27.56 Initial impulse of 117ms, settles down to about 25ms FQ_CODEL to PFIFO_FAST: 272.76 1.2ms, no spike QFQ+CODEL to PFIFO_FAST: 1002.39 .7ish, no spike QFQ+FQ_CODEL to PFIFO_FAST 1020.62 .95ms no spike (we are down at statistical noise levels) I haven't tried codel to codel or fq_codel to fq_codel much yet. I note also that I get best results when also applying fq_codel as an ingress qdisc. With that on it's hard to get any latencies in excess of 5ms in any direction. Without it, traffic in the incoming direction (via TCP_MAERTS on netperf) cracks 30ms worth of delay and 110 TCP_RR vs pfifo_fast. in debloat is script that does that using a three tier pfifo_fast superset. Using ifb is rocket science, as is tc, however. It would be very interesting to come up with a simpler way of using codel or fq_codel on ingress, perhaps with tbf. I just checked in the latest revision of debloat and the simple_qos and ingres scripts to the deBloat repository on github. >If the elephant flow is the 'high priority' traffic, dropping packets >from it is not what you wanted. >pfifo_fast is able to have strict priority. Which can lead to starvation of all flows, in pfifo_fast's case. Two possible compromises are using a 3 tier htb bucket to emulate pfifo_fasts behavior with the addition of codel for sane packet drop, or to wedge some interesting behavior into the extra 15 bits in the codel vars. > If you want Codel and strict > priority, you can use prio + codel (or prio + fq_codel) I'm not fond of strict prioritization. I am unsure of how useful mq_prio is at this point. > > > _______________________________________________ > Codel mailing list > Codel@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/codel