From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qa0-x22a.google.com (mail-qa0-x22a.google.com [IPv6:2607:f8b0:400d:c00::22a]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 1BD2821F13F for ; Mon, 3 Feb 2014 12:52:06 -0800 (PST) Received: by mail-qa0-f42.google.com with SMTP id k4so10973775qaq.15 for ; Mon, 03 Feb 2014 12:52:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=olQ80jv4cZq3xR/1e1mQLZHm+Q7UrCtZAnAormYp/lk=; b=tO9JSzLwTWn4uEAMdp13Drfnq7pwkhiGd4ftTdQO7y/PiXZVnaXwfAdqhqayhutEXI dnxA06xwg59H0OfxLVdqacEWcSw+MClRTyxE+v5NYzA0skGcItP379I7vt3f0YwrxrrZ 9an2wVZ30u9375ubLxwzhRNX/BRQjZip1XmF17+gQzaHVnybd0+syHb8gnei9h8dZKZM EPUOLjesPoD4frfmCPqJwS6a4lSbQroHCelTCRaJ/oFqjghTJEwrL5Svl/U4tFj0NZSa T/4Kn0GseYgN+8T7a+8HevUd2r8/qgzjf4fw1/IIk4jZysHpFJPtCBsQ+4YlthHHWdx9 R+Vw== MIME-Version: 1.0 X-Received: by 10.140.91.23 with SMTP id y23mr56903191qgd.3.1391460724687; Mon, 03 Feb 2014 12:52:04 -0800 (PST) Received: by 10.224.27.133 with HTTP; Mon, 3 Feb 2014 12:52:04 -0800 (PST) In-Reply-To: References: Date: Mon, 3 Feb 2014 12:52:04 -0800 Message-ID: From: Dave Taht To: Jim Gettys Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: bloat-ietf@lists.bufferbloat.net, "aqm@ietf.org" Subject: Re: [Bloat-ietf] [aqm] [FQ_CoDel] "Official" version X-BeenThere: bloat-ietf@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: IETF specific bufferbloat discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Feb 2014 20:52:06 -0000 On Mon, Feb 3, 2014 at 7:57 AM, Jim Gettys wrote: > > > On Mon, Feb 3, 2014 at 9:30 AM, Nicolas KUHN > wrote: >> >> Dear all, >> >> We would like to test FQ_CoDel and we wonder which version to evaluate. > > > fq_codel has been in Linux since about Linux 3.5, IIRC. For casual > experiments, a current Linux distro will have it available at the invocat= ion > of the "tc" command. But... Improvements as of 3.12 is the ability to set a default qdisc across the entire system via a single sysctl.conf option. > You should generally be testing the latest Linux release (not that I thin= k > there has been much change in fq_codel, as demonstrated in: > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/?id= =3Drefs%2Ftags%2Fv3.14-rc1&qt=3Dgrep&q=3Dfq_codel However there were some improvements to the jhash algorithm that landed in 3.8 and later to improve ipv6 hashing, and to more deeply inspect gre and 6in4 tunnels. > For serious bench marking getting results from an old system can make you= r > results easily invalid since there has been a large number of improvement= s > in Linux for bufferbloat everywhere over the last three years, and the qu= eue > discipline is only part of the equation. For example, many ethernet driv= ers > in SOC's found in home routers may still lack even BQL support. I note the sanest results are more consistently obtained using a SQM system like what's in cero, which leverages htb, thus moving control to htb and away from the drivers deeper in the stack. that said, htb also underwent improvements in 3.11 to make it faster, and there are some "dis-improvements" the current SQM system for cero that speed it up for the low powered hardware it's aimed at, above 20Mbit. Faster systems don't need those hacks... (and htb can be improved further, or replaced with hfsc.) > You should be testing the fq_codel queue discipline: while Dave Taht has > played with a number of variants on that theme, nothing to date has shown > itself (by testing) with enough better results to be worth merging upstre= am > into kernel.org. nfq_codel presently does mildly better at very low bandwidths (<2Mbit) as it is more sfq-like. Actually everything below 4Mbit is kind of finicy with everything (aqm, sfq, codel, fq_codel). I have been thinking of bringing back a patch from 3.5 in some form to make maxpacket handling saner at lower bandwidths (at a price in latency) > Pie finally entered kernel.org Linux a month or so ago. I have been backporting that as far back as 3.8. > > Note that there a large number of "traps" for the unwary in this > benchmarking/testing business, into which multiple groups have come to ba= d > ends causing questionable results to confuse everyone. For example, the > details of the underlying device driver can matter a lot (actually, > hugely!), and netem has to be used with exquisite care just to name a cou= ple > of the big traps. netem was hugely buggy prior to 3.11. I still only run it on a dedicated bo= x to insert delay and watch it carefully. > Fundamentally, you need to realize that the queue discipline in Linux (wh= ere > you can apply aqm) does *not* have control of all of the buffering in the > system; device drivers and even the hardware itself can do "interesting" > things to you. And a bunch of changes have taken place in the Linux TCP > stack to remove unneeded buffering since around Linux 3.2. At one point = I > think the list was up to a dozen or more significant improvements in the > Linux networking stack, independent of fq_codel. Yep. I look forward to hardware that can run this stuff directly. > We've come to learn that WiFi in particular needs a serious rewrite, and > before venturing far, you'd be wise to get in touch with us further. ATM= , > the ath9k driver is least problematic, but even there, aggregation causes > large amounts of buffering (32 packets!). > > Dave Taht and I tried to outline what you should be thinking about when > bench marking in: > > http://www.bufferbloat.net/projects/codel/wiki/Best_practices_for_benchma= rking_Codel_and_FQ_Codel Don't be too scared... > Pretty much all of these issues apply to anything you do, not just fq_cod= el. > > I think that page overs most of the problems though of course, Linux is = a > moving target so I'm sure there will be future problems :-(. > > >> >> Does anyone know where we could get the "official" version / source code >> of FQ_CoDel ? >> (linux and/or NS2 if possible) > > > www.kernel.org for the source. patches and backports to older kernel versions are available as part of the kernel-backports package. (YMMV, as we note lots of other changes in the stack elsewhere matter quite a lot) > > You can use git to see the exact revision history of the code. > > I forget off hand where to find the simulation code... There is a link > somewhere in the bufferbloat/codel wiki > > http://www.bufferbloat.net/projects/codel/wiki/Wiki ns2 and ns3 versions of the patches have been rotting in a repo on github. Hope someone makes it a google summer of code project to get these mainline= d. > Help from people to get the wiki up to date gratefully appreciated! > > Jim > > >> >> Thanks a lot, >> >> Kind regards, >> >> Nicolas >> >> _______________________________________________ >> aqm mailing list >> aqm@ietf.org >> https://www.ietf.org/mailman/listinfo/aqm > > > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > --=20 Dave T=E4ht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.= html