From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-iy0-f171.google.com (mail-iy0-f171.google.com [209.85.210.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 39832200762 for ; Mon, 2 Jan 2012 00:07:33 -0800 (PST) Received: by iagw33 with SMTP id w33so41867676iag.16 for ; Mon, 02 Jan 2012 00:07:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=iO04RyhA1q2eMUpZHl99qZjIQ3KB4T/T9A1otWmVdNQ=; b=DIV1vlUwpAgUoqfih7OAGT+CL7JoXlhBKuSc54AMYxm4zKL5yMm9ex321EuLsMyWSs FKxUTzjEYsq94RIr82PW9lnuFiv6VhwDX2lOz8oMy2oC25GC5RvjcMH1UT5Qu/qY7E1F emISl2vBplA/cfkvs0fq3g01DKzebzI6Om+Ao= MIME-Version: 1.0 Received: by 10.50.219.226 with SMTP id pr2mr55704892igc.30.1325491652240; Mon, 02 Jan 2012 00:07:32 -0800 (PST) Received: by 10.231.159.193 with HTTP; Mon, 2 Jan 2012 00:07:32 -0800 (PST) In-Reply-To: <1325481751.2526.23.camel@edumazet-laptop> References: <1325481751.2526.23.camel@edumazet-laptop> Date: Mon, 2 Jan 2012 09:07:32 +0100 Message-ID: From: Dave Taht To: Eric Dumazet Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: bloat Subject: Re: [Bloat] finally... winning on wired! X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jan 2012 08:07:33 -0000 On Mon, Jan 2, 2012 at 6:22 AM, Eric Dumazet wrote= : > Le lundi 02 janvier 2012 =E0 01:40 +0100, Dave Taht a =E9crit : > >> SFQ has generally been quite good in many respects. SFQ also does >> improved hashing on net-next. But: QFQ seemed very promising also, and >> it took until now to see it clearly, with BQL turned on. >> >> To look in more detail at sfq vs qfq, under even heavier load: >> >> http://www.teklibre.com/~d/bloat/pfifo_sfq_vs_qfq_linear50.png >> >> It's really very rare in my life that I've seen a win vs an existing >> system of these orders of magnitude. It's taken me a week to make sure >> the results were real, and repeatable... I thought about sitting on >> them for a while longer actually. I'd really like someone else to >> repeat these tests and tell me I'm not seeing things! >> >> I have hopes QFQ will do even better at 10Mbit vs SFQ. > > Happy New Year Dave > > This makes no sense to me. GOOD! I spent the holiday disbelieving the quality of the results and writing tests to automate getting them. Instead of having a holiday. > For most uses on a host or residential router, SFQ should perform the > same than QFQ. Well, I don't know. QFQ's scheduling mechanism is interesting, and the bursty quality of streams generated from a web browser and IW10 (web sites are about 1MB in size these days, 70+ tcp streams), is different from what we used to understand. Similarly, we get these enormous bursts of of TSO/GSO offloads that are hard to requeue.... But that's why we test... I have to tie the general architecture above into HTB and try it at 4Mbit or so, and then figure out a way to simulate web access. I kept seeing things like DNS, and tcp syn/syn acks dramatically 'leaping forward' in the queue, in the packet captures, in the QFQ case. QFQ kept 'winning big' for the past 6 weeks (between crashes), and I didn't understand why. The other thing I like about QFQ is the ability to play with pfifo_drop_hea= d and red as sub-qdiscs and to be able to 'see' multi-queue behavior with things such as ledbat, (bittorrent) which is the one thing that keeps me awake at night: http://tools.ietf.org/html/draft-ietf-ledbat-congestion-09 " Further study is required to fully understand the behaviour of LEDBAT with non-drop-tail, non-FIFO queues. " > QFQ is the thing you want to use on a big node, when SFQ limits are > reached (SFQ as implemented in linux : at most 127 concurrent flows, and > at most 127 packets in queue. This could be changed with an increase of > memory cost [ which are really small anyway : about 4 Kbytes per SFQ > queue ]. My primary interest in QFQ came from trying to find a solution for GigE and above for servers and desktops, as well as something that could be embedded on GigE (more notably 10GigE) ethernet cards themselves. The amount of extra cpu and memory QFQ eats turned out to be trivial. And hey, I found a bug. You solved it. A nice way to wake up this morning. >A "nolimit" implementation could use a dynamic memory allocator > schem, eventually consuming less memory on typical use :) The memory use on the router with either SFQ or QFQ is minimal, as is cpu. > Please try the patch I posted this morning to solve this SFQ bug. Hell. I'd hoped to go do the holiday that I just missed. :) > http://patchwork.ozlabs.org/patch/133793/ Yea, that seems like sanity. I just have to patch 2 sources trees, recompile, update 2 laptops, reflash two routers, and can test that... but I was seriously thinking of taking today off. > > --=20 Dave T=E4ht SKYPE: davetaht US Tel: 1-239-829-5608 FR Tel: 0638645374 http://www.bufferbloat.net