From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pd0-x230.google.com (mail-pd0-x230.google.com [IPv6:2607:f8b0:400e:c02::230]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 9CA2621F14C for ; Thu, 11 Jul 2013 11:54:10 -0700 (PDT) Received: by mail-pd0-f176.google.com with SMTP id t12so7712430pdi.7 for ; Thu, 11 Jul 2013 11:54:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:subject:from:to:cc:date:in-reply-to:references :content-type:x-mailer:content-transfer-encoding:mime-version; bh=8ME5P3Ph1/PofbUKAizAvhjwLPVWjZ+ZhrVb21RZMxo=; b=LfSyxGRnaq1eS50D1YE99BW+vi7Y0oL+1SqDZ8PM9NmcmZ9Xe0TiK0YdIsrKm18DZ4 pWIg9h314kPjIRWP8v/R+iD5eKzgDOaiKzexFHRK95d4rl+PEewgNqVZg8CFH0VN1Rhl pSfDomkqB9pRtE/qmgdV99mKryB+jTI76Qp9nzZwigOmaNLrqLFSTXPGlz6k0Tnn9Aol bKdAekefCZknyAI59JzPhv3CP2fVocbUZ5/pVd5lis8rUZCs6DsMKanDg5ax/eqEPiD0 9pInOfXZ+ZuvxOpcIlt87ICfsG2WkU13WZGGb5IaXcu+cbNGQ4suc8OcrbwnieYZ9UKR LSSQ== X-Received: by 10.68.253.161 with SMTP id ab1mr37673261pbd.76.1373568850112; Thu, 11 Jul 2013 11:54:10 -0700 (PDT) Received: from ?IPv6:2620:0:1000:3304:d49d:873e:bd9b:d524? ([2620:0:1000:3304:d49d:873e:bd9b:d524]) by mx.google.com with ESMTPSA id dc5sm1356271pbc.37.2013.07.11.11.54.09 for (version=SSLv3 cipher=RC4-SHA bits=128/128); Thu, 11 Jul 2013 11:54:09 -0700 (PDT) Message-ID: <1373568848.4600.66.camel@edumazet-glaptop> From: Eric Dumazet To: Dave Taht Date: Thu, 11 Jul 2013 11:54:08 -0700 In-Reply-To: References: <1373564673.4600.55.camel@edumazet-glaptop> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.3-0ubuntu6 Content-Transfer-Encoding: 7bit Mime-Version: 1.0 Cc: codel@lists.bufferbloat.net Subject: Re: [Codel] hardware multiqueue in fq_codel? X-BeenThere: codel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: CoDel AQM discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jul 2013 18:54:10 -0000 On Thu, 2013-07-11 at 11:06 -0700, Dave Taht wrote: > Gotcha. So what I actually did (felix did, in openwrt, actually) was > just make fq_codel the default qdisc to avoid having to inspect things > to set the number of queues in mq and mqprio. I see, for example, that > mq is the default for tg3... > > http://snapon.lab.bufferbloat.net/~cero2/deb/patches/0003-Use-FQ_codel-by-default.patch > > I just added it to htb and hfsc too: > > http://snapon.lab.bufferbloat.net/~cero2/deb/patches/0008-Make-fq_codel-the-default-qdisc-for-htb-and-hfsc.patch > > There's a patch to obsolete pfifo_fast entirely in openwrt, which is a > tad premature. > > A remaining concern is to what this affects: > > A) people that expect ifconfig X txqueuelen Y to do anything will be > misled. Perhaps this could be fixed by having the fq_codel default > limit be txqueuelen rather than the default (and overlarge) limit of > 10k, but as tons of people are supplying oddball txqueuelens, I tend > to think just ignoring txqueuelen going forward is more the right > thing. > > Do you actually get close to 10k packets outstanding in 10GigE under > any sane circumstances? 10GigE can send 10.000.000 packets per second. 10k is only 1ms of buffering, which is pretty low considering the cpu able to restart a queue might be blocked ~10 ms in a softirq handler. Whole point of codel is that number of packets in the queue is irrelevant. Only sojourn time is. Now if your host has memory concerns, that's a separate issue, and you can adjust the qdisc limit, or add a callback from mm to be able to shrink queue in case of memory pressure, if you deal with non elastic flows. > > B) people that expect pfifo_fast semantics, for which substituting > fq_codel behaves oddly in two ways - > > 1) if you are explicitly setting skb->priority for the default > pfifo_fast 3 bands and expecting a result, nothing happens - but in > the general case, people setting skb->priority are trying to get > better latency in the first place, and I really don't think almost > anybody will notice. > > 2) if you are using a filter on pfifo_fast that expects 3 bands, and > end up using fq_codel by default anyway we get DRR-like behavior over > codel rather than strict prioritization and lose fq_codel's full > benefits... which is still a win IMHO. I am not fond of being able to > starve the other two bands.... > > 3) trying to explicitly set pfifo_fast via tc doesn't work with this patch. > > 4) ECN processing is enabled by default (but off by default in sysctl) There is no 'one solution fits every needs'. codel is _not_ a replacement of pfifo_fast, its a replacement for pfifo. If you want to replace pfifo_fast, you want PRIO + 3 codel, because pfifo_fast is really PRIO + 3 pfifo.