From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-x230.google.com (mail-ob0-x230.google.com [IPv6:2607:f8b0:4003:c01::230]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id F01AB21F1FC for ; Thu, 11 Jul 2013 10:44:36 -0700 (PDT) Received: by mail-ob0-f176.google.com with SMTP id v19so10315225obq.35 for ; Thu, 11 Jul 2013 10:44:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:subject:from:to:cc:date:in-reply-to:references :content-type:x-mailer:content-transfer-encoding:mime-version; bh=sm1YeoQAnqrbJCQnwnhnqgJoLeziym1Jy6p6rePg9HM=; b=wQcFbL7MVAcfqXh5NArHTSLCx0FlOFVyjLrb3OADlyS7qzigIOwPI7B5kg8DN8Hwwt +oXN49AbttTlwQoYdnD/G89a+8ZWW56TOWJJD/pl7EaZdOngvUOfABHR4yrbOufxiaV3 JJZN9jtxMXQ60lUGSxc90RtbJOZ6wf1EfYY7CtiIrL8dqNCS3/vED0vn6xyWh1xov48o fcFTLY2e5x/6aZrTAGf+ZGY1X01DBVjn5oKUrprwhbn1mYSJfO3T4zBtO1TkAt1bbhvx BjAgdKGC67YlcVblzn46L12QHCBhFqOfWvZLV85FQGuFKSsnzGIYmtS6WzVDAME0CI8v xyTw== X-Received: by 10.60.37.74 with SMTP id w10mr32865295oej.30.1373564675979; Thu, 11 Jul 2013 10:44:35 -0700 (PDT) Received: from ?IPv6:2620:0:1000:3304:d49d:873e:bd9b:d524? ([2620:0:1000:3304:d49d:873e:bd9b:d524]) by mx.google.com with ESMTPSA id cp7sm50332368obb.0.2013.07.11.10.44.34 for (version=SSLv3 cipher=RC4-SHA bits=128/128); Thu, 11 Jul 2013 10:44:35 -0700 (PDT) Message-ID: <1373564673.4600.55.camel@edumazet-glaptop> From: Eric Dumazet To: Dave Taht Date: Thu, 11 Jul 2013 10:44:33 -0700 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.3-0ubuntu6 Content-Transfer-Encoding: 7bit Mime-Version: 1.0 Cc: codel@lists.bufferbloat.net Subject: Re: [Codel] hardware multiqueue in fq_codel? X-BeenThere: codel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: CoDel AQM discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jul 2013 17:44:37 -0000 On Thu, 2013-07-11 at 10:09 -0700, Dave Taht wrote: > In my default environments (wifi, mainly) the hardware queues have > very different properties. > > I'm under the impression that in at least a few ethernet devices they > are essentially the same. That said, in the sch_mq case, an entirely > separate qdisc is created per hardware queue, and it's always been > puzzling to me as to how to attempt to use them within a single qdisc > in the pull-through manner. > > logically, you should be able to take the fq_codel hash index (idx % > dev->num_tx_queues) and spread out across the hardware queues that > way, but I have no idea where that info would go (the skb? the flow?) > or even if it were possible as per the pull through problem... > > (This does not mean that I necessarily think hardware multiqueues are > a good idea... (certainly the results I get out of 802.11e are > terrible - but it would be nice to have a unified solution for hw > multiqueue devices) > We do not have a fixed/unified queue selection. It can be tweaked by many different things, depending on exact needs. MQ is not a qdisc per se, it's only a fake one, a demux if you want, so that each tx queue has a separate qdisc lock. If you stick one fq_codel at the top of the hierarchy (instead of MQ), then you loose all the pros of having multiple locks : sending packets from fq_codel to different queues on hardware makes no sense, since the single qdisc lock is the bottleneck. So if you want fq_codel and MQ, to be able to drive 40G links from many cpus, just use : ETH=eth0 NQUEUES=16 # or more, check how many tx queues your NIC supports tc qd del dev $ETH root 2>/dev/null tc qd add dev $ETH root handle 1: mq for i in `seq 1 $NQUEUES` do tc qd add dev $ETH parent 1:$i fq_codel done Thats only replaces the default pfifo_fast on each slave qdisc by fq_codel.