From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.taht.net (mail.taht.net [IPv6:2a01:7e00::f03c:91ff:feae:7028]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 47FC23B260 for ; Mon, 7 Nov 2016 12:41:45 -0500 (EST) Received: from dair-2506.local (access-63-249-67-130.static.cruzio.com [63.249.67.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.taht.net (Postfix) with ESMTPSA id 483F3213D4 for ; Mon, 7 Nov 2016 17:41:44 +0000 (UTC) To: cerowrt-devel@lists.bufferbloat.net References: From: =?UTF-8?Q?Dave_T=c3=a4ht?= Message-ID: <68f08a8d-df7e-6ba8-ec04-61fff65ad557@taht.net> Date: Mon, 7 Nov 2016 09:41:42 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Subject: Re: [Cerowrt-devel] Turris Omnia X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Nov 2016 17:41:45 -0000 On 11/7/16 6:30 AM, James Cloos wrote: > but with wshaper there were 8 rather than 4 lines matching parent for > each interface. There still are 8 for the 3 ethernet devices, but that > doesn't seem to be a problem.... The armada chipset in the omnia has 8 hardware queues on the ethernet devices, so you end up with 8 fq_codel instances on them. They did not support BQL, thus fq_codel was ineffective, until recently - a patch was submitted upstream to address this, I don't know if it made it into openwrt yet. I believed it shaved 10-20ms of latency off a saturated gigE network. Second problem is that BQL gives out an eventually even share of queue to the right hw queues, but that's "eventually". A third problem, even with BQL, is that you end up with the birthday problem with so few queues - two flows can easily hash into the same hw queue. I think the designer's original intent was not to have 8 equal weight queues, but to have them apply to levels of QoS. A secondary intent was to map queues to cores for processing. Not in the intent was to provide consistently low latency or network fairness. :/ Last problem - and why cake may be important to some - was that they had *aggressively* enabled (64k!) GRO offloads in the driver, which is helpful on benchmarks, but really hurts sqm-scripts w/fq_codel in some cases. > > So this seems solved. Yea! I note that tc -s qdisc show tends to be more revealing. If the BQL and GRO problem still exist, you generally will never see fq_codel mark or drop packets on the ethernet devices, even if you drive two ports into one. > Thanks. > > -JimC >