From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.taht.net (mail.taht.net [IPv6:2a01:7e00::f03c:91ff:feae:7028]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 193B83B29E for ; Thu, 16 Nov 2017 11:31:27 -0500 (EST) Received: from nemesis.taht.net (c-24-6-113-161.hsd1.ca.comcast.net [24.6.113.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.taht.net (Postfix) with ESMTPSA id B8C9121474; Thu, 16 Nov 2017 16:31:25 +0000 (UTC) From: Dave Taht To: Pete Heist Cc: cake@lists.bufferbloat.net References: Date: Thu, 16 Nov 2017 08:31:24 -0800 In-Reply-To: (Pete Heist's message of "Thu, 16 Nov 2017 10:14:37 +0100") Message-ID: <87shde2nj7.fsf@nemesis.taht.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] Cake upstream Planning X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 16:31:27 -0000 Pete Heist writes: >>> On Nov 15, 2017, at 9:04 PM, Dave Taht wrote: >>>=20 >>>=20 >>> Dave Taht writes: >>>=20 >>> TCP RTT ~=3D 8ms with default qdisc, throughput ~=3D 940= Mbit >>> TCP RTT ~=3D 4.5ms with =E2=80=98cake unlimited=E2=80=99, th= roughput ~=3D 920 Mbit >>> TCP RTT ~=3D 1ms with =E2=80=98cake unlimited lan=E2=80=99, = throughput ~=3D 920 Mbit >>>=20 >>>=20 >>> This was with BQL in play? Monitoring BQL's behavior might help. >>>=20 >>> I'd also love to know an exact setting for the shaper as a close as >>> possible to the underlying bandwidth of ethernet. However, I tend to= be >>> plagued with >>>=20 >>>=20 >>> Yes, with BQL (Intel I210 with igb driver on the APU2). An rrul_be test= with =E2=80=94 >>> socket-stats. >>=20 >> https://github.com/ffainelli/bqlmon was a tool for looking at bql more >> directly. >>=20 >> I had forked it for some reason or another: >>=20 >> https://github.com/dtaht/bqlmon > > Nice, that does work for me. It=E2=80=99s interesting that there are four= queues for the > igb driver, 00 - 03, and when I try an rrul_be_nflows test, not all four = queues > are necessarily used. Once I get >=3D 8 flows in each direction they usua= lly are > though. I suppose this is the driver deciding when to start using another= queue > or not. Usually it is selected via a hash. In more than a few cases, however, the designer of the hardware intended it as a strict priority queue. In other cases, it's based on the CPU. In all cases such a limited number of queues tends to cause oddities. I think it was the mvneta (?) that had the strict priority queue idea baked into it, which we ended up disabling entirely and going with just one hardware queue. > > Cake does seem to visibly reduce the size of the queues. I generally observe that TSO/GRO/etc tends to make BQL's queues 3-5 times larger than they are without those offloads - no way to fix it, short of doing what cake does to peel those apart. > For whatever > terminal/ncurses weirdness reason though, the bar graphs may be sometimes > blowing off the top of my 45 row screen, but it doesn=E2=80=99t entirely = ruin the > experience. Maybe that was why I forked it?