From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from full.lackof.org (full.lackof.org [204.13.164.203]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id ABEF23B29D for ; Mon, 2 May 2022 19:06:38 -0400 (EDT) Received: from [172.16.1.4] (97-126-20-163.tukw.qwest.net [97.126.20.163]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (Client did not present a certificate) by full.lackof.org (Postfix) with ESMTPSA id 4KsdyF351kzyjH for ; Mon, 2 May 2022 23:06:45 +0000 (UTC) Message-ID: <8886abdf-eff8-237b-be33-d57caa8b30cf@lackof.org> Date: Mon, 2 May 2022 16:06:37 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 Content-Language: en-US To: cerowrt-devel@lists.bufferbloat.net References: From: Matt Taggart In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Cerowrt-devel] Fwd: Realtek RTL8156 devices defaulting to CDC-NCM instead of vendor mode, resulting in reduced performance X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 May 2022 23:06:38 -0000 On 5/2/22 15:53, Dave Taht wrote: > .6ms considered good. Well they don't say to _where_ the round trip was, so I read it as the difference between 3ms to 0.6ms is 2.4ms that the non-vendor mode was adding. For the baseline of 0.6ms, we don't know where that's getting introduced. (probably still in the building, could be to their router or another host running who knows what). > > ---------- Forwarded message --------- > From: Forest Crossman > Date: Mon, May 2, 2022 at 3:49 PM > Subject: Realtek RTL8156 devices defaulting to CDC-NCM instead of > vendor mode, resulting in reduced performance > To: , , > Cc: , > > > Hi, all, > > I recently purchased a pair of USB to 2.5G Ethernet dongles based on > the RTL8156, and have so far been very happy with them, but only after > adding some udev rules[0] to to take advantage of the r8152 driver by > switching the devices from their default CDC-NCM mode to the vendor > mode. I was prompted to use those rules to switch the driver because > one of the adapters (based on the RTL8156A) would get very hot, up to > 120 F (49 C) even while idle, and the round-trip latency directly > between the pair of adapters was about 3 ms, and I couldn't help but > wonder if maybe the vendor mode might be more efficient. > > After performing some tests of latency and power consumption, testing > first with both adapters in NCM mode and then again with both in > vendor mode, I proved my hunch correct. I discovered that, in a > disconnected state, the RTL8156A adapter used about half as much power > (0.64 W -> 0.30 W) while the RTL8156B adapter saw a 21% reduction in > power (0.34 W -> 0.27 W). Similarly, in a connected-but-idle state the > RTL8156A again saw about a 55% savings in power consumption (2.17 W -> > 0.97 W) and a 40% savings in the RTL8156B adapter (0.94 W -> 0.56 W). > It was only under full load that the fewest power savings were seen, > with a reduction of only 15% in the RTL8156A (2.23 W -> 1.90 W) and no > savings for the RTL8156B (0.96 W). Similarly, round-trip latency while > idle went from 3 ms to 0.6 ms. I also tested under load and saw much > larger latency savings and reduced packet loss, but forgot to write > down the numbers (I can run the tests again if someone really wants me > too). Also, jumbo frames drastically reduced performance under NCM > mode, while vendor mode handled it like a champ (again, I forgot to > write down the numbers but can test again if asked). > > So, with all the benefits I've seen from using these adapters in their > vendor mode, is there still a reason to let the kernel prefer their > NCM mode? It'd be nice to be able to get the maximum performance from > these adapters on any Linux system I plug them into, without having to > install a udev rule on every one of those systems. > > If anyone would like to try replicating the results I listed here, or > to perform new tests, the specific RTL8156A adapter I used is the > Ugreen CM275[1] and the RTL8156B adapter is the Inateck ET1001[2]. > > > Curious to hear your thoughts on this, > > Forest > > > [0]: https://github.com/bb-qq/r8152/blob/160fb96d2319cdf64ae7597e8739972934ac83b2/50-usb-realtek-net.rules > [1]: https://www.amazon.com/gp/product/B081TY1WQX/ > [2]: https://www.amazon.com/gp/product/B08VN3DGK6/ > > -- Matt Taggart matt@lackof.org