From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 858E53B2A4 for ; Wed, 23 Dec 2020 02:23:09 -0500 (EST) Received: by mail-ed1-x533.google.com with SMTP id r5so15241326eda.12 for ; Tue, 22 Dec 2020 23:23:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=S4z/LzIZqPn6F0+fdZwZzUoU8ocyM8JYZuZbX181dTo=; b=JX0iFqvKOE+iUPOL3DbBx1k3X5kgP39vaG3uK2m449g1BKVwzqjzNv7wQjva5LE0jF bXiqXrl7NvnfqTkLEchZFONmj2aOpj8NjODtwQvq9EI1fqjNz6eI7ncFKxskffydRHkP fjHgUYZe9c/2f+7SFyfH6k6l/peXAQ0Ugqkb0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=S4z/LzIZqPn6F0+fdZwZzUoU8ocyM8JYZuZbX181dTo=; b=uU2GnyqQwgwKbG22hUkCT1iVBl33SR7qV4nxnITPBjRmYj9tE1lkoAQsuCpDVF0qgm kk9GTZ5NC18ng3R0Fo6fdaPGuVYvXTTZS0Qi1DdeLgwCd9QX6K1ebXNYOC0ORwVjcf4P VhS4JCNRKqKvijvEqIfPO00tOCjHWLq2qIpw3BmUW4ZkxhyrSi0zMBhcL+VrHjmjBXYC 6HxWudRzAuSQg3dxxkCbihtlCvihWDYRcW5PPv+v8URXLUu7Azoe9c+d3k+q7XhtbjBM 9kcVSqmIcFqKGJKu4LJ0h53T8/zAESoMRJGuDN8NBM4pgf0vOhgx43Gjc1Lj9SP6LGey dYhA== X-Gm-Message-State: AOAM531bHq5fPKkcDtTwtDhn0dCwGljEkGZkXeNqwG/QcOL3K91vXJGh 1Bi4qaC39DAV2j3YR9STAcHbFFiY4axOtRcFU7MTCbq6NT9WjiKuSXrfo71JO+0EsHbS75QfZq8 Hm795OpJAUVNN3+S/qgzmt5yOwz+Y0Ks/x/uY4GCz1a1jUtAypQ== X-Google-Smtp-Source: ABdhPJwkS8BbOXsdOvJrVT303CUO5BGCxKabOz2bOr6aR+1nPaMIxJzkdA5Qtz+sc7jDyJInxRnKbilEqXur5CyXxlI= X-Received: by 2002:a50:d88c:: with SMTP id p12mr23625703edj.370.1608708187973; Tue, 22 Dec 2020 23:23:07 -0800 (PST) MIME-Version: 1.0 From: Bob McMahon Date: Tue, 22 Dec 2020 23:22:57 -0800 Message-ID: To: Make-Wifi-fast Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-256; boundary="0000000000001eec2505b71c90e1" Subject: [Make-wifi-fast] iperf 2.0.14 experimental feature --near-congestion X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Dec 2020 07:23:09 -0000 --0000000000001eec2505b71c90e1 Content-Type: multipart/alternative; boundary="00000000000017c43a05b71c9023" --00000000000017c43a05b71c9023 Content-Type: text/plain; charset="UTF-8" Hi All, I've added an experimental feature to iperf 2.0.14 per option --near-congestion=. This is useful when measuring TCP write to read latencies at high throughput but not at bufferbloat. It requires a very much controlled network to be repeatable and, even then, still requires some knob tuning. *The idea is to rate limit the tcp writes by sampling the RTT after each write buffer completes (per -l.)* The delay is weighted per a multiplier of the RTT. This allows the queues to drain. The RTT multiplier can be any value zero or greater. It's loosely based off this paper . This feature is committed in version starting Dec 22 [rjmcmahon@localhost iperf2-code]$ iperf -v iperf version 2.0.14a (22 Dec 2020) pthreads Here's an example testing against a raspberry pi 4 wired gigE. *First with the delay being 0.4 of the RTT sampled after the -l write size.* Throughput is fairly high and the write to read latency averages 3.5 ms. [rjmcmahon@localhost Code]$ iperf -c 192.168.1.108 -i 1 --trip-times --near-congestion=0.4 -e ------------------------------------------------------------ Client connecting to 192.168.1.108, TCP port 5001 with pid 11219 (1 flows) Write buffer size: 131072 Byte TCP near-congestion delay weight set to 0.4000 TCP window size: 85.0 KByte (default) ------------------------------------------------------------ [ 1] local 192.168.1.62%enp2s0 port 48100 connected with 192.168.1.108 port 5001 (MSS=1448) (trip-times) (sock=3) (ct=0.28 ms) on 2020-12-22 22:32:39 (PST) [ ID] Interval Transfer Bandwidth Write/Err Rtry Cwnd/RTT NetPwr [ 1] 0.00-1.00 sec 112 MBytes 943 Mbits/sec 900/0 0 567K/2566 us 45921 [ 1] 1.00-2.00 sec 112 MBytes 941 Mbits/sec 897/0 0 567K/1653 us 71126 [ 1] 2.00-3.00 sec 111 MBytes 931 Mbits/sec 888/0 0 695K/2418 us 48136 [ 1] 3.00-4.00 sec 111 MBytes 932 Mbits/sec 889/0 0 695K/1407 us 82817 [ 1] 4.00-5.00 sec 109 MBytes 913 Mbits/sec 871/0 0 695K/2062 us 55366 [ 1] 5.00-6.00 sec 112 MBytes 935 Mbits/sec 892/0 0 695K/2484 us 47068 [ 1] 6.00-7.00 sec 112 MBytes 938 Mbits/sec 895/0 0 695K/2511 us 46718 [ 1] 7.00-8.00 sec 112 MBytes 938 Mbits/sec 895/0 0 695K/1895 us 61905 [ 1] 8.00-9.00 sec 111 MBytes 929 Mbits/sec 886/0 0 695K/2495 us 46545 [ 1] 9.00-10.00 sec 112 MBytes 940 Mbits/sec 896/0 0 695K/3403 us 34511 [ 1] 0.00-10.00 sec 1.09 GBytes 934 Mbits/sec 8912/0 0 695K/3403 us 34322 root@rpi4-rjm-1:/usr/local/src/iperf2-code# src/iperf -s -i 1 -e ------------------------------------------------------------ Server listening on TCP port 5001 with pid 3805 Read buffer size: 128 KByte (Dist bin width=16.0 KByte) TCP window size: 128 KByte (default) ------------------------------------------------------------ [ 1] local 192.168.1.108%eth0 port 5001 connected with 192.168.1.62 port 48100 (MSS=453) (trip-times) (sock=4) (peer 2.0.14-alpha) on 2020-12-22 22:32:39 (PST) [ ID] Interval Transfer Bandwidth Burst Latency avg/min/max/stdev (cnt/size) inP NetPwr Reads=Dist [ 1] 0.00-1.00 sec 112 MBytes 940 Mbits/sec 3.668/2.166/12.241/1.761 ms (896/131130) 423 KByte 32032 46203=46139:39:7:7:4:2:0:5 [ 1] 1.00-2.00 sec 112 MBytes 939 Mbits/sec 3.172/2.295/5.212/0.158 ms (896/131013) 364 KByte 37009 38631=38511:93:3:0:1:0:0:23 [ 1] 2.00-3.00 sec 111 MBytes 932 Mbits/sec 3.447/1.205/6.307/0.997 ms (889/131103) 391 KByte 33815 31142=30806:18:0:0:0:0:0:318 [ 1] 3.00-4.00 sec 111 MBytes 932 Mbits/sec 4.485/1.247/10.803/2.049 ms (889/131040) 512 KByte 25972 24613=24166:4:2:0:0:1:0:440 [ 1] 4.00-5.00 sec 109 MBytes 913 Mbits/sec 3.866/1.248/6.951/1.442 ms (871/131072) 431 KByte 29529 871=0:0:0:0:0:0:0:871 [ 1] 5.00-6.00 sec 112 MBytes 936 Mbits/sec 3.295/1.249/5.933/0.750 ms (892/131192) 376 KByte 35514 37172=36953:6:4:1:0:0:1:207 [ 1] 6.00-7.00 sec 112 MBytes 938 Mbits/sec 3.537/1.252/9.147/1.041 ms (895/131019) 405 KByte 33153 39414=39229:14:3:1:0:0:0:167 [ 1] 7.00-8.00 sec 112 MBytes 936 Mbits/sec 3.467/1.224/7.949/0.904 ms (893/131004) 396 KByte 33742 38755=38566:10:2:0:0:0:0:177 [ 1] 8.00-9.00 sec 111 MBytes 932 Mbits/sec 3.510/1.212/6.808/1.064 ms (888/131132) 399 KByte 33173 27034=26636:2:2:1:0:0:0:393 [ 1] 9.00-10.00 sec 112 MBytes 937 Mbits/sec 3.308/2.732/7.023/0.600 ms (894/131012) 379 KByte 35406 43303=43197:15:4:0:0:0:0:87 [ 1] 0.00-10.00 sec 1.09 GBytes 934 Mbits/sec 3.576/1.205/12.241/1.249 ms (8909/131072) 408 KByte 32644 327144=324203:201:27:10:5:3:1:2694 *Next, the delay is 0 so TCP drives to bufferbloat. Network power is 8512 vs 34322 and the average latency is 30 milliseconds vs 3.5.* *Trading off a little bit of top line throughput for a major improvement in latency.* [rjmcmahon@localhost Code]$ iperf -c 192.168.1.108 -i 1 --trip-times --near-congestion=0 -e ------------------------------------------------------------ Client connecting to 192.168.1.108, TCP port 5001 with pid 12140 (1 flows) Write buffer size: 131072 Byte TCP near-congestion delay weight set to 0.0000 TCP window size: 85.0 KByte (default) ------------------------------------------------------------ [ 1] local 192.168.1.62%enp2s0 port 48662 connected with 192.168.1.108 port 5001 (MSS=1448) (trip-times) (sock=3) (ct=0.19 ms) on 2020-12-22 22:36:18 (PST) [ ID] Interval Transfer Bandwidth Write/Err Rtry Cwnd/RTT NetPwr [ 1] 0.00-1.00 sec 116 MBytes 970 Mbits/sec 926/0 0 1528K/13054 us 9288 [ 1] 1.00-2.00 sec 112 MBytes 942 Mbits/sec 898/0 0 1671K/14330 us 8214 [ 1] 2.00-3.00 sec 112 MBytes 944 Mbits/sec 900/0 0 1784K/15225 us 7748 [ 1] 3.00-4.00 sec 112 MBytes 942 Mbits/sec 898/0 0 1870K/16005 us 7354 [ 1] 4.00-5.00 sec 112 MBytes 935 Mbits/sec 892/0 0 1934K/16513 us 7080 [ 1] 5.00-6.00 sec 112 MBytes 942 Mbits/sec 898/0 0 1442K/12335 us 9542 [ 1] 6.00-7.00 sec 112 MBytes 940 Mbits/sec 896/0 0 1518K/12872 us 9124 [ 1] 7.00-8.00 sec 113 MBytes 945 Mbits/sec 901/0 0 1571K/13423 us 8798 [ 1] 8.00-9.00 sec 112 MBytes 941 Mbits/sec 897/0 0 1607K/13718 us 8571 [ 1] 9.00-10.00 sec 112 MBytes 940 Mbits/sec 896/0 0 1631K/13863 us 8472 [ 1] 0.00-10.00 sec 1.10 GBytes 944 Mbits/sec 9005/0 70 1631K/13863 us 8512 root@rpi4-rjm-1:/usr/local/src/iperf2-code# src/iperf -s -i 1 -e ------------------------------------------------------------ Server listening on TCP port 5001 with pid 3820 Read buffer size: 128 KByte (Dist bin width=16.0 KByte) TCP window size: 128 KByte (default) ------------------------------------------------------------ [ 1] local 192.168.1.108%eth0 port 5001 connected with 192.168.1.62 port 48662 (MSS=453) (trip-times) (sock=4) (peer 2.0.14-alpha) on 2020-12-22 22:36:18 (PST) [ ID] Interval Transfer Bandwidth Burst Latency avg/min/max/stdev (cnt/size) inP NetPwr Reads=Dist [ 1] 0.00-1.00 sec 112 MBytes 940 Mbits/sec 28.251/2.674/50.071/7.127 ms (896/131160) 3.27 MByte 4160 50060=49977:31:13:8:3:3:5:20 [ 1] 1.00-2.00 sec 112 MBytes 942 Mbits/sec 30.198/24.293/36.232/3.449 ms (898/131058) 3.39 MByte 3897 54903=54902:1:0:0:0:0:0:0 [ 1] 2.00-3.00 sec 112 MBytes 941 Mbits/sec 30.184/24.276/36.088/3.467 ms (898/131053) 3.38 MByte 3899 54905=54903:2:0:0:0:0:0:0 [ 1] 3.00-4.00 sec 112 MBytes 942 Mbits/sec 30.172/24.259/36.306/3.444 ms (898/131060) 3.39 MByte 3901 55248=55245:3:0:0:0:0:0:0 [ 1] 4.00-5.00 sec 112 MBytes 941 Mbits/sec 30.170/24.267/36.208/3.433 ms (898/131053) 3.40 MByte 3901 55389=55383:6:0:0:0:0:0:0 [ 1] 5.00-6.00 sec 112 MBytes 942 Mbits/sec 30.302/15.238/48.700/3.973 ms (898/131056) 3.40 MByte 3884 53258=53224:7:0:0:0:1:1:25 [ 1] 6.00-7.00 sec 112 MBytes 942 Mbits/sec 30.140/24.269/36.058/3.454 ms (897/131204) 3.36 MByte 3905 55104=55102:2:0:0:0:0:0:0 [ 1] 7.00-8.00 sec 112 MBytes 941 Mbits/sec 30.267/24.299/36.244/3.430 ms (898/131053) 3.42 MByte 3888 55107=55106:1:0:0:0:0:0:0 [ 1] 8.00-9.00 sec 112 MBytes 942 Mbits/sec 30.179/24.241/36.158/3.446 ms (898/131056) 3.40 MByte 3900 55194=55192:1:1:0:0:0:0:0 [ 1] 9.00-10.00 sec 112 MBytes 942 Mbits/sec 30.170/24.273/36.415/3.452 ms (898/131056) 3.39 MByte 3901 55100=55095:5:0:0:0:0:0:0 [ 1] 0.00-10.03 sec 1.10 GBytes 941 Mbits/sec 30.005/2.674/50.071/4.059 ms (9002/131072) 3.37 MByte 3922 545735=545595:60:14:8:3:4:6:45 Bob -- This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it. --00000000000017c43a05b71c9023 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi All,

I've added an experimental feature to= =C2=A0iperf 2.0.14=C2=A0per option=C2=A0--near-congestion=3D<value>. = This is useful when measuring TCP write to read latencies at high throughpu= t but not at bufferbloat. It requires=C2=A0a very much controlled network t= o be repeatable and, even then, still requires some knob tuning.

= The idea is to rate limit the tcp writes by sampling the RTT after each wri= te buffer completes (per -l.)=C2=A0The delay is weighted per a multipli= er=C2=A0of the RTT. This allows the queues to drain.=C2=A0 The RTT multipli= er can be any value zero or greater. It's loosely based=C2=A0off this paper.
This feature is committed in version starting Dec 22

[rjmcmaho= n@localhost iperf2-code]$ iperf -v
iperf version 2.0.14a (22 Dec 2020) p= threads

Here's an example testing against a raspberry=C2=A0pi 4 = wired gigE.

First with the delay being 0.4 of t= he RTT sampled after=C2=A0the -l write size.=C2=A0 Throughput=C2=A0is f= airly high and the write to read latency averages 3.5 ms.

[rjmcmahon= @localhost Code]$ iperf -c 192.168.1.108 -i 1 --trip-times=C2=A0--near-congestion=3D0.4=C2=A0-e=
------------------------------------------------------------
Client = connecting to 192.168.1.108, TCP port 5001 with pid 11219 (1 flows)
Writ= e buffer size: 131072 Byte
TCP near-congestion delay weight set to 0.400= 0
TCP window size: 85.0 KByte (default)
-----------------------------= -------------------------------
[ =C2=A01] local 192.168.1.62%enp2s0 por= t 48100 connected with 192.168.1.108 port 5001 (MSS=3D1448) (trip-times) (s= ock=3D3) (ct=3D0.28 ms) on 2020-12-22 22:32:39 (PST)
[ ID] Interval =C2= =A0 =C2=A0 =C2=A0 =C2=A0Transfer =C2=A0 =C2=A0Bandwidth =C2=A0 =C2=A0 =C2= =A0 Write/Err =C2=A0Rtry =C2=A0 =C2=A0 Cwnd/RTT =C2=A0 =C2=A0 =C2=A0 =C2=A0= NetPwr
[ =C2=A01] 0.00-1.00 sec =C2=A0 112 MBytes =C2=A0 943 Mbits/sec = =C2=A0900/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 =C2=A0567K/25= 66 us =C2=A045921
[ =C2=A01] 1.00-2.00 sec =C2=A0 112 MBytes =C2=A0 941 = Mbits/sec =C2=A0897/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 =C2= =A0567K/1653 us =C2=A071126
[ =C2=A01] 2.00-3.00 sec =C2=A0 111 MBytes = =C2=A0 931 Mbits/sec =C2=A0888/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0= =C2=A0 =C2=A0695K/2418 us =C2=A048136
[ =C2=A01] 3.00-4.00 sec =C2=A0 1= 11 MBytes =C2=A0 932 Mbits/sec =C2=A0889/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A00 =C2=A0 =C2=A0 =C2=A0695K/1407 us =C2=A082817
[ =C2=A01] 4.00-5.00 s= ec =C2=A0 109 MBytes =C2=A0 913 Mbits/sec =C2=A0871/0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A00 =C2=A0 =C2=A0 =C2=A0695K/2062 us =C2=A055366
[ =C2=A01] 5= .00-6.00 sec =C2=A0 112 MBytes =C2=A0 935 Mbits/sec =C2=A0892/0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 =C2=A0695K/2484 us =C2=A047068
[= =C2=A01] 6.00-7.00 sec =C2=A0 112 MBytes =C2=A0 938 Mbits/sec =C2=A0895/0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 =C2=A0695K/2511 us =C2=A0= 46718
[ =C2=A01] 7.00-8.00 sec =C2=A0 112 MBytes =C2=A0 938 Mbits/sec = =C2=A0895/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 =C2=A0695K/18= 95 us =C2=A061905
[ =C2=A01] 8.00-9.00 sec =C2=A0 111 MBytes =C2=A0 929 = Mbits/sec =C2=A0886/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 =C2= =A0695K/2495 us =C2=A046545
[ =C2=A01] 9.00-10.00 sec =C2=A0 112 MBytes = =C2=A0 940 Mbits/sec =C2=A0896/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0= =C2=A0 =C2=A0695K/3403 us =C2=A034511
[ =C2=A01] 0.00-10.00 sec =C2=A01= .09 GBytes =C2=A0 934 Mbits/sec =C2=A08912/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A00 =C2=A0 =C2=A0 =C2=A0695K/3403 us =C2=A034322

root@rpi4-rjm-1:/usr/local/src/iper= f2-code# src/iperf -s -i 1 -e
------------------------------------------= ------------------
Server listening on TCP port 5001 with pid 3805
Re= ad buffer size: =C2=A0128 KByte (Dist bin width=3D16.0 KByte)
TCP window= size: =C2=A0128 KByte (default)
---------------------------------------= ---------------------
[ =C2=A01] local 192.168.1.108%eth0 port 5001 conn= ected with 192.168.1.62 port 48100 (MSS=3D453) (trip-times) (sock=3D4) (pee= r 2.0.14-alpha) on 2020-12-22 22:32:39 (PST)
[ ID] Interval =C2=A0 =C2= =A0 =C2=A0 =C2=A0Transfer =C2=A0 =C2=A0Bandwidth =C2=A0 =C2=A0Burst Latency= avg/min/max/stdev (cnt/size) inP NetPwr =C2=A0Reads=3DDist
[ =C2=A01] 0= .00-1.00 sec =C2=A0 112 MBytes =C2=A0 940 Mbits/sec =C2=A03.668/2.166/12.24= 1/1.761 ms (896/131130) =C2=A0423 KByte 32032 =C2=A046203=3D46139:39:7:7:4:= 2:0:5
[ =C2=A01] 1.00-2.00 sec =C2=A0 112 MBytes =C2=A0 939 Mbits/sec = =C2=A03.172/2.295/5.212/0.158 ms (896/131013) =C2=A0364 KByte 37009 =C2=A03= 8631=3D38511:93:3:0:1:0:0:23
[ =C2=A01] 2.00-3.00 sec =C2=A0 111 MBytes = =C2=A0 932 Mbits/sec =C2=A03.447/1.205/6.307/0.997 ms (889/131103) =C2=A039= 1 KByte 33815 =C2=A031142=3D30806:18:0:0:0:0:0:318
[ =C2=A01] 3.00-4.00 = sec =C2=A0 111 MBytes =C2=A0 932 Mbits/sec =C2=A04.485/1.247/10.803/2.049 m= s (889/131040) =C2=A0512 KByte 25972 =C2=A024613=3D24166:4:2:0:0:1:0:440[ =C2=A01] 4.00-5.00 sec =C2=A0 109 MBytes =C2=A0 913 Mbits/sec =C2=A03.86= 6/1.248/6.951/1.442 ms (871/131072) =C2=A0431 KByte 29529 =C2=A0871=3D0:0:0= :0:0:0:0:871
[ =C2=A01] 5.00-6.00 sec =C2=A0 112 MBytes =C2=A0 936 Mbits= /sec =C2=A03.295/1.249/5.933/0.750 ms (892/131192) =C2=A0376 KByte 35514 = =C2=A037172=3D36953:6:4:1:0:0:1:207
[ =C2=A01] 6.00-7.00 sec =C2=A0 112 = MBytes =C2=A0 938 Mbits/sec =C2=A03.537/1.252/9.147/1.041 ms (895/131019) = =C2=A0405 KByte 33153 =C2=A039414=3D39229:14:3:1:0:0:0:167
[ =C2=A01] 7.= 00-8.00 sec =C2=A0 112 MBytes =C2=A0 936 Mbits/sec =C2=A03.467/1.224/7.949/= 0.904 ms (893/131004) =C2=A0396 KByte 33742 =C2=A038755=3D38566:10:2:0:0:0:= 0:177
[ =C2=A01] 8.00-9.00 sec =C2=A0 111 MBytes =C2=A0 932 Mbits/sec = =C2=A03.510/1.212/6.808/1.064 ms (888/131132) =C2=A0399 KByte 33173 =C2=A02= 7034=3D26636:2:2:1:0:0:0:393
[ =C2=A01] 9.00-10.00 sec =C2=A0 112 MBytes= =C2=A0 937 Mbits/sec =C2=A03.308/2.732/7.023/0.600 ms (894/131012) =C2=A03= 79 KByte 35406 =C2=A043303=3D43197:15:4:0:0:0:0:87
[ =C2=A01] 0.00-10.00= sec =C2=A01.09 GBytes =C2=A0 934 Mbits/sec =C2=A03.576/1.205/12.241/1.249 ms (8909/131072) =C2= =A0408 KByte 32644 =C2=A0327144=3D324203:201:27:10:5:3:1:2694

=
Next, the delay is 0 so=C2=A0TCP drives to bufferbloat.=C2=A0Network power is 8512 vs 3= 4322 and the average latency is 30 milliseconds vs 3.5.

Trading off a little bit of top line t= hroughput for a major improvement in latency.

[rjmcmahon@localho= st Code]$ iperf -c 192.168.1.108 -i 1 --trip-times=C2=A0--near-congestion=3D0=C2=A0-e
------= ------------------------------------------------------
Client connecting= to 192.168.1.108, TCP port 5001 with pid 12140 (1 flows)
Write buffer s= ize: 131072 Byte
TCP near-congestion delay weight set to 0.0000
TCP w= indow size: 85.0 KByte (default)
---------------------------------------= ---------------------
[ =C2=A01] local 192.168.1.62%enp2s0 port 48662 co= nnected with 192.168.1.108 port 5001 (MSS=3D1448) (trip-times) (sock=3D3) (= ct=3D0.19 ms) on 2020-12-22 22:36:18 (PST)
[ ID] Interval =C2=A0 =C2=A0 = =C2=A0 =C2=A0Transfer =C2=A0 =C2=A0Bandwidth =C2=A0 =C2=A0 =C2=A0 Write/Err= =C2=A0Rtry =C2=A0 =C2=A0 Cwnd/RTT =C2=A0 =C2=A0 =C2=A0 =C2=A0NetPwr
[ = =C2=A01] 0.00-1.00 sec =C2=A0 116 MBytes =C2=A0 970 Mbits/sec =C2=A0926/0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 1528K/13054 us =C2=A09288=
[ =C2=A01] 1.00-2.00 sec =C2=A0 112 MBytes =C2=A0 942 Mbits/sec =C2=A08= 98/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 1671K/14330 us =C2= =A08214
[ =C2=A01] 2.00-3.00 sec =C2=A0 112 MBytes =C2=A0 944 Mbits/sec = =C2=A0900/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 1784K/15225 u= s =C2=A07748
[ =C2=A01] 3.00-4.00 sec =C2=A0 112 MBytes =C2=A0 942 Mbits= /sec =C2=A0898/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 1870K/16= 005 us =C2=A07354
[ =C2=A01] 4.00-5.00 sec =C2=A0 112 MBytes =C2=A0 935 = Mbits/sec =C2=A0892/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 193= 4K/16513 us =C2=A07080
[ =C2=A01] 5.00-6.00 sec =C2=A0 112 MBytes =C2=A0= 942 Mbits/sec =C2=A0898/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2= =A0 1442K/12335 us =C2=A09542
[ =C2=A01] 6.00-7.00 sec =C2=A0 112 MBytes= =C2=A0 940 Mbits/sec =C2=A0896/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2= =A0 =C2=A0 1518K/12872 us =C2=A09124
[ =C2=A01] 7.00-8.00 sec =C2=A0 113= MBytes =C2=A0 945 Mbits/sec =C2=A0901/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= 0 =C2=A0 =C2=A0 1571K/13423 us =C2=A08798
[ =C2=A01] 8.00-9.00 sec =C2= =A0 112 MBytes =C2=A0 941 Mbits/sec =C2=A0897/0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A00 =C2=A0 =C2=A0 1607K/13718 us =C2=A08571
[ =C2=A01] 9.00-10.00 s= ec =C2=A0 112 MBytes =C2=A0 940 Mbits/sec =C2=A0896/0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A00 =C2=A0 =C2=A0 1631K/13863 us =C2=A08472
[ =C2=A01] 0.00-1= 0.00 sec =C2=A01.10 GBytes =C2=A0 944 Mbits/sec =C2=A09005/0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 70 =C2=A0 =C2=A0 1631K/13863 us =C2=A08512

root@rpi4-rjm-1:/usr/local/src/= iperf2-code# src/iperf -s -i 1 -e
--------------------------------------= ----------------------
Server listening on TCP port 5001 with pid 3820Read buffer size: =C2=A0128 KByte (Dist bin width=3D16.0 KByte)
TCP wi= ndow size: =C2=A0128 KByte (default)
-----------------------------------= -------------------------
[ =C2=A01] local 192.168.1.108%eth0 port 5001 = connected with 192.168.1.62 port 48662 (MSS=3D453) (trip-times) (sock=3D4) = (peer 2.0.14-alpha) on 2020-12-22 22:36:18 (PST)
[ ID] Interval =C2=A0 = =C2=A0 =C2=A0 =C2=A0Transfer =C2=A0 =C2=A0Bandwidth =C2=A0 =C2=A0Burst Late= ncy avg/min/max/stdev (cnt/size) inP NetPwr =C2=A0Reads=3DDist
[ =C2=A01= ] 0.00-1.00 sec =C2=A0 112 MBytes =C2=A0 940 Mbits/sec =C2=A028.251/2.674/5= 0.071/7.127 ms (896/131160) 3.27 MByte 4160 =C2=A050060=3D49977:31:13:8:3:3= :5:20
[ =C2=A01] 1.00-2.00 sec =C2=A0 112 MBytes =C2=A0 942 Mbits/sec = =C2=A030.198/24.293/36.232/3.449 ms (898/131058) 3.39 MByte 3897 =C2=A05490= 3=3D54902:1:0:0:0:0:0:0
[ =C2=A01] 2.00-3.00 sec =C2=A0 112 MBytes =C2= =A0 941 Mbits/sec =C2=A030.184/24.276/36.088/3.467 ms (898/131053) 3.38 MBy= te 3899 =C2=A054905=3D54903:2:0:0:0:0:0:0
[ =C2=A01] 3.00-4.00 sec =C2= =A0 112 MBytes =C2=A0 942 Mbits/sec =C2=A030.172/24.259/36.306/3.444 ms (89= 8/131060) 3.39 MByte 3901 =C2=A055248=3D55245:3:0:0:0:0:0:0
[ =C2=A01] 4= .00-5.00 sec =C2=A0 112 MBytes =C2=A0 941 Mbits/sec =C2=A030.170/24.267/36.= 208/3.433 ms (898/131053) 3.40 MByte 3901 =C2=A055389=3D55383:6:0:0:0:0:0:0=
[ =C2=A01] 5.00-6.00 sec =C2=A0 112 MBytes =C2=A0 942 Mbits/sec =C2=A03= 0.302/15.238/48.700/3.973 ms (898/131056) 3.40 MByte 3884 =C2=A053258=3D532= 24:7:0:0:0:1:1:25
[ =C2=A01] 6.00-7.00 sec =C2=A0 112 MBytes =C2=A0 942 = Mbits/sec =C2=A030.140/24.269/36.058/3.454 ms (897/131204) 3.36 MByte 3905 = =C2=A055104=3D55102:2:0:0:0:0:0:0
[ =C2=A01] 7.00-8.00 sec =C2=A0 112 MB= ytes =C2=A0 941 Mbits/sec =C2=A030.267/24.299/36.244/3.430 ms (898/131053) = 3.42 MByte 3888 =C2=A055107=3D55106:1:0:0:0:0:0:0
[ =C2=A01] 8.00-9.00 s= ec =C2=A0 112 MBytes =C2=A0 942 Mbits/sec =C2=A030.179/24.241/36.158/3.446 = ms (898/131056) 3.40 MByte 3900 =C2=A055194=3D55192:1:1:0:0:0:0:0
[ =C2= =A01] 9.00-10.00 sec =C2=A0 112 MBytes =C2=A0 942 Mbits/sec =C2=A030.170/24= .273/36.415/3.452 ms (898/131056) 3.39 MByte 3901 =C2=A055100=3D55095:5:0:0= :0:0:0:0
[ =C2=A01] 0.00-10.03 sec =C2=A01.10 GBytes =C2=A0 941 Mbits/se= c =C2=A030.005/2.674= /50.071/4.059 ms (9002/131072) 3.37 MByte 3922 =C2=A0545735=3D545595:60:14:= 8:3:4:6:45

Bob

This ele= ctronic communication and the information and any files transmitted with it= , or attached to it, are confidential and are intended solely for the use o= f the individual or entity to whom it is addressed and may contain informat= ion that is confidential, legally privileged, protected by privacy laws, or= otherwise restricted from disclosure to anyone else. If you are not the in= tended recipient or the person responsible for delivering the e-mail to the= intended recipient, you are hereby notified that any use, copying, distrib= uting, dissemination, forwarding, printing, or copying of this e-mail is st= rictly prohibited. If you received this e-mail in error, please return the = e-mail to the sender, delete it from your computer, and destroy any printed= copy of it. --00000000000017c43a05b71c9023-- --0000000000001eec2505b71c90e1 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIIQPwYJKoZIhvcNAQcCoIIQMDCCECwCAQExDzANBglghkgBZQMEAgEFADALBgkqhkiG9w0BBwGg gg2UMIIE6DCCA9CgAwIBAgIOSBtqCRO9gCTKXSLwFPMwDQYJKoZIhvcNAQELBQAwTDEgMB4GA1UE CxMXR2xvYmFsU2lnbiBSb290IENBIC0gUjMxEzARBgNVBAoTCkdsb2JhbFNpZ24xEzARBgNVBAMT Ckdsb2JhbFNpZ24wHhcNMTYwNjE1MDAwMDAwWhcNMjQwNjE1MDAwMDAwWjBdMQswCQYDVQQGEwJC RTEZMBcGA1UEChMQR2xvYmFsU2lnbiBudi1zYTEzMDEGA1UEAxMqR2xvYmFsU2lnbiBQZXJzb25h bFNpZ24gMiBDQSAtIFNIQTI1NiAtIEczMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA tpZok2X9LAHsYqMNVL+Ly6RDkaKar7GD8rVtb9nw6tzPFnvXGeOEA4X5xh9wjx9sScVpGR5wkTg1 fgJIXTlrGESmaqXIdPRd9YQ+Yx9xRIIIPu3Jp/bpbiZBKYDJSbr/2Xago7sb9nnfSyjTSnucUcIP ZVChn6hKneVGBI2DT9yyyD3PmCEJmEzA8Y96qT83JmVH2GaPSSbCw0C+Zj1s/zqtKUbwE5zh8uuZ p4vC019QbaIOb8cGlzgvTqGORwK0gwDYpOO6QQdg5d03WvIHwTunnJdoLrfvqUg2vOlpqJmqR+nH 9lHS+bEstsVJtZieU1Pa+3LzfA/4cT7XA/pnwwIDAQABo4IBtTCCAbEwDgYDVR0PAQH/BAQDAgEG MGoGA1UdJQRjMGEGCCsGAQUFBwMCBggrBgEFBQcDBAYIKwYBBQUHAwkGCisGAQQBgjcUAgIGCisG AQQBgjcKAwQGCSsGAQQBgjcVBgYKKwYBBAGCNwoDDAYIKwYBBQUHAwcGCCsGAQUFBwMRMBIGA1Ud EwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFGlygmIxZ5VEhXeRgMQENkmdewthMB8GA1UdIwQYMBaA FI/wS3+oLkUkrk1Q+mOai97i3Ru8MD4GCCsGAQUFBwEBBDIwMDAuBggrBgEFBQcwAYYiaHR0cDov L29jc3AyLmdsb2JhbHNpZ24uY29tL3Jvb3RyMzA2BgNVHR8ELzAtMCugKaAnhiVodHRwOi8vY3Js Lmdsb2JhbHNpZ24uY29tL3Jvb3QtcjMuY3JsMGcGA1UdIARgMF4wCwYJKwYBBAGgMgEoMAwGCisG AQQBoDIBKAowQQYJKwYBBAGgMgFfMDQwMgYIKwYBBQUHAgEWJmh0dHBzOi8vd3d3Lmdsb2JhbHNp Z24uY29tL3JlcG9zaXRvcnkvMA0GCSqGSIb3DQEBCwUAA4IBAQConc0yzHxn4gtQ16VccKNm4iXv 6rS2UzBuhxI3XDPiwihW45O9RZXzWNgVcUzz5IKJFL7+pcxHvesGVII+5r++9eqI9XnEKCILjHr2 DgvjKq5Jmg6bwifybLYbVUoBthnhaFB0WLwSRRhPrt5eGxMw51UmNICi/hSKBKsHhGFSEaJQALZy 4HL0EWduE6ILYAjX6BSXRDtHFeUPddb46f5Hf5rzITGLsn9BIpoOVrgS878O4JnfUWQi29yBfn75 HajifFvPC+uqn+rcVnvrpLgsLOYG/64kWX/FRH8+mhVe+mcSX3xsUpcxK9q9vLTVtroU/yJUmEC4 OcH5dQsbHBqjMIIDXzCCAkegAwIBAgILBAAAAAABIVhTCKIwDQYJKoZIhvcNAQELBQAwTDEgMB4G A1UECxMXR2xvYmFsU2lnbiBSb290IENBIC0gUjMxEzARBgNVBAoTCkdsb2JhbFNpZ24xEzARBgNV BAMTCkdsb2JhbFNpZ24wHhcNMDkwMzE4MTAwMDAwWhcNMjkwMzE4MTAwMDAwWjBMMSAwHgYDVQQL ExdHbG9iYWxTaWduIFJvb3QgQ0EgLSBSMzETMBEGA1UEChMKR2xvYmFsU2lnbjETMBEGA1UEAxMK R2xvYmFsU2lnbjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMwldpB5BngiFvXAg7aE yiie/QV2EcWtiHL8RgJDx7KKnQRfJMsuS+FggkbhUqsMgUdwbN1k0ev1LKMPgj0MK66X17YUhhB5 uzsTgHeMCOFJ0mpiLx9e+pZo34knlTifBtc+ycsmWQ1z3rDI6SYOgxXG71uL0gRgykmmKPZpO/bL yCiR5Z2KYVc3rHQU3HTgOu5yLy6c+9C7v/U9AOEGM+iCK65TpjoWc4zdQQ4gOsC0p6Hpsk+QLjJg 6VfLuQSSaGjlOCZgdbKfd/+RFO+uIEn8rUAVSNECMWEZXriX7613t2Saer9fwRPvm2L7DWzgVGkW qQPabumDk3F2xmmFghcCAwEAAaNCMEAwDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8w HQYDVR0OBBYEFI/wS3+oLkUkrk1Q+mOai97i3Ru8MA0GCSqGSIb3DQEBCwUAA4IBAQBLQNvAUKr+ yAzv95ZURUm7lgAJQayzE4aGKAczymvmdLm6AC2upArT9fHxD4q/c2dKg8dEe3jgr25sbwMpjjM5 RcOO5LlXbKr8EpbsU8Yt5CRsuZRj+9xTaGdWPoO4zzUhw8lo/s7awlOqzJCK6fBdRoyV3XpYKBov Hd7NADdBj+1EbddTKJd+82cEHhXXipa0095MJ6RMG3NzdvQXmcIfeg7jLQitChws/zyrVQ4PkX42 68NXSb7hLi18YIvDQVETI53O9zJrlAGomecsMx86OyXShkDOOyyGeMlhLxS67ttVb9+E7gUJTb0o 2HLO02JQZR7rkpeDMdmztcpHWD9fMIIFQTCCBCmgAwIBAgIML7TfFWHfxluS5m0sMA0GCSqGSIb3 DQEBCwUAMF0xCzAJBgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMTMwMQYDVQQD EypHbG9iYWxTaWduIFBlcnNvbmFsU2lnbiAyIENBIC0gU0hBMjU2IC0gRzMwHhcNMjAwODMxMDgw OTQ5WhcNMjIwOTAxMDgwOTQ5WjCBjDELMAkGA1UEBhMCSU4xEjAQBgNVBAgTCUthcm5hdGFrYTES MBAGA1UEBxMJQmFuZ2Fsb3JlMRYwFAYDVQQKEw1Ccm9hZGNvbSBJbmMuMRQwEgYDVQQDEwtCb2Ig TWNNYWhvbjEnMCUGCSqGSIb3DQEJARYYYm9iLm1jbWFob25AYnJvYWRjb20uY29tMIIBIjANBgkq hkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzK1kauAQxAkeif97AJqv3QgJVDztfPjduswFJbcT0VbZ hr+E3gpqCnUm9TM4HZ2aviVSu9A9SkcpQBOaiOUPUkMrD6PoviodpwV2C5W+i7e/lexjsUdSuTZe h7GRKEOlhPnsW7RgKFR5+3Rrm1kAyEJ2x2ueyn3UhupxoNrYxZOss/+dPwQJ28kP7ICHBBJbcfmh doMU83sfJazBMsp+pMArApFdMwXOBL3dT4ZPkniaMilO+q2y7xQ212K+KlhaRWCWcImC/+pqZCyH /b1zx6vfBbdB/WD9+oi9rnXyd2ikVU2bvJ15VinB52kz7Kpj4A1e7zHgKxqNjFFycWXn2QIDAQAB o4IBzzCCAcswDgYDVR0PAQH/BAQDAgWgMIGeBggrBgEFBQcBAQSBkTCBjjBNBggrBgEFBQcwAoZB aHR0cDovL3NlY3VyZS5nbG9iYWxzaWduLmNvbS9jYWNlcnQvZ3NwZXJzb25hbHNpZ24yc2hhMmcz b2NzcC5jcnQwPQYIKwYBBQUHMAGGMWh0dHA6Ly9vY3NwMi5nbG9iYWxzaWduLmNvbS9nc3BlcnNv bmFsc2lnbjJzaGEyZzMwTQYDVR0gBEYwRDBCBgorBgEEAaAyASgKMDQwMgYIKwYBBQUHAgEWJmh0 dHBzOi8vd3d3Lmdsb2JhbHNpZ24uY29tL3JlcG9zaXRvcnkvMAkGA1UdEwQCMAAwRAYDVR0fBD0w OzA5oDegNYYzaHR0cDovL2NybC5nbG9iYWxzaWduLmNvbS9nc3BlcnNvbmFsc2lnbjJzaGEyZzMu Y3JsMCMGA1UdEQQcMBqBGGJvYi5tY21haG9uQGJyb2FkY29tLmNvbTATBgNVHSUEDDAKBggrBgEF BQcDBDAfBgNVHSMEGDAWgBRpcoJiMWeVRIV3kYDEBDZJnXsLYTAdBgNVHQ4EFgQUQ1s1Ocvee2Eh o0qY50aFmX9fHS0wDQYJKoZIhvcNAQELBQADggEBAFygcUiFPJ7uRXexxLYonpGCsztO0YcmuWCE jef/oS4oMYJyf+pv/dgdywaJ9U4FhYHg9wIPEcWwS1JpsUrEhx1zQ2JX8+HNHyuCiR11tgtzCr0z nxeMm3RAJPmM3cAy5mS7pDz1Ox081EqDQQzeesvMwwONjIpV1ORg17fpIHxz4KaKg8X5Yv6xYs0Z L+UVLQlmIJqoOHqzOelCjNTCSucYx/sYygzQcvUuX7qUgLOJq+o4JXhqu2dUJ6wu/NPApqyd8hSc YVBMvygG2Y3xBjxYC5Dr4+E1cmcOGEUQhkKo4cwv217+W4FXb4K4B/EM208EsUMXGTb7tV5zJ0rO IH4xggJvMIICawIBATBtMF0xCzAJBgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNh MTMwMQYDVQQDEypHbG9iYWxTaWduIFBlcnNvbmFsU2lnbiAyIENBIC0gU0hBMjU2IC0gRzMCDC+0 3xVh38ZbkuZtLDANBglghkgBZQMEAgEFAKCB1DAvBgkqhkiG9w0BCQQxIgQgzbp6/Xvmoj+XgcaI 02xwaPFqWH4F4UYMOWP9kbGVcL0wGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0B CQUxDxcNMjAxMjIzMDcyMzA4WjBpBgkqhkiG9w0BCQ8xXDBaMAsGCWCGSAFlAwQBKjALBglghkgB ZQMEARYwCwYJYIZIAWUDBAECMAoGCCqGSIb3DQMHMAsGCSqGSIb3DQEBCjALBgkqhkiG9w0BAQcw CwYJYIZIAWUDBAIBMA0GCSqGSIb3DQEBAQUABIIBAGiSTdlq3EG3dEpsgd/RJZL4PIaniilsNxIE JCDPR6DZsdBVJLrwxaEqaT7f9n4cff+JSAfCO0Nx3ralAHhrKPt1MYEm7rdozGjmrrN387i5zzEt VDr8qR7Uf7OboHCvvXV06gbX+eBREHcmDKyMjcY47ZrWGotSA+RDUVrn5NfTvsnIz+U1xp4p/JmO nuBGXJCoayctNrI9TMuhYMeCAgP/QKWUXkgkPyxaH10kl6Eqo/QMlGio4dtLXxww9D0zbH2DEISK Uu/C6B3KBhIuYfcmCOunAqHpoPs9kv0b2iIOQ/EmWYE8+90/4CRyx/v/YXzcikjhH6DN4bOKK6dE xmw= --0000000000001eec2505b71c90e1--