[LibreQoS] [Rpm] the grinch meets cloudflare's christmas present

rjmcmahon rjmcmahon at rjmcmahon.com
Fri Jan 6 15:38:32 EST 2023


Some thoughts are not to use UDP for testing here. Also, these speed 
tests have little to no information for network engineers about what's 
going on. Iperf 2 may better assist network engineers but then I'm 
biased ;)

Running iperf 2 https://sourceforge.net/projects/iperf2/ with 
--trip-times. Though the sampling and central limit theorem averaging is 
hiding the real distributions (use --histograms to get those)

Below are 4 parallel TCP streams from my home to one of my servers in 
the cloud. First where TCP is limited per CCA. Second is source side 
write rate limiting. Things to note:

o) connect times for both at 10-15 ms
o) multiple TCP retries on a few rites - one case is 4 with 5 writes. 
Source side pacing eliminates retries
o) Fairness with CCA isn't great but quite good with source side write 
pacing
o) Queue depth with CCA is about 150 Kbytes about 100K byte with source 
side pacing
o) min write to read is about 80 ms for both
o) max is 220 ms vs 97 ms
o) stdev for CCA write/read is 30 ms vs 3 ms
o) TCP RTT is 20ms w/CCA and 90 ms with ssp - seems odd here as 
TCP_QUICACK and TCP_NODELAY are both enabled.

[ CT] final connect times (min/avg/max/stdev) = 
10.326/13.522/14.986/2150.329 ms (tot/err) = 4/0
[rjmcmahon at ryzen3950 iperf2-code]$ iperf -c *** --hide-ips -e 
--trip-times -i 1 -P 4 -t 10 -w 4m --tcp-quickack -N
------------------------------------------------------------
Client connecting to (**hidden**), TCP port 5001 with pid 107678 (4 
flows)
Write buffer size: 131072 Byte
TOS set to 0x0 and nodelay (Nagle off)
TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte)
Event based writes (pending queue watermark at 16384 bytes)
------------------------------------------------------------
[  1] local *.*.*.85%enp4s0 port 42480 connected with *.*.*.123 port 
5001 (prefetch=16384) (trip-times) (sock=3) (qack) 
(icwnd/mss/irtt=14/1448/10534) (ct=10.63 ms) on 2023-01-06 12:17:56 
(PST)
[  4] local *.*.*.85%enp4s0 port 42488 connected with *.*.*.123 port 
5001 (prefetch=16384) (trip-times) (sock=5) (qack) 
(icwnd/mss/irtt=14/1448/14023) (ct=14.08 ms) on 2023-01-06 12:17:56 
(PST)
[  3] local *.*.*.85%enp4s0 port 42502 connected with *.*.*.123 port 
5001 (prefetch=16384) (trip-times) (sock=6) (qack) 
(icwnd/mss/irtt=14/1448/14642) (ct=14.70 ms) on 2023-01-06 12:17:56 
(PST)
[  2] local *.*.*.85%enp4s0 port 42484 connected with *.*.*.123 port 
5001 (prefetch=16384) (trip-times) (sock=4) (qack) 
(icwnd/mss/irtt=14/1448/14728) (ct=14.79 ms) on 2023-01-06 12:17:56 
(PST)
[ ID] Interval        Transfer    Bandwidth       Write/Err  Rtry     
Cwnd/RTT(var)        NetPwr
...
[  4] 4.00-5.00 sec  1.38 MBytes  11.5 Mbits/sec  11/0         3       
29K/21088(1142) us  68.37
[  2] 4.00-5.00 sec  1.62 MBytes  13.6 Mbits/sec  13/0         2       
31K/19284(612) us  88.36
[  1] 4.00-5.00 sec   896 KBytes  7.34 Mbits/sec  7/0         5       
16K/18996(658) us  48.30
[  3] 4.00-5.00 sec  1.00 MBytes  8.39 Mbits/sec  8/0         5       
18K/18133(208) us  57.83
[SUM] 4.00-5.00 sec  4.88 MBytes  40.9 Mbits/sec  39/0        15
[  4] 5.00-6.00 sec  1.25 MBytes  10.5 Mbits/sec  10/0         4       
29K/14717(489) us  89.06
[  1] 5.00-6.00 sec  1.00 MBytes  8.39 Mbits/sec  8/0         4       
16K/15874(408) us  66.06
[  3] 5.00-6.00 sec  1.12 MBytes  9.44 Mbits/sec  9/0         4       
16K/15826(382) us  74.54
[  2] 5.00-6.00 sec  1.50 MBytes  12.6 Mbits/sec  12/0         6        
9K/14878(557) us  106
[SUM] 5.00-6.00 sec  4.88 MBytes  40.9 Mbits/sec  39/0        18
[  4] 6.00-7.00 sec  1.75 MBytes  14.7 Mbits/sec  14/0         4       
25K/15472(496) us  119
[  2] 6.00-7.00 sec  1.00 MBytes  8.39 Mbits/sec  8/0         2       
26K/16417(427) us  63.87
[  1] 6.00-7.00 sec  1.25 MBytes  10.5 Mbits/sec  10/0         5       
16K/16268(679) us  80.57
[  3] 6.00-7.00 sec  1.00 MBytes  8.39 Mbits/sec  8/0         6       
15K/16629(799) us  63.06
[SUM] 6.00-7.00 sec  5.00 MBytes  41.9 Mbits/sec  40/0        17
[  4] 7.00-8.00 sec  1.75 MBytes  14.7 Mbits/sec  14/0         4       
22K/13986(519) us  131
[  1] 7.00-8.00 sec  1.12 MBytes  9.44 Mbits/sec  9/0         4       
16K/12679(377) us  93.04
[  3] 7.00-8.00 sec   896 KBytes  7.34 Mbits/sec  7/0         5       
14K/12971(367) us  70.74
[  2] 7.00-8.00 sec  1.12 MBytes  9.44 Mbits/sec  9/0         6       
15K/14740(779) us  80.03
[SUM] 7.00-8.00 sec  4.88 MBytes  40.9 Mbits/sec  39/0        19

[root at bobcat iperf2-code]# iperf -s -i 1 -e --hide-ips -w 4m
------------------------------------------------------------
Server listening on TCP port 5001 with pid 233615
Read buffer size:  128 KByte (Dist bin width=16.0 KByte)
TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte)
------------------------------------------------------------
[  1] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port 42480 
(trip-times) (sock=4) (peer 2.1.9-master) (qack) 
(icwnd/mss/irtt=14/1448/11636) on 2023-01-06 12:17:56 (PST)
[  2] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port 42502 
(trip-times) (sock=5) (peer 2.1.9-master) (qack) 
(icwnd/mss/irtt=14/1448/11898) on 2023-01-06 12:17:56 (PST)
[  3] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port 42484 
(trip-times) (sock=6) (peer 2.1.9-master) (qack) 
(icwnd/mss/irtt=14/1448/11938) on 2023-01-06 12:17:56 (PST)
[  4] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port 42488 
(trip-times) (sock=7) (peer 2.1.9-master) (qack) 
(icwnd/mss/irtt=14/1448/11919) on 2023-01-06 12:17:56 (PST)
[ ID] Interval        Transfer    Bandwidth    Burst Latency 
avg/min/max/stdev (cnt/size) inP NetPwr  Reads=Dist
...
[  2] 4.00-5.00 sec  1.06 MBytes  8.86 Mbits/sec  
129.819/90.391/186.075/31.346 ms (9/123080)  154 KByte 8.532803  
467=461:6:0:0:0:0:0:0
[  3] 4.00-5.00 sec  1.52 MBytes  12.8 Mbits/sec  
103.552/82.653/169.274/28.382 ms (12/132854)  149 KByte 15.40  
646=643:1:2:0:0:0:0:0
[  4] 4.00-5.00 sec  1.39 MBytes  11.6 Mbits/sec  
107.503/66.843/143.038/24.269 ms (11/132294)  149 KByte 13.54  
619=617:1:1:0:0:0:0:0
[  1] 4.00-5.00 sec   988 KBytes  8.10 Mbits/sec  
141.389/119.961/178.785/18.812 ms (7/144593)  170 KByte 7.158641  
409=404:5:0:0:0:0:0:0
[SUM] 4.00-5.00 sec  4.93 MBytes  41.4 Mbits/sec  
2141=2125:13:3:0:0:0:0:0
[  4] 5.00-6.00 sec  1.29 MBytes  10.8 Mbits/sec  
118.943/86.253/176.128/31.248 ms (10/135098)  164 KByte 11.36  
511=506:2:3:0:0:0:0:0
[  2] 5.00-6.00 sec  1.09 MBytes  9.17 Mbits/sec  
139.821/102.418/218.875/40.422 ms (9/127424)  148 KByte 8.202049  
487=484:2:1:0:0:0:0:0
[  3] 5.00-6.00 sec  1.51 MBytes  12.6 Mbits/sec  
102.146/77.085/140.893/18.441 ms (13/121520)  151 KByte 15.47  
640=636:1:3:0:0:0:0:0
[  1] 5.00-6.00 sec   981 KBytes  8.04 Mbits/sec  
161.901/105.582/219.931/36.260 ms (8/125614)  134 KByte 6.206944  
415=413:2:0:0:0:0:0:0
[SUM] 5.00-6.00 sec  4.85 MBytes  40.7 Mbits/sec  
2053=2039:7:7:0:0:0:0:0
[  4] 6.00-7.00 sec  1.74 MBytes  14.6 Mbits/sec  
88.846/74.297/101.859/7.118 ms (14/130526)  156 KByte 20.57  
711=707:3:1:0:0:0:0:0
[  1] 6.00-7.00 sec  1.22 MBytes  10.2 Mbits/sec  
120.639/100.257/157.567/21.770 ms (10/127568)  157 KByte 10.57  
494=488:5:1:0:0:0:0:0
[  2] 6.00-7.00 sec  1015 KBytes  8.32 Mbits/sec  
144.632/124.368/171.349/16.597 ms (8/129958)  143 KByte 7.188321  
408=403:5:0:0:0:0:0:0
[  3] 6.00-7.00 sec  1.02 MBytes  8.60 Mbits/sec  
143.516/102.322/173.001/24.089 ms (8/134302)  146 KByte 7.486359  
484=480:4:0:0:0:0:0:0
[SUM] 6.00-7.00 sec  4.98 MBytes  41.7 Mbits/sec  
2097=2078:17:2:0:0:0:0:0
[  4] 7.00-8.00 sec  1.77 MBytes  14.9 Mbits/sec  
85.406/65.797/103.418/12.609 ms (14/132595)  153 KByte 21.74  
692=687:2:3:0:0:0:0:0
[  2] 7.00-8.00 sec   957 KBytes  7.84 Mbits/sec  
153.936/131.452/191.464/19.361 ms (7/140042)  160 KByte 6.368199  
429=425:4:0:0:0:0:0:0
[  1] 7.00-8.00 sec  1.13 MBytes  9.44 Mbits/sec  
131.146/109.737/166.774/22.035 ms (9/131124)  146 KByte 8.998528  
520=516:4:0:0:0:0:0:0
[  3] 7.00-8.00 sec  1.13 MBytes  9.51 Mbits/sec  
126.512/88.404/220.175/42.237 ms (9/132089)  172 KByte 9.396784  
527=524:1:2:0:0:0:0:0
[SUM] 7.00-8.00 sec  4.96 MBytes  41.6 Mbits/sec  
2168=2152:11:5:0:0:0:0:0

With source side rate limiting to 9 mb/s per stream.

[rjmcmahon at ryzen3950 iperf2-code]$ iperf -c *** --hide-ips -e 
--trip-times -i 1 -P 4 -t 10 -w 4m --tcp-quickack -N -b 9m
------------------------------------------------------------
Client connecting to (**hidden**), TCP port 5001 with pid 108884 (4 
flows)
Write buffer size: 131072 Byte
TOS set to 0x0 and nodelay (Nagle off)
TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte)
Event based writes (pending queue watermark at 16384 bytes)
------------------------------------------------------------
[  1] local *.*.*.85%enp4s0 port 46448 connected with *.*.*.123 port 
5001 (prefetch=16384) (trip-times) (sock=3) (qack) 
(icwnd/mss/irtt=14/1448/10666) (ct=10.70 ms) on 2023-01-06 12:27:45 
(PST)
[  3] local *.*.*.85%enp4s0 port 46460 connected with *.*.*.123 port 
5001 (prefetch=16384) (trip-times) (sock=6) (qack) 
(icwnd/mss/irtt=14/1448/16499) (ct=16.54 ms) on 2023-01-06 12:27:45 
(PST)
[  2] local *.*.*.85%enp4s0 port 46454 connected with *.*.*.123 port 
5001 (prefetch=16384) (trip-times) (sock=4) (qack) 
(icwnd/mss/irtt=14/1448/16580) (ct=16.86 ms) on 2023-01-06 12:27:45 
(PST)
[  4] local *.*.*.85%enp4s0 port 46458 connected with *.*.*.123 port 
5001 (prefetch=16384) (trip-times) (sock=5) (qack) 
(icwnd/mss/irtt=14/1448/16802) (ct=16.83 ms) on 2023-01-06 12:27:45 
(PST)
[ ID] Interval        Transfer    Bandwidth       Write/Err  Rtry     
Cwnd/RTT(var)        NetPwr
...
[  2] 4.00-5.00 sec  1.00 MBytes  8.39 Mbits/sec  8/0         0      
134K/88055(12329) us  11.91
[  4] 4.00-5.00 sec  1.00 MBytes  8.39 Mbits/sec  8/0         0      
132K/74867(11755) us  14.01
[  1] 4.00-5.00 sec  1.00 MBytes  8.39 Mbits/sec  8/0         0      
134K/89101(13134) us  11.77
[  3] 4.00-5.00 sec  1.00 MBytes  8.39 Mbits/sec  8/0         0      
131K/91451(11938) us  11.47
[SUM] 4.00-5.00 sec  4.00 MBytes  33.6 Mbits/sec  32/0         0
[  2] 5.00-6.00 sec  1.12 MBytes  9.44 Mbits/sec  9/0         0      
134K/85135(14580) us  13.86
[  4] 5.00-6.00 sec  1.12 MBytes  9.44 Mbits/sec  9/0         0      
132K/85124(15654) us  13.86
[  1] 5.00-6.00 sec  1.12 MBytes  9.44 Mbits/sec  9/0         0      
134K/91336(11335) us  12.92
[  3] 5.00-6.00 sec  1.12 MBytes  9.44 Mbits/sec  9/0         0      
131K/89185(13499) us  13.23
[SUM] 5.00-6.00 sec  4.50 MBytes  37.7 Mbits/sec  36/0         0
[  2] 6.00-7.00 sec  1.12 MBytes  9.44 Mbits/sec  9/0         0      
134K/85687(13489) us  13.77
[  4] 6.00-7.00 sec  1.12 MBytes  9.44 Mbits/sec  9/0         0      
132K/82803(13001) us  14.25
[  1] 6.00-7.00 sec  1.12 MBytes  9.44 Mbits/sec  9/0         0      
134K/86869(15186) us  13.58
[  3] 6.00-7.00 sec  1.12 MBytes  9.44 Mbits/sec  9/0         0      
131K/91447(12515) us  12.90
[SUM] 6.00-7.00 sec  4.50 MBytes  37.7 Mbits/sec  36/0         0
[  2] 7.00-8.00 sec  1.00 MBytes  8.39 Mbits/sec  8/0         0      
134K/81814(13168) us  12.82
[  4] 7.00-8.00 sec  1.00 MBytes  8.39 Mbits/sec  8/0         0      
132K/89008(13283) us  11.78
[  1] 7.00-8.00 sec  1.00 MBytes  8.39 Mbits/sec  8/0         0      
134K/89494(12151) us  11.72
[  3] 7.00-8.00 sec  1.00 MBytes  8.39 Mbits/sec  8/0         0      
131K/91083(12797) us  11.51
[SUM] 7.00-8.00 sec  4.00 MBytes  33.6 Mbits/sec  32/0         0

[root at bobcat iperf2-code]# iperf -s -i 1 -e --hide-ips -w 4m
------------------------------------------------------------
Server listening on TCP port 5001 with pid 233981
Read buffer size:  128 KByte (Dist bin width=16.0 KByte)
TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte)
------------------------------------------------------------
[  1] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port 46448 
(trip-times) (sock=4) (peer 2.1.9-master) (qack) 
(icwnd/mss/irtt=14/1448/11987) on 2023-01-06 12:27:45 (PST)
[  2] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port 46454 
(trip-times) (sock=5) (peer 2.1.9-master) (qack) 
(icwnd/mss/irtt=14/1448/11132) on 2023-01-06 12:27:45 (PST)
[  3] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port 46460 
(trip-times) (sock=6) (peer 2.1.9-master) (qack) 
(icwnd/mss/irtt=14/1448/11097) on 2023-01-06 12:27:45 (PST)
[  4] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port 46458 
(trip-times) (sock=7) (peer 2.1.9-master) (qack) 
(icwnd/mss/irtt=14/1448/17823) on 2023-01-06 12:27:45 (PST)
[ ID] Interval        Transfer    Bandwidth    Burst Latency 
avg/min/max/stdev (cnt/size) inP NetPwr  Reads=Dist
[  4] 0.00-1.00 sec  1.09 MBytes  9.15 Mbits/sec  
93.383/90.103/95.661/2.232 ms (8/143028)  128 KByte 12.25  
451=451:0:0:0:0:0:0:0
[  3] 0.00-1.00 sec  1.08 MBytes  9.06 Mbits/sec  
96.834/95.229/102.645/2.442 ms (8/141580)  131 KByte 11.70  
472=472:0:0:0:0:0:0:0
[  1] 0.00-1.00 sec  1.10 MBytes  9.19 Mbits/sec  
95.183/92.623/97.579/1.431 ms (8/143571)  131 KByte 12.07  
495=495:0:0:0:0:0:0:0
[  2] 0.00-1.00 sec  1.09 MBytes  9.15 Mbits/sec  
89.317/84.865/94.906/3.674 ms (8/143028)  122 KByte 12.81  
489=489:0:0:0:0:0:0:0
[ ID] Interval        Transfer    Bandwidth       Reads=Dist
[SUM] 0.00-1.00 sec  4.36 MBytes  36.6 Mbits/sec  
1907=1907:0:0:0:0:0:0:0
[  4] 1.00-2.00 sec  1.07 MBytes  8.95 Mbits/sec  
92.649/89.987/95.036/1.828 ms (9/124314) 96.5 KByte 12.08  
492=492:0:0:0:0:0:0:0
[  3] 1.00-2.00 sec  1.06 MBytes  8.93 Mbits/sec  
96.305/95.647/96.794/0.432 ms (9/123992)  100 KByte 11.59  
480=480:0:0:0:0:0:0:0
[  1] 1.00-2.00 sec  1.06 MBytes  8.89 Mbits/sec  
92.578/90.866/94.145/1.371 ms (9/123510) 95.8 KByte 12.01  
513=513:0:0:0:0:0:0:0
[  2] 1.00-2.00 sec  1.07 MBytes  8.96 Mbits/sec  
90.767/87.984/94.352/1.944 ms (9/124475) 94.7 KByte 12.34  
489=489:0:0:0:0:0:0:0
[SUM] 1.00-2.00 sec  4.26 MBytes  35.7 Mbits/sec  
1974=1974:0:0:0:0:0:0:0
[  4] 2.00-3.00 sec  1.09 MBytes  9.13 Mbits/sec  
93.977/91.795/96.561/1.693 ms (8/142656)  112 KByte 12.14  
497=497:0:0:0:0:0:0:0
[  3] 2.00-3.00 sec  1.08 MBytes  9.04 Mbits/sec  
96.544/95.815/97.798/0.693 ms (8/141208)  114 KByte 11.70  
503=503:0:0:0:0:0:0:0
[  1] 2.00-3.00 sec  1.07 MBytes  9.01 Mbits/sec  
93.970/91.193/96.325/1.796 ms (8/140846)  111 KByte 11.99  
509=509:0:0:0:0:0:0:0
[  2] 2.00-3.00 sec  1.08 MBytes  9.10 Mbits/sec  
92.843/90.216/96.355/2.040 ms (8/142113)  111 KByte 12.25  
509=509:0:0:0:0:0:0:0
[SUM] 2.00-3.00 sec  4.32 MBytes  36.3 Mbits/sec  
2018=2018:0:0:0:0:0:0:0
[  4] 3.00-4.00 sec  1.06 MBytes  8.86 Mbits/sec  
93.222/89.063/96.104/2.346 ms (9/123027) 96.1 KByte 11.88  
487=487:0:0:0:0:0:0:0
[  3] 3.00-4.00 sec  1.07 MBytes  8.97 Mbits/sec  
96.277/95.051/97.230/0.767 ms (9/124636)  101 KByte 11.65  
489=489:0:0:0:0:0:0:0
[  1] 3.00-4.00 sec  1.08 MBytes  9.02 Mbits/sec  
93.899/88.732/96.972/2.737 ms (9/125280) 98.6 KByte 12.01  
493=493:0:0:0:0:0:0:0
[  2] 3.00-4.00 sec  1.07 MBytes  8.97 Mbits/sec  
92.490/89.862/95.265/1.796 ms (9/124636) 96.6 KByte 12.13  
493=493:0:0:0:0:0:0:0
[SUM] 3.00-4.00 sec  4.27 MBytes  35.8 Mbits/sec  
1962=1962:0:0:0:0:0:0:0
[  4] 4.00-5.00 sec  1.07 MBytes  9.00 Mbits/sec  
92.431/81.888/96.221/4.524 ms (9/124958) 96.8 KByte 12.17  
498=498:0:0:0:0:0:0:0
[  1] 4.00-5.00 sec  1.07 MBytes  8.97 Mbits/sec  
95.018/93.445/96.200/0.957 ms (9/124636) 99.3 KByte 11.81  
490=490:0:0:0:0:0:0:0
[  2] 4.00-5.00 sec  1.06 MBytes  8.90 Mbits/sec  
93.874/86.485/95.672/2.810 ms (9/123671) 97.3 KByte 11.86  
481=481:0:0:0:0:0:0:0
[  3] 4.00-5.00 sec  1.08 MBytes  9.09 Mbits/sec  
95.737/93.881/97.197/0.972 ms (9/126245)  101 KByte 11.87  
484=484:0:0:0:0:0:0:0
[SUM] 4.00-5.00 sec  4.29 MBytes  36.0 Mbits/sec  
1953=1953:0:0:0:0:0:0:0
[  4] 5.00-6.00 sec  1.09 MBytes  9.13 Mbits/sec  
92.908/86.844/95.994/3.012 ms (8/142656)  111 KByte 12.28  
467=467:0:0:0:0:0:0:0
[  3] 5.00-6.00 sec  1.07 MBytes  8.94 Mbits/sec  
96.593/95.343/97.660/0.876 ms (8/139760)  113 KByte 11.58  
478=478:0:0:0:0:0:0:0
[  1] 5.00-6.00 sec  1.08 MBytes  9.03 Mbits/sec  
95.021/91.421/97.167/1.893 ms (8/141027)  112 KByte 11.87  
491=491:0:0:0:0:0:0:0
[  2] 5.00-6.00 sec  1.08 MBytes  9.06 Mbits/sec  
92.162/82.720/97.692/5.060 ms (8/141570)  109 KByte 12.29  
488=488:0:0:0:0:0:0:0
[SUM] 5.00-6.00 sec  4.31 MBytes  36.2 Mbits/sec  
1924=1924:0:0:0:0:0:0:0
[  4] 6.00-7.00 sec  1.04 MBytes  8.70 Mbits/sec  
92.793/85.343/96.967/3.552 ms (9/120775) 93.9 KByte 11.71  
485=485:0:0:0:0:0:0:0
[  2] 6.00-7.00 sec  1.05 MBytes  8.79 Mbits/sec  
91.679/84.479/96.760/3.975 ms (9/122062) 93.8 KByte 11.98  
472=472:0:0:0:0:0:0:0
[  3] 6.00-7.00 sec  1.06 MBytes  8.88 Mbits/sec  
96.982/95.933/98.371/0.680 ms (9/123349)  100 KByte 11.45  
477=477:0:0:0:0:0:0:0
[  1] 6.00-7.00 sec  1.05 MBytes  8.80 Mbits/sec  
94.342/91.660/96.025/1.660 ms (9/122223) 96.7 KByte 11.66  
494=494:0:0:0:0:0:0:0
[SUM] 6.00-7.00 sec  4.19 MBytes  35.2 Mbits/sec  
1928=1928:0:0:0:0:0:0:0
[  4] 7.00-8.00 sec  1.10 MBytes  9.25 Mbits/sec  
92.515/88.182/96.351/2.538 ms (8/144466)  112 KByte 12.49  
510=510:0:0:0:0:0:0:0
[  3] 7.00-8.00 sec  1.09 MBytes  9.13 Mbits/sec  
96.580/95.737/98.977/1.098 ms (8/142656)  115 KByte 11.82  
480=480:0:0:0:0:0:0:0
[  1] 7.00-8.00 sec  1.10 MBytes  9.21 Mbits/sec  
95.269/91.719/97.514/2.126 ms (8/143923)  115 KByte 12.09  
515=515:0:0:0:0:0:0:0
[  2] 7.00-8.00 sec  1.11 MBytes  9.29 Mbits/sec  
90.073/84.700/96.176/4.324 ms (8/145190)  110 KByte 12.90  
508=508:0:0:0:0:0:0:0
[SUM] 7.00-8.00 sec  4.40 MBytes  36.9 Mbits/sec  
2013=2013:0:0:0:0:0:0:0

Bob

>> -----Original Message-----
> 
>> From: LibreQoS <libreqos-bounces at lists.bufferbloat.net> On Behalf Of
> Dave Taht
> 
>> via LibreQoS
> 
>> Sent: Wednesday, January 4, 2023 12:26 PM
> 
>> Subject: [LibreQoS] the grinch meets cloudflare's christmas present
> 
>> 
> 
>> Please try the new, the shiny, the really wonderful test here:
> 
>> 
> https://urldefense.com/v3/__https://speed.cloudflare.com/__;!!BhdT!iZcFJ8WVU9S
> [1]
> 
>> 
> 9zz5t456oxkfObrC5Xb9j5AG8UO3DqD5x4GAJkawZr0iGwEUtF0_09U8mCDnAkrJ9QEMHGbCMKVw$
> [1]
> 
>> 
> 
>> I would really appreciate some independent verification of
> 
>> measurements using this tool. In my brief experiments it appears -
> as
> 
>> all the commercial tools to date - to dramatically understate the
> 
>> bufferbloat, on my LTE, (and my starlink terminal is out being
> 
>> hacked^H^H^H^H^H^Hworked on, so I can't measure that)
> 
> [acm]
> 
> Hi Dave, I made some time to test "cloudflare's christmas present"
> yesterday.
> 
> I'm on DOCSIS 3.1 service with 1Gbps Down. The Upstream has a "turbo"
> mode with 40-50Mbps for the first ~3 sec, then steady-state about
> 23Mbps.
> 
> When I saw the ~620Mbps Downstream measurement, I was ready to
> complain that even the IP-Layer Capacity was grossly underestimated.
> In addition, the Latency measurements seem very low (as you asserted),
> although the cloud server was “nearby”.
> 
> However, I found that Ookla and the ISP-provided measurement were also
> reporting ~600Mbps! So the cloudflare Downstream capacity (or
> throughput?) measurement was consistent with others. Our UDPST server
> was unreachable, otherwise I would have added that measurement, too.
> 
> The Upstream measurement graph seems to illustrate the “turbo”
> mode, with the dip after attaining 44.5Mbps.
> 
> UDPST saturates the uplink and we measure the full 250ms of the
> Upstream buffer. Cloudflare’s latency measurements don’t even come
> close.
> 
> Al
> 
> 
> 
> Links:
> ------
> [1] 
> https://urldefense.com/v3/__https:/speed.cloudflare.com/__;!!BhdT!iZcFJ8WVU9S9zz5t456oxkfObrC5Xb9j5AG8UO3DqD5x4GAJkawZr0iGwEUtF0_09U8mCDnAkrJ9QEMHGbCMKVw$
> _______________________________________________
> Rpm mailing list
> Rpm at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/rpm


More information about the LibreQoS mailing list