[Cake] [Bloat] [Starlink] [Rpm] [LibreQoS] the grinch meets cloudflare'schristmas present
Luis A. Cornejo
luis.a.cornejo at gmail.com
Tue Jan 10 12:25:03 EST 2023
Here is my VZ HSI
No SQMm on
On Sat, Jan 7, 2023 at 6:38 PM Dick Roy via Bloat <
bloat at lists.bufferbloat.net> wrote:
>
>
>
>
> -----Original Message-----
> From: rjmcmahon [mailto:rjmcmahon at rjmcmahon.com]
> Sent: Friday, January 6, 2023 3:45 PM
> To: dickroy at alum.mit.edu
> Cc: 'MORTON JR., AL'; 'IETF IPPM WG'; 'libreqos'; 'Cake List'; 'Rpm';
> 'bloat'
> Subject: Re: [Starlink] [Rpm] [LibreQoS] the grinch meets
> cloudflare'schristmas present
>
>
>
> yeah, I'd prefer not to output CLT sample groups at all but the
>
> histograms aren't really human readable and users constantly ask for
>
> them. I thought about providing a distance from the gaussian as output
>
> too but so far few would understand it and nobody I found would act upon
>
> it.
>
> *[RR] Understandable until such metrics are “actionable”, and that’s “up
> to us to find/define/figure out” it seems to me. Metrics that are not
> actionable are write-only memory and good for little but historical record**J
> *
>
> The tool produces the full histograms so no information is really
>
> missing except for maybe better time series analysis.
>
> *[RR] Isn’t that in fact what we are trying to extract from the e2e stats
> we collect? i.e., infer the time evolution of the system from its I/O
> behavior? As you point out, it’s really hard to do without probes in the
> guts of the system, nd yes, synchronization is important **J*
>
>
>
> The open source flows python code also released with iperf 2 does use
>
> the komogorov-smirnov distances & distance matrices to cluster when the
>
> number of histograms are just too much. We've analyzed 1M runs to fault
>
> isolate the "unexpected interruptions" or "bugs" and without statistical
>
> support it is just not doable. This does require instrumentation of the
>
> full path with mapping to a common clock domain (e.g. GPS) and not just
>
> e2e stats. I find an e2e complaint by an end user about "poor speed" as
>
> useful as telling a pharmacist I have a fever. Not much diagnostically
>
> is going on. Take an aspirin.
>
> *[RR] That’s AWESOME!!!!!!!!!!!!!!!!!! I love that analogy!*
>
>
>
> *RR*
>
>
>
> https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test
>
> https://sourceforge.net/p/iperf2/code/ci/master/tree/flows/flows.py
>
>
>
> Bob
>
> > See below …
>
> >
>
> > -----Original Message-----
>
> > From: Starlink [mailto:starlink-bounces at lists.bufferbloat.net] On
>
> > Behalf Of rjmcmahon via Starlink
>
> > Sent: Friday, January 6, 2023 12:39 PM
>
> > To: MORTON JR., AL
>
> > Cc: Dave Taht via Starlink; IETF IPPM WG; libreqos; Cake List; Rpm;
>
> > bloat
>
> > Subject: Re: [Starlink] [Rpm] [LibreQoS] the grinch meets
>
> > cloudflare'schristmas present
>
> >
>
> > Some thoughts are not to use UDP for testing here. Also, these speed
>
> >
>
> > tests have little to no information for network engineers about what's
>
> >
>
> >
>
> > going on. Iperf 2 may better assist network engineers but then I'm
>
> >
>
> > biased ;)
>
> >
>
> > Running iperf 2 https://sourceforge.net/projects/iperf2/ with
>
> >
>
> > --trip-times. Though the sampling and central limit theorem averaging
>
> > is
>
> >
>
> > hiding the real distributions (use --histograms to get those)
>
> >
>
> > _[RR] FWIW (IMNBWM __J)… If the output/final histograms indicate the
>
> > PDF is NOT Gaussian, then any application of the CLT is
>
> > inappropriate/contra-indicated! The CLT is a "proof under certain
>
> > regularity conditions/assumptions of underlying/constituent PDFs, that
>
> > the resulting PDF (after all the necessary convolutions are performed
>
> > to get to the PDF of the output) will asymptotically approach a
>
> > Gaussian with only a mean and a std. dev. left to specify. _
>
> >
>
> > Below are 4 parallel TCP streams from my home to one of my servers in
>
> >
>
> > the cloud. First where TCP is limited per CCA. Second is source side
>
> >
>
> > write rate limiting. Things to note:
>
> >
>
> > o) connect times for both at 10-15 ms
>
> >
>
> > o) multiple TCP retries on a few rites - one case is 4 with 5 writes.
>
> >
>
> > Source side pacing eliminates retries
>
> >
>
> > o) Fairness with CCA isn't great but quite good with source side write
>
> >
>
> >
>
> > pacing
>
> >
>
> > o) Queue depth with CCA is about 150 Kbytes about 100K byte with
>
> > source
>
> >
>
> > side pacing
>
> >
>
> > o) min write to read is about 80 ms for both
>
> >
>
> > o) max is 220 ms vs 97 ms
>
> >
>
> > o) stdev for CCA write/read is 30 ms vs 3 ms
>
> >
>
> > o) TCP RTT is 20ms w/CCA and 90 ms with ssp - seems odd here as
>
> >
>
> > TCP_QUICACK and TCP_NODELAY are both enabled.
>
> >
>
> > [ CT] final connect times (min/avg/max/stdev) =
>
> >
>
> > 10.326/13.522/14.986/2150.329 ms (tot/err) = 4/0
>
> >
>
> > [rjmcmahon at ryzen3950 iperf2-code]$ iperf -c *** --hide-ips -e
>
> >
>
> > --trip-times -i 1 -P 4 -t 10 -w 4m --tcp-quickack -N
>
> >
>
> > ------------------------------------------------------------
>
> >
>
> > Client connecting to (**hidden**), TCP port 5001 with pid 107678 (4
>
> >
>
> > flows)
>
> >
>
> > Write buffer size: 131072 Byte
>
> >
>
> > TOS set to 0x0 and nodelay (Nagle off)
>
> >
>
> > TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte)
>
> >
>
> > Event based writes (pending queue watermark at 16384 bytes)
>
> >
>
> > ------------------------------------------------------------
>
> >
>
> > [ 1] local *.*.*.85%enp4s0 port 42480 connected with *.*.*.123 port
>
> >
>
> > 5001 (prefetch=16384) (trip-times) (sock=3) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/10534) (ct=10.63 ms) on 2023-01-06 12:17:56
>
> >
>
> > (PST)
>
> >
>
> > [ 4] local *.*.*.85%enp4s0 port 42488 connected with *.*.*.123 port
>
> >
>
> > 5001 (prefetch=16384) (trip-times) (sock=5) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/14023) (ct=14.08 ms) on 2023-01-06 12:17:56
>
> >
>
> > (PST)
>
> >
>
> > [ 3] local *.*.*.85%enp4s0 port 42502 connected with *.*.*.123 port
>
> >
>
> > 5001 (prefetch=16384) (trip-times) (sock=6) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/14642) (ct=14.70 ms) on 2023-01-06 12:17:56
>
> >
>
> > (PST)
>
> >
>
> > [ 2] local *.*.*.85%enp4s0 port 42484 connected with *.*.*.123 port
>
> >
>
> > 5001 (prefetch=16384) (trip-times) (sock=4) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/14728) (ct=14.79 ms) on 2023-01-06 12:17:56
>
> >
>
> > (PST)
>
> >
>
> > [ ID] Interval Transfer Bandwidth Write/Err Rtry
>
> >
>
> > Cwnd/RTT(var) NetPwr
>
> >
>
> > ...
>
> >
>
> > [ 4] 4.00-5.00 sec 1.38 MBytes 11.5 Mbits/sec 11/0 3
>
> >
>
> >
>
> > 29K/21088(1142) us 68.37
>
> >
>
> > [ 2] 4.00-5.00 sec 1.62 MBytes 13.6 Mbits/sec 13/0 2
>
> >
>
> >
>
> > 31K/19284(612) us 88.36
>
> >
>
> > [ 1] 4.00-5.00 sec 896 KBytes 7.34 Mbits/sec 7/0 5
>
> >
>
> > 16K/18996(658) us 48.30
>
> >
>
> > [ 3] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 5
>
> >
>
> > 18K/18133(208) us 57.83
>
> >
>
> > [SUM] 4.00-5.00 sec 4.88 MBytes 40.9 Mbits/sec 39/0 15
>
> >
>
> > [ 4] 5.00-6.00 sec 1.25 MBytes 10.5 Mbits/sec 10/0 4
>
> >
>
> >
>
> > 29K/14717(489) us 89.06
>
> >
>
> > [ 1] 5.00-6.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 4
>
> >
>
> > 16K/15874(408) us 66.06
>
> >
>
> > [ 3] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 4
>
> >
>
> > 16K/15826(382) us 74.54
>
> >
>
> > [ 2] 5.00-6.00 sec 1.50 MBytes 12.6 Mbits/sec 12/0 6
>
> >
>
> >
>
> > 9K/14878(557) us 106
>
> >
>
> > [SUM] 5.00-6.00 sec 4.88 MBytes 40.9 Mbits/sec 39/0 18
>
> >
>
> > [ 4] 6.00-7.00 sec 1.75 MBytes 14.7 Mbits/sec 14/0 4
>
> >
>
> >
>
> > 25K/15472(496) us 119
>
> >
>
> > [ 2] 6.00-7.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 2
>
> >
>
> > 26K/16417(427) us 63.87
>
> >
>
> > [ 1] 6.00-7.00 sec 1.25 MBytes 10.5 Mbits/sec 10/0 5
>
> >
>
> >
>
> > 16K/16268(679) us 80.57
>
> >
>
> > [ 3] 6.00-7.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 6
>
> >
>
> > 15K/16629(799) us 63.06
>
> >
>
> > [SUM] 6.00-7.00 sec 5.00 MBytes 41.9 Mbits/sec 40/0 17
>
> >
>
> > [ 4] 7.00-8.00 sec 1.75 MBytes 14.7 Mbits/sec 14/0 4
>
> >
>
> >
>
> > 22K/13986(519) us 131
>
> >
>
> > [ 1] 7.00-8.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 4
>
> >
>
> > 16K/12679(377) us 93.04
>
> >
>
> > [ 3] 7.00-8.00 sec 896 KBytes 7.34 Mbits/sec 7/0 5
>
> >
>
> > 14K/12971(367) us 70.74
>
> >
>
> > [ 2] 7.00-8.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 6
>
> >
>
> > 15K/14740(779) us 80.03
>
> >
>
> > [SUM] 7.00-8.00 sec 4.88 MBytes 40.9 Mbits/sec 39/0 19
>
> >
>
> > [root at bobcat iperf2-code]# iperf -s -i 1 -e --hide-ips -w 4m
>
> >
>
> > ------------------------------------------------------------
>
> >
>
> > Server listening on TCP port 5001 with pid 233615
>
> >
>
> > Read buffer size: 128 KByte (Dist bin width=16.0 KByte)
>
> >
>
> > TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte)
>
> >
>
> > ------------------------------------------------------------
>
> >
>
> > [ 1] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
>
> > 42480
>
> >
>
> > (trip-times) (sock=4) (peer 2.1.9-master) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/11636) on 2023-01-06 12:17:56 (PST)
>
> >
>
> > [ 2] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
>
> > 42502
>
> >
>
> > (trip-times) (sock=5) (peer 2.1.9-master) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/11898) on 2023-01-06 12:17:56 (PST)
>
> >
>
> > [ 3] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
>
> > 42484
>
> >
>
> > (trip-times) (sock=6) (peer 2.1.9-master) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/11938) on 2023-01-06 12:17:56 (PST)
>
> >
>
> > [ 4] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
>
> > 42488
>
> >
>
> > (trip-times) (sock=7) (peer 2.1.9-master) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/11919) on 2023-01-06 12:17:56 (PST)
>
> >
>
> > [ ID] Interval Transfer Bandwidth Burst Latency
>
> >
>
> > avg/min/max/stdev (cnt/size) inP NetPwr Reads=Dist
>
> >
>
> > ...
>
> >
>
> > [ 2] 4.00-5.00 sec 1.06 MBytes 8.86 Mbits/sec
>
> >
>
> > 129.819/90.391/186.075/31.346 ms (9/123080) 154 KByte 8.532803
>
> >
>
> > 467=461:6:0:0:0:0:0:0
>
> >
>
> > [ 3] 4.00-5.00 sec 1.52 MBytes 12.8 Mbits/sec
>
> >
>
> > 103.552/82.653/169.274/28.382 ms (12/132854) 149 KByte 15.40
>
> >
>
> > 646=643:1:2:0:0:0:0:0
>
> >
>
> > [ 4] 4.00-5.00 sec 1.39 MBytes 11.6 Mbits/sec
>
> >
>
> > 107.503/66.843/143.038/24.269 ms (11/132294) 149 KByte 13.54
>
> >
>
> > 619=617:1:1:0:0:0:0:0
>
> >
>
> > [ 1] 4.00-5.00 sec 988 KBytes 8.10 Mbits/sec
>
> >
>
> > 141.389/119.961/178.785/18.812 ms (7/144593) 170 KByte 7.158641
>
> >
>
> > 409=404:5:0:0:0:0:0:0
>
> >
>
> > [SUM] 4.00-5.00 sec 4.93 MBytes 41.4 Mbits/sec
>
> >
>
> > 2141=2125:13:3:0:0:0:0:0
>
> >
>
> > [ 4] 5.00-6.00 sec 1.29 MBytes 10.8 Mbits/sec
>
> >
>
> > 118.943/86.253/176.128/31.248 ms (10/135098) 164 KByte 11.36
>
> >
>
> > 511=506:2:3:0:0:0:0:0
>
> >
>
> > [ 2] 5.00-6.00 sec 1.09 MBytes 9.17 Mbits/sec
>
> >
>
> > 139.821/102.418/218.875/40.422 ms (9/127424) 148 KByte 8.202049
>
> >
>
> > 487=484:2:1:0:0:0:0:0
>
> >
>
> > [ 3] 5.00-6.00 sec 1.51 MBytes 12.6 Mbits/sec
>
> >
>
> > 102.146/77.085/140.893/18.441 ms (13/121520) 151 KByte 15.47
>
> >
>
> > 640=636:1:3:0:0:0:0:0
>
> >
>
> > [ 1] 5.00-6.00 sec 981 KBytes 8.04 Mbits/sec
>
> >
>
> > 161.901/105.582/219.931/36.260 ms (8/125614) 134 KByte 6.206944
>
> >
>
> > 415=413:2:0:0:0:0:0:0
>
> >
>
> > [SUM] 5.00-6.00 sec 4.85 MBytes 40.7 Mbits/sec
>
> >
>
> > 2053=2039:7:7:0:0:0:0:0
>
> >
>
> > [ 4] 6.00-7.00 sec 1.74 MBytes 14.6 Mbits/sec
>
> >
>
> > 88.846/74.297/101.859/7.118 ms (14/130526) 156 KByte 20.57
>
> >
>
> > 711=707:3:1:0:0:0:0:0
>
> >
>
> > [ 1] 6.00-7.00 sec 1.22 MBytes 10.2 Mbits/sec
>
> >
>
> > 120.639/100.257/157.567/21.770 ms (10/127568) 157 KByte 10.57
>
> >
>
> > 494=488:5:1:0:0:0:0:0
>
> >
>
> > [ 2] 6.00-7.00 sec 1015 KBytes 8.32 Mbits/sec
>
> >
>
> > 144.632/124.368/171.349/16.597 ms (8/129958) 143 KByte 7.188321
>
> >
>
> > 408=403:5:0:0:0:0:0:0
>
> >
>
> > [ 3] 6.00-7.00 sec 1.02 MBytes 8.60 Mbits/sec
>
> >
>
> > 143.516/102.322/173.001/24.089 ms (8/134302) 146 KByte 7.486359
>
> >
>
> > 484=480:4:0:0:0:0:0:0
>
> >
>
> > [SUM] 6.00-7.00 sec 4.98 MBytes 41.7 Mbits/sec
>
> >
>
> > 2097=2078:17:2:0:0:0:0:0
>
> >
>
> > [ 4] 7.00-8.00 sec 1.77 MBytes 14.9 Mbits/sec
>
> >
>
> > 85.406/65.797/103.418/12.609 ms (14/132595) 153 KByte 21.74
>
> >
>
> > 692=687:2:3:0:0:0:0:0
>
> >
>
> > [ 2] 7.00-8.00 sec 957 KBytes 7.84 Mbits/sec
>
> >
>
> > 153.936/131.452/191.464/19.361 ms (7/140042) 160 KByte 6.368199
>
> >
>
> > 429=425:4:0:0:0:0:0:0
>
> >
>
> > [ 1] 7.00-8.00 sec 1.13 MBytes 9.44 Mbits/sec
>
> >
>
> > 131.146/109.737/166.774/22.035 ms (9/131124) 146 KByte 8.998528
>
> >
>
> > 520=516:4:0:0:0:0:0:0
>
> >
>
> > [ 3] 7.00-8.00 sec 1.13 MBytes 9.51 Mbits/sec
>
> >
>
> > 126.512/88.404/220.175/42.237 ms (9/132089) 172 KByte 9.396784
>
> >
>
> > 527=524:1:2:0:0:0:0:0
>
> >
>
> > [SUM] 7.00-8.00 sec 4.96 MBytes 41.6 Mbits/sec
>
> >
>
> > 2168=2152:11:5:0:0:0:0:0
>
> >
>
> > With source side rate limiting to 9 mb/s per stream.
>
> >
>
> > [rjmcmahon at ryzen3950 iperf2-code]$ iperf -c *** --hide-ips -e
>
> >
>
> > --trip-times -i 1 -P 4 -t 10 -w 4m --tcp-quickack -N -b 9m
>
> >
>
> > ------------------------------------------------------------
>
> >
>
> > Client connecting to (**hidden**), TCP port 5001 with pid 108884 (4
>
> >
>
> > flows)
>
> >
>
> > Write buffer size: 131072 Byte
>
> >
>
> > TOS set to 0x0 and nodelay (Nagle off)
>
> >
>
> > TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte)
>
> >
>
> > Event based writes (pending queue watermark at 16384 bytes)
>
> >
>
> > ------------------------------------------------------------
>
> >
>
> > [ 1] local *.*.*.85%enp4s0 port 46448 connected with *.*.*.123 port
>
> >
>
> > 5001 (prefetch=16384) (trip-times) (sock=3) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/10666) (ct=10.70 ms) on 2023-01-06 12:27:45
>
> >
>
> > (PST)
>
> >
>
> > [ 3] local *.*.*.85%enp4s0 port 46460 connected with *.*.*.123 port
>
> >
>
> > 5001 (prefetch=16384) (trip-times) (sock=6) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/16499) (ct=16.54 ms) on 2023-01-06 12:27:45
>
> >
>
> > (PST)
>
> >
>
> > [ 2] local *.*.*.85%enp4s0 port 46454 connected with *.*.*.123 port
>
> >
>
> > 5001 (prefetch=16384) (trip-times) (sock=4) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/16580) (ct=16.86 ms) on 2023-01-06 12:27:45
>
> >
>
> > (PST)
>
> >
>
> > [ 4] local *.*.*.85%enp4s0 port 46458 connected with *.*.*.123 port
>
> >
>
> > 5001 (prefetch=16384) (trip-times) (sock=5) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/16802) (ct=16.83 ms) on 2023-01-06 12:27:45
>
> >
>
> > (PST)
>
> >
>
> > [ ID] Interval Transfer Bandwidth Write/Err Rtry
>
> >
>
> > Cwnd/RTT(var) NetPwr
>
> >
>
> > ...
>
> >
>
> > [ 2] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
>
> >
>
> > 134K/88055(12329) us 11.91
>
> >
>
> > [ 4] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
>
> >
>
> > 132K/74867(11755) us 14.01
>
> >
>
> > [ 1] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
>
> >
>
> > 134K/89101(13134) us 11.77
>
> >
>
> > [ 3] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
>
> >
>
> > 131K/91451(11938) us 11.47
>
> >
>
> > [SUM] 4.00-5.00 sec 4.00 MBytes 33.6 Mbits/sec 32/0 0
>
> >
>
> > [ 2] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
>
> >
>
> > 134K/85135(14580) us 13.86
>
> >
>
> > [ 4] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
>
> >
>
> > 132K/85124(15654) us 13.86
>
> >
>
> > [ 1] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
>
> >
>
> > 134K/91336(11335) us 12.92
>
> >
>
> > [ 3] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
>
> >
>
> > 131K/89185(13499) us 13.23
>
> >
>
> > [SUM] 5.00-6.00 sec 4.50 MBytes 37.7 Mbits/sec 36/0 0
>
> >
>
> > [ 2] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
>
> >
>
> > 134K/85687(13489) us 13.77
>
> >
>
> > [ 4] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
>
> >
>
> > 132K/82803(13001) us 14.25
>
> >
>
> > [ 1] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
>
> >
>
> > 134K/86869(15186) us 13.58
>
> >
>
> > [ 3] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
>
> >
>
> > 131K/91447(12515) us 12.90
>
> >
>
> > [SUM] 6.00-7.00 sec 4.50 MBytes 37.7 Mbits/sec 36/0 0
>
> >
>
> > [ 2] 7.00-8.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
>
> >
>
> > 134K/81814(13168) us 12.82
>
> >
>
> > [ 4] 7.00-8.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
>
> >
>
> > 132K/89008(13283) us 11.78
>
> >
>
> > [ 1] 7.00-8.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
>
> >
>
> > 134K/89494(12151) us 11.72
>
> >
>
> > [ 3] 7.00-8.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
>
> >
>
> > 131K/91083(12797) us 11.51
>
> >
>
> > [SUM] 7.00-8.00 sec 4.00 MBytes 33.6 Mbits/sec 32/0 0
>
> >
>
> > [root at bobcat iperf2-code]# iperf -s -i 1 -e --hide-ips -w 4m
>
> >
>
> > ------------------------------------------------------------
>
> >
>
> > Server listening on TCP port 5001 with pid 233981
>
> >
>
> > Read buffer size: 128 KByte (Dist bin width=16.0 KByte)
>
> >
>
> > TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte)
>
> >
>
> > ------------------------------------------------------------
>
> >
>
> > [ 1] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
>
> > 46448
>
> >
>
> > (trip-times) (sock=4) (peer 2.1.9-master) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/11987) on 2023-01-06 12:27:45 (PST)
>
> >
>
> > [ 2] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
>
> > 46454
>
> >
>
> > (trip-times) (sock=5) (peer 2.1.9-master) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/11132) on 2023-01-06 12:27:45 (PST)
>
> >
>
> > [ 3] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
>
> > 46460
>
> >
>
> > (trip-times) (sock=6) (peer 2.1.9-master) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/11097) on 2023-01-06 12:27:45 (PST)
>
> >
>
> > [ 4] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
>
> > 46458
>
> >
>
> > (trip-times) (sock=7) (peer 2.1.9-master) (qack)
>
> >
>
> > (icwnd/mss/irtt=14/1448/17823) on 2023-01-06 12:27:45 (PST)
>
> >
>
> > [ ID] Interval Transfer Bandwidth Burst Latency
>
> >
>
> > avg/min/max/stdev (cnt/size) inP NetPwr Reads=Dist
>
> >
>
> > [ 4] 0.00-1.00 sec 1.09 MBytes 9.15 Mbits/sec
>
> >
>
> > 93.383/90.103/95.661/2.232 ms (8/143028) 128 KByte 12.25
>
> >
>
> > 451=451:0:0:0:0:0:0:0
>
> >
>
> > [ 3] 0.00-1.00 sec 1.08 MBytes 9.06 Mbits/sec
>
> >
>
> > 96.834/95.229/102.645/2.442 ms (8/141580) 131 KByte 11.70
>
> >
>
> > 472=472:0:0:0:0:0:0:0
>
> >
>
> > [ 1] 0.00-1.00 sec 1.10 MBytes 9.19 Mbits/sec
>
> >
>
> > 95.183/92.623/97.579/1.431 ms (8/143571) 131 KByte 12.07
>
> >
>
> > 495=495:0:0:0:0:0:0:0
>
> >
>
> > [ 2] 0.00-1.00 sec 1.09 MBytes 9.15 Mbits/sec
>
> >
>
> > 89.317/84.865/94.906/3.674 ms (8/143028) 122 KByte 12.81
>
> >
>
> > 489=489:0:0:0:0:0:0:0
>
> >
>
> > [ ID] Interval Transfer Bandwidth Reads=Dist
>
> >
>
> > [SUM] 0.00-1.00 sec 4.36 MBytes 36.6 Mbits/sec
>
> >
>
> > 1907=1907:0:0:0:0:0:0:0
>
> >
>
> > [ 4] 1.00-2.00 sec 1.07 MBytes 8.95 Mbits/sec
>
> >
>
> > 92.649/89.987/95.036/1.828 ms (9/124314) 96.5 KByte 12.08
>
> >
>
> > 492=492:0:0:0:0:0:0:0
>
> >
>
> > [ 3] 1.00-2.00 sec 1.06 MBytes 8.93 Mbits/sec
>
> >
>
> > 96.305/95.647/96.794/0.432 ms (9/123992) 100 KByte 11.59
>
> >
>
> > 480=480:0:0:0:0:0:0:0
>
> >
>
> > [ 1] 1.00-2.00 sec 1.06 MBytes 8.89 Mbits/sec
>
> >
>
> > 92.578/90.866/94.145/1.371 ms (9/123510) 95.8 KByte 12.01
>
> >
>
> > 513=513:0:0:0:0:0:0:0
>
> >
>
> > [ 2] 1.00-2.00 sec 1.07 MBytes 8.96 Mbits/sec
>
> >
>
> > 90.767/87.984/94.352/1.944 ms (9/124475) 94.7 KByte 12.34
>
> >
>
> > 489=489:0:0:0:0:0:0:0
>
> >
>
> > [SUM] 1.00-2.00 sec 4.26 MBytes 35.7 Mbits/sec
>
> >
>
> > 1974=1974:0:0:0:0:0:0:0
>
> >
>
> > [ 4] 2.00-3.00 sec 1.09 MBytes 9.13 Mbits/sec
>
> >
>
> > 93.977/91.795/96.561/1.693 ms (8/142656) 112 KByte 12.14
>
> >
>
> > 497=497:0:0:0:0:0:0:0
>
> >
>
> > [ 3] 2.00-3.00 sec 1.08 MBytes 9.04 Mbits/sec
>
> >
>
> > 96.544/95.815/97.798/0.693 ms (8/141208) 114 KByte 11.70
>
> >
>
> > 503=503:0:0:0:0:0:0:0
>
> >
>
> > [ 1] 2.00-3.00 sec 1.07 MBytes 9.01 Mbits/sec
>
> >
>
> > 93.970/91.193/96.325/1.796 ms (8/140846) 111 KByte 11.99
>
> >
>
> > 509=509:0:0:0:0:0:0:0
>
> >
>
> > [ 2] 2.00-3.00 sec 1.08 MBytes 9.10 Mbits/sec
>
> >
>
> > 92.843/90.216/96.355/2.040 ms (8/142113) 111 KByte 12.25
>
> >
>
> > 509=509:0:0:0:0:0:0:0
>
> >
>
> > [SUM] 2.00-3.00 sec 4.32 MBytes 36.3 Mbits/sec
>
> >
>
> > 2018=2018:0:0:0:0:0:0:0
>
> >
>
> > [ 4] 3.00-4.00 sec 1.06 MBytes 8.86 Mbits/sec
>
> >
>
> > 93.222/89.063/96.104/2.346 ms (9/123027) 96.1 KByte 11.88
>
> >
>
> > 487=487:0:0:0:0:0:0:0
>
> >
>
> > [ 3] 3.00-4.00 sec 1.07 MBytes 8.97 Mbits/sec
>
> >
>
> > 96.277/95.051/97.230/0.767 ms (9/124636) 101 KByte 11.65
>
> >
>
> > 489=489:0:0:0:0:0:0:0
>
> >
>
> > [ 1] 3.00-4.00 sec 1.08 MBytes 9.02 Mbits/sec
>
> >
>
> > 93.899/88.732/96.972/2.737 ms (9/125280) 98.6 KByte 12.01
>
> >
>
> > 493=493:0:0:0:0:0:0:0
>
> >
>
> > [ 2] 3.00-4.00 sec 1.07 MBytes 8.97 Mbits/sec
>
> >
>
> > 92.490/89.862/95.265/1.796 ms (9/124636) 96.6 KByte 12.13
>
> >
>
> > 493=493:0:0:0:0:0:0:0
>
> >
>
> > [SUM] 3.00-4.00 sec 4.27 MBytes 35.8 Mbits/sec
>
> >
>
> > 1962=1962:0:0:0:0:0:0:0
>
> >
>
> > [ 4] 4.00-5.00 sec 1.07 MBytes 9.00 Mbits/sec
>
> >
>
> > 92.431/81.888/96.221/4.524 ms (9/124958) 96.8 KByte 12.17
>
> >
>
> > 498=498:0:0:0:0:0:0:0
>
> >
>
> > [ 1] 4.00-5.00 sec 1.07 MBytes 8.97 Mbits/sec
>
> >
>
> > 95.018/93.445/96.200/0.957 ms (9/124636) 99.3 KByte 11.81
>
> >
>
> > 490=490:0:0:0:0:0:0:0
>
> >
>
> > [ 2] 4.00-5.00 sec 1.06 MBytes 8.90 Mbits/sec
>
> >
>
> > 93.874/86.485/95.672/2.810 ms (9/123671) 97.3 KByte 11.86
>
> >
>
> > 481=481:0:0:0:0:0:0:0
>
> >
>
> > [ 3] 4.00-5.00 sec 1.08 MBytes 9.09 Mbits/sec
>
> >
>
> > 95.737/93.881/97.197/0.972 ms (9/126245) 101 KByte 11.87
>
> >
>
> > 484=484:0:0:0:0:0:0:0
>
> >
>
> > [SUM] 4.00-5.00 sec 4.29 MBytes 36.0 Mbits/sec
>
> >
>
> > 1953=1953:0:0:0:0:0:0:0
>
> >
>
> > [ 4] 5.00-6.00 sec 1.09 MBytes 9.13 Mbits/sec
>
> >
>
> > 92.908/86.844/95.994/3.012 ms (8/142656) 111 KByte 12.28
>
> >
>
> > 467=467:0:0:0:0:0:0:0
>
> >
>
> > [ 3] 5.00-6.00 sec 1.07 MBytes 8.94 Mbits/sec
>
> >
>
> > 96.593/95.343/97.660/0.876 ms (8/139760) 113 KByte 11.58
>
> >
>
> > 478=478:0:0:0:0:0:0:0
>
> >
>
> > [ 1] 5.00-6.00 sec 1.08 MBytes 9.03 Mbits/sec
>
> >
>
> > 95.021/91.421/97.167/1.893 ms (8/141027) 112 KByte 11.87
>
> >
>
> > 491=491:0:0:0:0:0:0:0
>
> >
>
> > [ 2] 5.00-6.00 sec 1.08 MBytes 9.06 Mbits/sec
>
> >
>
> > 92.162/82.720/97.692/5.060 ms (8/141570) 109 KByte 12.29
>
> >
>
> > 488=488:0:0:0:0:0:0:0
>
> >
>
> > [SUM] 5.00-6.00 sec 4.31 MBytes 36.2 Mbits/sec
>
> >
>
> > 1924=1924:0:0:0:0:0:0:0
>
> >
>
> > [ 4] 6.00-7.00 sec 1.04 MBytes 8.70 Mbits/sec
>
> >
>
> > 92.793/85.343/96.967/3.552 ms (9/120775) 93.9 KByte 11.71
>
> >
>
> > 485=485:0:0:0:0:0:0:0
>
> >
>
> > [ 2] 6.00-7.00 sec 1.05 MBytes 8.79 Mbits/sec
>
> >
>
> > 91.679/84.479/96.760/3.975 ms (9/122062) 93.8 KByte 11.98
>
> >
>
> > 472=472:0:0:0:0:0:0:0
>
> >
>
> > [ 3] 6.00-7.00 sec 1.06 MBytes 8.88 Mbits/sec
>
> >
>
> > 96.982/95.933/98.371/0.680 ms (9/123349) 100 KByte 11.45
>
> >
>
> > 477=477:0:0:0:0:0:0:0
>
> >
>
> > [ 1] 6.00-7.00 sec 1.05 MBytes 8.80 Mbits/sec
>
> >
>
> > 94.342/91.660/96.025/1.660 ms (9/122223) 96.7 KByte 11.66
>
> >
>
> > 494=494:0:0:0:0:0:0:0
>
> >
>
> > [SUM] 6.00-7.00 sec 4.19 MBytes 35.2 Mbits/sec
>
> >
>
> > 1928=1928:0:0:0:0:0:0:0
>
> >
>
> > [ 4] 7.00-8.00 sec 1.10 MBytes 9.25 Mbits/sec
>
> >
>
> > 92.515/88.182/96.351/2.538 ms (8/144466) 112 KByte 12.49
>
> >
>
> > 510=510:0:0:0:0:0:0:0
>
> >
>
> > [ 3] 7.00-8.00 sec 1.09 MBytes 9.13 Mbits/sec
>
> >
>
> > 96.580/95.737/98.977/1.098 ms (8/142656) 115 KByte 11.82
>
> >
>
> > 480=480:0:0:0:0:0:0:0
>
> >
>
> > [ 1] 7.00-8.00 sec 1.10 MBytes 9.21 Mbits/sec
>
> >
>
> > 95.269/91.719/97.514/2.126 ms (8/143923) 115 KByte 12.09
>
> >
>
> > 515=515:0:0:0:0:0:0:0
>
> >
>
> > [ 2] 7.00-8.00 sec 1.11 MBytes 9.29 Mbits/sec
>
> >
>
> > 90.073/84.700/96.176/4.324 ms (8/145190) 110 KByte 12.90
>
> >
>
> > 508=508:0:0:0:0:0:0:0
>
> >
>
> > [SUM] 7.00-8.00 sec 4.40 MBytes 36.9 Mbits/sec
>
> >
>
> > 2013=2013:0:0:0:0:0:0:0
>
> >
>
> > Bob
>
> >
>
> >>> -----Original Message-----
>
> >
>
> >>
>
> >
>
> >>> From: LibreQoS <libreqos-bounces at lists.bufferbloat.net> On Behalf
>
> > Of
>
> >
>
> >> Dave Taht
>
> >
>
> >>
>
> >
>
> >>> via LibreQoS
>
> >
>
> >>
>
> >
>
> >>> Sent: Wednesday, January 4, 2023 12:26 PM
>
> >
>
> >>
>
> >
>
> >>> Subject: [LibreQoS] the grinch meets cloudflare's christmas present
>
> >
>
> >
>
> >>
>
> >
>
> >>>
>
> >
>
> >>
>
> >
>
> >>> Please try the new, the shiny, the really wonderful test here:
>
> >
>
> >>
>
> >
>
> >>>
>
> >
>
> >>
>
> >
> https://urldefense.com/v3/__https://speed.cloudflare.com/__;!!BhdT!iZcFJ8WVU9S
>
> >
>
> >
>
> >> [1]
>
> >
>
> >>
>
> >
>
> >>>
>
> >
>
> >>
>
> >
> 9zz5t456oxkfObrC5Xb9j5AG8UO3DqD5x4GAJkawZr0iGwEUtF0_09U8mCDnAkrJ9QEMHGbCMKVw$
>
> >
>
> >
>
> >> [1]
>
> >
>
> >>
>
> >
>
> >>>
>
> >
>
> >>
>
> >
>
> >>> I would really appreciate some independent verification of
>
> >
>
> >>
>
> >
>
> >>> measurements using this tool. In my brief experiments it appears -
>
> >
>
> >> as
>
> >
>
> >>
>
> >
>
> >>> all the commercial tools to date - to dramatically understate the
>
> >
>
> >>
>
> >
>
> >>> bufferbloat, on my LTE, (and my starlink terminal is out being
>
> >
>
> >>
>
> >
>
> >>> hacked^H^H^H^H^H^Hworked on, so I can't measure that)
>
> >
>
> >>
>
> >
>
> >> [acm]
>
> >
>
> >>
>
> >
>
> >> Hi Dave, I made some time to test "cloudflare's christmas present"
>
> >
>
> >> yesterday.
>
> >
>
> >>
>
> >
>
> >> I'm on DOCSIS 3.1 service with 1Gbps Down. The Upstream has a
>
> > "turbo"
>
> >
>
> >> mode with 40-50Mbps for the first ~3 sec, then steady-state about
>
> >
>
> >> 23Mbps.
>
> >
>
> >>
>
> >
>
> >> When I saw the ~620Mbps Downstream measurement, I was ready to
>
> >
>
> >> complain that even the IP-Layer Capacity was grossly underestimated.
>
> >
>
> >
>
> >> In addition, the Latency measurements seem very low (as you
>
> > asserted),
>
> >
>
> >> although the cloud server was "nearby".
>
> >
>
> >>
>
> >
>
> >> However, I found that Ookla and the ISP-provided measurement were
>
> > also
>
> >
>
> >> reporting ~600Mbps! So the cloudflare Downstream capacity (or
>
> >
>
> >> throughput?) measurement was consistent with others. Our UDPST
>
> > server
>
> >
>
> >> was unreachable, otherwise I would have added that measurement, too.
>
> >
>
> >
>
> >>
>
> >
>
> >> The Upstream measurement graph seems to illustrate the "turbo"
>
> >
>
> >> mode, with the dip after attaining 44.5Mbps.
>
> >
>
> >>
>
> >
>
> >> UDPST saturates the uplink and we measure the full 250ms of the
>
> >
>
> >> Upstream buffer. Cloudflare's latency measurements don't even come
>
> >
>
> >> close.
>
> >
>
> >>
>
> >
>
> >> Al
>
> >
>
> >>
>
> >
>
> >>
>
> >
>
> >>
>
> >
>
> >> Links:
>
> >
>
> >> ------
>
> >
>
> >> [1]
>
> >
>
> >>
>
> >
> https://urldefense.com/v3/__https:/speed.cloudflare.com/__;!!BhdT!iZcFJ8WVU9S9zz5t456oxkfObrC5Xb9j5AG8UO3DqD5x4GAJkawZr0iGwEUtF0_09U8mCDnAkrJ9QEMHGbCMKVw$
>
> >
>
> >
>
> >> _______________________________________________
>
> >
>
> >> Rpm mailing list
>
> >
>
> >> Rpm at lists.bufferbloat.net
>
> >
>
> >> https://lists.bufferbloat.net/listinfo/rpm
>
> >
>
> > _______________________________________________
>
> >
>
> > Starlink mailing list
>
> >
>
> > Starlink at lists.bufferbloat.net
>
> >
>
> > https://lists.bufferbloat.net/listinfo/starlink
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20230110/d7fca33e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot 2023-01-10 at 11-19-57 Internet Speed Test - Measure Network Performance Cloudflare.png
Type: image/png
Size: 631718 bytes
Desc: not available
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20230110/d7fca33e/attachment-0001.png>
More information about the Cake
mailing list