From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 858763CB4C for ; Wed, 11 Jan 2023 06:05:42 -0500 (EST) Received: by mail-ej1-x634.google.com with SMTP id u9so35966607ejo.0 for ; Wed, 11 Jan 2023 03:05:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tp.org; s=google; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=m59OwkbDf7SqTYNbU45nSjuZNfvEKaad3rOGf0cXud8=; b=LBq72mkmH/vIPSGq48QsrGKtullbOXwOi/NjiPAlBFgkCMTYBWT1w1aOU0xJpQwoge dxYzXioG+j+OduYQu1acVv404cLymmM/YA5aSSUqOG4cVVBlfibMLJodTDbIG4FEs0AD rE1V9qSZojWZpGAIq3mWCkdJ4tBF+zQnKcWK0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=m59OwkbDf7SqTYNbU45nSjuZNfvEKaad3rOGf0cXud8=; b=pjZ7GEssao8OsBSZqdMHNDs0iDwM5VJpdjTBtiU9Iu7Ia8x0CChdVVN99ZRJdPOFnR nIaPiR2uB31ZQ7wyi+RRTy1kTqLuhFo61X6GuazlCdWDYTfe4/+KJCL70pvavcG6zU9J hlwtpNhEpFjvJ2kQkbx+Cq22HtUiOYQdJ7G6vwL4eGJzpPgjcuyKHEd2+7fHr0e2RDpL nxAzp5rNZnWwYU2LLTLUZewXn0Ew+l7VXVyT0Ju/vHcvorgF90OfjIk7XPpMsURC8xtc 2dmvZqF029a1j/NlxPOIIx+WW7CXdSO73RMMBfwr5PK14mZZnoWMBs0eIbKVS2Pby88o SNUA== X-Gm-Message-State: AFqh2kphoQDM80kZySU5P3dHjyEZ28uHWVIC/daDDUqtclVARmjV8MSG WULmz+O5VIaYTx0FTMAIua5/LdWZevl2G2Pi/pOj X-Google-Smtp-Source: AMrXdXtSOZRuUHWNEA3Says+Zs37AR/lHVg4QmvflS3b0KL/1y+VLDBfBFJNBu6+1JbLxuAUSushxbExPoWF6lM1ntk= X-Received: by 2002:a17:906:5a0b:b0:84d:3b4a:6ace with SMTP id mx11-20020a1709065a0b00b0084d3b4a6acemr791350ejc.641.1673435140964; Wed, 11 Jan 2023 03:05:40 -0800 (PST) MIME-Version: 1.0 References: <89D796E75967416B9723211C183A8396@SRA6> <3696AEA5409D4303ABCBC439727A5E40@SRA6> In-Reply-To: From: Jay Moran Date: Wed, 11 Jan 2023 06:05:29 -0500 Message-ID: To: Dave Taht Cc: Cake List , IETF IPPM WG , "Luis A. Cornejo" , "MORTON JR., AL" , Rpm , bloat , dickroy@alum.mit.edu, libreqos Content-Type: multipart/alternative; boundary="00000000000021e47f05f1fafb8a" X-Mailman-Approved-At: Wed, 25 Jan 2023 08:09:34 -0500 Subject: Re: [Bloat] [Rpm] [Starlink] [LibreQoS] the grinch meets cloudflare'schristmas present X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jan 2023 11:05:42 -0000 --00000000000021e47f05f1fafb8a Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Quick note from reading your blog entry. Last night, I played with the Cloudflare Speedtest a little. It downloads 25MB and a 50MB (or 100MB, can=E2=80=99t remember) as well on a =E2=80=9Csp= eedier=E2=80=9D network after it does the 10MB file. I was getting 1.2Gbs down and 760Mbs up, 4ms of LUL, and seeing those larger file sizes. I was trying to screenshot and noticed I had those extra file sizes I had to scroll down for. I ended up getting distracted and not taking the shot to send. But, it will do a longer/bigger test under right conditions. Network here at the house is AT&T Fiber, 5Gbs up/down - limited to 3.6Gbs down from Ubiquity UDM SE router/firewall with all IPS/Geo-blocking turned on. 4.7Gbs non-blocking up. I am building a pfSense box to eliminate the bottleneck. Couldn=E2=80=99t be happier, good job AS7018. The machine I was testing from was Win10 wired 10Gbs and gets ~2.2Gbs up/down for fast.com/speedtest.net. I haven=E2=80=99t take time to test int= ernally or try and tune that system, or might be CAT5e cabling issue=E2=80=A6 is fa= st enough for me for that system. Jay On Wed, Jan 11, 2023 at 12:07 AM Dave Taht via Bloat < bloat@lists.bufferbloat.net> wrote: > Dear Luis: > > You hit 17 seconds of delay on your test. > > I got you beat, today, on my LTE connection, I cracked 182 seconds. > > I'd like to thank Verizon for making it possible for me to spew 4000 > words on my kvetches about the current speedtest regimes of speedtest, > cloudflare, and so on, by making my network connection so lousy today > that I sat in front of emacs to rant - and y'all for helping tone > down, a little, this blog entry: > > https://blog.cerowrt.org/post/speedtests/ > > On Tue, Jan 10, 2023 at 9:25 AM Luis A. Cornejo via Rpm > wrote: > > > > Here is my VZ HSI > > > > > > No SQMm on > > > > On Sat, Jan 7, 2023 at 6:38 PM Dick Roy via Bloat < > bloat@lists.bufferbloat.net> wrote: > >> > >> > >> > >> > >> > >> -----Original Message----- > >> From: rjmcmahon [mailto:rjmcmahon@rjmcmahon.com] > >> Sent: Friday, January 6, 2023 3:45 PM > >> To: dickroy@alum.mit.edu > >> Cc: 'MORTON JR., AL'; 'IETF IPPM WG'; 'libreqos'; 'Cake List'; 'Rpm'; > 'bloat' > >> Subject: Re: [Starlink] [Rpm] [LibreQoS] the grinch meets > cloudflare'schristmas present > >> > >> > >> > >> yeah, I'd prefer not to output CLT sample groups at all but the > >> > >> histograms aren't really human readable and users constantly ask for > >> > >> them. I thought about providing a distance from the gaussian as output > >> > >> too but so far few would understand it and nobody I found would act up= on > >> > >> it. > >> > >> [RR] Understandable until such metrics are =E2=80=9Cactionable=E2=80= =9D, and that=E2=80=99s =E2=80=9Cup > to us to find/define/figure out=E2=80=9D it seems to me. Metrics that are= not > actionable are write-only memory and good for little but historical recor= dJ > >> > >> The tool produces the full histograms so no information is really > >> > >> missing except for maybe better time series analysis. > >> > >> [RR] Isn=E2=80=99t that in fact what we are trying to extract from the= e2e > stats we collect? i.e., infer the time evolution of the system from its > I/O behavior? As you point out, it=E2=80=99s really hard to do without pr= obes in > the guts of the system, nd yes, synchronization is important J > >> > >> > >> > >> The open source flows python code also released with iperf 2 does use > >> > >> the komogorov-smirnov distances & distance matrices to cluster when th= e > >> > >> number of histograms are just too much. We've analyzed 1M runs to faul= t > >> > >> isolate the "unexpected interruptions" or "bugs" and without statistic= al > >> > >> support it is just not doable. This does require instrumentation of th= e > >> > >> full path with mapping to a common clock domain (e.g. GPS) and not jus= t > >> > >> e2e stats. I find an e2e complaint by an end user about "poor speed" a= s > >> > >> useful as telling a pharmacist I have a fever. Not much diagnostically > >> > >> is going on. Take an aspirin. > >> > >> [RR] That=E2=80=99s AWESOME!!!!!!!!!!!!!!!!!! I love that analogy! > >> > >> > >> > >> RR > >> > >> > >> > >> https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test > >> > >> https://sourceforge.net/p/iperf2/code/ci/master/tree/flows/flows.py > >> > >> > >> > >> Bob > >> > >> > See below =E2=80=A6 > >> > >> > > >> > >> > -----Original Message----- > >> > >> > From: Starlink [mailto:starlink-bounces@lists.bufferbloat.net] On > >> > >> > Behalf Of rjmcmahon via Starlink > >> > >> > Sent: Friday, January 6, 2023 12:39 PM > >> > >> > To: MORTON JR., AL > >> > >> > Cc: Dave Taht via Starlink; IETF IPPM WG; libreqos; Cake List; Rpm; > >> > >> > bloat > >> > >> > Subject: Re: [Starlink] [Rpm] [LibreQoS] the grinch meets > >> > >> > cloudflare'schristmas present > >> > >> > > >> > >> > Some thoughts are not to use UDP for testing here. Also, these speed > >> > >> > > >> > >> > tests have little to no information for network engineers about what= 's > >> > >> > > >> > >> > > >> > >> > going on. Iperf 2 may better assist network engineers but then I'm > >> > >> > > >> > >> > biased ;) > >> > >> > > >> > >> > Running iperf 2 https://sourceforge.net/projects/iperf2/ with > >> > >> > > >> > >> > --trip-times. Though the sampling and central limit theorem averagin= g > >> > >> > is > >> > >> > > >> > >> > hiding the real distributions (use --histograms to get those) > >> > >> > > >> > >> > _[RR] FWIW (IMNBWM __J)=E2=80=A6 If the output/final histograms indi= cate the > >> > >> > PDF is NOT Gaussian, then any application of the CLT is > >> > >> > inappropriate/contra-indicated! The CLT is a "proof under certain > >> > >> > regularity conditions/assumptions of underlying/constituent PDFs, th= at > >> > >> > the resulting PDF (after all the necessary convolutions are performe= d > >> > >> > to get to the PDF of the output) will asymptotically approach a > >> > >> > Gaussian with only a mean and a std. dev. left to specify. _ > >> > >> > > >> > >> > Below are 4 parallel TCP streams from my home to one of my servers i= n > >> > >> > > >> > >> > the cloud. First where TCP is limited per CCA. Second is source side > >> > >> > > >> > >> > write rate limiting. Things to note: > >> > >> > > >> > >> > o) connect times for both at 10-15 ms > >> > >> > > >> > >> > o) multiple TCP retries on a few rites - one case is 4 with 5 writes= . > >> > >> > > >> > >> > Source side pacing eliminates retries > >> > >> > > >> > >> > o) Fairness with CCA isn't great but quite good with source side wri= te > >> > >> > > >> > >> > > >> > >> > pacing > >> > >> > > >> > >> > o) Queue depth with CCA is about 150 Kbytes about 100K byte with > >> > >> > source > >> > >> > > >> > >> > side pacing > >> > >> > > >> > >> > o) min write to read is about 80 ms for both > >> > >> > > >> > >> > o) max is 220 ms vs 97 ms > >> > >> > > >> > >> > o) stdev for CCA write/read is 30 ms vs 3 ms > >> > >> > > >> > >> > o) TCP RTT is 20ms w/CCA and 90 ms with ssp - seems odd here as > >> > >> > > >> > >> > TCP_QUICACK and TCP_NODELAY are both enabled. > >> > >> > > >> > >> > [ CT] final connect times (min/avg/max/stdev) =3D > >> > >> > > >> > >> > 10.326/13.522/14.986/2150.329 ms (tot/err) =3D 4/0 > >> > >> > > >> > >> > [rjmcmahon@ryzen3950 iperf2-code]$ iperf -c *** --hide-ips -e > >> > >> > > >> > >> > --trip-times -i 1 -P 4 -t 10 -w 4m --tcp-quickack -N > >> > >> > > >> > >> > ------------------------------------------------------------ > >> > >> > > >> > >> > Client connecting to (**hidden**), TCP port 5001 with pid 107678 (4 > >> > >> > > >> > >> > flows) > >> > >> > > >> > >> > Write buffer size: 131072 Byte > >> > >> > > >> > >> > TOS set to 0x0 and nodelay (Nagle off) > >> > >> > > >> > >> > TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte) > >> > >> > > >> > >> > Event based writes (pending queue watermark at 16384 bytes) > >> > >> > > >> > >> > ------------------------------------------------------------ > >> > >> > > >> > >> > [ 1] local *.*.*.85%enp4s0 port 42480 connected with *.*.*.123 port > >> > >> > > >> > >> > 5001 (prefetch=3D16384) (trip-times) (sock=3D3) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/10534) (ct=3D10.63 ms) on 2023-01-06 12:17= :56 > >> > >> > > >> > >> > (PST) > >> > >> > > >> > >> > [ 4] local *.*.*.85%enp4s0 port 42488 connected with *.*.*.123 port > >> > >> > > >> > >> > 5001 (prefetch=3D16384) (trip-times) (sock=3D5) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/14023) (ct=3D14.08 ms) on 2023-01-06 12:17= :56 > >> > >> > > >> > >> > (PST) > >> > >> > > >> > >> > [ 3] local *.*.*.85%enp4s0 port 42502 connected with *.*.*.123 port > >> > >> > > >> > >> > 5001 (prefetch=3D16384) (trip-times) (sock=3D6) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/14642) (ct=3D14.70 ms) on 2023-01-06 12:17= :56 > >> > >> > > >> > >> > (PST) > >> > >> > > >> > >> > [ 2] local *.*.*.85%enp4s0 port 42484 connected with *.*.*.123 port > >> > >> > > >> > >> > 5001 (prefetch=3D16384) (trip-times) (sock=3D4) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/14728) (ct=3D14.79 ms) on 2023-01-06 12:17= :56 > >> > >> > > >> > >> > (PST) > >> > >> > > >> > >> > [ ID] Interval Transfer Bandwidth Write/Err Rtry > >> > >> > > >> > >> > Cwnd/RTT(var) NetPwr > >> > >> > > >> > >> > ... > >> > >> > > >> > >> > [ 4] 4.00-5.00 sec 1.38 MBytes 11.5 Mbits/sec 11/0 3 > >> > >> > > >> > >> > > >> > >> > 29K/21088(1142) us 68.37 > >> > >> > > >> > >> > [ 2] 4.00-5.00 sec 1.62 MBytes 13.6 Mbits/sec 13/0 2 > >> > >> > > >> > >> > > >> > >> > 31K/19284(612) us 88.36 > >> > >> > > >> > >> > [ 1] 4.00-5.00 sec 896 KBytes 7.34 Mbits/sec 7/0 5 > >> > >> > > >> > >> > 16K/18996(658) us 48.30 > >> > >> > > >> > >> > [ 3] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 5 > >> > >> > > >> > >> > 18K/18133(208) us 57.83 > >> > >> > > >> > >> > [SUM] 4.00-5.00 sec 4.88 MBytes 40.9 Mbits/sec 39/0 15 > >> > >> > > >> > >> > [ 4] 5.00-6.00 sec 1.25 MBytes 10.5 Mbits/sec 10/0 4 > >> > >> > > >> > >> > > >> > >> > 29K/14717(489) us 89.06 > >> > >> > > >> > >> > [ 1] 5.00-6.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 4 > >> > >> > > >> > >> > 16K/15874(408) us 66.06 > >> > >> > > >> > >> > [ 3] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 4 > >> > >> > > >> > >> > 16K/15826(382) us 74.54 > >> > >> > > >> > >> > [ 2] 5.00-6.00 sec 1.50 MBytes 12.6 Mbits/sec 12/0 6 > >> > >> > > >> > >> > > >> > >> > 9K/14878(557) us 106 > >> > >> > > >> > >> > [SUM] 5.00-6.00 sec 4.88 MBytes 40.9 Mbits/sec 39/0 18 > >> > >> > > >> > >> > [ 4] 6.00-7.00 sec 1.75 MBytes 14.7 Mbits/sec 14/0 4 > >> > >> > > >> > >> > > >> > >> > 25K/15472(496) us 119 > >> > >> > > >> > >> > [ 2] 6.00-7.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 2 > >> > >> > > >> > >> > 26K/16417(427) us 63.87 > >> > >> > > >> > >> > [ 1] 6.00-7.00 sec 1.25 MBytes 10.5 Mbits/sec 10/0 5 > >> > >> > > >> > >> > > >> > >> > 16K/16268(679) us 80.57 > >> > >> > > >> > >> > [ 3] 6.00-7.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 6 > >> > >> > > >> > >> > 15K/16629(799) us 63.06 > >> > >> > > >> > >> > [SUM] 6.00-7.00 sec 5.00 MBytes 41.9 Mbits/sec 40/0 17 > >> > >> > > >> > >> > [ 4] 7.00-8.00 sec 1.75 MBytes 14.7 Mbits/sec 14/0 4 > >> > >> > > >> > >> > > >> > >> > 22K/13986(519) us 131 > >> > >> > > >> > >> > [ 1] 7.00-8.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 4 > >> > >> > > >> > >> > 16K/12679(377) us 93.04 > >> > >> > > >> > >> > [ 3] 7.00-8.00 sec 896 KBytes 7.34 Mbits/sec 7/0 5 > >> > >> > > >> > >> > 14K/12971(367) us 70.74 > >> > >> > > >> > >> > [ 2] 7.00-8.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 6 > >> > >> > > >> > >> > 15K/14740(779) us 80.03 > >> > >> > > >> > >> > [SUM] 7.00-8.00 sec 4.88 MBytes 40.9 Mbits/sec 39/0 19 > >> > >> > > >> > >> > [root@bobcat iperf2-code]# iperf -s -i 1 -e --hide-ips -w 4m > >> > >> > > >> > >> > ------------------------------------------------------------ > >> > >> > > >> > >> > Server listening on TCP port 5001 with pid 233615 > >> > >> > > >> > >> > Read buffer size: 128 KByte (Dist bin width=3D16.0 KByte) > >> > >> > > >> > >> > TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte) > >> > >> > > >> > >> > ------------------------------------------------------------ > >> > >> > > >> > >> > [ 1] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port > >> > >> > 42480 > >> > >> > > >> > >> > (trip-times) (sock=3D4) (peer 2.1.9-master) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/11636) on 2023-01-06 12:17:56 (PST) > >> > >> > > >> > >> > [ 2] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port > >> > >> > 42502 > >> > >> > > >> > >> > (trip-times) (sock=3D5) (peer 2.1.9-master) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/11898) on 2023-01-06 12:17:56 (PST) > >> > >> > > >> > >> > [ 3] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port > >> > >> > 42484 > >> > >> > > >> > >> > (trip-times) (sock=3D6) (peer 2.1.9-master) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/11938) on 2023-01-06 12:17:56 (PST) > >> > >> > > >> > >> > [ 4] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port > >> > >> > 42488 > >> > >> > > >> > >> > (trip-times) (sock=3D7) (peer 2.1.9-master) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/11919) on 2023-01-06 12:17:56 (PST) > >> > >> > > >> > >> > [ ID] Interval Transfer Bandwidth Burst Latency > >> > >> > > >> > >> > avg/min/max/stdev (cnt/size) inP NetPwr Reads=3DDist > >> > >> > > >> > >> > ... > >> > >> > > >> > >> > [ 2] 4.00-5.00 sec 1.06 MBytes 8.86 Mbits/sec > >> > >> > > >> > >> > 129.819/90.391/186.075/31.346 ms (9/123080) 154 KByte 8.532803 > >> > >> > > >> > >> > 467=3D461:6:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 3] 4.00-5.00 sec 1.52 MBytes 12.8 Mbits/sec > >> > >> > > >> > >> > 103.552/82.653/169.274/28.382 ms (12/132854) 149 KByte 15.40 > >> > >> > > >> > >> > 646=3D643:1:2:0:0:0:0:0 > >> > >> > > >> > >> > [ 4] 4.00-5.00 sec 1.39 MBytes 11.6 Mbits/sec > >> > >> > > >> > >> > 107.503/66.843/143.038/24.269 ms (11/132294) 149 KByte 13.54 > >> > >> > > >> > >> > 619=3D617:1:1:0:0:0:0:0 > >> > >> > > >> > >> > [ 1] 4.00-5.00 sec 988 KBytes 8.10 Mbits/sec > >> > >> > > >> > >> > 141.389/119.961/178.785/18.812 ms (7/144593) 170 KByte 7.158641 > >> > >> > > >> > >> > 409=3D404:5:0:0:0:0:0:0 > >> > >> > > >> > >> > [SUM] 4.00-5.00 sec 4.93 MBytes 41.4 Mbits/sec > >> > >> > > >> > >> > 2141=3D2125:13:3:0:0:0:0:0 > >> > >> > > >> > >> > [ 4] 5.00-6.00 sec 1.29 MBytes 10.8 Mbits/sec > >> > >> > > >> > >> > 118.943/86.253/176.128/31.248 ms (10/135098) 164 KByte 11.36 > >> > >> > > >> > >> > 511=3D506:2:3:0:0:0:0:0 > >> > >> > > >> > >> > [ 2] 5.00-6.00 sec 1.09 MBytes 9.17 Mbits/sec > >> > >> > > >> > >> > 139.821/102.418/218.875/40.422 ms (9/127424) 148 KByte 8.202049 > >> > >> > > >> > >> > 487=3D484:2:1:0:0:0:0:0 > >> > >> > > >> > >> > [ 3] 5.00-6.00 sec 1.51 MBytes 12.6 Mbits/sec > >> > >> > > >> > >> > 102.146/77.085/140.893/18.441 ms (13/121520) 151 KByte 15.47 > >> > >> > > >> > >> > 640=3D636:1:3:0:0:0:0:0 > >> > >> > > >> > >> > [ 1] 5.00-6.00 sec 981 KBytes 8.04 Mbits/sec > >> > >> > > >> > >> > 161.901/105.582/219.931/36.260 ms (8/125614) 134 KByte 6.206944 > >> > >> > > >> > >> > 415=3D413:2:0:0:0:0:0:0 > >> > >> > > >> > >> > [SUM] 5.00-6.00 sec 4.85 MBytes 40.7 Mbits/sec > >> > >> > > >> > >> > 2053=3D2039:7:7:0:0:0:0:0 > >> > >> > > >> > >> > [ 4] 6.00-7.00 sec 1.74 MBytes 14.6 Mbits/sec > >> > >> > > >> > >> > 88.846/74.297/101.859/7.118 ms (14/130526) 156 KByte 20.57 > >> > >> > > >> > >> > 711=3D707:3:1:0:0:0:0:0 > >> > >> > > >> > >> > [ 1] 6.00-7.00 sec 1.22 MBytes 10.2 Mbits/sec > >> > >> > > >> > >> > 120.639/100.257/157.567/21.770 ms (10/127568) 157 KByte 10.57 > >> > >> > > >> > >> > 494=3D488:5:1:0:0:0:0:0 > >> > >> > > >> > >> > [ 2] 6.00-7.00 sec 1015 KBytes 8.32 Mbits/sec > >> > >> > > >> > >> > 144.632/124.368/171.349/16.597 ms (8/129958) 143 KByte 7.188321 > >> > >> > > >> > >> > 408=3D403:5:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 3] 6.00-7.00 sec 1.02 MBytes 8.60 Mbits/sec > >> > >> > > >> > >> > 143.516/102.322/173.001/24.089 ms (8/134302) 146 KByte 7.486359 > >> > >> > > >> > >> > 484=3D480:4:0:0:0:0:0:0 > >> > >> > > >> > >> > [SUM] 6.00-7.00 sec 4.98 MBytes 41.7 Mbits/sec > >> > >> > > >> > >> > 2097=3D2078:17:2:0:0:0:0:0 > >> > >> > > >> > >> > [ 4] 7.00-8.00 sec 1.77 MBytes 14.9 Mbits/sec > >> > >> > > >> > >> > 85.406/65.797/103.418/12.609 ms (14/132595) 153 KByte 21.74 > >> > >> > > >> > >> > 692=3D687:2:3:0:0:0:0:0 > >> > >> > > >> > >> > [ 2] 7.00-8.00 sec 957 KBytes 7.84 Mbits/sec > >> > >> > > >> > >> > 153.936/131.452/191.464/19.361 ms (7/140042) 160 KByte 6.368199 > >> > >> > > >> > >> > 429=3D425:4:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 1] 7.00-8.00 sec 1.13 MBytes 9.44 Mbits/sec > >> > >> > > >> > >> > 131.146/109.737/166.774/22.035 ms (9/131124) 146 KByte 8.998528 > >> > >> > > >> > >> > 520=3D516:4:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 3] 7.00-8.00 sec 1.13 MBytes 9.51 Mbits/sec > >> > >> > > >> > >> > 126.512/88.404/220.175/42.237 ms (9/132089) 172 KByte 9.396784 > >> > >> > > >> > >> > 527=3D524:1:2:0:0:0:0:0 > >> > >> > > >> > >> > [SUM] 7.00-8.00 sec 4.96 MBytes 41.6 Mbits/sec > >> > >> > > >> > >> > 2168=3D2152:11:5:0:0:0:0:0 > >> > >> > > >> > >> > With source side rate limiting to 9 mb/s per stream. > >> > >> > > >> > >> > [rjmcmahon@ryzen3950 iperf2-code]$ iperf -c *** --hide-ips -e > >> > >> > > >> > >> > --trip-times -i 1 -P 4 -t 10 -w 4m --tcp-quickack -N -b 9m > >> > >> > > >> > >> > ------------------------------------------------------------ > >> > >> > > >> > >> > Client connecting to (**hidden**), TCP port 5001 with pid 108884 (4 > >> > >> > > >> > >> > flows) > >> > >> > > >> > >> > Write buffer size: 131072 Byte > >> > >> > > >> > >> > TOS set to 0x0 and nodelay (Nagle off) > >> > >> > > >> > >> > TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte) > >> > >> > > >> > >> > Event based writes (pending queue watermark at 16384 bytes) > >> > >> > > >> > >> > ------------------------------------------------------------ > >> > >> > > >> > >> > [ 1] local *.*.*.85%enp4s0 port 46448 connected with *.*.*.123 port > >> > >> > > >> > >> > 5001 (prefetch=3D16384) (trip-times) (sock=3D3) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/10666) (ct=3D10.70 ms) on 2023-01-06 12:27= :45 > >> > >> > > >> > >> > (PST) > >> > >> > > >> > >> > [ 3] local *.*.*.85%enp4s0 port 46460 connected with *.*.*.123 port > >> > >> > > >> > >> > 5001 (prefetch=3D16384) (trip-times) (sock=3D6) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/16499) (ct=3D16.54 ms) on 2023-01-06 12:27= :45 > >> > >> > > >> > >> > (PST) > >> > >> > > >> > >> > [ 2] local *.*.*.85%enp4s0 port 46454 connected with *.*.*.123 port > >> > >> > > >> > >> > 5001 (prefetch=3D16384) (trip-times) (sock=3D4) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/16580) (ct=3D16.86 ms) on 2023-01-06 12:27= :45 > >> > >> > > >> > >> > (PST) > >> > >> > > >> > >> > [ 4] local *.*.*.85%enp4s0 port 46458 connected with *.*.*.123 port > >> > >> > > >> > >> > 5001 (prefetch=3D16384) (trip-times) (sock=3D5) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/16802) (ct=3D16.83 ms) on 2023-01-06 12:27= :45 > >> > >> > > >> > >> > (PST) > >> > >> > > >> > >> > [ ID] Interval Transfer Bandwidth Write/Err Rtry > >> > >> > > >> > >> > Cwnd/RTT(var) NetPwr > >> > >> > > >> > >> > ... > >> > >> > > >> > >> > [ 2] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0 > >> > >> > > >> > >> > 134K/88055(12329) us 11.91 > >> > >> > > >> > >> > [ 4] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0 > >> > >> > > >> > >> > 132K/74867(11755) us 14.01 > >> > >> > > >> > >> > [ 1] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0 > >> > >> > > >> > >> > 134K/89101(13134) us 11.77 > >> > >> > > >> > >> > [ 3] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0 > >> > >> > > >> > >> > 131K/91451(11938) us 11.47 > >> > >> > > >> > >> > [SUM] 4.00-5.00 sec 4.00 MBytes 33.6 Mbits/sec 32/0 0 > >> > >> > > >> > >> > [ 2] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0 > >> > >> > > >> > >> > 134K/85135(14580) us 13.86 > >> > >> > > >> > >> > [ 4] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0 > >> > >> > > >> > >> > 132K/85124(15654) us 13.86 > >> > >> > > >> > >> > [ 1] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0 > >> > >> > > >> > >> > 134K/91336(11335) us 12.92 > >> > >> > > >> > >> > [ 3] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0 > >> > >> > > >> > >> > 131K/89185(13499) us 13.23 > >> > >> > > >> > >> > [SUM] 5.00-6.00 sec 4.50 MBytes 37.7 Mbits/sec 36/0 0 > >> > >> > > >> > >> > [ 2] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0 > >> > >> > > >> > >> > 134K/85687(13489) us 13.77 > >> > >> > > >> > >> > [ 4] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0 > >> > >> > > >> > >> > 132K/82803(13001) us 14.25 > >> > >> > > >> > >> > [ 1] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0 > >> > >> > > >> > >> > 134K/86869(15186) us 13.58 > >> > >> > > >> > >> > [ 3] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0 > >> > >> > > >> > >> > 131K/91447(12515) us 12.90 > >> > >> > > >> > >> > [SUM] 6.00-7.00 sec 4.50 MBytes 37.7 Mbits/sec 36/0 0 > >> > >> > > >> > >> > [ 2] 7.00-8.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0 > >> > >> > > >> > >> > 134K/81814(13168) us 12.82 > >> > >> > > >> > >> > [ 4] 7.00-8.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0 > >> > >> > > >> > >> > 132K/89008(13283) us 11.78 > >> > >> > > >> > >> > [ 1] 7.00-8.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0 > >> > >> > > >> > >> > 134K/89494(12151) us 11.72 > >> > >> > > >> > >> > [ 3] 7.00-8.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0 > >> > >> > > >> > >> > 131K/91083(12797) us 11.51 > >> > >> > > >> > >> > [SUM] 7.00-8.00 sec 4.00 MBytes 33.6 Mbits/sec 32/0 0 > >> > >> > > >> > >> > [root@bobcat iperf2-code]# iperf -s -i 1 -e --hide-ips -w 4m > >> > >> > > >> > >> > ------------------------------------------------------------ > >> > >> > > >> > >> > Server listening on TCP port 5001 with pid 233981 > >> > >> > > >> > >> > Read buffer size: 128 KByte (Dist bin width=3D16.0 KByte) > >> > >> > > >> > >> > TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte) > >> > >> > > >> > >> > ------------------------------------------------------------ > >> > >> > > >> > >> > [ 1] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port > >> > >> > 46448 > >> > >> > > >> > >> > (trip-times) (sock=3D4) (peer 2.1.9-master) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/11987) on 2023-01-06 12:27:45 (PST) > >> > >> > > >> > >> > [ 2] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port > >> > >> > 46454 > >> > >> > > >> > >> > (trip-times) (sock=3D5) (peer 2.1.9-master) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/11132) on 2023-01-06 12:27:45 (PST) > >> > >> > > >> > >> > [ 3] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port > >> > >> > 46460 > >> > >> > > >> > >> > (trip-times) (sock=3D6) (peer 2.1.9-master) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/11097) on 2023-01-06 12:27:45 (PST) > >> > >> > > >> > >> > [ 4] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port > >> > >> > 46458 > >> > >> > > >> > >> > (trip-times) (sock=3D7) (peer 2.1.9-master) (qack) > >> > >> > > >> > >> > (icwnd/mss/irtt=3D14/1448/17823) on 2023-01-06 12:27:45 (PST) > >> > >> > > >> > >> > [ ID] Interval Transfer Bandwidth Burst Latency > >> > >> > > >> > >> > avg/min/max/stdev (cnt/size) inP NetPwr Reads=3DDist > >> > >> > > >> > >> > [ 4] 0.00-1.00 sec 1.09 MBytes 9.15 Mbits/sec > >> > >> > > >> > >> > 93.383/90.103/95.661/2.232 ms (8/143028) 128 KByte 12.25 > >> > >> > > >> > >> > 451=3D451:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 3] 0.00-1.00 sec 1.08 MBytes 9.06 Mbits/sec > >> > >> > > >> > >> > 96.834/95.229/102.645/2.442 ms (8/141580) 131 KByte 11.70 > >> > >> > > >> > >> > 472=3D472:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 1] 0.00-1.00 sec 1.10 MBytes 9.19 Mbits/sec > >> > >> > > >> > >> > 95.183/92.623/97.579/1.431 ms (8/143571) 131 KByte 12.07 > >> > >> > > >> > >> > 495=3D495:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 2] 0.00-1.00 sec 1.09 MBytes 9.15 Mbits/sec > >> > >> > > >> > >> > 89.317/84.865/94.906/3.674 ms (8/143028) 122 KByte 12.81 > >> > >> > > >> > >> > 489=3D489:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ ID] Interval Transfer Bandwidth Reads=3DDist > >> > >> > > >> > >> > [SUM] 0.00-1.00 sec 4.36 MBytes 36.6 Mbits/sec > >> > >> > > >> > >> > 1907=3D1907:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 4] 1.00-2.00 sec 1.07 MBytes 8.95 Mbits/sec > >> > >> > > >> > >> > 92.649/89.987/95.036/1.828 ms (9/124314) 96.5 KByte 12.08 > >> > >> > > >> > >> > 492=3D492:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 3] 1.00-2.00 sec 1.06 MBytes 8.93 Mbits/sec > >> > >> > > >> > >> > 96.305/95.647/96.794/0.432 ms (9/123992) 100 KByte 11.59 > >> > >> > > >> > >> > 480=3D480:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 1] 1.00-2.00 sec 1.06 MBytes 8.89 Mbits/sec > >> > >> > > >> > >> > 92.578/90.866/94.145/1.371 ms (9/123510) 95.8 KByte 12.01 > >> > >> > > >> > >> > 513=3D513:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 2] 1.00-2.00 sec 1.07 MBytes 8.96 Mbits/sec > >> > >> > > >> > >> > 90.767/87.984/94.352/1.944 ms (9/124475) 94.7 KByte 12.34 > >> > >> > > >> > >> > 489=3D489:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [SUM] 1.00-2.00 sec 4.26 MBytes 35.7 Mbits/sec > >> > >> > > >> > >> > 1974=3D1974:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 4] 2.00-3.00 sec 1.09 MBytes 9.13 Mbits/sec > >> > >> > > >> > >> > 93.977/91.795/96.561/1.693 ms (8/142656) 112 KByte 12.14 > >> > >> > > >> > >> > 497=3D497:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 3] 2.00-3.00 sec 1.08 MBytes 9.04 Mbits/sec > >> > >> > > >> > >> > 96.544/95.815/97.798/0.693 ms (8/141208) 114 KByte 11.70 > >> > >> > > >> > >> > 503=3D503:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 1] 2.00-3.00 sec 1.07 MBytes 9.01 Mbits/sec > >> > >> > > >> > >> > 93.970/91.193/96.325/1.796 ms (8/140846) 111 KByte 11.99 > >> > >> > > >> > >> > 509=3D509:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 2] 2.00-3.00 sec 1.08 MBytes 9.10 Mbits/sec > >> > >> > > >> > >> > 92.843/90.216/96.355/2.040 ms (8/142113) 111 KByte 12.25 > >> > >> > > >> > >> > 509=3D509:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [SUM] 2.00-3.00 sec 4.32 MBytes 36.3 Mbits/sec > >> > >> > > >> > >> > 2018=3D2018:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 4] 3.00-4.00 sec 1.06 MBytes 8.86 Mbits/sec > >> > >> > > >> > >> > 93.222/89.063/96.104/2.346 ms (9/123027) 96.1 KByte 11.88 > >> > >> > > >> > >> > 487=3D487:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 3] 3.00-4.00 sec 1.07 MBytes 8.97 Mbits/sec > >> > >> > > >> > >> > 96.277/95.051/97.230/0.767 ms (9/124636) 101 KByte 11.65 > >> > >> > > >> > >> > 489=3D489:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 1] 3.00-4.00 sec 1.08 MBytes 9.02 Mbits/sec > >> > >> > > >> > >> > 93.899/88.732/96.972/2.737 ms (9/125280) 98.6 KByte 12.01 > >> > >> > > >> > >> > 493=3D493:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 2] 3.00-4.00 sec 1.07 MBytes 8.97 Mbits/sec > >> > >> > > >> > >> > 92.490/89.862/95.265/1.796 ms (9/124636) 96.6 KByte 12.13 > >> > >> > > >> > >> > 493=3D493:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [SUM] 3.00-4.00 sec 4.27 MBytes 35.8 Mbits/sec > >> > >> > > >> > >> > 1962=3D1962:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 4] 4.00-5.00 sec 1.07 MBytes 9.00 Mbits/sec > >> > >> > > >> > >> > 92.431/81.888/96.221/4.524 ms (9/124958) 96.8 KByte 12.17 > >> > >> > > >> > >> > 498=3D498:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 1] 4.00-5.00 sec 1.07 MBytes 8.97 Mbits/sec > >> > >> > > >> > >> > 95.018/93.445/96.200/0.957 ms (9/124636) 99.3 KByte 11.81 > >> > >> > > >> > >> > 490=3D490:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 2] 4.00-5.00 sec 1.06 MBytes 8.90 Mbits/sec > >> > >> > > >> > >> > 93.874/86.485/95.672/2.810 ms (9/123671) 97.3 KByte 11.86 > >> > >> > > >> > >> > 481=3D481:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 3] 4.00-5.00 sec 1.08 MBytes 9.09 Mbits/sec > >> > >> > > >> > >> > 95.737/93.881/97.197/0.972 ms (9/126245) 101 KByte 11.87 > >> > >> > > >> > >> > 484=3D484:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [SUM] 4.00-5.00 sec 4.29 MBytes 36.0 Mbits/sec > >> > >> > > >> > >> > 1953=3D1953:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 4] 5.00-6.00 sec 1.09 MBytes 9.13 Mbits/sec > >> > >> > > >> > >> > 92.908/86.844/95.994/3.012 ms (8/142656) 111 KByte 12.28 > >> > >> > > >> > >> > 467=3D467:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 3] 5.00-6.00 sec 1.07 MBytes 8.94 Mbits/sec > >> > >> > > >> > >> > 96.593/95.343/97.660/0.876 ms (8/139760) 113 KByte 11.58 > >> > >> > > >> > >> > 478=3D478:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 1] 5.00-6.00 sec 1.08 MBytes 9.03 Mbits/sec > >> > >> > > >> > >> > 95.021/91.421/97.167/1.893 ms (8/141027) 112 KByte 11.87 > >> > >> > > >> > >> > 491=3D491:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 2] 5.00-6.00 sec 1.08 MBytes 9.06 Mbits/sec > >> > >> > > >> > >> > 92.162/82.720/97.692/5.060 ms (8/141570) 109 KByte 12.29 > >> > >> > > >> > >> > 488=3D488:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [SUM] 5.00-6.00 sec 4.31 MBytes 36.2 Mbits/sec > >> > >> > > >> > >> > 1924=3D1924:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 4] 6.00-7.00 sec 1.04 MBytes 8.70 Mbits/sec > >> > >> > > >> > >> > 92.793/85.343/96.967/3.552 ms (9/120775) 93.9 KByte 11.71 > >> > >> > > >> > >> > 485=3D485:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 2] 6.00-7.00 sec 1.05 MBytes 8.79 Mbits/sec > >> > >> > > >> > >> > 91.679/84.479/96.760/3.975 ms (9/122062) 93.8 KByte 11.98 > >> > >> > > >> > >> > 472=3D472:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 3] 6.00-7.00 sec 1.06 MBytes 8.88 Mbits/sec > >> > >> > > >> > >> > 96.982/95.933/98.371/0.680 ms (9/123349) 100 KByte 11.45 > >> > >> > > >> > >> > 477=3D477:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 1] 6.00-7.00 sec 1.05 MBytes 8.80 Mbits/sec > >> > >> > > >> > >> > 94.342/91.660/96.025/1.660 ms (9/122223) 96.7 KByte 11.66 > >> > >> > > >> > >> > 494=3D494:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [SUM] 6.00-7.00 sec 4.19 MBytes 35.2 Mbits/sec > >> > >> > > >> > >> > 1928=3D1928:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 4] 7.00-8.00 sec 1.10 MBytes 9.25 Mbits/sec > >> > >> > > >> > >> > 92.515/88.182/96.351/2.538 ms (8/144466) 112 KByte 12.49 > >> > >> > > >> > >> > 510=3D510:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 3] 7.00-8.00 sec 1.09 MBytes 9.13 Mbits/sec > >> > >> > > >> > >> > 96.580/95.737/98.977/1.098 ms (8/142656) 115 KByte 11.82 > >> > >> > > >> > >> > 480=3D480:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 1] 7.00-8.00 sec 1.10 MBytes 9.21 Mbits/sec > >> > >> > > >> > >> > 95.269/91.719/97.514/2.126 ms (8/143923) 115 KByte 12.09 > >> > >> > > >> > >> > 515=3D515:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [ 2] 7.00-8.00 sec 1.11 MBytes 9.29 Mbits/sec > >> > >> > > >> > >> > 90.073/84.700/96.176/4.324 ms (8/145190) 110 KByte 12.90 > >> > >> > > >> > >> > 508=3D508:0:0:0:0:0:0:0 > >> > >> > > >> > >> > [SUM] 7.00-8.00 sec 4.40 MBytes 36.9 Mbits/sec > >> > >> > > >> > >> > 2013=3D2013:0:0:0:0:0:0:0 > >> > >> > > >> > >> > Bob > >> > >> > > >> > >> >>> -----Original Message----- > >> > >> > > >> > >> >> > >> > >> > > >> > >> >>> From: LibreQoS On Behalf > >> > >> > Of > >> > >> > > >> > >> >> Dave Taht > >> > >> > > >> > >> >> > >> > >> > > >> > >> >>> via LibreQoS > >> > >> > > >> > >> >> > >> > >> > > >> > >> >>> Sent: Wednesday, January 4, 2023 12:26 PM > >> > >> > > >> > >> >> > >> > >> > > >> > >> >>> Subject: [LibreQoS] the grinch meets cloudflare's christmas presen= t > >> > >> > > >> > >> > > >> > >> >> > >> > >> > > >> > >> >>> > >> > >> > > >> > >> >> > >> > >> > > >> > >> >>> Please try the new, the shiny, the really wonderful test here: > >> > >> > > >> > >> >> > >> > >> > > >> > >> >>> > >> > >> > > >> > >> >> > >> > >> > > https://urldefense.com/v3/__https://speed.cloudflare.com/__;!!BhdT!iZcFJ8= WVU9S > >> > >> > > >> > >> > > >> > >> >> [1] > >> > >> > > >> > >> >> > >> > >> > > >> > >> >>> > >> > >> > > >> > >> >> > >> > >> > > 9zz5t456oxkfObrC5Xb9j5AG8UO3DqD5x4GAJkawZr0iGwEUtF0_09U8mCDnAkrJ9QEMHGbCM= KVw$ > >> > >> > > >> > >> > > >> > >> >> [1] > >> > >> > > >> > >> >> > >> > >> > > >> > >> >>> > >> > >> > > >> > >> >> > >> > >> > > >> > >> >>> I would really appreciate some independent verification of > >> > >> > > >> > >> >> > >> > >> > > >> > >> >>> measurements using this tool. In my brief experiments it appears - > >> > >> > > >> > >> >> as > >> > >> > > >> > >> >> > >> > >> > > >> > >> >>> all the commercial tools to date - to dramatically understate the > >> > >> > > >> > >> >> > >> > >> > > >> > >> >>> bufferbloat, on my LTE, (and my starlink terminal is out being > >> > >> > > >> > >> >> > >> > >> > > >> > >> >>> hacked^H^H^H^H^H^Hworked on, so I can't measure that) > >> > >> > > >> > >> >> > >> > >> > > >> > >> >> [acm] > >> > >> > > >> > >> >> > >> > >> > > >> > >> >> Hi Dave, I made some time to test "cloudflare's christmas present" > >> > >> > > >> > >> >> yesterday. > >> > >> > > >> > >> >> > >> > >> > > >> > >> >> I'm on DOCSIS 3.1 service with 1Gbps Down. The Upstream has a > >> > >> > "turbo" > >> > >> > > >> > >> >> mode with 40-50Mbps for the first ~3 sec, then steady-state about > >> > >> > > >> > >> >> 23Mbps. > >> > >> > > >> > >> >> > >> > >> > > >> > >> >> When I saw the ~620Mbps Downstream measurement, I was ready to > >> > >> > > >> > >> >> complain that even the IP-Layer Capacity was grossly underestimated= . > >> > >> > > >> > >> > > >> > >> >> In addition, the Latency measurements seem very low (as you > >> > >> > asserted), > >> > >> > > >> > >> >> although the cloud server was "nearby". > >> > >> > > >> > >> >> > >> > >> > > >> > >> >> However, I found that Ookla and the ISP-provided measurement were > >> > >> > also > >> > >> > > >> > >> >> reporting ~600Mbps! So the cloudflare Downstream capacity (or > >> > >> > > >> > >> >> throughput?) measurement was consistent with others. Our UDPST > >> > >> > server > >> > >> > > >> > >> >> was unreachable, otherwise I would have added that measurement, too= . > >> > >> > > >> > >> > > >> > >> >> > >> > >> > > >> > >> >> The Upstream measurement graph seems to illustrate the "turbo" > >> > >> > > >> > >> >> mode, with the dip after attaining 44.5Mbps. > >> > >> > > >> > >> >> > >> > >> > > >> > >> >> UDPST saturates the uplink and we measure the full 250ms of the > >> > >> > > >> > >> >> Upstream buffer. Cloudflare's latency measurements don't even come > >> > >> > > >> > >> >> close. > >> > >> > > >> > >> >> > >> > >> > > >> > >> >> Al > >> > >> > > >> > >> >> > >> > >> > > >> > >> >> > >> > >> > > >> > >> >> > >> > >> > > >> > >> >> Links: > >> > >> > > >> > >> >> ------ > >> > >> > > >> > >> >> [1] > >> > >> > > >> > >> >> > >> > >> > > https://urldefense.com/v3/__https:/speed.cloudflare.com/__;!!BhdT!iZcFJ8W= VU9S9zz5t456oxkfObrC5Xb9j5AG8UO3DqD5x4GAJkawZr0iGwEUtF0_09U8mCDnAkrJ9QEMHGb= CMKVw$ > >> > >> > > >> > >> > > >> > >> >> _______________________________________________ > >> > >> > > >> > >> >> Rpm mailing list > >> > >> > > >> > >> >> Rpm@lists.bufferbloat.net > >> > >> > > >> > >> >> https://lists.bufferbloat.net/listinfo/rpm > >> > >> > > >> > >> > _______________________________________________ > >> > >> > > >> > >> > Starlink mailing list > >> > >> > > >> > >> > Starlink@lists.bufferbloat.net > >> > >> > > >> > >> > https://lists.bufferbloat.net/listinfo/starlink > >> > >> _______________________________________________ > >> Bloat mailing list > >> Bloat@lists.bufferbloat.net > >> https://lists.bufferbloat.net/listinfo/bloat > > > > _______________________________________________ > > Rpm mailing list > > Rpm@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/rpm > > > > -- > This song goes out to all the folk that thought Stadia would work: > > https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-698136666= 5607352320-FXtz > Dave T=C3=A4ht CEO, TekLibre, LLC > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > --=20 --=20 Jay Moran http://linkedin.com/in/jaycmoran --00000000000021e47f05f1fafb8a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Quick note from reading your blog entry.

Last night, I played with the Cloudflare= Speedtest a little. It downloads 25MB and a 50MB (or 100MB, can=E2=80=99t = remember) as well on a =E2=80=9Cspeedier=E2=80=9D network after it does the= 10MB file.

I was gettin= g 1.2Gbs down and 760Mbs up, 4ms of LUL, and seeing those larger file sizes= . I was trying to screenshot and noticed I had those extra file sizes I had= to scroll down for. I ended up getting distracted and not taking the shot = to send. But, it will do a longer/bigger test under right conditions.
=

Network here at the house is = AT&T Fiber, 5Gbs up/down - limited to 3.6Gbs down from Ubiquity UDM SE = router/firewall with all IPS/Geo-blocking turned on. 4.7Gbs non-blocking up= . I am building a pfSense box to eliminate the bottleneck. Couldn=E2=80=99t= be happier, good job AS7018.

The machine I was testing from was Win10 wired 10Gbs and gets ~2.2Gbs= up/down for fast.com/speedtest.n= et. I haven=E2=80=99t take time to test internally or try and tune that= system, or might be CAT5e cabling issue=E2=80=A6 is fast enough for me for= that system.

Jay
<= div>
On= Wed, Jan 11, 2023 at 12:07 AM Dave Taht via Bloat <bloat@lists.bufferbloat.net> wrote:
Dear Luis:

You hit 17 seconds of delay on your test.

I got you beat, today, on my LTE connection, I cracked 182 seconds.

I'd like to thank Verizon for making it possible for me to spew 4000 words on my kvetches about the current speedtest regimes of speedtest,
cloudflare, and so on, by making my network connection so lousy today
that I sat in front of emacs to rant - and y'all for helping tone
down, a little, this blog entry:

https://blog.cerowrt.org/post/speedtests/

On Tue, Jan 10, 2023 at 9:25 AM Luis A. Cornejo via Rpm
<rpm@list= s.bufferbloat.net> wrote:
>
> Here is my VZ HSI
>
>
> No SQMm on
>
> On Sat, Jan 7, 2023 at 6:38 PM Dick Roy via Bloat <bloat@lists.bufferbloat.ne= t> wrote:
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: rjmcmahon [mailto:rjmcmahon@rjmcmahon.com]
>> Sent: Friday, January 6, 2023 3:45 PM
>> To: dick= roy@alum.mit.edu
>> Cc: 'MORTON JR., AL'; 'IETF IPPM WG'; 'libreqo= s'; 'Cake List'; 'Rpm'; 'bloat'
>> Subject: Re: [Starlink] [Rpm] [LibreQoS] the grinch meets cloudfla= re'schristmas present
>>
>>
>>
>> yeah, I'd prefer not to output CLT sample groups at all but th= e
>>
>> histograms aren't really human readable and users constantly a= sk for
>>
>> them. I thought about providing a distance from the gaussian as ou= tput
>>
>> too but so far few would understand it and nobody I found would ac= t upon
>>
>> it.
>>
>> [RR] Understandable until such metrics are =E2=80=9Cactionable=E2= =80=9D, and that=E2=80=99s =E2=80=9Cup to us to find/define/figure out=E2= =80=9D it seems to me. Metrics that are not actionable are write-only memor= y and good for little but historical recordJ
>>
>> The tool produces the full histograms so no information is really<= br> >>
>> missing except for maybe better time series analysis.
>>
>> [RR] Isn=E2=80=99t that in fact what we are trying to extract from= the e2e stats we collect?=C2=A0 i.e., infer the time evolution of the syst= em from its I/O behavior? As you point out, it=E2=80=99s really hard to do = without probes in the guts of the system, nd yes, synchronization is import= ant J
>>
>>
>>
>> The open source flows python code also released with iperf 2 does = use
>>
>> the komogorov-smirnov distances & distance matrices to cluster= when the
>>
>> number of histograms are just too much. We've analyzed 1M runs= to fault
>>
>> isolate the "unexpected interruptions" or "bugs&quo= t; and without statistical
>>
>> support it is just not doable. This does require instrumentation o= f the
>>
>> full path with mapping to a common clock domain (e.g. GPS) and not= just
>>
>> e2e stats. I find an e2e complaint by an end user about "poor= speed" as
>>
>> useful as telling a pharmacist I have a fever. Not much diagnostic= ally
>>
>> is going on. Take an aspirin.
>>
>> [RR] That=E2=80=99s AWESOME!!!!!!!!!!!!!!!!!! I love that analogy!=
>>
>>
>>
>> RR
>>
>>
>>
>> https://en.wikipedia.org/wiki/= Kolmogorov%E2%80%93Smirnov_test
>>
>> https://sourceforge.net/= p/iperf2/code/ci/master/tree/flows/flows.py
>>
>>
>>
>> Bob
>>
>> > See below =E2=80=A6
>>
>> >
>>
>> > -----Original Message-----
>>
>> > From: Starlink [mailto:starlink-bounces@lists.bufferbloat.ne= t] On
>>
>> > Behalf Of rjmcmahon via Starlink
>>
>> > Sent: Friday, January 6, 2023 12:39 PM
>>
>> > To: MORTON JR., AL
>>
>> > Cc: Dave Taht via Starlink; IETF IPPM WG; libreqos; Cake List= ; Rpm;
>>
>> > bloat
>>
>> > Subject: Re: [Starlink] [Rpm] [LibreQoS] the grinch meets
>>
>> > cloudflare'schristmas present
>>
>> >
>>
>> > Some thoughts are not to use UDP for testing here. Also, thes= e speed
>>
>> >
>>
>> > tests have little to no information for network engineers abo= ut what's
>>
>> >
>>
>> >
>>
>> > going on. Iperf 2 may better assist network engineers but the= n I'm
>>
>> >
>>
>> > biased ;)
>>
>> >
>>
>> > Running iperf 2 https://sourceforge.net/projec= ts/iperf2/ with
>>
>> >
>>
>> > --trip-times. Though the sampling and central limit theorem a= veraging
>>
>> > is
>>
>> >
>>
>> > hiding the real distributions (use --histograms to get those)=
>>
>> >
>>
>> > _[RR] FWIW (IMNBWM __J)=E2=80=A6 If the output/final histogra= ms indicate the
>>
>> > PDF is NOT Gaussian, then any application of the CLT is
>>
>> > inappropriate/contra-indicated! The CLT is a "proof unde= r certain
>>
>> > regularity conditions/assumptions of underlying/constituent P= DFs, that
>>
>> > the resulting PDF (after all the necessary convolutions are p= erformed
>>
>> > to get to the PDF of the output) will asymptotically approach= a
>>
>> > Gaussian with only a mean and a std. dev. left to specify. _<= br> >>
>> >
>>
>> > Below are 4 parallel TCP streams from my home to one of my se= rvers in
>>
>> >
>>
>> > the cloud. First where TCP is limited per CCA. Second is sour= ce side
>>
>> >
>>
>> > write rate limiting. Things to note:
>>
>> >
>>
>> > o) connect times for both at 10-15 ms
>>
>> >
>>
>> > o) multiple TCP retries on a few rites - one case is 4 with 5= writes.
>>
>> >
>>
>> > Source side pacing eliminates retries
>>
>> >
>>
>> > o) Fairness with CCA isn't great but quite good with sour= ce side write
>>
>> >
>>
>> >
>>
>> > pacing
>>
>> >
>>
>> > o) Queue depth with CCA is about 150 Kbytes about 100K byte w= ith
>>
>> > source
>>
>> >
>>
>> > side pacing
>>
>> >
>>
>> > o) min write to read is about 80 ms for both
>>
>> >
>>
>> > o) max is 220 ms vs 97 ms
>>
>> >
>>
>> > o) stdev for CCA write/read is 30 ms vs 3 ms
>>
>> >
>>
>> > o) TCP RTT is 20ms w/CCA and 90 ms with ssp - seems odd here = as
>>
>> >
>>
>> > TCP_QUICACK and TCP_NODELAY are both enabled.
>>
>> >
>>
>> > [ CT] final connect times (min/avg/max/stdev) =3D
>>
>> >
>>
>> > 10.326/13.522/14.986/2150.329 ms (tot/err) =3D 4/0
>>
>> >
>>
>> > [rjmcmahon@ryzen3950 iperf2-code]$ iperf -c *** --hide-ips -e=
>>
>> >
>>
>> > --trip-times -i 1 -P 4 -t 10 -w 4m --tcp-quickack -N
>>
>> >
>>
>> > ------------------------------------------------------------<= br> >>
>> >
>>
>> > Client connecting to (**hidden**), TCP port 5001 with pid 107= 678 (4
>>
>> >
>>
>> > flows)
>>
>> >
>>
>> > Write buffer size: 131072 Byte
>>
>> >
>>
>> > TOS set to 0x0 and nodelay (Nagle off)
>>
>> >
>>
>> > TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte) >>
>> >
>>
>> > Event based writes (pending queue watermark at 16384 bytes) >>
>> >
>>
>> > ------------------------------------------------------------<= br> >>
>> >
>>
>> > [=C2=A0 1] local *.*.*.85%enp4s0 port 42480 connected with *.= *.*.123 port
>>
>> >
>>
>> > 5001 (prefetch=3D16384) (trip-times) (sock=3D3) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/10534) (ct=3D10.63 ms) on 2023-01-0= 6 12:17:56
>>
>> >
>>
>> > (PST)
>>
>> >
>>
>> > [=C2=A0 4] local *.*.*.85%enp4s0 port 42488 connected with *.= *.*.123 port
>>
>> >
>>
>> > 5001 (prefetch=3D16384) (trip-times) (sock=3D5) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/14023) (ct=3D14.08 ms) on 2023-01-0= 6 12:17:56
>>
>> >
>>
>> > (PST)
>>
>> >
>>
>> > [=C2=A0 3] local *.*.*.85%enp4s0 port 42502 connected with *.= *.*.123 port
>>
>> >
>>
>> > 5001 (prefetch=3D16384) (trip-times) (sock=3D6) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/14642) (ct=3D14.70 ms) on 2023-01-0= 6 12:17:56
>>
>> >
>>
>> > (PST)
>>
>> >
>>
>> > [=C2=A0 2] local *.*.*.85%enp4s0 port 42484 connected with *.= *.*.123 port
>>
>> >
>>
>> > 5001 (prefetch=3D16384) (trip-times) (sock=3D4) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/14728) (ct=3D14.79 ms) on 2023-01-0= 6 12:17:56
>>
>> >
>>
>> > (PST)
>>
>> >
>>
>> > [ ID] Interval=C2=A0 =C2=A0 =C2=A0 =C2=A0 Transfer=C2=A0 =C2= =A0 Bandwidth=C2=A0 =C2=A0 =C2=A0 =C2=A0Write/Err=C2=A0 Rtry
>>
>> >
>>
>> > Cwnd/RTT(var)=C2=A0 =C2=A0 =C2=A0 =C2=A0 NetPwr
>>
>> >
>>
>> > ...
>>
>> >
>>
>> > [=C2=A0 4] 4.00-5.00 sec=C2=A0 1.38 MBytes=C2=A0 11.5 Mbits/s= ec=C2=A0 11/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A03
>>
>> >
>>
>> >
>>
>> > 29K/21088(1142) us=C2=A0 68.37
>>
>> >
>>
>> > [=C2=A0 2] 4.00-5.00 sec=C2=A0 1.62 MBytes=C2=A0 13.6 Mbits/s= ec=C2=A0 13/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A02
>>
>> >
>>
>> >
>>
>> > 31K/19284(612) us=C2=A0 88.36
>>
>> >
>>
>> > [=C2=A0 1] 4.00-5.00 sec=C2=A0 =C2=A0896 KBytes=C2=A0 7.34 Mb= its/sec=C2=A0 7/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A05
>>
>> >
>>
>> > 16K/18996(658) us=C2=A0 48.30
>>
>> >
>>
>> > [=C2=A0 3] 4.00-5.00 sec=C2=A0 1.00 MBytes=C2=A0 8.39 Mbits/s= ec=C2=A0 8/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A05
>>
>> >
>>
>> > 18K/18133(208) us=C2=A0 57.83
>>
>> >
>>
>> > [SUM] 4.00-5.00 sec=C2=A0 4.88 MBytes=C2=A0 40.9 Mbits/sec=C2= =A0 39/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 15
>>
>> >
>>
>> > [=C2=A0 4] 5.00-6.00 sec=C2=A0 1.25 MBytes=C2=A0 10.5 Mbits/s= ec=C2=A0 10/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A04
>>
>> >
>>
>> >
>>
>> > 29K/14717(489) us=C2=A0 89.06
>>
>> >
>>
>> > [=C2=A0 1] 5.00-6.00 sec=C2=A0 1.00 MBytes=C2=A0 8.39 Mbits/s= ec=C2=A0 8/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A04
>>
>> >
>>
>> > 16K/15874(408) us=C2=A0 66.06
>>
>> >
>>
>> > [=C2=A0 3] 5.00-6.00 sec=C2=A0 1.12 MBytes=C2=A0 9.44 Mbits/s= ec=C2=A0 9/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A04
>>
>> >
>>
>> > 16K/15826(382) us=C2=A0 74.54
>>
>> >
>>
>> > [=C2=A0 2] 5.00-6.00 sec=C2=A0 1.50 MBytes=C2=A0 12.6 Mbits/s= ec=C2=A0 12/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A06
>>
>> >
>>
>> >
>>
>> > 9K/14878(557) us=C2=A0 106
>>
>> >
>>
>> > [SUM] 5.00-6.00 sec=C2=A0 4.88 MBytes=C2=A0 40.9 Mbits/sec=C2= =A0 39/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 18
>>
>> >
>>
>> > [=C2=A0 4] 6.00-7.00 sec=C2=A0 1.75 MBytes=C2=A0 14.7 Mbits/s= ec=C2=A0 14/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A04
>>
>> >
>>
>> >
>>
>> > 25K/15472(496) us=C2=A0 119
>>
>> >
>>
>> > [=C2=A0 2] 6.00-7.00 sec=C2=A0 1.00 MBytes=C2=A0 8.39 Mbits/s= ec=C2=A0 8/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A02
>>
>> >
>>
>> > 26K/16417(427) us=C2=A0 63.87
>>
>> >
>>
>> > [=C2=A0 1] 6.00-7.00 sec=C2=A0 1.25 MBytes=C2=A0 10.5 Mbits/s= ec=C2=A0 10/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A05
>>
>> >
>>
>> >
>>
>> > 16K/16268(679) us=C2=A0 80.57
>>
>> >
>>
>> > [=C2=A0 3] 6.00-7.00 sec=C2=A0 1.00 MBytes=C2=A0 8.39 Mbits/s= ec=C2=A0 8/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A06
>>
>> >
>>
>> > 15K/16629(799) us=C2=A0 63.06
>>
>> >
>>
>> > [SUM] 6.00-7.00 sec=C2=A0 5.00 MBytes=C2=A0 41.9 Mbits/sec=C2= =A0 40/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 17
>>
>> >
>>
>> > [=C2=A0 4] 7.00-8.00 sec=C2=A0 1.75 MBytes=C2=A0 14.7 Mbits/s= ec=C2=A0 14/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A04
>>
>> >
>>
>> >
>>
>> > 22K/13986(519) us=C2=A0 131
>>
>> >
>>
>> > [=C2=A0 1] 7.00-8.00 sec=C2=A0 1.12 MBytes=C2=A0 9.44 Mbits/s= ec=C2=A0 9/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A04
>>
>> >
>>
>> > 16K/12679(377) us=C2=A0 93.04
>>
>> >
>>
>> > [=C2=A0 3] 7.00-8.00 sec=C2=A0 =C2=A0896 KBytes=C2=A0 7.34 Mb= its/sec=C2=A0 7/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A05
>>
>> >
>>
>> > 14K/12971(367) us=C2=A0 70.74
>>
>> >
>>
>> > [=C2=A0 2] 7.00-8.00 sec=C2=A0 1.12 MBytes=C2=A0 9.44 Mbits/s= ec=C2=A0 9/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A06
>>
>> >
>>
>> > 15K/14740(779) us=C2=A0 80.03
>>
>> >
>>
>> > [SUM] 7.00-8.00 sec=C2=A0 4.88 MBytes=C2=A0 40.9 Mbits/sec=C2= =A0 39/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 19
>>
>> >
>>
>> > [root@bobcat iperf2-code]# iperf -s -i 1 -e --hide-ips -w 4m<= br> >>
>> >
>>
>> > ------------------------------------------------------------<= br> >>
>> >
>>
>> > Server listening on TCP port 5001 with pid 233615
>>
>> >
>>
>> > Read buffer size:=C2=A0 128 KByte (Dist bin width=3D16.0 KByt= e)
>>
>> >
>>
>> > TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte) >>
>> >
>>
>> > ------------------------------------------------------------<= br> >>
>> >
>>
>> > [=C2=A0 1] local *.*.*.123%eth0 port 5001 connected with *.*.= *.171 port
>>
>> > 42480
>>
>> >
>>
>> > (trip-times) (sock=3D4) (peer 2.1.9-master) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/11636) on 2023-01-06 12:17:56 (PST)=
>>
>> >
>>
>> > [=C2=A0 2] local *.*.*.123%eth0 port 5001 connected with *.*.= *.171 port
>>
>> > 42502
>>
>> >
>>
>> > (trip-times) (sock=3D5) (peer 2.1.9-master) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/11898) on 2023-01-06 12:17:56 (PST)=
>>
>> >
>>
>> > [=C2=A0 3] local *.*.*.123%eth0 port 5001 connected with *.*.= *.171 port
>>
>> > 42484
>>
>> >
>>
>> > (trip-times) (sock=3D6) (peer 2.1.9-master) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/11938) on 2023-01-06 12:17:56 (PST)=
>>
>> >
>>
>> > [=C2=A0 4] local *.*.*.123%eth0 port 5001 connected with *.*.= *.171 port
>>
>> > 42488
>>
>> >
>>
>> > (trip-times) (sock=3D7) (peer 2.1.9-master) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/11919) on 2023-01-06 12:17:56 (PST)=
>>
>> >
>>
>> > [ ID] Interval=C2=A0 =C2=A0 =C2=A0 =C2=A0 Transfer=C2=A0 =C2= =A0 Bandwidth=C2=A0 =C2=A0 Burst Latency
>>
>> >
>>
>> > avg/min/max/stdev (cnt/size) inP NetPwr=C2=A0 Reads=3DDist >>
>> >
>>
>> > ...
>>
>> >
>>
>> > [=C2=A0 2] 4.00-5.00 sec=C2=A0 1.06 MBytes=C2=A0 8.86 Mbits/s= ec
>>
>> >
>>
>> > 129.819/90.391/186.075/31.346 ms (9/123080)=C2=A0 154 KByte 8= .532803
>>
>> >
>>
>> > 467=3D461:6:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 3] 4.00-5.00 sec=C2=A0 1.52 MBytes=C2=A0 12.8 Mbits/s= ec
>>
>> >
>>
>> > 103.552/82.653/169.274/28.382 ms (12/132854)=C2=A0 149 KByte = 15.40
>>
>> >
>>
>> > 646=3D643:1:2:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 4] 4.00-5.00 sec=C2=A0 1.39 MBytes=C2=A0 11.6 Mbits/s= ec
>>
>> >
>>
>> > 107.503/66.843/143.038/24.269 ms (11/132294)=C2=A0 149 KByte = 13.54
>>
>> >
>>
>> > 619=3D617:1:1:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 1] 4.00-5.00 sec=C2=A0 =C2=A0988 KBytes=C2=A0 8.10 Mb= its/sec
>>
>> >
>>
>> > 141.389/119.961/178.785/18.812 ms (7/144593)=C2=A0 170 KByte = 7.158641
>>
>> >
>>
>> > 409=3D404:5:0:0:0:0:0:0
>>
>> >
>>
>> > [SUM] 4.00-5.00 sec=C2=A0 4.93 MBytes=C2=A0 41.4 Mbits/sec >>
>> >
>>
>> > 2141=3D2125:13:3:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 4] 5.00-6.00 sec=C2=A0 1.29 MBytes=C2=A0 10.8 Mbits/s= ec
>>
>> >
>>
>> > 118.943/86.253/176.128/31.248 ms (10/135098)=C2=A0 164 KByte = 11.36
>>
>> >
>>
>> > 511=3D506:2:3:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 2] 5.00-6.00 sec=C2=A0 1.09 MBytes=C2=A0 9.17 Mbits/s= ec
>>
>> >
>>
>> > 139.821/102.418/218.875/40.422 ms (9/127424)=C2=A0 148 KByte = 8.202049
>>
>> >
>>
>> > 487=3D484:2:1:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 3] 5.00-6.00 sec=C2=A0 1.51 MBytes=C2=A0 12.6 Mbits/s= ec
>>
>> >
>>
>> > 102.146/77.085/140.893/18.441 ms (13/121520)=C2=A0 151 KByte = 15.47
>>
>> >
>>
>> > 640=3D636:1:3:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 1] 5.00-6.00 sec=C2=A0 =C2=A0981 KBytes=C2=A0 8.04 Mb= its/sec
>>
>> >
>>
>> > 161.901/105.582/219.931/36.260 ms (8/125614)=C2=A0 134 KByte = 6.206944
>>
>> >
>>
>> > 415=3D413:2:0:0:0:0:0:0
>>
>> >
>>
>> > [SUM] 5.00-6.00 sec=C2=A0 4.85 MBytes=C2=A0 40.7 Mbits/sec >>
>> >
>>
>> > 2053=3D2039:7:7:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 4] 6.00-7.00 sec=C2=A0 1.74 MBytes=C2=A0 14.6 Mbits/s= ec
>>
>> >
>>
>> > 88.846/74.297/101.859/7.118 ms (14/130526)=C2=A0 156 KByte 20= .57
>>
>> >
>>
>> > 711=3D707:3:1:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 1] 6.00-7.00 sec=C2=A0 1.22 MBytes=C2=A0 10.2 Mbits/s= ec
>>
>> >
>>
>> > 120.639/100.257/157.567/21.770 ms (10/127568)=C2=A0 157 KByte= 10.57
>>
>> >
>>
>> > 494=3D488:5:1:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 2] 6.00-7.00 sec=C2=A0 1015 KBytes=C2=A0 8.32 Mbits/s= ec
>>
>> >
>>
>> > 144.632/124.368/171.349/16.597 ms (8/129958)=C2=A0 143 KByte = 7.188321
>>
>> >
>>
>> > 408=3D403:5:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 3] 6.00-7.00 sec=C2=A0 1.02 MBytes=C2=A0 8.60 Mbits/s= ec
>>
>> >
>>
>> > 143.516/102.322/173.001/24.089 ms (8/134302)=C2=A0 146 KByte = 7.486359
>>
>> >
>>
>> > 484=3D480:4:0:0:0:0:0:0
>>
>> >
>>
>> > [SUM] 6.00-7.00 sec=C2=A0 4.98 MBytes=C2=A0 41.7 Mbits/sec >>
>> >
>>
>> > 2097=3D2078:17:2:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 4] 7.00-8.00 sec=C2=A0 1.77 MBytes=C2=A0 14.9 Mbits/s= ec
>>
>> >
>>
>> > 85.406/65.797/103.418/12.609 ms (14/132595)=C2=A0 153 KByte 2= 1.74
>>
>> >
>>
>> > 692=3D687:2:3:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 2] 7.00-8.00 sec=C2=A0 =C2=A0957 KBytes=C2=A0 7.84 Mb= its/sec
>>
>> >
>>
>> > 153.936/131.452/191.464/19.361 ms (7/140042)=C2=A0 160 KByte = 6.368199
>>
>> >
>>
>> > 429=3D425:4:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 1] 7.00-8.00 sec=C2=A0 1.13 MBytes=C2=A0 9.44 Mbits/s= ec
>>
>> >
>>
>> > 131.146/109.737/166.774/22.035 ms (9/131124)=C2=A0 146 KByte = 8.998528
>>
>> >
>>
>> > 520=3D516:4:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 3] 7.00-8.00 sec=C2=A0 1.13 MBytes=C2=A0 9.51 Mbits/s= ec
>>
>> >
>>
>> > 126.512/88.404/220.175/42.237 ms (9/132089)=C2=A0 172 KByte 9= .396784
>>
>> >
>>
>> > 527=3D524:1:2:0:0:0:0:0
>>
>> >
>>
>> > [SUM] 7.00-8.00 sec=C2=A0 4.96 MBytes=C2=A0 41.6 Mbits/sec >>
>> >
>>
>> > 2168=3D2152:11:5:0:0:0:0:0
>>
>> >
>>
>> > With source side rate limiting to 9 mb/s per stream.
>>
>> >
>>
>> > [rjmcmahon@ryzen3950 iperf2-code]$ iperf -c *** --hide-ips -e=
>>
>> >
>>
>> > --trip-times -i 1 -P 4 -t 10 -w 4m --tcp-quickack -N -b 9m >>
>> >
>>
>> > ------------------------------------------------------------<= br> >>
>> >
>>
>> > Client connecting to (**hidden**), TCP port 5001 with pid 108= 884 (4
>>
>> >
>>
>> > flows)
>>
>> >
>>
>> > Write buffer size: 131072 Byte
>>
>> >
>>
>> > TOS set to 0x0 and nodelay (Nagle off)
>>
>> >
>>
>> > TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte) >>
>> >
>>
>> > Event based writes (pending queue watermark at 16384 bytes) >>
>> >
>>
>> > ------------------------------------------------------------<= br> >>
>> >
>>
>> > [=C2=A0 1] local *.*.*.85%enp4s0 port 46448 connected with *.= *.*.123 port
>>
>> >
>>
>> > 5001 (prefetch=3D16384) (trip-times) (sock=3D3) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/10666) (ct=3D10.70 ms) on 2023-01-0= 6 12:27:45
>>
>> >
>>
>> > (PST)
>>
>> >
>>
>> > [=C2=A0 3] local *.*.*.85%enp4s0 port 46460 connected with *.= *.*.123 port
>>
>> >
>>
>> > 5001 (prefetch=3D16384) (trip-times) (sock=3D6) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/16499) (ct=3D16.54 ms) on 2023-01-0= 6 12:27:45
>>
>> >
>>
>> > (PST)
>>
>> >
>>
>> > [=C2=A0 2] local *.*.*.85%enp4s0 port 46454 connected with *.= *.*.123 port
>>
>> >
>>
>> > 5001 (prefetch=3D16384) (trip-times) (sock=3D4) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/16580) (ct=3D16.86 ms) on 2023-01-0= 6 12:27:45
>>
>> >
>>
>> > (PST)
>>
>> >
>>
>> > [=C2=A0 4] local *.*.*.85%enp4s0 port 46458 connected with *.= *.*.123 port
>>
>> >
>>
>> > 5001 (prefetch=3D16384) (trip-times) (sock=3D5) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/16802) (ct=3D16.83 ms) on 2023-01-0= 6 12:27:45
>>
>> >
>>
>> > (PST)
>>
>> >
>>
>> > [ ID] Interval=C2=A0 =C2=A0 =C2=A0 =C2=A0 Transfer=C2=A0 =C2= =A0 Bandwidth=C2=A0 =C2=A0 =C2=A0 =C2=A0Write/Err=C2=A0 Rtry
>>
>> >
>>
>> > Cwnd/RTT(var)=C2=A0 =C2=A0 =C2=A0 =C2=A0 NetPwr
>>
>> >
>>
>> > ...
>>
>> >
>>
>> > [=C2=A0 2] 4.00-5.00 sec=C2=A0 1.00 MBytes=C2=A0 8.39 Mbits/s= ec=C2=A0 8/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 134K/88055(12329) us=C2=A0 11.91
>>
>> >
>>
>> > [=C2=A0 4] 4.00-5.00 sec=C2=A0 1.00 MBytes=C2=A0 8.39 Mbits/s= ec=C2=A0 8/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 132K/74867(11755) us=C2=A0 14.01
>>
>> >
>>
>> > [=C2=A0 1] 4.00-5.00 sec=C2=A0 1.00 MBytes=C2=A0 8.39 Mbits/s= ec=C2=A0 8/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 134K/89101(13134) us=C2=A0 11.77
>>
>> >
>>
>> > [=C2=A0 3] 4.00-5.00 sec=C2=A0 1.00 MBytes=C2=A0 8.39 Mbits/s= ec=C2=A0 8/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 131K/91451(11938) us=C2=A0 11.47
>>
>> >
>>
>> > [SUM] 4.00-5.00 sec=C2=A0 4.00 MBytes=C2=A0 33.6 Mbits/sec=C2= =A0 32/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > [=C2=A0 2] 5.00-6.00 sec=C2=A0 1.12 MBytes=C2=A0 9.44 Mbits/s= ec=C2=A0 9/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 134K/85135(14580) us=C2=A0 13.86
>>
>> >
>>
>> > [=C2=A0 4] 5.00-6.00 sec=C2=A0 1.12 MBytes=C2=A0 9.44 Mbits/s= ec=C2=A0 9/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 132K/85124(15654) us=C2=A0 13.86
>>
>> >
>>
>> > [=C2=A0 1] 5.00-6.00 sec=C2=A0 1.12 MBytes=C2=A0 9.44 Mbits/s= ec=C2=A0 9/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 134K/91336(11335) us=C2=A0 12.92
>>
>> >
>>
>> > [=C2=A0 3] 5.00-6.00 sec=C2=A0 1.12 MBytes=C2=A0 9.44 Mbits/s= ec=C2=A0 9/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 131K/89185(13499) us=C2=A0 13.23
>>
>> >
>>
>> > [SUM] 5.00-6.00 sec=C2=A0 4.50 MBytes=C2=A0 37.7 Mbits/sec=C2= =A0 36/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > [=C2=A0 2] 6.00-7.00 sec=C2=A0 1.12 MBytes=C2=A0 9.44 Mbits/s= ec=C2=A0 9/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 134K/85687(13489) us=C2=A0 13.77
>>
>> >
>>
>> > [=C2=A0 4] 6.00-7.00 sec=C2=A0 1.12 MBytes=C2=A0 9.44 Mbits/s= ec=C2=A0 9/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 132K/82803(13001) us=C2=A0 14.25
>>
>> >
>>
>> > [=C2=A0 1] 6.00-7.00 sec=C2=A0 1.12 MBytes=C2=A0 9.44 Mbits/s= ec=C2=A0 9/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 134K/86869(15186) us=C2=A0 13.58
>>
>> >
>>
>> > [=C2=A0 3] 6.00-7.00 sec=C2=A0 1.12 MBytes=C2=A0 9.44 Mbits/s= ec=C2=A0 9/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 131K/91447(12515) us=C2=A0 12.90
>>
>> >
>>
>> > [SUM] 6.00-7.00 sec=C2=A0 4.50 MBytes=C2=A0 37.7 Mbits/sec=C2= =A0 36/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > [=C2=A0 2] 7.00-8.00 sec=C2=A0 1.00 MBytes=C2=A0 8.39 Mbits/s= ec=C2=A0 8/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 134K/81814(13168) us=C2=A0 12.82
>>
>> >
>>
>> > [=C2=A0 4] 7.00-8.00 sec=C2=A0 1.00 MBytes=C2=A0 8.39 Mbits/s= ec=C2=A0 8/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 132K/89008(13283) us=C2=A0 11.78
>>
>> >
>>
>> > [=C2=A0 1] 7.00-8.00 sec=C2=A0 1.00 MBytes=C2=A0 8.39 Mbits/s= ec=C2=A0 8/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 134K/89494(12151) us=C2=A0 11.72
>>
>> >
>>
>> > [=C2=A0 3] 7.00-8.00 sec=C2=A0 1.00 MBytes=C2=A0 8.39 Mbits/s= ec=C2=A0 8/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > 131K/91083(12797) us=C2=A0 11.51
>>
>> >
>>
>> > [SUM] 7.00-8.00 sec=C2=A0 4.00 MBytes=C2=A0 33.6 Mbits/sec=C2= =A0 32/0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>>
>> >
>>
>> > [root@bobcat iperf2-code]# iperf -s -i 1 -e --hide-ips -w 4m<= br> >>
>> >
>>
>> > ------------------------------------------------------------<= br> >>
>> >
>>
>> > Server listening on TCP port 5001 with pid 233981
>>
>> >
>>
>> > Read buffer size:=C2=A0 128 KByte (Dist bin width=3D16.0 KByt= e)
>>
>> >
>>
>> > TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte) >>
>> >
>>
>> > ------------------------------------------------------------<= br> >>
>> >
>>
>> > [=C2=A0 1] local *.*.*.123%eth0 port 5001 connected with *.*.= *.171 port
>>
>> > 46448
>>
>> >
>>
>> > (trip-times) (sock=3D4) (peer 2.1.9-master) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/11987) on 2023-01-06 12:27:45 (PST)=
>>
>> >
>>
>> > [=C2=A0 2] local *.*.*.123%eth0 port 5001 connected with *.*.= *.171 port
>>
>> > 46454
>>
>> >
>>
>> > (trip-times) (sock=3D5) (peer 2.1.9-master) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/11132) on 2023-01-06 12:27:45 (PST)=
>>
>> >
>>
>> > [=C2=A0 3] local *.*.*.123%eth0 port 5001 connected with *.*.= *.171 port
>>
>> > 46460
>>
>> >
>>
>> > (trip-times) (sock=3D6) (peer 2.1.9-master) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/11097) on 2023-01-06 12:27:45 (PST)=
>>
>> >
>>
>> > [=C2=A0 4] local *.*.*.123%eth0 port 5001 connected with *.*.= *.171 port
>>
>> > 46458
>>
>> >
>>
>> > (trip-times) (sock=3D7) (peer 2.1.9-master) (qack)
>>
>> >
>>
>> > (icwnd/mss/irtt=3D14/1448/17823) on 2023-01-06 12:27:45 (PST)=
>>
>> >
>>
>> > [ ID] Interval=C2=A0 =C2=A0 =C2=A0 =C2=A0 Transfer=C2=A0 =C2= =A0 Bandwidth=C2=A0 =C2=A0 Burst Latency
>>
>> >
>>
>> > avg/min/max/stdev (cnt/size) inP NetPwr=C2=A0 Reads=3DDist >>
>> >
>>
>> > [=C2=A0 4] 0.00-1.00 sec=C2=A0 1.09 MBytes=C2=A0 9.15 Mbits/s= ec
>>
>> >
>>
>> > 93.383/90.103/95.661/2.232 ms (8/143028)=C2=A0 128 KByte 12.2= 5
>>
>> >
>>
>> > 451=3D451:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 3] 0.00-1.00 sec=C2=A0 1.08 MBytes=C2=A0 9.06 Mbits/s= ec
>>
>> >
>>
>> > 96.834/95.229/102.645/2.442 ms (8/141580)=C2=A0 131 KByte 11.= 70
>>
>> >
>>
>> > 472=3D472:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 1] 0.00-1.00 sec=C2=A0 1.10 MBytes=C2=A0 9.19 Mbits/s= ec
>>
>> >
>>
>> > 95.183/92.623/97.579/1.431 ms (8/143571)=C2=A0 131 KByte 12.0= 7
>>
>> >
>>
>> > 495=3D495:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 2] 0.00-1.00 sec=C2=A0 1.09 MBytes=C2=A0 9.15 Mbits/s= ec
>>
>> >
>>
>> > 89.317/84.865/94.906/3.674 ms (8/143028)=C2=A0 122 KByte 12.8= 1
>>
>> >
>>
>> > 489=3D489:0:0:0:0:0:0:0
>>
>> >
>>
>> > [ ID] Interval=C2=A0 =C2=A0 =C2=A0 =C2=A0 Transfer=C2=A0 =C2= =A0 Bandwidth=C2=A0 =C2=A0 =C2=A0 =C2=A0Reads=3DDist
>>
>> >
>>
>> > [SUM] 0.00-1.00 sec=C2=A0 4.36 MBytes=C2=A0 36.6 Mbits/sec >>
>> >
>>
>> > 1907=3D1907:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 4] 1.00-2.00 sec=C2=A0 1.07 MBytes=C2=A0 8.95 Mbits/s= ec
>>
>> >
>>
>> > 92.649/89.987/95.036/1.828 ms (9/124314) 96.5 KByte 12.08
>>
>> >
>>
>> > 492=3D492:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 3] 1.00-2.00 sec=C2=A0 1.06 MBytes=C2=A0 8.93 Mbits/s= ec
>>
>> >
>>
>> > 96.305/95.647/96.794/0.432 ms (9/123992)=C2=A0 100 KByte 11.5= 9
>>
>> >
>>
>> > 480=3D480:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 1] 1.00-2.00 sec=C2=A0 1.06 MBytes=C2=A0 8.89 Mbits/s= ec
>>
>> >
>>
>> > 92.578/90.866/94.145/1.371 ms (9/123510) 95.8 KByte 12.01
>>
>> >
>>
>> > 513=3D513:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 2] 1.00-2.00 sec=C2=A0 1.07 MBytes=C2=A0 8.96 Mbits/s= ec
>>
>> >
>>
>> > 90.767/87.984/94.352/1.944 ms (9/124475) 94.7 KByte 12.34
>>
>> >
>>
>> > 489=3D489:0:0:0:0:0:0:0
>>
>> >
>>
>> > [SUM] 1.00-2.00 sec=C2=A0 4.26 MBytes=C2=A0 35.7 Mbits/sec >>
>> >
>>
>> > 1974=3D1974:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 4] 2.00-3.00 sec=C2=A0 1.09 MBytes=C2=A0 9.13 Mbits/s= ec
>>
>> >
>>
>> > 93.977/91.795/96.561/1.693 ms (8/142656)=C2=A0 112 KByte 12.1= 4
>>
>> >
>>
>> > 497=3D497:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 3] 2.00-3.00 sec=C2=A0 1.08 MBytes=C2=A0 9.04 Mbits/s= ec
>>
>> >
>>
>> > 96.544/95.815/97.798/0.693 ms (8/141208)=C2=A0 114 KByte 11.7= 0
>>
>> >
>>
>> > 503=3D503:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 1] 2.00-3.00 sec=C2=A0 1.07 MBytes=C2=A0 9.01 Mbits/s= ec
>>
>> >
>>
>> > 93.970/91.193/96.325/1.796 ms (8/140846)=C2=A0 111 KByte 11.9= 9
>>
>> >
>>
>> > 509=3D509:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 2] 2.00-3.00 sec=C2=A0 1.08 MBytes=C2=A0 9.10 Mbits/s= ec
>>
>> >
>>
>> > 92.843/90.216/96.355/2.040 ms (8/142113)=C2=A0 111 KByte 12.2= 5
>>
>> >
>>
>> > 509=3D509:0:0:0:0:0:0:0
>>
>> >
>>
>> > [SUM] 2.00-3.00 sec=C2=A0 4.32 MBytes=C2=A0 36.3 Mbits/sec >>
>> >
>>
>> > 2018=3D2018:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 4] 3.00-4.00 sec=C2=A0 1.06 MBytes=C2=A0 8.86 Mbits/s= ec
>>
>> >
>>
>> > 93.222/89.063/96.104/2.346 ms (9/123027) 96.1 KByte 11.88
>>
>> >
>>
>> > 487=3D487:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 3] 3.00-4.00 sec=C2=A0 1.07 MBytes=C2=A0 8.97 Mbits/s= ec
>>
>> >
>>
>> > 96.277/95.051/97.230/0.767 ms (9/124636)=C2=A0 101 KByte 11.6= 5
>>
>> >
>>
>> > 489=3D489:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 1] 3.00-4.00 sec=C2=A0 1.08 MBytes=C2=A0 9.02 Mbits/s= ec
>>
>> >
>>
>> > 93.899/88.732/96.972/2.737 ms (9/125280) 98.6 KByte 12.01
>>
>> >
>>
>> > 493=3D493:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 2] 3.00-4.00 sec=C2=A0 1.07 MBytes=C2=A0 8.97 Mbits/s= ec
>>
>> >
>>
>> > 92.490/89.862/95.265/1.796 ms (9/124636) 96.6 KByte 12.13
>>
>> >
>>
>> > 493=3D493:0:0:0:0:0:0:0
>>
>> >
>>
>> > [SUM] 3.00-4.00 sec=C2=A0 4.27 MBytes=C2=A0 35.8 Mbits/sec >>
>> >
>>
>> > 1962=3D1962:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 4] 4.00-5.00 sec=C2=A0 1.07 MBytes=C2=A0 9.00 Mbits/s= ec
>>
>> >
>>
>> > 92.431/81.888/96.221/4.524 ms (9/124958) 96.8 KByte 12.17
>>
>> >
>>
>> > 498=3D498:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 1] 4.00-5.00 sec=C2=A0 1.07 MBytes=C2=A0 8.97 Mbits/s= ec
>>
>> >
>>
>> > 95.018/93.445/96.200/0.957 ms (9/124636) 99.3 KByte 11.81
>>
>> >
>>
>> > 490=3D490:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 2] 4.00-5.00 sec=C2=A0 1.06 MBytes=C2=A0 8.90 Mbits/s= ec
>>
>> >
>>
>> > 93.874/86.485/95.672/2.810 ms (9/123671) 97.3 KByte 11.86
>>
>> >
>>
>> > 481=3D481:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 3] 4.00-5.00 sec=C2=A0 1.08 MBytes=C2=A0 9.09 Mbits/s= ec
>>
>> >
>>
>> > 95.737/93.881/97.197/0.972 ms (9/126245)=C2=A0 101 KByte 11.8= 7
>>
>> >
>>
>> > 484=3D484:0:0:0:0:0:0:0
>>
>> >
>>
>> > [SUM] 4.00-5.00 sec=C2=A0 4.29 MBytes=C2=A0 36.0 Mbits/sec >>
>> >
>>
>> > 1953=3D1953:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 4] 5.00-6.00 sec=C2=A0 1.09 MBytes=C2=A0 9.13 Mbits/s= ec
>>
>> >
>>
>> > 92.908/86.844/95.994/3.012 ms (8/142656)=C2=A0 111 KByte 12.2= 8
>>
>> >
>>
>> > 467=3D467:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 3] 5.00-6.00 sec=C2=A0 1.07 MBytes=C2=A0 8.94 Mbits/s= ec
>>
>> >
>>
>> > 96.593/95.343/97.660/0.876 ms (8/139760)=C2=A0 113 KByte 11.5= 8
>>
>> >
>>
>> > 478=3D478:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 1] 5.00-6.00 sec=C2=A0 1.08 MBytes=C2=A0 9.03 Mbits/s= ec
>>
>> >
>>
>> > 95.021/91.421/97.167/1.893 ms (8/141027)=C2=A0 112 KByte 11.8= 7
>>
>> >
>>
>> > 491=3D491:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 2] 5.00-6.00 sec=C2=A0 1.08 MBytes=C2=A0 9.06 Mbits/s= ec
>>
>> >
>>
>> > 92.162/82.720/97.692/5.060 ms (8/141570)=C2=A0 109 KByte 12.2= 9
>>
>> >
>>
>> > 488=3D488:0:0:0:0:0:0:0
>>
>> >
>>
>> > [SUM] 5.00-6.00 sec=C2=A0 4.31 MBytes=C2=A0 36.2 Mbits/sec >>
>> >
>>
>> > 1924=3D1924:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 4] 6.00-7.00 sec=C2=A0 1.04 MBytes=C2=A0 8.70 Mbits/s= ec
>>
>> >
>>
>> > 92.793/85.343/96.967/3.552 ms (9/120775) 93.9 KByte 11.71
>>
>> >
>>
>> > 485=3D485:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 2] 6.00-7.00 sec=C2=A0 1.05 MBytes=C2=A0 8.79 Mbits/s= ec
>>
>> >
>>
>> > 91.679/84.479/96.760/3.975 ms (9/122062) 93.8 KByte 11.98
>>
>> >
>>
>> > 472=3D472:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 3] 6.00-7.00 sec=C2=A0 1.06 MBytes=C2=A0 8.88 Mbits/s= ec
>>
>> >
>>
>> > 96.982/95.933/98.371/0.680 ms (9/123349)=C2=A0 100 KByte 11.4= 5
>>
>> >
>>
>> > 477=3D477:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 1] 6.00-7.00 sec=C2=A0 1.05 MBytes=C2=A0 8.80 Mbits/s= ec
>>
>> >
>>
>> > 94.342/91.660/96.025/1.660 ms (9/122223) 96.7 KByte 11.66
>>
>> >
>>
>> > 494=3D494:0:0:0:0:0:0:0
>>
>> >
>>
>> > [SUM] 6.00-7.00 sec=C2=A0 4.19 MBytes=C2=A0 35.2 Mbits/sec >>
>> >
>>
>> > 1928=3D1928:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 4] 7.00-8.00 sec=C2=A0 1.10 MBytes=C2=A0 9.25 Mbits/s= ec
>>
>> >
>>
>> > 92.515/88.182/96.351/2.538 ms (8/144466)=C2=A0 112 KByte 12.4= 9
>>
>> >
>>
>> > 510=3D510:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 3] 7.00-8.00 sec=C2=A0 1.09 MBytes=C2=A0 9.13 Mbits/s= ec
>>
>> >
>>
>> > 96.580/95.737/98.977/1.098 ms (8/142656)=C2=A0 115 KByte 11.8= 2
>>
>> >
>>
>> > 480=3D480:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 1] 7.00-8.00 sec=C2=A0 1.10 MBytes=C2=A0 9.21 Mbits/s= ec
>>
>> >
>>
>> > 95.269/91.719/97.514/2.126 ms (8/143923)=C2=A0 115 KByte 12.0= 9
>>
>> >
>>
>> > 515=3D515:0:0:0:0:0:0:0
>>
>> >
>>
>> > [=C2=A0 2] 7.00-8.00 sec=C2=A0 1.11 MBytes=C2=A0 9.29 Mbits/s= ec
>>
>> >
>>
>> > 90.073/84.700/96.176/4.324 ms (8/145190)=C2=A0 110 KByte 12.9= 0
>>
>> >
>>
>> > 508=3D508:0:0:0:0:0:0:0
>>
>> >
>>
>> > [SUM] 7.00-8.00 sec=C2=A0 4.40 MBytes=C2=A0 36.9 Mbits/sec >>
>> >
>>
>> > 2013=3D2013:0:0:0:0:0:0:0
>>
>> >
>>
>> > Bob
>>
>> >
>>
>> >>> -----Original Message-----
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>> From: LibreQoS <libreqos-bounces@lists.bufferbloa= t.net> On Behalf
>>
>> > Of
>>
>> >
>>
>> >> Dave Taht
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>> via LibreQoS
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>> Sent: Wednesday, January 4, 2023 12:26 PM
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>> Subject: [LibreQoS] the grinch meets cloudflare's= christmas present
>>
>> >
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>>
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>> Please try the new, the shiny, the really wonderful t= est here:
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>>
>>
>> >
>>
>> >>
>>
>> > https://= urldefense.com/v3/__https://speed.cloudflare.com/__;!!BhdT!iZcFJ8WVU9S<= br> >>
>> >
>>
>> >
>>
>> >> [1]
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>>
>>
>> >
>>
>> >>
>>
>> > 9zz5t456oxkfObrC5Xb9j5AG8UO3DqD5x4GAJkawZr0iGwEUtF0_09U8mCDnA= krJ9QEMHGbCMKVw$
>>
>> >
>>
>> >
>>
>> >> [1]
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>>
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>> I would really appreciate some independent verificati= on of
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>> measurements using this tool. In my brief experiments= it appears -
>>
>> >
>>
>> >> as
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>> all the commercial tools to date - to dramatically un= derstate the
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>> bufferbloat, on my LTE, (and my starlink terminal is = out being
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>> hacked^H^H^H^H^H^Hworked on, so I can't measure t= hat)
>>
>> >
>>
>> >>
>>
>> >
>>
>> >> [acm]
>>
>> >
>>
>> >>
>>
>> >
>>
>> >> Hi Dave, I made some time to test "cloudflare's = christmas present"
>>
>> >
>>
>> >> yesterday.
>>
>> >
>>
>> >>
>>
>> >
>>
>> >> I'm on DOCSIS 3.1 service with 1Gbps Down. The Upstre= am has a
>>
>> > "turbo"
>>
>> >
>>
>> >> mode with 40-50Mbps for the first ~3 sec, then steady-sta= te about
>>
>> >
>>
>> >> 23Mbps.
>>
>> >
>>
>> >>
>>
>> >
>>
>> >> When I saw the ~620Mbps Downstream measurement, I was rea= dy to
>>
>> >
>>
>> >> complain that even the IP-Layer Capacity was grossly unde= restimated.
>>
>> >
>>
>> >
>>
>> >> In addition, the Latency measurements seem very low (as y= ou
>>
>> > asserted),
>>
>> >
>>
>> >> although the cloud server was "nearby".
>>
>> >
>>
>> >>
>>
>> >
>>
>> >> However, I found that Ookla and the ISP-provided measurem= ent were
>>
>> > also
>>
>> >
>>
>> >> reporting ~600Mbps! So the cloudflare Downstream capacity= (or
>>
>> >
>>
>> >> throughput?) measurement was consistent with others. Our = UDPST
>>
>> > server
>>
>> >
>>
>> >> was unreachable, otherwise I would have added that measur= ement, too.
>>
>> >
>>
>> >
>>
>> >>
>>
>> >
>>
>> >> The Upstream measurement graph seems to illustrate the &q= uot;turbo"
>>
>> >
>>
>> >> mode, with the dip after attaining 44.5Mbps.
>>
>> >
>>
>> >>
>>
>> >
>>
>> >> UDPST saturates the uplink and we measure the full 250ms = of the
>>
>> >
>>
>> >> Upstream buffer. Cloudflare's latency measurements do= n't even come
>>
>> >
>>
>> >> close.
>>
>> >
>>
>> >>
>>
>> >
>>
>> >> Al
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>
>>
>> >
>>
>> >>
>>
>> >
>>
>> >> Links:
>>
>> >
>>
>> >> ------
>>
>> >
>>
>> >> [1]
>>
>> >
>>
>> >>
>>
>> > https:/= /urldefense.com/v3/__https:/speed.cloudflare.com/__;!!BhdT!iZcFJ8WVU9S9zz5t= 456oxkfObrC5Xb9j5AG8UO3DqD5x4GAJkawZr0iGwEUtF0_09U8mCDnAkrJ9QEMHGbCMKVw$
>>
>> >
>>
>> >
>>
>> >> _______________________________________________
>>
>> >
>>
>> >> Rpm mailing list
>>
>> >
>>
>> >> Rpm@lists.bufferbloat.net
>>
>> >
>>
>> >> https://lists.bufferbloat.net/listinfo/r= pm
>>
>> >
>>
>> > _______________________________________________
>>
>> >
>>
>> > Starlink mailing list
>>
>> >
>>
>> > Starlink@lists.bufferbloat.net
>>
>> >
>>
>> > https://lists.bufferbloat.net/listinfo/= starlink
>>
>> _______________________________________________
>> Bloat mailing list
>> B= loat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>
> _______________________________________________
> Rpm mailing list
> Rpm@lis= ts.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/rpm



--
This song goes out to all the folk that thought Stadia would work:
https://www.= linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXt= z
Dave T=C3=A4ht CEO, TekLibre, LLC
_______________________________________________
Bloat mailing list
Bloat@list= s.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat
--
--00000000000021e47f05f1fafb8a--