From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id EBB8D3CB37 for ; Tue, 9 Jan 2024 03:31:52 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1704789109; x=1705393909; i=moeller0@gmx.de; bh=8miyHQVqw2mybFdGzKp4xZ+MJKTjyDOAjJO1s6t9T7I=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References: To; b=rZVnd1tah9c5DlFA3Vhcpb3sfv0bvRRF9+mw8toBRmx+AS+wfpGLoSEqx7m7qSIW hncpRAazrfkMiyjWaxy8AEP8kyT2v82wXMuQ+hv6YKDJebs+EhhOBRXgY4cmI5hP4 AZ95qBkhYewmaE2LxeyQPAP74A09z1PlYtul7hEzvVqpEWrohwa4gDSeaw6ZU4fup EKZEGePtdXYw4un5naraUHYUjcPqCDTnjJtHjOce/fGNuBlKfILkRQQb7aF6O3dCZ aGHJe9izenBNMIgEkBXybrIfVJvdfMGYBJoYREzpbrH0UK4WuaIlJr2bSmR/wRhOg QSy/H4FdVv03HoXQlw== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from smtpclient.apple ([134.76.241.253]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1MAwXr-1rTUNA30HZ-00BMBb; Tue, 09 Jan 2024 09:31:49 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.4\)) From: Sebastian Moeller In-Reply-To: <2b067151-e866-4fc0-bc3e-7cbecfd88d32@indexexchange.com> Date: Tue, 9 Jan 2024 09:31:48 +0100 Cc: starlink@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: <1503DC33-6A46-4D96-BCD9-1C3EDF3FF213@gmx.de> References: <2b067151-e866-4fc0-bc3e-7cbecfd88d32@indexexchange.com> To: Dave Collier-Brown X-Mailer: Apple Mail (2.3696.120.41.1.4) X-Provags-ID: V03:K1:GOcgo+KNCQ/RtnXHf7YEmDz3VC81IHG3CCZhOL2/9KHAjzqeGBo TSPiuNTZ7V87KGpM2k63ZUBibZwtWNynUYasnazOBZEqgnTfhrqbxkGDfxtaq7xVR2G5xli Uela42qXwOnP2iDNRB760Txz71QWEZfFUOQ85q39L4NkEkBUJx1a9yh2C9uDwT6TzLl1Fwi 4HlePXUZ+aCRebimAFkfw== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:4wVh3Wf3QWE=;7EG/glJ8R42vRu+8pDpYxy1NsHO vv3MwV4b6NVGtoRHgO/vD33TKw6bv7YSTmTvYQKEVkjVhQ96SoZqlZYDZLidzU8U9Xu7tkIys 6CQK8EAI45wvIh7/A+/Qy9yslAj3wcuquuujhuNMP2EPUGWTWx1rhMaj9t1ShgnNhsUyiQeU8 gXL3w/J2o9b4xdxzBz39qtIPPZSdrEv10SSaV9XeAJCy5msc21Iyso+Wzu04mjV7rYlg3ymO2 Ejzttk9/zbzqWGLIugPBhLkvK9/vEv1jk7w86mFYpId0ToF+zyPqkc7M5LGa9K1IGTSGfdYH2 klLyQf/mrwb4R2sF/IFaysge6nWvUtEIE3l4ZCFleC+XQ7FbuZ/5QmoA5G+hFpOZUcfp6XIF8 8OJRkYFG6tReahF86gHwHjCxhpRkEn/FIjUpfpKvGITqTi1qFl/c4hu1CaddFRIuC8qPFSzxC eBxHiRFcbQI2hi1fgXYbFoaNZNwV26Yt9v7/+pPsUS1O9cBTxQrgybvledLqRxrwdqm20e2tb ck4sr0oTdtqCF6w4H8w0Ur2SbYgBtXn8jo4JzFNxhzka6d5uPMtETKCT9L8/DPUWqPl4EmFpv CGWyoCFVJiCN2ZzpI7J2jLHfGruEXsU0ANJhpV0eVszFEO5pRuoXI5FSd5/n743RK64NuMeRF x08C07B1T889zM6hhxjN47jLkG8kIoQ1GOsVl6ANGDT9wPTYCC7LvtHoRY4RwHVUjScQAPlax ZoK094O/PegzX+HQmj+fjdgczBiLDenR2i6JXfFFtJ2yaxZZFNqegiWIxudNwwaLUOH9agRLD w1UqvyGyUvBJXM7g1SxURBVESeEi6nr0vuHnbBRaQPRKEDAWAXMMLK5F7HdQhTG112YLnI8qG e7OjZmbisZQFZG4v+/R2chXaTyNVvhj3ye3C+mpVV2OtzHb25raicMsy3rzefcZdvDWhu5Mn5 9mWgr4b88h1fYRwvF0SLRVERnXo= Subject: Re: [Starlink] [Bloat] goresponsiveness learned a few tricks... X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Jan 2024 08:31:53 -0000 Hi Dave, > On Jan 8, 2024, at 22:08, Dave Collier-Brown via Starlink = wrote: >=20 > Cool: I got=20 >=20 > 01-08-2024 21:04:41 UTC Go Responsiveness to = mensura.cdn-apple.com:443... > Results: > =3D=3D=3D=3D=3D=3D=3D=3D > Download: > Throughput: 150.234 Mbps (18.779 MBps), using 8 parallel = connections. > RPM: 624 (P90) > RPM: 1005 (Single-Sided 5% Trimmed Mean) > =3D=3D=3D=3D=3D=3D=3D=3D > Upload: > Throughput: 13.124 Mbps (1.641 MBps), using 8 parallel = connections. > RPM: 360 (P90) > RPM: 1210 (Single-Sided 5% Trimmed Mean) > =3D=3D=3D=3D=3D=3D=3D=3D > Final RPM: 392 (P90) > Final RPM: 1124 (Single-Sided 5% Trimmed Mean) >=20 >=20 > Now all I have to do is figure out what it means (;-)) RPM is essentially defined as the inverse of latency averaged over: download saturation: self latency, foreign latency upload saturation: self latency, foreign latency self latency is measuring latency from within the load bearing flows, = while foreign latency measures against flows not (deeply) involved in = the saturating load. The actual transformation is 60000/RTT[ms] =3D> RPM or 60000/RPM =3D> = RTT[ms]. So your numbers mean: Download: RTT(P90) 60000/624 =3D 96.15 ms RTT(sTR5) 60000/1005 =3D 59.70 ms Upload: RTT(P90) 60000/360 =3D 166.67 ms RTT(sTR5) 60000/1210 =3D 49.59 ms Download: RTT(P90) 60000/392 =3D 153.06 ms RTT(sTR5) 60000/1124 =3D 53.38 ms As expected the 90 percentile values are considerably lower than the = (fancy) averages. Without splitting out self and foreign values it is = hard to figure out whether that is mostly caused by inter- or intra-flow = congestion, but there is a current branch (final_self_and_foreign_rpms) = that helps out there: bash-3.2$ ./networkQuality --relative-rpm --rpm.parallel --config = mensura.cdn-apple.com --port 443 --path /api/v1/gm/config --rpm.timeout = 120 --extended-stats --rpm.mnp 32 --logger-filename = go_networkQuality_20231228_223707 --detailed 01-09-2024 07:20:11 UTC Go Responsiveness to = mensura.cdn-apple.com:443... Baseline RPM: 4786 (P90) Baseline RPM: 8385 (Single-Sided 5% Trimmed Mean) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Warning: Could not fetch the extended stats for a probe: OOPS: Could not = get the TCP info for the connection (not a TLS connection) Results: =3D=3D=3D=3D=3D=3D=3D=3D Download: Note: Test did not run to stability, these results are = estimates. Throughput: 15.364 Mbps (1.920 MBps), using 32 parallel = connections. Extended Statistics: Maximum Segment Size: 1368 Total Bytes Retransmitted: 2811 Retransmission Ratio: 0.11% Total Bytes Reordered: 333592209 Average RTT: 192.0625 RPM Calculation stats: Total Self Probes: 4944 Total Foreign Probes: 14832 Trimmed Self Probes Count: 247 Trimmed Foreign Probes Count: 741 P90 Self RTT: 6.849031s P90 Foreign RTT: 0.601590s P90 RTT: 3.725311s Trimmed Mean Self RTT: 1.906785s Trimmed Mean Foreign RTT: 0.132828s Trimmed Mean RTT: 1.019806s RPM: 16 (P90) RPM: 59 (Single-Sided 5% Trimmed Mean) =3D=3D=3D=3D=3D=3D=3D=3D Upload: Throughput: 32.503 Mbps (4.063 MBps), using 29 parallel = connections. Extended Statistics: Maximum Segment Size: 1368 Total Bytes Retransmitted: 1024727 Retransmission Ratio: 0.24% Total Bytes Reordered: 461083 Average RTT: 176.17241379310346 RPM Calculation stats: Total Self Probes: 7063 Total Foreign Probes: 21189 Trimmed Self Probes Count: 353 Trimmed Foreign Probes Count: 1059 P90 Self RTT: 0.770294s P90 Foreign RTT: 0.667592s P90 RTT: 0.718943s Trimmed Mean Self RTT: 0.112447s Trimmed Mean Foreign RTT: 0.120068s Trimmed Mean RTT: 0.116258s RPM: 83 (P90) RPM: 516 (Single-Sided 5% Trimmed Mean) =3D=3D=3D=3D=3D=3D=3D=3D Final RPM Calculation stats: Total Self Probes: 12007 Total Foreign Probes: 36021 Trimmed Self Probes Count: 600 Trimmed Foreign Probes Count: 1801 P90 Self RTT: 5.504198s P90 Foreign RTT: 0.635901s P90 RTT: 3.070049s Trimmed Mean Self RTT: 0.119163s Trimmed Mean Foreign RTT: 0.123304s Trimmed Mean RTT: 0.121233s Final RPM: 20 (P90) RTT: 3.070049s Final RPM: 495 (Single-Sided 5% Trimmed Mean) RTT: 0.121233s Final RPM (Self Only): 11 (P90) RTT: 5.504198s Final RPM (Self Only): 504 (Single-Sided 5% Trimmed Mean) RTT: 0.119163s Final RPM (Foreign Only): 94 (P90) RTT: 0.635901s Final RPM (Foreign Only): 487 (Single-Sided 5% Trimmed Mean) RTT: = 0.123304s Working Conditions RPM Effect: 198% (P90) Working Conditions RPM Effect: 178% (Single-Sided 5% Trimmed Mean) >=20 > --dave >=20 > On 2024-01-08 14:45, Dave Taht via Starlink wrote: >> ---------- Forwarded message --------- >> From: Sebastian Moeller via Bloat=20 >> >>=20 >> Date: Mon, Jan 8, 2024 at 2:41=E2=80=AFPM >> Subject: [Bloat] goresponsiveness learned a few tricks... >> To: Dave Taht via Bloat=20 >> >>=20 >>=20 >>=20 >> Just a quick shoutout to Will Hawkins goresponsiveness effort >> (h++ps://github.com/network-quality/goresponsiveness: open source go >> implementation along the lines of the RPM IETF responsiveness draft >> (h++ps://github.com/network-quality/draft-ietf-ippm-responsiveness). >>=20 >> The goal I think is a quick latency under working conditions/latency >> under load for the rest of us. I think the draft likely will get a >> last call soon, so whoever wants to add something to the spec or just >> wants to comment, now would be a good time: >>=20 >> goresponsiveness is currently acquiring a few nice features, like the >> ability to compare the latency between idle and working conditions, >> run both saturating loads concurrently, and to report the sub results >> for the "self" probes (measuring latency within the load generating >> connection/flow) and the "foreign" probes (measuring latency outside >> of the load generating flows) >>=20 >> bash-3.2$ ./networkQuality --relative-rpm --rpm.parallel --config >> mensura.cdn-apple.com --port 443 --path /api/v1/gm/config >> --rpm.timeout 120 --extended-stats --rpm.mnp 32 --logger-filename >> go_networkQuality_20231228_223707 --detailed >> 01-08-2024 19:37:07 UTC Go Responsiveness to = mensura.cdn-apple.com:443... >> Baseline RPM: 2813 (P90) >> Baseline RPM: 3888 (Single-Sided 5% Trimmed Mean) >> Results: >> =3D=3D=3D=3D=3D=3D=3D=3D >> Download: >> Throughput: 80.367 Mbps (10.046 MBps), using 13 parallel = connections. >> Extended Statistics: >> Maximum Segment Size: 1208 >> Total Bytes Retransmitted: 2860 >> Retransmission Ratio: 0.25% >> Total Bytes Reordered: 140968234 >> Average RTT: 34.30769230769231 >> RPM: 264 (P90) >> RPM: 509 (Single-Sided 5% Trimmed Mean) >> =3D=3D=3D=3D=3D=3D=3D=3D >> Upload: >> Throughput: 30.124 Mbps (3.766 MBps), using 9 parallel = connections. >> Extended Statistics: >> Maximum Segment Size: 1208 >> Total Bytes Retransmitted: 466855 >> Retransmission Ratio: 0.95% >> Total Bytes Reordered: 700 >> Average RTT: 37 >> RPM: 676 (P90) >> RPM: 2536 (Single-Sided 5% Trimmed Mean) >> =3D=3D=3D=3D=3D=3D=3D=3D >> Final RPM Calculation stats: >> Total Self Probes: 933 >> Total Foreign Probes: 2799 >> Trimmed Self Probes Count: 46 >> Trimmed Foreign Probes Count: 139 >> P90 Self RTT: 0.379444841 >> P90 Foreign RTT: 0.031294024999999996 >> Trimmed Mean Self RTT: 0.035124 >> Trimmed Mean Foreign RTT: 0.017979 >>=20 >> Final RPM: 292 (P90) >> Final RPM: 2260 (Single-Sided 5% Trimmed Mean) >> Final RPM (Self Only): 158 (P90) >> Final RPM (Self Only): 1708 (Single-Sided 5% Trimmed Mean) >> Final RPM (Foreign Only): 1917 (P90) >> Final RPM (Foreign Only): 3337 (Single-Sided 5% Trimmed Mean) >> Working Conditions RPM Effect: 162% (P90) >> Working Conditions RPM Effect: 53% (Single-Sided 5% Trimmed Mean) >>=20 >>=20 >> The beauty of the self and foreign reports is that here we can see >> that my cake'd upstream does a decent job to preserve foreign >> responsiveness while in the self responsiveness we see how TCP >> self-congests like there is no tomorrow... Please ignore the download >> direction, this is over a 100 Mbps USB ethernet dongle and my true >> download shaper is set to 105 Mbps, so this is mostly the bad USB >> dongle in action... >>=20 >>=20 >>=20 >> _______________________________________________ >> Bloat mailing list >>=20 >> Bloat@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/bloat >>=20 >>=20 >>=20 >>=20 > --=20 > David Collier-Brown, | Always do right. This will gratify > System Programmer and Author | some people and astonish the rest >=20 > dave.collier-brown@indexexchange.com | -- Mark Twain >=20 > CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, = including any and all attachments, contains confidential information = intended only for the person(s) to whom it is addressed. Any = dissemination, distribution, copying or disclosure is strictly = prohibited and is not a waiver of confidentiality. If you have received = this telecommunication in error, please notify the sender immediately by = return electronic mail and delete the message from your inbox and = deleted items folders. This telecommunication does not constitute an = express or implied agreement to conduct transactions by electronic = means, nor does it constitute a contract offer, a contract amendment or = an acceptance of a contract offer. Contract terms contained in this = telecommunication are subject to legal review and the completion of = formal documentation and are not binding until same is confirmed in = writing and has been signed by an authorized signatory. >=20 > _______________________________________________ > Starlink mailing list > Starlink@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/starlink