From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id BDB163B2A4 for ; Sun, 23 Oct 2022 09:53:01 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1666533178; bh=pl4oUxTcIs2XmTCflbM2sCaQAfLG/bUExjCujuDOEbE=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=cZJRyhjgNe7vnLZKCmLydB9STVlsRrdDnKqV6PGa+Jbcp9iM/bvexEjB6w6sclsuv qATE1WrS+//nu6qL8jX4bwHIeJXtq+Afd9k4+rZTqlqn7a5KoEF0mbuzGj8Q3VCQCf t7e4d7os9wbLuyw6ewzfZf9FDjH9iOZtBFzUUVY4= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from smtpclient.apple ([95.116.66.209]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MVvLB-1ocO910eqj-00Rrex; Sun, 23 Oct 2022 15:52:58 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) From: Sebastian Moeller In-Reply-To: <78ae1fce-b23a-ef67-e486-72c26d2bbe30@indexexchange.com> Date: Sun, 23 Oct 2022 15:52:57 +0200 Cc: starlink@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: <93A92B00-AB22-414A-9A05-B887262E5A8E@gmx.de> References: <339AB8BC-9628-40E2-9339-77FCFA74488D@gmx.de> <0e811787-cc55-8db9-2a4b-7706813769da@indexexchange.com> <850ACED5-3345-441E-9A92-BF27B21EABF5@gmx.de> <78ae1fce-b23a-ef67-e486-72c26d2bbe30@indexexchange.com> To: Dave Collier-Brown X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Provags-ID: V03:K1:k3AaV4upq1x3QsTK/bwHkgeJFBEHiUKiX6ZZ1OhaVx41bkXobzl KS8XEcSwx6C0e+oE5K8LItsPsZVymvkw0YGAJJ58B17PcLR91qOpju9f+dUfZPoRXbobJcz ODOnKAhRn31CgbE9GkYfhPFnOIb4VGZ6BCq/8Ul2SZ/R+dJVpDcCE5krlIO5EpK+xxoxJfL Q2TvJh2pfe8YI7Q7TvH4A== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:L9HgdOZM0q0=:9Gvg1LidcP7ToHuQqU77KY VJt/g+QlDJsX9IKg9xoysetgTMiLPfdnQgm4QlUqcHA/BbJWHrZrpofSOYkUXESORDxPGaeDH 8jPnCm/pK0WG7ZHhHgeUsBg9r+1wdjs6OTBzG4fXWlYYLahpKTEKyC9OUX6PnAr6gAx7JpHks IJu5VmWmKFWk/qM7adpYl6BK+VrgB4EY1SGOaG0e34AMwqCLqVt8leHVd7f6bnUcWvE5IMbqS DWAbT8VKsyfCCd+CdcoT+860Jq5LtqNRdpg8abdHePT98s49IOcrXszNIadyC6gWcUVqiA95p 9oNRIguD1QWnb69R68kaTmS9MH2cCueeQ3+OL55+CRptJ20U0Tq+kAp8TGuDS1N3R0uoy3h8q TEA0G9uNKp2fcy73rA2uYRhMW+HQNyy8uFIhiW7nSwUa1xUsdS4tnRvpz02a2Deargn5lBoec j5xEwSA/rlg2IDv9ShgJcE2L7eC8M+BRd6KvVCqd4AcfMZ3sEEuuVSgxR9Fl1aqfiluTRIIm4 /l7PPxzmLbegN1aaveE5ZQTbaK8PJgntK6UtdvJL3tmhIxj7IVc8SUh9ZZgHNFFHkAAxVZBu2 UdwOC/gfT+smY6rdJBYc5DwLgE0naymnJE6MeNh95MmuGkgDOzDpeP3Kzd8exZt5m8q0HXMuc P5cTONWSrbcJpDmr5FHfUBY8RBEuIBQrxr7Etpzk3sw9EhekpM8LqI3mZ8Awzd7ywpwyqZhqd kbts2GcyaBsU7gSeUevfILfWA25jnEmVlFtc+5tfw++S7Bf4F/G8Ctx4BT1y0wVUuVu8AjUb9 lzVpIIB+RcVFi/CdChZQ3OkgYXAnids4nuuRh+qWP0sNxGBTDEobP5LKzStvOLtBwXc1xAVaQ 0fwGPqhQHipD+lxv9/HYKIlbox/qMYm/9n1NVb/kTZQe6ESf+NZkUSWsRIldJc723Lzi5zPO3 EMydm/nl2IuRC6W/CnRqr2kxFnht6Sw6ZTAiCk6CSbHegRWMkhyxAZrCXXHaYohrZuAKNFOVP 5CLRvCs7Q40igYcbV/K2mVOsdketxVo05E46NuP5W89hHFwLsvkdPhBAtM8sPPUcVzHXcE37+ WrGk/9bROIBpoGccTynHXZlIxd23VhRpL7mgZaJaGB32ugCWjanTkbT6ye360Kp+q3yDbcYxo K3qSntmjuPyL6egN+dBUfJq+o12423SzGmyLVf46xiqX6cbyHIf3fO087we5HWHBgy6uz+BtP /f5MCw/tqu3T3Ma44qXvp9IdxKaWjp/rj17/JM7lqpXaLw90T/vtLWwzrzMesQSsHY9aBpRUj xLg0r7Xbi4m393HgssBIP/7HTTC847BiLneUiapwydg8wJJ7QwSxYQ+rKt1oVmxirJqM9/fNJ FOqIeM0404BJNk5VNCe0rZfKQw5zAzs/YmDd82zXskFfTvqprJuF7cX2mHzYgge4QrCpB/a1D c5W3KKpO5IP34s5+vXlauwKku2nCVtf+dClMP2bIARXIIQ8RLcTDe+FJacfvE3hZhht05AKlO QqwobJo+mjuBEv7ksp1DSCwg3Sfmpf53bWaZezMbJawP+hISNsvq0RkOa9yKMxmb6wRxyhdYQ zYEYV6/DZiJKhoguMTQ6yjJpXzXYKODdGZ7E2YjbyfdwpxVaCEvRErSuDm+JiTwkjngBpOWxU fijdQyX4TpAOFVvlXc5r8BvAVsjhCgCDoRf3us48jVC9x9jHw== Subject: Re: [Starlink] [Rpm] [M-Lab-Discuss] misery metrics & consequences X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Oct 2022 13:53:02 -0000 Hi David, > On Oct 23, 2022, at 15:11, Dave Collier-Brown = wrote: >=20 > On 10/23/22 08:26, Sebastian Moeller wrote: >=20 >> [SM] Kathy Nichols' pping = (https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithu= b.com%2Fpollere%2Fpping&data=3D05%7C01%7C%7C678b8216944a43dba97508dab4= f1c261%7Cb07c069022b843668d8d7b845d088e18%7C1%7C0%7C638021247687178024%7CU= nknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL= CJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3DLZLefdgL2yTM%2F5jOLeKLCRcWokWA4o= x4Vs0RwYScmqg%3D&reserved=3D0) might be an option, either on the ISP = side or run on CPEs with some method to harvest the collected data from = the ISP side.=20 > Yes: I use pping to investigate occasional problems at work, but I was = thinking more about home networks, where some big speed-changes happen = and local congestion happens. [SM] Okay. In the context of cake-autorate = (https://github.com/lynxthecat/CAKE-autorate/blob/main/README.md) we = implemented a flight recorder type logging that continuously logs the = last X (configurable) epoch and stores bot shaper and achieved rates as = well as the results from the latency probes. This script can be used = with rate setting disabled to record relevant data and the user just = needs to remember to export the data after experiencing = interesting/abnormal events. Sure this does not have per application = resolution, but should give some idea about current latency as well as = current traffic. I will admit though that this logging is not exactly = cheap CPU-wise and lacks the precision of packet captures... but it can = be operated as flight recorder where relevant events can be = exported/stored post-hoc... >=20 >> Protocols with less fields readable like QUIC would require special = care to evaluate the spin-bit if that exists. Or just resort to active = polling and ping* each CPE once per second or so (for a course = resolution, you could increase the polling rate on detecting anomalies = thereby risking to make congestion slightly worse). None of this will = allow to measure within home network congestion though, but it might = still be a wortwhile diagnostic to know that the access link is OK, = while the user reports latency issues. > If one has a good way to capture RTT and data rate for one problematic = app, say zoom, then one could see that network problems were happening = at the same time as lags and dropouts.=20 [SM] As above logging all traffic is relatively easy, per = application or per flow will require different tools or packet = captures... > ISPs would positively hate that, of course. [SM] Assuming they come out of this looking bad, if the outcome = is to imply the local WiFi being the root cause ISPs might actually = appreciate it ;) Regards Sebastian >=20 > --dave >=20 >=20 >=20 >=20 >>=20 >> Regards >> Sebastian >>=20 >> *) I think there are dedicated devices available that allow to ping = large numbers of IPs in a periodic fashion. >>=20 >>=20 >>=20 >>> --dave >>>=20 >>>=20 >>>=20 >>> On 10/23/22 07:57, Sebastian Moeller via Starlink wrote: >>>=20 >>>> Hi Glenn, >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>>> On Oct 23, 2022, at 02:17, Glenn Fishbine via Rpm = >>>>>=20 >>>>> wrote: >>>>>=20 >>>>> As a classic died in the wool empiricist, granted that you can = identify "misery" factors, given a population of 1,000 users, how do you = propose deriving a misery index for that population? >>>>>=20 >>>>> We can measure download, upload, ping, jitter pretty much without = user intervention. For the measurements you hypothesize, how you you = automatically extract those indecies without subjective user = contamination. >>>>>=20 >>>>> I.e. my download speed sucks. Measure the download speed. >>>>>=20 >>>>> My isp doesn't fix my problem. Measure what? How? >>>>>=20 >>>>> Human survey technology is 70+ years old and it still has problems = figuring out how to correlate opinion with fact. >>>>>=20 >>>>> Without an objective measurement scheme that doesn't require human = interaction, the misery index is a cool hypothesis with no way to link = to actual data. What objective measurements can be made? Answer that = and the index becomes useful. Otherwise it's just consumer whining. >>>>>=20 >>>>> Not trying to be combative here, in fact I like the concept you = support, but I'm hard pressed to see how the concept can lead to data, = and the data lead to policy proposals. >>>>>=20 >>>>>=20 >>>> [SM] So it seems that outside of seemingly simple to test = throughput numbers*, the next most important quality number (or the most = important depending on subjective ranking) is how does latency change = under "load". Absolute latency is also important albeit static high = latency can be worked around within limits so the change under load = seems more relevant. >>>> All of flent's RRUL test, apple's networkQuality/RPM, and = iperf2's bounceback test offer methods to asses latency change under = load**, as do waveforms bufferbloat tests and even to a degree Ookla's = speedtest.net. IMHO something like latency increase under load or = apple's responsiveness measure RPM (basically the inverse of the latency = under load calculated on a per minute basis, so it scales in the typical = higher numbers are better way, unlike raw latency under load numbers = where smaller is better). >>>> IMHO what networkQuality is missing ATM is to measure and = report the unloaded RPM as well as the loaded the first gives a measure = over the static latency the second over how well things keep working if = capacity gets tight. They report the base RTT which can be converted to = RPM. As an example: >>>>=20 >>>> macbook:~ user$ networkQuality -v >>>> =3D=3D=3D=3D SUMMARY =3D=3D=3D=3D >>>> Upload capacity: 24.341 Mbps >>>> Download capacity: 91.951 Mbps >>>> Upload flows: 20 >>>> Download flows: 16 >>>> Responsiveness: High (2123 RPM) >>>> Base RTT: 16 >>>> Start: 10/23/22, 13:44:39 >>>> End: 10/23/22, 13:44:53 >>>> OS Version: Version 12.6 (Build 21G115) >>>>=20 >>>> Here RPM 2133 corresponds to 60000/2123 =3D 28.26 ms latency under = load, while the Base RTT of 16ms corresponds to 60000/16 =3D 3750 RPM, = son on this link load reduces the responsiveness by 3750-2123 =3D 1627 = RPM a reduction by 100-100*2123/3750 =3D 43.4%, and that is with = competent AQM and scheduling on the router. >>>>=20 >>>> Without competent AQM/shaping I get: >>>> =3D=3D=3D=3D SUMMARY =3D=3D=3D=3D >>>> Upload capacity: 15.101 Mbps >>>> Download capacity: 97.664 Mbps >>>> Upload flows: 20 >>>> Download flows: 12 >>>> Responsiveness: Medium (427 RPM) >>>> Base RTT: 16 >>>> Start: 10/23/22, 13:51:50 >>>> End: 10/23/22, 13:52:06 >>>> OS Version: Version 12.6 (Build 21G115) >>>> latency under load: 60000/427 =3D 140.52 ms >>>> base RPM: 60000/16 =3D 3750 RPM >>>> reduction RPM: 100-100*427/3750 =3D 88.6% >>>>=20 >>>>=20 >>>> I understand apple's desire to have a single reported number with a = single qualifier medium/high/... because in the end a link is only = reliably usable if responsiveness under load stays acceptable, but with = two numbers it is easier to see what one's ISP could do to help. (I = guess some ISPs might already be unhappy with the single number, so this = needs some diplomacy/tact) >>>>=20 >>>> Regards >>>> Sebastian >>>>=20 >>>>=20 >>>>=20 >>>> *) Seemingly as quite some ISPs operate their own speedtest servers = in their network and ignore customers not reaching the contracted rates = into speedtest-servers located in different ASs. As the product is = called internet access I a inclined to expect that my ISP maintains = sufficient peering/transit capacity to reach the next tier of AS at my = contracted rate (the EU legislative seems to agree, see EU directive = 2015/2120). >>>>=20 >>>> **) Most do by creating load themselves and measuring throughput at = the same time, bounceback IIUC will focus on the latency measurement and = leave the load generation optional (so offers a mode to measure = responsiveness of a live network with minimal measurement traffic). = @Bob, please correct me if this is wrong. >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>>> On Fri, Oct 21, 2022, 5:20 PM Dave Taht >>>>>=20 >>>>> >>>>>=20 >>>>> wrote: >>>>> One of the best talks I've ever seen on how to measure customer >>>>> satisfaction properly just went up after the P99 Conference. >>>>>=20 >>>>> It's called Misery Metrics. >>>>>=20 >>>>> After going through a deep dive as to why and how we think and act = on >>>>> percentiles, bins, and other statistical methods as to how we use = the >>>>> web and internet are *so wrong* (well worth watching and thinking >>>>> about if you are relying on or creating network metrics today), it >>>>> then points to the real metrics that matter to users and the = ultimate >>>>> success of an internet business: Timeouts, retries, misses, failed >>>>> queries, angry phone calls, abandoned shopping carts and loss of >>>>> engagement. >>>>>=20 >>>>>=20 >>>>>=20 >>>>> = https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fwww.p9= 9conf.io%2Fsession%2Fmisery-metrics-consequences%2F&data=3D05%7C01%7C%= 7C678b8216944a43dba97508dab4f1c261%7Cb07c069022b843668d8d7b845d088e18%7C1%= 7C0%7C638021247687178024%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQI= joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3D%2Bc= MJsNSiXRF%2F77x%2FADA88rnaFK8YbIIBKPOua2Rz41s%3D&reserved=3D0 >>>>>=20 >>>>>=20 >>>>>=20 >>>>> The ending advice was - don't aim to make a specific percentile >>>>> acceptable, aim for an acceptable % of misery. >>>>>=20 >>>>> I enjoyed the p99 conference more than any conference I've = attended in years. >>>>>=20 >>>>> -- >>>>> This song goes out to all the folk that thought Stadia would work: >>>>>=20 >>>>>=20 >>>>> = https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fwww.li= nkedin.com%2Fposts%2Fdtaht_the-mushroom-song-activity-6981366665607352320-= FXtz&data=3D05%7C01%7C%7C678b8216944a43dba97508dab4f1c261%7Cb07c069022= b843668d8d7b845d088e18%7C1%7C0%7C638021247687178024%7CUnknown%7CTWFpbGZsb3= d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C300= 0%7C%7C%7C&sdata=3DaSrboZRnm30gb6ZRrFtZ01Gl65axo1vmxaouBE1%2FK9k%3D&am= p;reserved=3D0 >>>>>=20 >>>>>=20 >>>>> Dave T=C3=A4ht CEO, TekLibre, LLC >>>>>=20 >>>>> -- >>>>> You received this message because you are subscribed to the Google = Groups "discuss" group. >>>>> To unsubscribe from this group and stop receiving emails from it, = send an email to >>>>>=20 >>>>> discuss+unsubscribe@measurementlab.net >>>>>=20 >>>>> . >>>>> To view this discussion on the web visit >>>>>=20 >>>>> = https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgroups= .google.com%2Fa%2Fmeasurementlab.net%2Fd%2Fmsgid%2Fdiscuss%2FCAA93jw4w27a1= EO_QQG7NNkih%252BC3QQde5%253D_7OqGeS9xy9nB6wkg%2540mail.gmail.com&data= =3D05%7C01%7C%7C678b8216944a43dba97508dab4f1c261%7Cb07c069022b843668d8d7b8= 45d088e18%7C1%7C0%7C638021247687178024%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4= wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&am= p;sdata=3Da2Yru9HMRhkHuP6M8qsA5pgB20uw11w%2BdiyX%2Fy9VYTQ%3D&reserved=3D= 0 >>>>>=20 >>>>> . >>>>> _______________________________________________ >>>>> Rpm mailing list >>>>>=20 >>>>>=20 >>>>> Rpm@lists.bufferbloat.net >>>>> = https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists.= bufferbloat.net%2Flistinfo%2Frpm&data=3D05%7C01%7C%7C678b8216944a43dba= 97508dab4f1c261%7Cb07c069022b843668d8d7b845d088e18%7C1%7C0%7C6380212476871= 78024%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6= Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3D51HHEyIB0moDJPIsjDnhNNT= 4YxMvIAGiyh3he5WguVU%3D&reserved=3D0 >>>> _______________________________________________ >>>> Starlink mailing list >>>>=20 >>>>=20 >>>> Starlink@lists.bufferbloat.net >>>> = https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists.= bufferbloat.net%2Flistinfo%2Fstarlink&data=3D05%7C01%7C%7C678b8216944a= 43dba97508dab4f1c261%7Cb07c069022b843668d8d7b845d088e18%7C1%7C0%7C63802124= 7687178024%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJ= BTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3DfLB3ojY%2FXNZ2%2Fh= Wc%2B2WfwOhHz1vLIrC653g2ZmlLRrA%3D&reserved=3D0 >>> -- >>> David Collier-Brown, | Always do right. This will gratify >>> System Programmer and Author | some people and astonish the rest >>>=20 >>>=20 >>> dave.collier-brown@indexexchange.com >>> | -- Mark Twain >>>=20 >>> CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, = including any and all attachments, contains confidential information = intended only for the person(s) to whom it is addressed. Any = dissemination, distribution, copying or disclosure is strictly = prohibited and is not a waiver of confidentiality. If you have received = this telecommunication in error, please notify the sender immediately by = return electronic mail and delete the message from your inbox and = deleted items folders. This telecommunication does not constitute an = express or implied agreement to conduct transactions by electronic = means, nor does it constitute a contract offer, a contract amendment or = an acceptance of a contract offer. Contract terms contained in this = telecommunication are subject to legal review and the completion of = formal documentation and are not binding until same is confirmed in = writing and has been signed by an authorized signatory. >>>=20 >>> _______________________________________________ >>> Starlink mailing list >>>=20 >>> Starlink@lists.bufferbloat.net >>> = https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists.= bufferbloat.net%2Flistinfo%2Fstarlink&data=3D05%7C01%7C%7C678b8216944a= 43dba97508dab4f1c261%7Cb07c069022b843668d8d7b845d088e18%7C1%7C0%7C63802124= 7687178024%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJ= BTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3DfLB3ojY%2FXNZ2%2Fh= Wc%2B2WfwOhHz1vLIrC653g2ZmlLRrA%3D&reserved=3D0 > --=20 > David Collier-Brown, | Always do right. This will gratify > System Programmer and Author | some people and astonish the rest >=20 > dave.collier-brown@indexexchange.com | -- Mark Twain