From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp93.iad3b.emailsrvr.com (smtp93.iad3b.emailsrvr.com [146.20.161.93]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 934703B2A4 for ; Sun, 23 Oct 2022 11:01:23 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=evslin.com; s=20220608-p431o605; t=1666537283; bh=pZAvmWbXSsdhfuLkAiq90xUbIED4q0pUSBkZ2SC/Tl0=; h=From:To:Subject:Date:From; b=nKUO8EYp0YyyW60hNlsw0/feLGNR/3ORtq7In9E5VWgt9s3vaUvD10GMsrNP4Psu6 q16Ue025oxa8DIQtS9pu9xmqeYzRF4KwO5otvA7lzmZMKC0jcU4fnRFmlO7r1Qi0Wv KK/DRZ0iMl/LLrlMZxRHKKp0XbQHBni5V3mxsy8w= X-Auth-ID: tom@evslin.com Received: by smtp4.relay.iad3b.emailsrvr.com (Authenticated sender: tom-AT-evslin.com) with ESMTPSA id 5071F2006C; Sun, 23 Oct 2022 11:01:22 -0400 (EDT) From: To: "'Dave Collier-Brown'" , "'Sebastian Moeller'" Cc: References: <339AB8BC-9628-40E2-9339-77FCFA74488D@gmx.de> <0e811787-cc55-8db9-2a4b-7706813769da@indexexchange.com> <850ACED5-3345-441E-9A92-BF27B21EABF5@gmx.de> <78ae1fce-b23a-ef67-e486-72c26d2bbe30@indexexchange.com> <93A92B00-AB22-414A-9A05-B887262E5A8E@gmx.de> <1575107a-ce9f-e5ce-7611-74ea1496cbd5@indexexchange.com> In-Reply-To: <1575107a-ce9f-e5ce-7611-74ea1496cbd5@indexexchange.com> Date: Sun, 23 Oct 2022 11:01:21 -0400 Message-ID: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_ECE5_01D8E6CE.C99C1310" X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQG3oT8FZnnA05/3JUWy5ZzAifhEawGDBoByAkIGcdICamzeFgIYT3s3AjiYQN4BAyeRwAHIMukJrfQ/SJA= Content-Language: en-us X-Classification-ID: d16cd4fd-696c-46ea-8ab0-eca9a5f62a1e-1-1 Subject: Re: [Starlink] [Rpm] [M-Lab-Discuss] misery metrics & consequences X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Oct 2022 15:01:23 -0000 This is a multipart message in MIME format. ------=_NextPart_000_ECE5_01D8E6CE.C99C1310 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I think a =E2=80=9Cgood reason=E2=80=9D is a way to meaningfully compare = the usefulness of various ISPs for teleconferencing. Needs obviously = different from web browsing, email, or, steaming but critical to many = people. Lots of federal dollars going out to build out = =E2=80=9Cbroadband=E2=80=9D but generally with up and download speed as = the only metrics and now acknowledgement of the =E2=80=9Cmisery=E2=80=9D = jitter or just log latency can impose. =20 From: Starlink On Behalf Of = Dave Collier-Brown via Starlink Sent: Sunday, October 23, 2022 10:08 AM To: Sebastian Moeller Cc: starlink@lists.bufferbloat.net Subject: Re: [Starlink] [Rpm] [M-Lab-Discuss] misery metrics & = consequences =20 OK, it's pretty clear that we're already measuring and adapting to = misery, does anyone have a good reason to want to provide a "misery = meter"? =20 I'd normally be tempted, but I'm working in the ML team in a startup, = and have been having trouble even reading email this year (;-)) --dave =20 =20 On 10/23/22 09:52, Sebastian Moeller wrote: [EXTERNAL] This email originated from outside the organization. Do not = click links or open attachments unless you recognize the sender and know = the content is safe. =20 Hi David, =20 =20 On Oct 23, 2022, at 15:11, Dave Collier-Brown = = wrote: =20 On 10/23/22 08:26, Sebastian Moeller wrote: =20 [SM] Kathy Nichols' pping = (https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgith= ub.com%2Fpollere%2Fpping = = &data=3D05%7C01%7C%7C0de1c0ddf51f41aef70c08dab4fde5ad%7Cb07c069022b84= 3668d8d7b845d088e18%7C1%7C0%7C638021299817872156%7CUnknown%7CTWFpbGZsb3d8= eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000= %7C%7C%7C&sdata=3DJsNkkEDTzwCintX7H2KhsviIuc2S4r7RFYIevucnKeA%3D&= reserved=3D0) might be an option, either on the ISP side or run on CPEs = with some method to harvest the collected data from the ISP side. Yes: I use pping to investigate occasional problems at work, but I was = thinking more about home networks, where some big speed-changes happen = and local congestion happens. =20 [SM] Okay. In the context of cake-autorate = (https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgith= ub.com%2Flynxthecat%2FCAKE-autorate%2Fblob%2Fmain%2FREADME.md = = &data=3D05%7C01%7C%7C0de1c0ddf51f41aef70c08dab4fde5ad%7Cb07c069022b84= 3668d8d7b845d088e18%7C1%7C0%7C638021299817872156%7CUnknown%7CTWFpbGZsb3d8= eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000= %7C%7C%7C&sdata=3DLwP%2F45%2FIdIJgdFGITFEgYa2egg4xyQEqkE21HYKb0nM%3D&= amp;reserved=3D0) we implemented a flight recorder type logging that = continuously logs the last X (configurable) epoch and stores bot shaper = and achieved rates as well as the results from the latency probes. This = script can be used with rate setting disabled to record relevant data = and the user just needs to remember to export the data after = experiencing interesting/abnormal events. Sure this does not have per = application resolution, but should give some idea about current latency = as well as current traffic. I will admit though that this logging is not = exactly cheap CPU-wise and lacks the precision of packet captures... but = it can be operated as flight recorder where relevant events can be = exported/stored post-hoc... =20 =20 =20 Protocols with less fields readable like QUIC would require special care = to evaluate the spin-bit if that exists. Or just resort to active = polling and ping* each CPE once per second or so (for a course = resolution, you could increase the polling rate on detecting anomalies = thereby risking to make congestion slightly worse). None of this will = allow to measure within home network congestion though, but it might = still be a wortwhile diagnostic to know that the access link is OK, = while the user reports latency issues. If one has a good way to capture RTT and data rate for one problematic = app, say zoom, then one could see that network problems were happening = at the same time as lags and dropouts. =20 [SM] As above logging all traffic is relatively easy, per = application or per flow will require different tools or packet = captures... =20 ISPs would positively hate that, of course. =20 [SM] Assuming they come out of this looking bad, if the outcome = is to imply the local WiFi being the root cause ISPs might actually = appreciate it ;) =20 Regards Sebastian =20 =20 --dave =20 =20 =20 =20 =20 Regards Sebastian =20 *) I think there are dedicated devices available that allow to ping = large numbers of IPs in a periodic fashion. =20 =20 =20 --dave =20 =20 =20 On 10/23/22 07:57, Sebastian Moeller via Starlink wrote: =20 Hi Glenn, =20 =20 =20 =20 On Oct 23, 2022, at 02:17, Glenn Fishbine via Rpm = =20 wrote: =20 As a classic died in the wool empiricist, granted that you can identify = "misery" factors, given a population of 1,000 users, how do you propose = deriving a misery index for that population? =20 We can measure download, upload, ping, jitter pretty much without user = intervention. For the measurements you hypothesize, how you you = automatically extract those indecies without subjective user = contamination. =20 I.e. my download speed sucks. Measure the download speed. =20 My isp doesn't fix my problem. Measure what? How? =20 Human survey technology is 70+ years old and it still has problems = figuring out how to correlate opinion with fact. =20 Without an objective measurement scheme that doesn't require human = interaction, the misery index is a cool hypothesis with no way to link = to actual data. What objective measurements can be made? Answer that = and the index becomes useful. Otherwise it's just consumer whining. =20 Not trying to be combative here, in fact I like the concept you support, = but I'm hard pressed to see how the concept can lead to data, and the = data lead to policy proposals. =20 =20 [SM] So it seems that outside of seemingly simple to test = throughput numbers*, the next most important quality number (or the most = important depending on subjective ranking) is how does latency change = under "load". Absolute latency is also important albeit static high = latency can be worked around within limits so the change under load = seems more relevant. All of flent's RRUL test, apple's networkQuality/RPM, and iperf2's = bounceback test offer methods to asses latency change under load**, as = do waveforms bufferbloat tests and even to a degree Ookla's = speedtest.net. IMHO something like latency increase under load or = apple's responsiveness measure RPM (basically the inverse of the latency = under load calculated on a per minute basis, so it scales in the typical = higher numbers are better way, unlike raw latency under load numbers = where smaller is better). IMHO what networkQuality is missing ATM is to measure and report = the unloaded RPM as well as the loaded the first gives a measure over = the static latency the second over how well things keep working if = capacity gets tight. They report the base RTT which can be converted to = RPM. As an example: =20 macbook:~ user$ networkQuality -v =3D=3D=3D=3D SUMMARY =3D=3D=3D=3D Upload capacity: 24.341 Mbps Download capacity: 91.951 Mbps Upload flows: 20 Download flows: 16 Responsiveness: High (2123 RPM) Base RTT: 16 Start: 10/23/22, 13:44:39 End: 10/23/22, 13:44:53 OS Version: Version 12.6 (Build 21G115) =20 Here RPM 2133 corresponds to 60000/2123 =3D 28.26 ms latency under load, = while the Base RTT of 16ms corresponds to 60000/16 =3D 3750 RPM, son on = this link load reduces the responsiveness by 3750-2123 =3D 1627 RPM a = reduction by 100-100*2123/3750 =3D 43.4%, and that is with competent AQM = and scheduling on the router. =20 Without competent AQM/shaping I get: =3D=3D=3D=3D SUMMARY =3D=3D=3D=3D Upload capacity: 15.101 Mbps Download capacity: 97.664 Mbps Upload flows: 20 Download flows: 12 Responsiveness: Medium (427 RPM) Base RTT: 16 Start: 10/23/22, 13:51:50 End: 10/23/22, 13:52:06 OS Version: Version 12.6 (Build 21G115) latency under load: 60000/427 =3D 140.52 ms base RPM: 60000/16 =3D 3750 RPM reduction RPM: 100-100*427/3750 =3D 88.6% =20 =20 I understand apple's desire to have a single reported number with a = single qualifier medium/high/... because in the end a link is only = reliably usable if responsiveness under load stays acceptable, but with = two numbers it is easier to see what one's ISP could do to help. (I = guess some ISPs might already be unhappy with the single number, so this = needs some diplomacy/tact) =20 Regards Sebastian =20 =20 =20 *) Seemingly as quite some ISPs operate their own speedtest servers in = their network and ignore customers not reaching the contracted rates = into speedtest-servers located in different ASs. As the product is = called internet access I a inclined to expect that my ISP maintains = sufficient peering/transit capacity to reach the next tier of AS at my = contracted rate (the EU legislative seems to agree, see EU directive = 2015/2120). =20 **) Most do by creating load themselves and measuring throughput at the = same time, bounceback IIUC will focus on the latency measurement and = leave the load generation optional (so offers a mode to measure = responsiveness of a live network with minimal measurement traffic). = @Bob, please correct me if this is wrong. =20 =20 =20 =20 On Fri, Oct 21, 2022, 5:20 PM Dave Taht =20 =20 wrote: One of the best talks I've ever seen on how to measure customer satisfaction properly just went up after the P99 Conference. =20 It's called Misery Metrics. =20 After going through a deep dive as to why and how we think and act on percentiles, bins, and other statistical methods as to how we use the web and internet are *so wrong* (well worth watching and thinking about if you are relying on or creating network metrics today), it then points to the real metrics that matter to users and the ultimate success of an internet business: Timeouts, retries, misses, failed queries, angry phone calls, abandoned shopping carts and loss of engagement. =20 =20 =20 https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fwww.p= 99conf.io%2Fsession%2Fmisery-metrics-consequences%2F = = &data=3D05%7C01%7C%7C0de1c0ddf51f41aef70c08dab4fde5ad%7Cb07c069022b84= 3668d8d7b845d088e18%7C1%7C0%7C638021299817872156%7CUnknown%7CTWFpbGZsb3d8= eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000= %7C%7C%7C&sdata=3D%2FOe2eo9f7JQ8bnQRB23HEaeXq6G9QxSQ%2FZkNb%2F6ctyU%3= D&reserved=3D0 =20 =20 =20 The ending advice was - don't aim to make a specific percentile acceptable, aim for an acceptable % of misery. =20 I enjoyed the p99 conference more than any conference I've attended in = years. =20 -- This song goes out to all the folk that thought Stadia would work: =20 =20 https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fwww.l= inkedin.com%2Fposts%2Fdtaht_the-mushroom-song-activity-698136666560735232= 0-FXtz = = &data=3D05%7C01%7C%7C0de1c0ddf51f41aef70c08dab4fde5ad%7Cb07c069022b84= 3668d8d7b845d088e18%7C1%7C0%7C638021299817872156%7CUnknown%7CTWFpbGZsb3d8= eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000= %7C%7C%7C&sdata=3DALKX4qknTgJBAiBET9j2yfdyhuEmM5rs2Ng3%2B09rat4%3D&am= p;reserved=3D0 =20 =20 Dave T=C3=A4ht CEO, TekLibre, LLC =20 -- You received this message because you are subscribed to the Google = Groups "discuss" group. To unsubscribe from this group and stop receiving emails from it, send = an email to =20 discuss+unsubscribe@measurementlab.net = =20 =20 . To view this discussion on the web visit =20 https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgroup= s.google.com%2Fa%2Fmeasurementlab.net%2Fd%2Fmsgid%2Fdiscuss%2FCAA93jw4w27= a1EO_QQG7NNkih%252BC3QQde5%253D_7OqGeS9xy9nB6wkg%2540mail.gmail.com&d= ata=3D05%7C01%7C%7C0de1c0ddf51f41aef70c08dab4fde5ad%7Cb07c069022b843668d8= d7b845d088e18%7C1%7C0%7C638021299817872156%7CUnknown%7CTWFpbGZsb3d8eyJWIj= oiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C= %7C&sdata=3DHVk9tgu97ElRdvdHiiE3PSuEzT6PM731Ag4XMIVDJIU%3D&reserv= ed=3D0 =20 . _______________________________________________ Rpm mailing list =20 =20 Rpm@lists.bufferbloat.net =20 https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists= .bufferbloat.net%2Flistinfo%2Frpm = = &data=3D05%7C01%7C%7C0de1c0ddf51f41aef70c08dab4fde5ad%7Cb07c069022b84= 3668d8d7b845d088e18%7C1%7C0%7C638021299817872156%7CUnknown%7CTWFpbGZsb3d8= eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000= %7C%7C%7C&sdata=3D9Qd2WIP0ONe2zt%2FX3r0ws3QQMkRNjfmeY7dl9LH6T9k%3D&am= p;reserved=3D0 _______________________________________________ Starlink mailing list =20 =20 Starlink@lists.bufferbloat.net =20 https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists= .bufferbloat.net%2Flistinfo%2Fstarlink = = &data=3D05%7C01%7C%7C0de1c0ddf51f41aef70c08dab4fde5ad%7Cb07c069022b84= 3668d8d7b845d088e18%7C1%7C0%7C638021299817872156%7CUnknown%7CTWFpbGZsb3d8= eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000= %7C%7C%7C&sdata=3DhEuwM2IalFt67cx%2FkqQuHNR%2FL%2B8pwH0PKtMCiFMb6yU%3= D&reserved=3D0 -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest =20 =20 dave.collier-brown@indexexchange.com = =20 | -- Mark Twain =20 CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, = including any and all attachments, contains confidential information = intended only for the person(s) to whom it is addressed. Any = dissemination, distribution, copying or disclosure is strictly = prohibited and is not a waiver of confidentiality. If you have received = this telecommunication in error, please notify the sender immediately by = return electronic mail and delete the message from your inbox and = deleted items folders. This telecommunication does not constitute an = express or implied agreement to conduct transactions by electronic = means, nor does it constitute a contract offer, a contract amendment or = an acceptance of a contract offer. Contract terms contained in this = telecommunication are subject to legal review and the completion of = formal documentation and are not binding until same is confirmed in = writing and has been signed by an authorized signatory. =20 _______________________________________________ Starlink mailing list =20 Starlink@lists.bufferbloat.net =20 https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists= .bufferbloat.net%2Flistinfo%2Fstarlink = = &data=3D05%7C01%7C%7C0de1c0ddf51f41aef70c08dab4fde5ad%7Cb07c069022b84= 3668d8d7b845d088e18%7C1%7C0%7C638021299817872156%7CUnknown%7CTWFpbGZsb3d8= eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000= %7C%7C%7C&sdata=3DhEuwM2IalFt67cx%2FkqQuHNR%2FL%2B8pwH0PKtMCiFMb6yU%3= D&reserved=3D0 -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest =20 dave.collier-brown@indexexchange.com = | -- Mark = Twain =20 --=20 David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest dave.collier-brown@indexexchange.com = | -- Mark = Twain ------=_NextPart_000_ECE5_01D8E6CE.C99C1310 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

I think a =E2=80=9Cgood = reason=E2=80=9D is a way to meaningfully compare the usefulness of = various ISPs for teleconferencing. Needs obviously different from web = browsing, email, or, steaming but critical to many people. Lots of = federal dollars going out to build out =E2=80=9Cbroadband=E2=80=9D but = generally with up and download speed as the only metrics and now = acknowledgement of the =E2=80=9Cmisery=E2=80=9D jitter or just log = latency can impose.

 

From: Starlink = <starlink-bounces@lists.bufferbloat.net> On Behalf Of Dave = Collier-Brown via Starlink
Sent: Sunday, October 23, 2022 = 10:08 AM
To: Sebastian Moeller = <moeller0@gmx.de>
Cc: = starlink@lists.bufferbloat.net
Subject: Re: [Starlink] [Rpm] = [M-Lab-Discuss] misery metrics & = consequences

 

OK, it's pretty clear that = we're already measuring and adapting to misery, does anyone have a good = reason to want to provide a "misery meter"?  =

I'd normally be tempted, but I'm working in the ML = team in a startup, and have been having trouble even reading = email this year = (;-))

--dave

 

 

On 10/23/22 09:52, = Sebastian Moeller wrote:

[EXTERNAL] This =
email originated from outside the organization. Do not click links or =
open attachments unless you recognize the sender and know the content is =
safe.
 
Hi =
David,
 
 =

On =
Oct 23, 2022, at 15:11, Dave Collier-Brown <dave.collier-Bro=
wn@indexexchange.com> =
wrote:
 
On 10/23/22 =
08:26, Sebastian Moeller =
wrote:
 
=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 [SM] Kathy Nichols' pping (https://can01.sa=
felinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithub.com%2Fpollere%=
2Fpping&amp;data=3D05%7C01%7C%7C0de1c0ddf51f41aef70c08dab4fde5ad%7Cb0=
7c069022b843668d8d7b845d088e18%7C1%7C0%7C638021299817872156%7CUnknown%7CT=
WFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn=
0%3D%7C3000%7C%7C%7C&amp;sdata=3DJsNkkEDTzwCintX7H2KhsviIuc2S4r7RFYIe=
vucnKeA%3D&amp;reserved=3D0) might be an option, either on the =
ISP side or run on CPEs with some method to harvest the collected data =
from the ISP side.
Yes: I use pping to =
investigate occasional problems at work, but I was thinking more about =
home networks, where some big speed-changes happen and local congestion =
happens.
 
=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 [SM] Okay. In the context of =
cake-autorate (https://can01.safelinks.protection.outlook.com/?=
url=3Dhttps%3A%2F%2Fgithub.com%2Flynxthecat%2FCAKE-autorate%2Fblob%2Fmain=
%2FREADME.md&amp;data=3D05%7C01%7C%7C0de1c0ddf51f41aef70c08dab4fde5ad=
%7Cb07c069022b843668d8d7b845d088e18%7C1%7C0%7C638021299817872156%7CUnknow=
n%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV=
CI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=3DLwP%2F45%2FIdIJgdFGITFEgYa2egg4=
xyQEqkE21HYKb0nM%3D&amp;reserved=3D0) we implemented a flight =
recorder type logging that continuously logs the last X (configurable) =
epoch and stores bot shaper and achieved rates as well as the results =
from the latency probes. This script can be used with rate setting =
disabled to record relevant data and the user just needs to remember to =
export the data after experiencing interesting/abnormal events. Sure =
this does not have per application resolution, but should give some idea =
about current latency as well as current traffic. I will admit though =
that this logging is not exactly cheap CPU-wise and lacks the precision =
of packet captures... but it can be operated as flight recorder where =
relevant events can be exported/stored =
post-hoc...
 
 <=
/o:p>
 
Protocols with less =
fields readable like QUIC would require special care to evaluate the =
spin-bit if that exists. Or just resort to active polling and ping* each =
CPE once per second or so (for a course resolution, you could increase =
the polling rate on detecting anomalies thereby risking to make =
congestion slightly worse). None of this will allow to measure within =
home network congestion though, but it might still be a wortwhile =
diagnostic to know that the access link is OK, while the user reports =
latency issues.
If one has a good way =
to capture RTT and data rate for one problematic app, say zoom, then one =
could see that network problems were happening at the same time as lags =
and =
dropouts.
 
=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 [SM] As above logging all =
traffic is relatively easy, per application or per flow will require =
different tools or packet =
captures...
 
ISPs would =
positively hate that, of =
course.
 
=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 [SM] Assuming they come out of =
this looking bad, if the outcome is to imply the local WiFi being the =
root cause ISPs might actually appreciate it =
;)
 
Regards
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
Sebastian
 
 
--dave
 
 =
;
 
 
 
Regards
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 Sebastian
 
*) I =
think there are dedicated devices available that allow to ping large =
numbers of IPs in a periodic =
fashion.
 
 
 
--dave
 
 
 =
;
On 10/23/22 07:57, Sebastian Moeller via Starlink =
wrote:
 
Hi =
Glenn,
 
 =

 
 
On Oct 23, 2022, at =
02:17, Glenn Fishbine via Rpm <rpm@lists.bufferbloat.net&g=
t;
 
 =
wrote:
 
As a classic =
died in the wool empiricist, granted that you can identify =
"misery" factors, given a population of 1,000 users, how do =
you propose deriving a misery index for that =
population?
 
We can =
measure download, upload, ping, jitter pretty much without user =
intervention.=C2=A0 For the measurements you hypothesize, how you you =
automatically extract those indecies without subjective user =
contamination.
 
I.e.=C2=A0=
 my download speed sucks. Measure the download =
speed.
 
My isp doesn't =
fix my problem. Measure what? =
How?
 
Human survey =
technology is 70+ years old and it still has problems figuring out how =
to correlate opinion with =
fact.
 
Without an =
objective measurement scheme that doesn't require human interaction, the =
misery index is a cool hypothesis with no way to link to actual =
data.=C2=A0 What objective measurements can be made?=C2=A0 Answer that =
and the index becomes useful. Otherwise it's just consumer =
whining.
 
Not trying to =
be combative here, in fact I like the concept you support, but I'm hard =
pressed to see how the concept can lead to data, and the data lead to =
policy =
proposals.
 
 
=C2=A0=C2=A0=C2=A0=C2=A0 [SM] So it seems =
that outside of seemingly simple to test throughput numbers*, the next =
most important quality number (or the most important depending on =
subjective ranking) is how does latency change under "load". =
Absolute latency is also important albeit static high latency can be =
worked around within limits so the change under load seems more =
relevant.
=C2=A0=C2=A0=C2=A0=C2=A0 All of flent's =
RRUL test, apple's networkQuality/RPM, and iperf2's bounceback test =
offer methods to asses latency change under load**, as do waveforms =
bufferbloat tests and even to a degree Ookla's speedtest.net. IMHO =
something like latency increase under load or apple's responsiveness =
measure RPM (basically the inverse of the latency under load calculated =
on a per minute basis, so it scales in the typical higher numbers are =
better way, unlike raw latency under load numbers where smaller is =
better).
=C2=A0=C2=A0=C2=A0=C2=A0 IMHO what =
networkQuality is missing ATM is to measure and report the unloaded RPM =
as well as the loaded the first gives a measure over the static latency =
the second over how well things keep working if capacity gets tight. =
They report the base RTT which can be converted to RPM. As an =
example:
 
macbook:~ =
user$ networkQuality -v
=3D=3D=3D=3D SUMMARY =
=3D=3D=3D=3D
Upload capacity: 24.341 =
Mbps
Download capacity: 91.951 =
Mbps
Upload flows: 20
Download =
flows: 16
Responsiveness: High (2123 =
RPM)
Base RTT: 16
Start: =
10/23/22, 13:44:39
End: 10/23/22, =
13:44:53
OS Version: Version 12.6 (Build =
21G115)
 
Here RPM 2133 =
corresponds to 60000/2123 =3D 28.26 ms latency under load, while the =
Base RTT of 16ms corresponds to 60000/16 =3D 3750 RPM, son on this link =
load reduces the responsiveness by 3750-2123 =3D 1627 RPM a reduction by =
100-100*2123/3750 =3D 43.4%, and that is with competent AQM and =
scheduling on the =
router.
 
Without =
competent AQM/shaping I get:
=3D=3D=3D=3D SUMMARY =
=3D=3D=3D=3D
Upload capacity: 15.101 =
Mbps
Download capacity: 97.664 =
Mbps
Upload flows: 20
Download =
flows: 12
Responsiveness: Medium (427 =
RPM)
Base RTT: 16
Start: =
10/23/22, 13:51:50
End: 10/23/22, =
13:52:06
OS Version: Version 12.6 (Build =
21G115)
latency under load: 60000/427 =3D 140.52 =
ms
base RPM: 60000/16 =3D 3750 =
RPM
reduction RPM: 100-100*427/3750 =3D =
88.6%
 
 <=
/pre>
I understand apple's desire to have a single reported number =
with a single qualifier medium/high/... because in the end a link is =
only reliably usable if responsiveness under load stays acceptable, but =
with two numbers it is easier to see what one's ISP could do to help. (I =
guess some ISPs might already be unhappy with the single number, so this =
needs some =
diplomacy/tact)
 
Regards<=
o:p>
=C2=A0=C2=A0=C2=A0=C2=A0 =
Sebastian
 
 
 
*) Seemingly as quite some =
ISPs operate their own speedtest servers in their network and ignore =
customers not reaching the contracted rates into speedtest-servers =
located in different ASs. As the product is called internet access I a =
inclined to expect that my ISP maintains sufficient peering/transit =
capacity to reach the next tier of AS at my contracted rate (the EU =
legislative seems to agree, see EU directive =
2015/2120).
 
**) Most do =
by creating load themselves and measuring throughput at the same time, =
bounceback IIUC will focus on the latency measurement and leave the load =
generation optional (so offers a mode to measure responsiveness of a =
live network with minimal measurement traffic). @Bob, please correct me =
if this is =
wrong.
 
 =

 
 
On Fri, Oct 21, =
2022, 5:20 PM Dave =
Taht
 
<dave.taht@gmail.com><=
/o:p>
 
 =
wrote:
One of the best talks I've ever seen on how =
to measure customer
satisfaction properly just went =
up after the P99 =
Conference.
 
It's called =
Misery Metrics.
 
After =
going through a deep dive as to why and how we think and act =
on
percentiles, bins, and other statistical methods =
as to how we use the
web and internet are *so =
wrong* (well worth watching and thinking
about if =
you are relying on or creating network metrics today), =
it
then points to the real metrics that matter to =
users and the ultimate
success of an internet =
business: Timeouts, retries, misses, =
failed
queries, angry phone calls, abandoned =
shopping carts and loss =
of
engagement.
 
 
 
https://can01.safelinks.protection.outlook.com/?url=3D=
https%3A%2F%2Fwww.p99conf.io%2Fsession%2Fmisery-metrics-consequences%2F&a=
mp;amp;data=3D05%7C01%7C%7C0de1c0ddf51f41aef70c08dab4fde5ad%7Cb07c069022b=
843668d8d7b845d088e18%7C1%7C0%7C638021299817872156%7CUnknown%7CTWFpbGZsb3=
d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C30=
00%7C%7C%7C&amp;sdata=3D%2FOe2eo9f7JQ8bnQRB23HEaeXq6G9QxSQ%2FZkNb%2F6=
ctyU%3D&amp;reserved=3D0
 
 
 
The =
ending advice was - don't aim to make a specific =
percentile
acceptable, aim for an acceptable % of =
misery.
 
I enjoyed the =
p99 conference more than any conference I've attended in =
years.
 
--
This song goes out to all the folk that thought Stadia would =
work:
 
 <=
/pre>
https://can01.safelinks.protect=
ion.outlook.com/?url=3Dhttps%3A%2F%2Fwww.linkedin.com%2Fposts%2Fdtaht_the=
-mushroom-song-activity-6981366665607352320-FXtz&amp;data=3D05%7C01%7=
C%7C0de1c0ddf51f41aef70c08dab4fde5ad%7Cb07c069022b843668d8d7b845d088e18%7=
C1%7C0%7C638021299817872156%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL=
CJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sda=
ta=3DALKX4qknTgJBAiBET9j2yfdyhuEmM5rs2Ng3%2B09rat4%3D&amp;reserved=3D=
0
 
 <=
/pre>
Dave T=C3=A4ht CEO, TekLibre, =
LLC
 
--
<=
pre>You received this message because you are subscribed to the Google =
Groups "discuss" group.
To unsubscribe =
from this group and stop receiving emails from it, send an email =
to
 
discuss+unsubscrib=
e@measurementlab.net
 
.
To view this discussion on the web =
visit
 
https://can01.safelinks.protection.outlook.co=
m/?url=3Dhttps%3A%2F%2Fgroups.google.com%2Fa%2Fmeasurementlab.net%2Fd%2Fm=
sgid%2Fdiscuss%2FCAA93jw4w27a1EO_QQG7NNkih%252BC3QQde5%253D_7OqGeS9xy9nB6=
wkg%2540mail.gmail.com&amp;data=3D05%7C01%7C%7C0de1c0ddf51f41aef70c08=
dab4fde5ad%7Cb07c069022b843668d8d7b845d088e18%7C1%7C0%7C63802129981787215=
6%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1=
haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=3DHVk9tgu97ElRdvdHiiE3P=
SuEzT6PM731Ag4XMIVDJIU%3D&amp;reserved=3D0
<=
o:p> 
.
_______________________=
________________________
Rpm mailing =
list
 
 
Rpm@lists.bufferbloat.net
http=
s://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists.buf=
ferbloat.net%2Flistinfo%2Frpm&amp;data=3D05%7C01%7C%7C0de1c0ddf51f41a=
ef70c08dab4fde5ad%7Cb07c069022b843668d8d7b845d088e18%7C1%7C0%7C6380212998=
17872156%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJB=
TiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=3D9Qd2WIP0ONe2zt=
%2FX3r0ws3QQMkRNjfmeY7dl9LH6T9k%3D&amp;reserved=3D0
_______________________________________________=

Starlink mailing =
list
 
 
Starlink@lists.bufferbloat=
.net
https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fli=
sts.bufferbloat.net%2Flistinfo%2Fstarlink&amp;data=3D05%7C01%7C%7C0de=
1c0ddf51f41aef70c08dab4fde5ad%7Cb07c069022b843668d8d7b845d088e18%7C1%7C0%=
7C638021299817872156%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi=
V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=3DhE=
uwM2IalFt67cx%2FkqQuHNR%2FL%2B8pwH0PKtMCiFMb6yU%3D&amp;reserved=3D0
--
David =
Collier-Brown,=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | Always =
do right. This will gratify
System Programmer and =
Author | some people and astonish the =
rest
 
 
dave.collier-brown@i=
ndexexchange.com
 =
|=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 -- Mark =
Twain
 
CONFIDENTIALITY =
NOTICE AND DISCLAIMER : This telecommunication, including any and all =
attachments, contains confidential information intended only for the =
person(s) to whom it is addressed. Any dissemination, distribution, =
copying or disclosure is strictly prohibited and is not a waiver of =
confidentiality. If you have received this telecommunication in error, =
please notify the sender immediately by return electronic mail and =
delete the message from your inbox and deleted items folders. This =
telecommunication does not constitute an express or implied agreement to =
conduct transactions by electronic means, nor does it constitute a =
contract offer, a contract amendment or an acceptance of a contract =
offer. Contract terms contained in this telecommunication are subject to =
legal review and the completion of formal documentation and are not =
binding until same is confirmed in writing and has been signed by an =
authorized =
signatory.
 
_____________=
__________________________________
Starlink mailing =
list
 
Starlink@lists.bufferbloat=
.net
https://can01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fli=
sts.bufferbloat.net%2Flistinfo%2Fstarlink&amp;data=3D05%7C01%7C%7C0de=
1c0ddf51f41aef70c08dab4fde5ad%7Cb07c069022b843668d8d7b845d088e18%7C1%7C0%=
7C638021299817872156%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi=
V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=3DhE=
uwM2IalFt67cx%2FkqQuHNR%2FL%2B8pwH0PKtMCiFMb6yU%3D&amp;reserved=3D0
--
David Collier-Brown,=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | = Always do right. This will gratify
System =
Programmer and Author | some people and astonish the =
rest
 
dave.collier-brown@i=
ndexexchange.com =
|=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 -- Mark =
Twain
 
-- 
David =
Collier-Brown,=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | Always =
do right. This will gratify
System Programmer and =
Author | some people and astonish the rest
dave.collier-brown@i=
ndexexchange.com =
|=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 -- Mark Twain
------=_NextPart_000_ECE5_01D8E6CE.C99C1310--