From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 8FC693B2A4 for ; Sun, 23 Oct 2022 08:26:08 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1666527965; bh=XKu+6Ff8CpzfMT8J2XrR7/DLPyapvdPXaHvhQ6eO8DE=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=Hu5do/vNXzIJjSNh+75Z8BJFbhnZX1gcMYw3fdEme5Sq/Er1Zx7bijcvPCf0i8OSt BHzZ6IgI65eg/G6LIeP9rkWPa3McNOqeP0yr34NCIYQzhfIOHsD3uDeELgU2VtaInk RVUrSQB/QqSUhvsE+zX3WLC4S1/FFLb++3+N6oRY= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from smtpclient.apple ([95.116.66.209]) by mail.gmx.net (mrgmx104 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MTiTt-1oeecS0a7h-00U4VZ; Sun, 23 Oct 2022 14:26:05 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) From: Sebastian Moeller In-Reply-To: <0e811787-cc55-8db9-2a4b-7706813769da@indexexchange.com> Date: Sun, 23 Oct 2022 14:26:04 +0200 Cc: starlink@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: <850ACED5-3345-441E-9A92-BF27B21EABF5@gmx.de> References: <339AB8BC-9628-40E2-9339-77FCFA74488D@gmx.de> <0e811787-cc55-8db9-2a4b-7706813769da@indexexchange.com> To: Dave Collier-Brown X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Provags-ID: V03:K1:L5bHFS7o+tC/KecIuAO/OCp7973ISDG30o3/Llsh2+d52wz4220 nBMS3trE5f49zcH6jJu4FN4iCkVmTbMrQ/HKj2yDgEylkR0cPDMuG2WMJxIOpCHSMA3j6BV fPRJ/WawZcNqyjVArIHHLKP8f3+H0po7kMQ1aAt/O0ve1DXZyVS2+c7A8JDjquPy6++Vdug adjbKZqKsLzn4VPJQSJXg== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:XCbvJcZ4FgI=:WecuNxGifa7xKbR1c2v5yh xsAUMp4ZO4OhV9JDegvr41AmsoXIOyafw5FnM1QDhqr8oyyWudrLw8jqyFBU4G8rTrhm4bAru rmB8eqU29SKYJ15lf4Cy4fPy+G34qUdvDxD09LAO3bLbsvcb192uxxiKaajkz4MUlI7p/Ow53 v3pfHCoMdCmvvkcSGfpxyNatucuXKYIrHBnyEqGani8E3QCKZJlvDJh/VvxBCBl9TzM+Iihid tu5xp1MGyuEt1YnqcuIkA8Iw6PZD2Q1rpSDPeT6iOfxnuwJlVntzy0GnBE4pRvWICgEJcFcZg aB3R0vdM2EKFUDiSEsXj+2KMo4UAlbfcG/RUvRgGzW2gtINun7EGXMsCqdD9FOOXXtjfUna/3 8o0WO9VYqNpXZlT2Rp5uZ3XB62/9ixY59jrNHIwa5olBDip+fSzYX5JzBB9WBEqVO9XVzDewX pRoZy3ONg0YtHuP7jIi0g5fUKOG5qEi/kYYZ1JtjV2ochWHLT4ceAppMq4Cr+QWHe0qzscNPZ dgMcD2LtYWkrJi/UavqUlNjlY+0SugbToy3oewBI62+YRuXqWyatTTzpte7UdFl68McT5HzuF dfS3+8Drb9BU/Kn9LVhomRptuV/Y1aMu7n3DDTYyWZDZsTix37gvj4EIHlXDtODs1/CdpcMgM V49e1pnieP7Y1M23DWUGwNhFoxfUEjlEkxifRUOak4z14NhI17utT0DcCkMmRSb+EQZQPde/P 7JB48HHv6mFtG3CS7aY9Gvtl1+OsKwFtYxUcHAD48MnNrAeYeDzXdaZnvbv2Fj/1lbXrhQ11v xbDtqVWtzOgu5txj5MRdwbOPB1+nhraHVLHTh68hbsKq3o+2advNuWJT8o2GS9b3/e4Mod6U3 O9HC58jGZQg53LmCSZyVuVNfhGmpGoNx+wp85LI3X/7MFBwLonxyKRDNYYfsRuGNPAvIo8Vul KD5bvBgK6xtSiGYIYQt87IhgUgZJciqkQ/4d8HR0tNtLym/XtNL+vtP34GN+UZCQ0zh1gKqjL jqFOV+7CpUtFKLLveuaebTrIOLAQjnHPJmLzv905ZTwl+IMhYda6XzIlMAYH/cp/Io6oKW226 MhYTIEo9GeoWrE4JngczBrimTkW276Kbq7Na37R2G5TFhRCvqFtly1CQUoop5PTaH1ToWaL8D i+BIT92m14W0TPVlARAvZiStjNxmgZRHn0NS9+6FdMEIDZMmuV3K0HzB4ECw4Enhirn2FnUVy Om254i4PhaBllB7v3Y2PckZ7wyxSWuNgze9vRjsaycd8VvV9Zqs8oq4we8EAVtoQemtWsteM+ VM4ikCgFSDwXqx9ss2wGo/5M4uayHCOipcLh7+ClLs9R3tSE2kDQvjDFDcWw4sjkVnmnAIyFK pjbfoDrq65umw2S7b/NhArotdw39v6aLBj04juAdLeFiZfUM2z1tABwpn3WBZ79wRWWT4m+/K i0l0KmakgA74WfUBUZzHr0IAOvEdD62j1gLIWnUpRthZPVGJMy1WzopvAVg4p8uiZ4AbLP0qU lK2FkjeOPWGfayk+vfdOFR/gQ9HLKfiZZEFndFfJqR682 Subject: Re: [Starlink] [Rpm] [M-Lab-Discuss] misery metrics & consequences X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Oct 2022 12:26:09 -0000 Hi David, > On Oct 23, 2022, at 14:17, Dave Collier-Brown via Starlink = wrote: >=20 > If our business-transaction customers are made miserable by timeouts, = by analogy it follows that home internet users are made miserable by >=20 > =E2=80=A2 stalls, "buffering" and complete disappearance in = conference-calls > =E2=80=A2 "shouting down the well" distortion in any kind of = audio > Disappearance is probably disconnection, and easy to measure >=20 > The others are delay-related, and can be computed from timestampe and = sequence numbers. >=20 > Can we provide a tool to expose the latter? A "miserometer"? [SM] Kathy Nichols' pping (https://github.com/pollere/pping) = might be an option, either on the ISP side or run on CPEs with some = method to harvest the collected data from the ISP side. Protocols with = less fields readable like QUIC would require special care to evaluate = the spin-bit if that exists. Or just resort to active polling and ping* = each CPE once per second or so (for a course resolution, you could = increase the polling rate on detecting anomalies thereby risking to make = congestion slightly worse). None of this will allow to measure within = home network congestion though, but it might still be a wortwhile = diagnostic to know that the access link is OK, while the user reports = latency issues. Regards Sebastian *) I think there are dedicated devices available that allow to ping = large numbers of IPs in a periodic fashion. >=20 > --dave >=20 >=20 >=20 > On 10/23/22 07:57, Sebastian Moeller via Starlink wrote: >> Hi Glenn, >>=20 >>=20 >>=20 >>> On Oct 23, 2022, at 02:17, Glenn Fishbine via Rpm = >>> wrote: >>>=20 >>> As a classic died in the wool empiricist, granted that you can = identify "misery" factors, given a population of 1,000 users, how do you = propose deriving a misery index for that population? >>>=20 >>> We can measure download, upload, ping, jitter pretty much without = user intervention. For the measurements you hypothesize, how you you = automatically extract those indecies without subjective user = contamination. >>>=20 >>> I.e. my download speed sucks. Measure the download speed. >>>=20 >>> My isp doesn't fix my problem. Measure what? How? >>>=20 >>> Human survey technology is 70+ years old and it still has problems = figuring out how to correlate opinion with fact. >>>=20 >>> Without an objective measurement scheme that doesn't require human = interaction, the misery index is a cool hypothesis with no way to link = to actual data. What objective measurements can be made? Answer that = and the index becomes useful. Otherwise it's just consumer whining. >>>=20 >>> Not trying to be combative here, in fact I like the concept you = support, but I'm hard pressed to see how the concept can lead to data, = and the data lead to policy proposals. >>>=20 >> [SM] So it seems that outside of seemingly simple to test = throughput numbers*, the next most important quality number (or the most = important depending on subjective ranking) is how does latency change = under "load". Absolute latency is also important albeit static high = latency can be worked around within limits so the change under load = seems more relevant.=20 >> All of flent's RRUL test, apple's networkQuality/RPM, and = iperf2's bounceback test offer methods to asses latency change under = load**, as do waveforms bufferbloat tests and even to a degree Ookla's = speedtest.net. IMHO something like latency increase under load or = apple's responsiveness measure RPM (basically the inverse of the latency = under load calculated on a per minute basis, so it scales in the typical = higher numbers are better way, unlike raw latency under load numbers = where smaller is better). >> IMHO what networkQuality is missing ATM is to measure and report = the unloaded RPM as well as the loaded the first gives a measure over = the static latency the second over how well things keep working if = capacity gets tight. They report the base RTT which can be converted to = RPM. As an example: >>=20 >> macbook:~ user$ networkQuality -v >> =3D=3D=3D=3D SUMMARY =3D=3D=3D=3D = =20 >> Upload capacity: 24.341 Mbps >> Download capacity: 91.951 Mbps >> Upload flows: 20 >> Download flows: 16 >> Responsiveness: High (2123 RPM) >> Base RTT: 16 >> Start: 10/23/22, 13:44:39 >> End: 10/23/22, 13:44:53 >> OS Version: Version 12.6 (Build 21G115) >>=20 >> Here RPM 2133 corresponds to 60000/2123 =3D 28.26 ms latency under = load, while the Base RTT of 16ms corresponds to 60000/16 =3D 3750 RPM, = son on this link load reduces the responsiveness by 3750-2123 =3D 1627 = RPM a reduction by 100-100*2123/3750 =3D 43.4%, and that is with = competent AQM and scheduling on the router. >>=20 >> Without competent AQM/shaping I get: >> =3D=3D=3D=3D SUMMARY =3D=3D=3D=3D = =20 >> Upload capacity: 15.101 Mbps >> Download capacity: 97.664 Mbps >> Upload flows: 20 >> Download flows: 12 >> Responsiveness: Medium (427 RPM) >> Base RTT: 16 >> Start: 10/23/22, 13:51:50 >> End: 10/23/22, 13:52:06 >> OS Version: Version 12.6 (Build 21G115) >> latency under load: 60000/427 =3D 140.52 ms=20 >> base RPM: 60000/16 =3D 3750 RPM >> reduction RPM: 100-100*427/3750 =3D 88.6% >>=20 >>=20 >> I understand apple's desire to have a single reported number with a = single qualifier medium/high/... because in the end a link is only = reliably usable if responsiveness under load stays acceptable, but with = two numbers it is easier to see what one's ISP could do to help. (I = guess some ISPs might already be unhappy with the single number, so this = needs some diplomacy/tact) >>=20 >> Regards >> Sebastian >>=20 >>=20 >>=20 >> *) Seemingly as quite some ISPs operate their own speedtest servers = in their network and ignore customers not reaching the contracted rates = into speedtest-servers located in different ASs. As the product is = called internet access I a inclined to expect that my ISP maintains = sufficient peering/transit capacity to reach the next tier of AS at my = contracted rate (the EU legislative seems to agree, see EU directive = 2015/2120). >>=20 >> **) Most do by creating load themselves and measuring throughput at = the same time, bounceback IIUC will focus on the latency measurement and = leave the load generation optional (so offers a mode to measure = responsiveness of a live network with minimal measurement traffic). = @Bob, please correct me if this is wrong. >>=20 >>=20 >>=20 >>>=20 >>> On Fri, Oct 21, 2022, 5:20 PM Dave Taht=20 >>> >>> wrote: >>> One of the best talks I've ever seen on how to measure customer >>> satisfaction properly just went up after the P99 Conference. >>>=20 >>> It's called Misery Metrics. >>>=20 >>> After going through a deep dive as to why and how we think and act = on >>> percentiles, bins, and other statistical methods as to how we use = the >>> web and internet are *so wrong* (well worth watching and thinking >>> about if you are relying on or creating network metrics today), it >>> then points to the real metrics that matter to users and the = ultimate >>> success of an internet business: Timeouts, retries, misses, failed >>> queries, angry phone calls, abandoned shopping carts and loss of >>> engagement. >>>=20 >>>=20 >>> https://www.p99conf.io/session/misery-metrics-consequences/ >>>=20 >>>=20 >>> The ending advice was - don't aim to make a specific percentile >>> acceptable, aim for an acceptable % of misery. >>>=20 >>> I enjoyed the p99 conference more than any conference I've attended = in years. >>>=20 >>> --=20 >>> This song goes out to all the folk that thought Stadia would work: >>>=20 >>> = https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665= 607352320-FXtz >>>=20 >>> Dave T=C3=A4ht CEO, TekLibre, LLC >>>=20 >>> --=20 >>> You received this message because you are subscribed to the Google = Groups "discuss" group. >>> To unsubscribe from this group and stop receiving emails from it, = send an email to=20 >>> discuss+unsubscribe@measurementlab.net >>> . >>> To view this discussion on the web visit=20 >>> = https://groups.google.com/a/measurementlab.net/d/msgid/discuss/CAA93jw4w27= a1EO_QQG7NNkih%2BC3QQde5%3D_7OqGeS9xy9nB6wkg%40mail.gmail.com >>> . >>> _______________________________________________ >>> Rpm mailing list >>>=20 >>> Rpm@lists.bufferbloat.net >>> https://lists.bufferbloat.net/listinfo/rpm >> _______________________________________________ >> Starlink mailing list >>=20 >> Starlink@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/starlink > --=20 > David Collier-Brown, | Always do right. This will gratify > System Programmer and Author | some people and astonish the rest >=20 > dave.collier-brown@indexexchange.com | -- Mark Twain >=20 > CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, = including any and all attachments, contains confidential information = intended only for the person(s) to whom it is addressed. Any = dissemination, distribution, copying or disclosure is strictly = prohibited and is not a waiver of confidentiality. If you have received = this telecommunication in error, please notify the sender immediately by = return electronic mail and delete the message from your inbox and = deleted items folders. This telecommunication does not constitute an = express or implied agreement to conduct transactions by electronic = means, nor does it constitute a contract offer, a contract amendment or = an acceptance of a contract offer. Contract terms contained in this = telecommunication are subject to legal review and the completion of = formal documentation and are not binding until same is confirmed in = writing and has been signed by an authorized signatory. >=20 > _______________________________________________ > Starlink mailing list > Starlink@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/starlink