From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from CAN01-YT3-obe.outbound.protection.outlook.com (mail-yt3can01on2135.outbound.protection.outlook.com [40.107.115.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 253BE3B2A4 for ; Sun, 23 Oct 2022 08:17:38 -0400 (EDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MEVFawv9JfYGXZe3nE63nDjAhjbUYdIbDpGKHhxnRWtAZFXxlqxJ5KGaRUsvRHZngqe/whjqvQKqPX3FQbJtmProxDS/LXCd7ZnSU6cZQHIxW6zfsSjO3IpOWt/UvPj9DvseMGEZ2cMcEdu+gtj0CUBhZ4Zcn1ZBqhiQlOYZ4hlxxxTjutFzNNdxNflIMgmFEzDpNwOmQZNYfpeggpVgBoG63Dzzxf1ox35cdvWXkmmbgffFigk73NTSI4UkslGITvEVCr7J9ObAQWN9pKfJNUU2iMaLvYIkhhkWvMwqfXUS+EqTaSmQprvUKjcSRV0/kbh76JcfhnJZX+iUQAr+lA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2nPnxgv0HFTKhzHCaoRxbyon88hPFcIl1w8T4ierGno=; b=eiflslz8aD9QLcru3iVB6uMo+EsNAYgRnnMe8hofjSXmeVDcMoN6Dq14ns7rp1PC1JgH8UsJxGrSUjZWEzPMVma6Zd5PKmMzM61LnWU/jvmURYXsVrL1+P8lvF6pkJZo8o0g4HEyZQelD/IWtyNhDp2/LHY9LHR1cxmFpb9Nx3cO0v8PrG3qLBgFU+Sw6ay7cpvBpqohnKYsv2KIyDh8VlCDvQuhXB6sboPklFBnxIBHsA0PAPSbSj1xwgS9wzeEJYuTGD2DV3j+Pn2dEv5aeWwE3Y8UdBvVRxZy6vyXUCzyPQdpkBcKvENenrzUwn6mUE6guEx0Q6rC4w72yLMu3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=indexexchange.com; dmarc=pass action=none header.from=indexexchange.com; dkim=pass header.d=indexexchange.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=indexexchange.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2nPnxgv0HFTKhzHCaoRxbyon88hPFcIl1w8T4ierGno=; b=hnofPAvjysfVVPB8/9M/dnFJOacBb3Z1/GHRhi/jalW8aeqlOyzRJdUiRzoopIwO9C4KKLaYtae+ODG0Y0TWVeJD4LiRj6Dxh5GBUJ+csNzmDECOVrDOwtDlwLXO/B/WvOXH9Xu2S61HKp3vD8BCD8QE7V5JTD4HiN0bm9RQBbY= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=indexexchange.com; Received: from YQXPR01MB4756.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c01:1e::12) by YT2PR01MB5789.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:55::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.22; Sun, 23 Oct 2022 12:17:36 +0000 Received: from YQXPR01MB4756.CANPRD01.PROD.OUTLOOK.COM ([fe80::d3d7:5363:e565:8944]) by YQXPR01MB4756.CANPRD01.PROD.OUTLOOK.COM ([fe80::d3d7:5363:e565:8944%9]) with mapi id 15.20.5746.023; Sun, 23 Oct 2022 12:17:36 +0000 Content-Type: multipart/alternative; boundary="------------7RkUp0xG7nFufmdOh8o1UGPU" Message-ID: <0e811787-cc55-8db9-2a4b-7706813769da@indexexchange.com> Date: Sun, 23 Oct 2022 08:17:35 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.2.2 To: starlink@lists.bufferbloat.net References: <339AB8BC-9628-40E2-9339-77FCFA74488D@gmx.de> Content-Language: en-US From: Dave Collier-Brown In-Reply-To: <339AB8BC-9628-40E2-9339-77FCFA74488D@gmx.de> X-ClientProxiedBy: YT3PR01CA0041.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:82::21) To YQXPR01MB4756.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c01:1e::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: YQXPR01MB4756:EE_|YT2PR01MB5789:EE_ X-MS-Office365-Filtering-Correlation-Id: 87050e98-1d8d-4b1b-c302-08dab4f09213 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: XIQctDwkfRSk5m/2ZqX1qQ0diEk7ERgyGGl37QSKHkrj3lP5pvz3Nvb8gTJS5wBXj5TuGAbOhPTcMGQvKCyNIJkVQhbeB9BMV1mNtPGb5wLisYzEViB+wY8+7LMSyoZlge9Z5NXPY4BX9644fyPm5COo9mPesLpHL9uICb5Hcs82xYPJZtcRMw0D5cqX9KdHFZwmO0kRA9w0yFNC9kIexFnergmZ2NaC5O2Zd5zznHW/Cuwvk/AP2Cez2aWCHkqCfJzdGyOKCL7Dh3fLR+FzrhVgyE5FV+6Qd68FYdGHj6GPGNcYrLt1OLs0fZo+GePDZsBryD2rtAPRihnPITwOlI4/fldqIcVywIU7m3AUJzuezXSEz3gqwjTFYq33oTFSXiUs/ryq7icZvQCO4+GLGkWSOotYtsHEGu48+aOB64peiywkMbqhPWDcmZkJVIFJrp9mIiH8Xcv8xDTE+VoXE6f1DHp/yK/+t479psx09yvj10h4rZEdQmbpc2DXuRvtf8vDhL2zkj7P4sj2U5pK/3e49ClG9zM3OnJkT2TmMQGrHBq0aCUUXJJj12Dx4B+iNUOC9OhTJzZHR4iryh0xQbCY9pqXFFtuovchA7l1fTGKIt+jOEySdULaWE/111S42HH4VcQhi+CFqxHxLOa21gt8c8UlKE6qbG5fjZOMpeihZo8fLt8aT7+AREtVXNzYEthjNi/BjUWRjwXTSUNbRQH+nOQr+Om0YTbw8kk7U7JyyXpXnNtBBBZl88Myx8E1I4aQB+C2lbIX1ssJGhnhxAUG+Dd8GOMngudpJi97mol5e+sAhguOVpbuDxeCZyhL X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:YQXPR01MB4756.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(39850400004)(136003)(396003)(346002)(376002)(451199015)(26005)(8936002)(41300700001)(6512007)(6916009)(66946007)(66556008)(66476007)(8676002)(53546011)(6506007)(36756003)(33964004)(5660300002)(166002)(316002)(40140700001)(38100700002)(66574015)(2616005)(30864003)(2906002)(186003)(31696002)(86362001)(83380400001)(966005)(31686004)(6486002)(478600001)(66899015)(45080400002)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bHp6ZU8rVEg0Qko0TlBTVDR0cTA2ZTJLNTRXVU8xeVJTZkwwM0xGNXFxcTJw?= =?utf-8?B?S1JCWTJMS2VhaXpNcC9td3MwbDczckFkTm9kYmtLN0hhOHpYd2NYUzAySGpP?= =?utf-8?B?WjRtTktGQklLWjNHSk1xOFhJQWhDOHlDRll6dEpjRFdrVlY5MkE3V0gyOXl4?= =?utf-8?B?ZXNGSlBqZ0pTZjNhNmU3OEhNaEdRUU8wNUFLOTVMTFVJR2p5L0d2Mll6cDBk?= =?utf-8?B?M25lc3BUZGVVTWJRRExkaW0rWlcxRHEyaEgzdFp4Q0dtbUN3M0J3aDd4VXhn?= =?utf-8?B?Z3VuempOdHhTUWM0emJsaUoxdkgrSW5udU5MVno1eEJkbjJVbG9pWFlhUld1?= =?utf-8?B?UTRsaTcra2l6eUd6RlJkYnZqSXVGaUpOdWZJbFZjKzhKdnN5YVNmVlpzbzhs?= =?utf-8?B?TWdZenFMT2QzTUxlWHlrcGNOUTVjRy90UHFEVDhOeEY4ajAzVi9WeWk2b1VR?= =?utf-8?B?N21BU3pacnRYUnNtSFdhMzdzT2RLYVpQVHNMQ0dzaVd1M2g1bm1nUlQvRHJU?= =?utf-8?B?T3RDOHhCZ01RQjdLMnB1S3J6RTBXMmhwc0VLNHQ0RUhDaUpiOVExUUFRR2pJ?= =?utf-8?B?citKVDNHQVBEK3RiVXNlRmJEdDNPcWlyeVhYczNOTTZJWHhOanRNQlpVTTd4?= =?utf-8?B?UVk0NUhIeTI5RkV1MVJSWmlnOE50andZbHBVOGIyamVpeWFNYlQzT3VxUnlk?= =?utf-8?B?SmF3UkdlTVhZK2M3RWtZa0V5SHhOblFzeEljU0IyM2d1OW5IR25kb2dOVzU0?= =?utf-8?B?ZjVpNEFUYmJPUVZkOHpqaktGVXpQYTBkQ0lac3BERHkzdnlqMmVNTmdvQ0ZT?= =?utf-8?B?OU5PMVFxM3hUVWc0ZGkzdnJ1SEtkTDhUWnNnMXRCRG5oYzVMNzIrTlYzWmQr?= =?utf-8?B?Q00vVmsvVzRtUTBkM3NnU05vVXo2U21mMGdOVjdMT1k0MDRkRENtcE1saEk1?= =?utf-8?B?eStCaVd4Y2tRRm91eldSdnVhREdqNjBpTk1rSUc2a0FITW1odTllc3VWMTFQ?= =?utf-8?B?VEErQTB2MmRHakEwSFFpaTJXaUcxRnR3Tk44M2FoZml6NXpYRmhSZFFuMVhl?= =?utf-8?B?ZDNVUkdZQUtiUkYwWUJDTThERmNoTjN6YzNKUktqSC8wNUVSMVlMZHJpc09Q?= =?utf-8?B?SGVFVjQ4S01KTFNsNC9zcXpVM2ovSjR2ejg1WldTV1FqMVJ5bVJYeWM5NE5v?= =?utf-8?B?U1JUbFJ3SDRTOURHQ0labjhLWExBUzFhZVhhUHFjclhNL2JJUUxXVXpRSGF4?= =?utf-8?B?K1VXMU11Smt6SFIwVitzd2lKajJib0ovWENicURuMUVNTEc4RHlZdi9tVGU5?= =?utf-8?B?K1JsR3dDS1NVVzYxUnFpQ2ZiM1QxQjE1TmVHZGdnK0VYRzNqL01kblZBRWxr?= =?utf-8?B?dkphbERtSDNJdjVqWjNVeEIwSjJLYVZUaUlWN3dtWG91NXVENnFoN2hVeWIx?= =?utf-8?B?STk4SjFZeEVFMWVud1UybUdOc3VZN2NwN3RGSEZWS3hyUnEzVm5ORUlpK0JR?= =?utf-8?B?UTU0MXo0MGEzQWUzbzlXR2ZNRk1CM3k0b0RsUklodHpzMmk5S2Q1S29Vazh0?= =?utf-8?B?cU1kME1aWGRBV1ZPNCtneUY5cllqdHpmSXVzNEdtRWNyTEp5endhdEEyU1Jr?= =?utf-8?B?UmRFN29BZnJlTFVNMGNYcnJTNHNvQTdnUUtqMDhyMVgxYTZTTVdXRy93NEJ4?= =?utf-8?B?K3NFNHMwemRoMU56cEJFTXNkVUFiQlZXU3RHMThNV2xjaVNqTW9JY3JyUEhY?= =?utf-8?B?bm5Nd29rWFlHaFdrQW1SNk5VVXQ2emZJYTdwelBabmRyM2xVeXlOZm9nWE1P?= =?utf-8?B?OExzL2pzazNJK2Vjb2FURjhDa3VMc01oZTA3elFWWm00MG40L25EdnV5RFRM?= =?utf-8?B?MTdDWGZiYjRLTUdPRk1pVzJML1dRT3BXZFIrRms4M1RLR2UxbmJUK3lZRTZ0?= =?utf-8?B?U0xES2t0dlA4MDZhQWhGZUJSRnRWYTU1ckR1MDZmWWdqWGNOUkl2Tnpnaitv?= =?utf-8?B?aFVHK2NxRkw1S3lneEtqR1BxdFBBWkg2Y1oxd2NWM3VJODJBM1piUnl2TVA5?= =?utf-8?B?L3RoaVlGUlY3SmNjbzdjeEVXRDJQYW11N2MyT2h1MHd4MmptZjVRNUJxNGxk?= =?utf-8?B?bllUdEZIeW91Uytwdy9pb3QxZXR5bHQzTi9aaG81LzJrb1J2UGJEa21SV0FG?= =?utf-8?Q?FecQ+sPVRybqphD0DA1SjE0=3D?= X-OriginatorOrg: indexexchange.com X-MS-Exchange-CrossTenant-Network-Message-Id: 87050e98-1d8d-4b1b-c302-08dab4f09213 X-MS-Exchange-CrossTenant-AuthSource: YQXPR01MB4756.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2022 12:17:36.3515 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b07c0690-22b8-4366-8d8d-7b845d088e18 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ROvYJUdJ0HyPq4LT1vH9/Jrw/+Q2ATc9+SCVrbMJAHEb9VpCBLHE3LLF03TQibjO5lSVspit9z8GaqnQUD2UsXTitKnUh5VV09T4K/o1iJY7z0Agb9WSWXCAGngSpZBL X-MS-Exchange-Transport-CrossTenantHeadersStamped: YT2PR01MB5789 Subject: Re: [Starlink] [Rpm] [M-Lab-Discuss] misery metrics & consequences X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Oct 2022 12:17:38 -0000 --------------7RkUp0xG7nFufmdOh8o1UGPU Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: quoted-printable If our business-transaction customers are made miserable by timeouts, by an= alogy it follows that home internet users are made miserable by * stalls, "buffering" and complete disappearance in conference-calls * "shouting down the well" distortion in any kind of audio Disappearance is probably disconnection, and easy to measure The others are delay-related, and can be computed from timestampe and seque= nce numbers. Can we provide a tool to expose the latter? A "miserometer"? --dave On 10/23/22 07:57, Sebastian Moeller via Starlink wrote: Hi Glenn, On Oct 23, 2022, at 02:17, Glenn Fishbine via Rpm wrote: As a classic died in the wool empiricist, granted that you can identify "mi= sery" factors, given a population of 1,000 users, how do you propose derivi= ng a misery index for that population? We can measure download, upload, ping, jitter pretty much without user inte= rvention. For the measurements you hypothesize, how you you automatically = extract those indecies without subjective user contamination. I.e. my download speed sucks. Measure the download speed. My isp doesn't fix my problem. Measure what? How? Human survey technology is 70+ years old and it still has problems figuring= out how to correlate opinion with fact. Without an objective measurement scheme that doesn't require human interact= ion, the misery index is a cool hypothesis with no way to link to actual da= ta. What objective measurements can be made? Answer that and the index be= comes useful. Otherwise it's just consumer whining. Not trying to be combative here, in fact I like the concept you support, bu= t I'm hard pressed to see how the concept can lead to data, and the data le= ad to policy proposals. [SM] So it seems that outside of seemingly simple to test throughpu= t numbers*, the next most important quality number (or the most important d= epending on subjective ranking) is how does latency change under "load". Ab= solute latency is also important albeit static high latency can be worked a= round within limits so the change under load seems more relevant. All of flent's RRUL test, apple's networkQuality/RPM, and iperf2's = bounceback test offer methods to asses latency change under load**, as do w= aveforms bufferbloat tests and even to a degree Ookla's speedtest.net. IMHO= something like latency increase under load or apple's responsiveness measu= re RPM (basically the inverse of the latency under load calculated on a per= minute basis, so it scales in the typical higher numbers are better way, u= nlike raw latency under load numbers where smaller is better). IMHO what networkQuality is missing ATM is to measure and report th= e unloaded RPM as well as the loaded the first gives a measure over the sta= tic latency the second over how well things keep working if capacity gets t= ight. They report the base RTT which can be converted to RPM. As an example= : macbook:~ user$ networkQuality -v =3D=3D=3D=3D SUMMARY =3D=3D=3D=3D Upload capacity: 24.341 Mbps Download capacity: 91.951 Mbps Upload flows: 20 Download flows: 16 Responsiveness: High (2123 RPM) Base RTT: 16 Start: 10/23/22, 13:44:39 End: 10/23/22, 13:44:53 OS Version: Version 12.6 (Build 21G115) Here RPM 2133 corresponds to 60000/2123 =3D 28.26 ms latency under load, wh= ile the Base RTT of 16ms corresponds to 60000/16 =3D 3750 RPM, son on this = link load reduces the responsiveness by 3750-2123 =3D 1627 RPM a reduction = by 100-100*2123/3750 =3D 43.4%, and that is with competent AQM and scheduli= ng on the router. Without competent AQM/shaping I get: =3D=3D=3D=3D SUMMARY =3D=3D=3D=3D Upload capacity: 15.101 Mbps Download capacity: 97.664 Mbps Upload flows: 20 Download flows: 12 Responsiveness: Medium (427 RPM) Base RTT: 16 Start: 10/23/22, 13:51:50 End: 10/23/22, 13:52:06 OS Version: Version 12.6 (Build 21G115) latency under load: 60000/427 =3D 140.52 ms base RPM: 60000/16 =3D 3750 RPM reduction RPM: 100-100*427/3750 =3D 88.6% I understand apple's desire to have a single reported number with a single = qualifier medium/high/... because in the end a link is only reliably usable= if responsiveness under load stays acceptable, but with two numbers it is = easier to see what one's ISP could do to help. (I guess some ISPs might alr= eady be unhappy with the single number, so this needs some diplomacy/tact) Regards Sebastian *) Seemingly as quite some ISPs operate their own speedtest servers in thei= r network and ignore customers not reaching the contracted rates into speed= test-servers located in different ASs. As the product is called internet ac= cess I a inclined to expect that my ISP maintains sufficient peering/transi= t capacity to reach the next tier of AS at my contracted rate (the EU legis= lative seems to agree, see EU directive 2015/2120). **) Most do by creating load themselves and measuring throughput at the sam= e time, bounceback IIUC will focus on the latency measurement and leave the= load generation optional (so offers a mode to measure responsiveness of a = live network with minimal measurement traffic). @Bob, please correct me if = this is wrong. On Fri, Oct 21, 2022, 5:20 PM Dave Taht wrote: One of the best talks I've ever seen on how to measure customer satisfaction properly just went up after the P99 Conference. It's called Misery Metrics. After going through a deep dive as to why and how we think and act on percentiles, bins, and other statistical methods as to how we use the web and internet are *so wrong* (well worth watching and thinking about if you are relying on or creating network metrics today), it then points to the real metrics that matter to users and the ultimate success of an internet business: Timeouts, retries, misses, failed queries, angry phone calls, abandoned shopping carts and loss of engagement. https://www.p99conf.io/session/misery-metrics-consequences/ The ending advice was - don't aim to make a specific percentile acceptable, aim for an acceptable % of misery. I enjoyed the p99 conference more than any conference I've attended in year= s. -- This song goes out to all the folk that thought Stadia would work: https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-69813666656= 07352320-FXtz Dave T=C3=A4ht CEO, TekLibre, LLC -- You received this message because you are subscribed to the Google Groups "= discuss" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to discuss+unsubscribe@measurementlab.net. To view this discussion on the web visit https://groups.google.com/a/measur= ementlab.net/d/msgid/discuss/CAA93jw4w27a1EO_QQG7NNkih%2BC3QQde5%3D_7OqGeS9= xy9nB6wkg%40mail.gmail.com. _______________________________________________ Rpm mailing list Rpm@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/rpm _______________________________________________ Starlink mailing list Starlink@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/starlink -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest dave.collier-brown@indexexchange.com | -- Mark Twain CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, including a= ny and all attachments, contains confidential information intended only for= the person(s) to whom it is addressed. Any dissemination, distribution, co= pying or disclosure is strictly prohibited and is not a waiver of confident= iality. If you have received this telecommunication in error, please notify= the sender immediately by return electronic mail and delete the message fr= om your inbox and deleted items folders. This telecommunication does not co= nstitute an express or implied agreement to conduct transactions by electro= nic means, nor does it constitute a contract offer, a contract amendment or= an acceptance of a contract offer. Contract terms contained in this teleco= mmunication are subject to legal review and the completion of formal docume= ntation and are not binding until same is confirmed in writing and has been= signed by an authorized signatory. --------------7RkUp0xG7nFufmdOh8o1UGPU Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

If our business-transaction customers are made miserable by timeouts, by= analogy it follows that home internet users are made miserable by

  • stalls, "buffering" and complete disappearance in conference-= calls
  • "shouting down the well" distortion in any kind of = audio

Disappearance is probably disconnection, and easy to measure

The others are delay-related, and can be computed from timestampe and se= quence numbers.

Can we provide a tool to expose the latter? A "miserometer"?

--dave


On 10/23/22 07:57, Sebastian Moeller via Sta= rlink wrote:
Hi Glenn,



On Oct 23, 2022, at 02:17, Glenn Fis=
hbine via Rpm <rpm@lists.bufferbloat.net> wrote:

As a classic died in the wool empiricist, granted that you can identify &qu=
ot;misery" factors, given a population of 1,000 users, how do you prop=
ose deriving a misery index for that population?

We can measure download, upload, ping, jitter pretty much without user inte=
rvention.  For the measurements you hypothesize, how you you automatically =
extract those indecies without subjective user contamination.

I.e.  my download speed sucks. Measure the download speed.

My isp doesn't fix my problem. Measure what? How?

Human survey technology is 70+ years old and it still has problems figuring=
 out how to correlate opinion with fact.

Without an objective measurement scheme that doesn't require human interact=
ion, the misery index is a cool hypothesis with no way to link to actual da=
ta.  What objective measurements can be made?  Answer that and the index be=
comes useful. Otherwise it's just consumer whining.

Not trying to be combative here, in fact I like the concept you support, bu=
t I'm hard pressed to see how the concept can lead to data, and the data le=
ad to policy proposals.

	[SM] So it seems that outside of seemingly simple to test throughput numbe=
rs*, the next most important quality number (or the most important dependin=
g on subjective ranking) is how does latency change under "load".=
 Absolute latency is also important albeit static high latency can be worke=
d around within limits so the change under load seems more relevant.=20
	All of flent's RRUL test, apple's networkQuality/RPM, and iperf2's bounceb=
ack test offer methods to asses latency change under load**, as do waveform=
s bufferbloat tests and even to a degree Ookla's speedtest.net. IMHO someth=
ing like latency increase under load or apple's responsiveness measure RPM =
(basically the inverse of the latency under load calculated on a per minute=
 basis, so it scales in the typical higher numbers are better way, unlike r=
aw latency under load numbers where smaller is better).
	IMHO what networkQuality is missing ATM is to measure and report the unloa=
ded RPM as well as the loaded the first gives a measure over the static lat=
ency the second over how well things keep working if capacity gets tight. T=
hey report the base RTT which can be converted to RPM. As an example:

macbook:~ user$ networkQuality -v
=3D=3D=3D=3D SUMMARY =3D=3D=3D=3D                                          =
                                              =20
Upload capacity: 24.341 Mbps
Download capacity: 91.951 Mbps
Upload flows: 20
Download flows: 16
Responsiveness: High (2123 RPM)
Base RTT: 16
Start: 10/23/22, 13:44:39
End: 10/23/22, 13:44:53
OS Version: Version 12.6 (Build 21G115)

Here RPM 2133 corresponds to 60000/2123 =3D 28.26 ms latency under load, wh=
ile the Base RTT of 16ms corresponds to 60000/16 =3D 3750 RPM, son on this =
link load reduces the responsiveness by 3750-2123 =3D 1627 RPM a reduction =
by 100-100*2123/3750 =3D 43.4%, and that is with competent AQM and scheduli=
ng on the router.

Without competent AQM/shaping I get:
=3D=3D=3D=3D SUMMARY =3D=3D=3D=3D                                          =
                                              =20
Upload capacity: 15.101 Mbps
Download capacity: 97.664 Mbps
Upload flows: 20
Download flows: 12
Responsiveness: Medium (427 RPM)
Base RTT: 16
Start: 10/23/22, 13:51:50
End: 10/23/22, 13:52:06
OS Version: Version 12.6 (Build 21G115)
latency under load: 60000/427 =3D 140.52 ms=20
base RPM: 60000/16 =3D 3750 RPM
reduction RPM: 100-100*427/3750 =3D 88.6%


I understand apple's desire to have a single reported number with a single =
qualifier medium/high/... because in the end a link is only reliably usable=
 if responsiveness under load stays acceptable, but with two numbers it is =
easier to see what one's ISP could do to help. (I guess some ISPs might alr=
eady be unhappy with the single number, so this needs some diplomacy/tact)

Regards
	Sebastian



*) Seemingly as quite some ISPs operate their own speedtest servers in thei=
r network and ignore customers not reaching the contracted rates into speed=
test-servers located in different ASs. As the product is called internet ac=
cess I a inclined to expect that my ISP maintains sufficient peering/transi=
t capacity to reach the next tier of AS at my contracted rate (the EU legis=
lative seems to agree, see EU directive 2015/2120).

**) Most do by creating load themselves and measuring throughput at the sam=
e time, bounceback IIUC will focus on the latency measurement and leave the=
 load generation optional (so offers a mode to measure responsiveness of a =
live network with minimal measurement traffic). @Bob, please correct me if =
this is wrong.




On Fri, Oct 21, 2022, 5:20 PM Dave Taht <dave.taht@gmail.com> wrote:
One of the best talks I've ever seen on how to measure customer
satisfaction properly just went up after the P99 Conference.

It's called Misery Metrics.

After going through a deep dive as to why and how we think and act on
percentiles, bins, and other statistical methods as to how we use the
web and internet are *so wrong* (well worth watching and thinking
about if you are relying on or creating network metrics today), it
then points to the real metrics that matter to users and the ultimate
success of an internet business: Timeouts, retries, misses, failed
queries, angry phone calls, abandoned shopping carts and loss of
engagement.

https://www.p99conf.io/session/misery-metrics-=
consequences/

The ending advice was - don't aim to make a specific percentile
acceptable, aim for an acceptable % of misery.

I enjoyed the p99 conference more than any conference I've attended in year=
s.

--=20
This song goes out to all the folk that thought Stadia would work:
https://www.linke=
din.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave T=C3=A4ht CEO, TekLibre, LLC

--=20
You received this message because you are subscribed to the Google Groups &=
quot;discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to discuss+unsubscribe@measurementlab.net.
To view this discussion on the web visit http=
s://groups.google.com/a/measurementlab.net/d/msgid/discuss/CAA93jw4w27a1EO_=
QQG7NNkih%2BC3QQde5%3D_7OqGeS9xy9nB6wkg%40mail.gmail.com.
_______________________________________________
Rpm mailing list
Rpm@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/rpm

_______________________________________________
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink

--=20
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
dave.collier-brown@indexexchange.com |              -- =
Mark Twain

CONFIDENTIALITY NOTICE AND D= ISCLAIMER : T= his telecommunication, including any and all attachments, contains confiden= tial information intended only for the person(s) to whom it is addressed. Any dissemination, distribution, copying or discl= osure is strictly prohibited and is not a waiver of confidentiality. If you= have received this telecommunication in error, please notify the sender im= mediately by return electronic mail and delete the message from your inbox and deleted items folders. This tel= ecommunication does not constitute an express or implied agreement to condu= ct transactions by electronic means, nor does it constitute a contract offe= r, a contract amendment or an acceptance of a contract offer. Contract terms contained in this telecommunication ar= e subject to legal review and the completion of formal documentation and ar= e not binding until same is confirmed in writing and has been signed by an = authorized signatory.

--------------7RkUp0xG7nFufmdOh8o1UGPU--