From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from CAN01-YT3-obe.outbound.protection.outlook.com (mail-yt3can01on2125.outbound.protection.outlook.com [40.107.115.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 9B9303B29E for ; Mon, 24 Oct 2022 13:37:59 -0400 (EDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=e7DnQwf9uDn75vk8FVQvPGiGccP0vgH3jnmXvqaeEuCajGsQjf/0SGvi3xk266cqta9OiFD6SyBtCs4F5ST89KZOmUTBCz3QVEDoGMxeN+TLeHpn6sb9LFtk0Ge37+BlMQwIG2MukrpcTqgxjkpXIbl7Y0NHAi/ECJr05Xj5fL2ZAfJvs5ORIPqB5ijSBuSe+8sCsuSC/3tanaiod6m+n6u+5kUOkq2462+23OUIUVIFXbygO/qsx094hJQBgF0DhJB3y28FSMHo1ieULRogJbB83IkY9iFifHO8Q3JXI9b/qkfgHze75W6c8T5uNNJWi/V7zqr7QBEqOMuWMoMkMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=83uLkEaYSsmnEdu+ADUC6bfeisZ4ToXWFapkjIFoxgI=; b=Maz6TTOhxVcclgdUjZKoJLlIJ6T+vMporY8Gl9f7M+oLFFGjOWNEChKJBwMVoY+Gp6omK8oFDGkDLfKEyIPMzgTAhga+ZqDRw+gUcnKcsBIOY3Hmj3EaOEHle+/NXNmpqukEQMurnIj6Ejp4xptvbvllSE0VEyvw1FnObdhww2daH1DwZ8K+RhGNJkFJGMoa5SqaF8nZDIXsxAhJc06lAPHgAmCrvJbonV/B8vd+14HBrgtjQvfwd1/+kcURCQTAtn6s6Mwz+KYs1grIqxRAYWkmg0Qc0V3w51Teuadnr2FnHXk9Ce1VGI6OupZwzT2Yxpp/JoCV+l04pxHzBpYpmA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=indexexchange.com; dmarc=pass action=none header.from=indexexchange.com; dkim=pass header.d=indexexchange.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=indexexchange.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=83uLkEaYSsmnEdu+ADUC6bfeisZ4ToXWFapkjIFoxgI=; b=ONJr1H3nvahhop8aq+jhILVT17o7llAuVVsPK/JxJ3JtiHiU1tDmDls5kJHMj4AoSTr5evRCXQ8zmVTWR6OsNlm8W52ycIaNh6xTEv493vOkqOHPS3bJcOMmqeDVQEgFhoe5zsu12GeJJ/LoYCGCS13QKzauPC7XoqT4BuQLY5A= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=indexexchange.com; Received: from YQXPR01MB4756.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c01:1e::12) by YQBPR0101MB8559.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c01:56::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.35; Mon, 24 Oct 2022 17:37:57 +0000 Received: from YQXPR01MB4756.CANPRD01.PROD.OUTLOOK.COM ([fe80::d3d7:5363:e565:8944]) by YQXPR01MB4756.CANPRD01.PROD.OUTLOOK.COM ([fe80::d3d7:5363:e565:8944%9]) with mapi id 15.20.5746.026; Mon, 24 Oct 2022 17:37:57 +0000 Content-Type: multipart/alternative; boundary="------------lU9ijHc7INm5PtsmfxAYE3De" Message-ID: Date: Mon, 24 Oct 2022 13:37:55 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.2.2 Content-Language: en-US To: starlink@lists.bufferbloat.net References: From: Dave Collier-Brown In-Reply-To: X-ClientProxiedBy: YT3PR01CA0041.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:82::21) To YQXPR01MB4756.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c01:1e::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: YQXPR01MB4756:EE_|YQBPR0101MB8559:EE_ X-MS-Office365-Filtering-Correlation-Id: 909fea5e-4bea-4899-4d83-08dab5e67d13 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1B450HXtiE5IM/iPnN/dUnqGxNoCaen+9FJ+UNuDpGexU6CKYV/+Qqg+NUGqpyg4GUR/steTSy9QB0EB6o++xsHkYpgf5j7FQXPBYPfU/oJYeY/rJFFPkKJGvKeQTCuPYwi9Ohy9godjyWfwzcR0XAwaDYAIDxSTMFzGE0+tVpJbl1yvUjMee+rpCgDbLeN83uH/qkGNe5sHr0VffJmtzx96eQusbHCYDTzjcBNkP//jVQ7K/irMXG0iSWBFhJ8Z3ZQisQ6PyKWiPwvUvCxhDi583z2VSTFXiD6FBbLgAVTUtp2aJ13e/HO8Q4Yt3snIc4bKmzMalDyveAA8a9zpt3WvDZhTrEs2MEf7Xp0zwF1rTZ2Ye7L13wvPz+uX6uQORYjxP5rxFLU032Fd3HWuTw6cuHv3las5ynE3PafUsobejQ+wbW6oA1IQY0xW28yFLAWRHtjqUDxtZ+thzM7XZgCtxTI776qjGkafnrwFulkVHULTdTViXeUt1+jIIIrbh/CboYkbn478Z7N4FU309WcfrPd+mQWsJ7llbefYuWP/OSimyJjtoEMiCZwg15rsIo/1mnxLaF7wth+lslsm8LTTHk56oEe7KDxRk3fECs8y7xJa4sj6uinCh1uCeW6eZ2rcaYYtah2Bzb8Sgi2TLZw576iX90lGvUsY1VkBFI2+hqrhGBx7Vu1PrbKPPfCPWVzv+wNPyHyT7iH7grQcr+hXRKCcAmZ7J9aEGmnyTF+r4BlQOcqGHutU1bo0iqOrvPj949w4YJNY0VIG1X12srsBTJPIqU/TKWUzfYG9U8Ihz/rSg4qiL4Wj6pYAkrNV X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:YQXPR01MB4756.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(13230022)(4636009)(346002)(376002)(136003)(366004)(39850400004)(396003)(451199015)(26005)(83380400001)(53546011)(6506007)(6512007)(186003)(33964004)(2616005)(5660300002)(6486002)(316002)(6916009)(966005)(8676002)(41300700001)(66946007)(478600001)(66476007)(66574015)(2906002)(45080400002)(8936002)(66556008)(66899015)(31696002)(36756003)(86362001)(38100700002)(31686004)(166002)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?YjlubTZqOUVta213OHM5NlltNjZHVnJmNmZTL3I1V25UWVZSdFdhRUwvTjB2?= =?utf-8?B?b2NyU09XZ3R1ZHBZQm5weGFYcFEyS2xjbXN5ZC9hb0RTUnNITmQyV3dybG93?= =?utf-8?B?OW45TUwwSk5udnd2aEdPZ2tpVm1WY1Zrby9GM0JGMU5hZnd3QXJJcmNMcWRj?= =?utf-8?B?NUErdkhGSUhLN1kxRlh2TVRua1hnZmlNUERneDlwK0I1cEVITjVkNnM0L2VZ?= =?utf-8?B?NHBkVE5EaHJudTl4SnhUSUZ0Yk9oOXNWSGlDU2w2QlZvUXY1eldZdDdXQ0hr?= =?utf-8?B?M0svMVdEbG5aV3J5WXNtRGVsMGZLWmJYUW93MzFRR3NzWlZub0J5K01xV05s?= =?utf-8?B?RkFNWG5kVlF2cGkrRWIvV3cyZlEvMEZlZytzVzdPeFlrYU1qR0x2dSt1SmJh?= =?utf-8?B?aFk3UWw0UXpzZFpGK3RXM0ZneGUxMzJxdFFNRVVrWDVJQit5VFZXcXdtRjRF?= =?utf-8?B?SEtPVEhDQzd1SjBIaFM4MTNycDdVYjg5OGsvak13MzJlZ29qTlRMRU5QZUlS?= =?utf-8?B?eG9XcG51TmVVWktEbU01bE5FWVlUbm55VmhQd1J2TWVHaWxtVncrZlo0akxj?= =?utf-8?B?UHJzM3BYcENNRWwzTlVvUlhnV1JYbmF4T3F0WlVWUlZleVR1anozN29TMyth?= =?utf-8?B?SEpCeVNxaXF5TVhzSnJRSDZ5eG1uUGdlVzZyQWlWNGhORXlFVElWVjlMNTJW?= =?utf-8?B?c1JJcW1xc1RIOFQ5MVZxNTBqcEphV1E4SFl2eXBxSGRoNDdLbHQvSE9obmk0?= =?utf-8?B?cjROY2ZFMzRtaGw0TERNaUxPU3dXT2h5d1BpUmVHUytNNjlIQTRFa3dNbS9z?= =?utf-8?B?MW1WbkVNOE0zRmJmZ0RrdGtreENXZUhPb3JIV0Nua3g1R1VxZzhDb2psQjha?= =?utf-8?B?WUxqSUY5VUp4UlU0ajJGcW1hMVppalBXT2liSk5QSW5kZjZNWDhFNklWY2N3?= =?utf-8?B?MXF5OHdCSHJpUXprYUtIdk03eUpHWURZTEdzOXFSRHRFem91OXpRWFZmVWEr?= =?utf-8?B?VmovN3RNYnViRTE0K0xVMEYrd291eGNFOG9tU2R4RENJVkNLMUowWVJaOG9P?= =?utf-8?B?aVpIcnVpQVRzWGtRQzd4WjM2TkJmZ3FIOU1RK1hCN09xb2JiZXk0T2pkWWQv?= =?utf-8?B?T3NKU2tJTTBjSDFnSlVucUt5eEVjL1M4am90RkUxeXJxWmsxSXJzTExHYjBY?= =?utf-8?B?TDNlTnRBaS81dW9Cd0RNa0NNeXRxMFJXajVyajF1ejd6M3hwUkdRWXAxckFq?= =?utf-8?B?Z3FGckJjVVdhR2xVckpLMnltaWt3QmcxUTB5L1hvRUtLRTR0WnpiZ2QzczNY?= =?utf-8?B?YUJNZkxZdWNFNG5EbER5eVNMQ0EyOGVxVkM2UHlXSldhS0lNMjlGZ2FCclEv?= =?utf-8?B?Rmk2cTFpd1pFb1Z0MjgzTkM0ODFUbFpzdFVJb1FRZXlrQ2hjUjFjVVVhRldG?= =?utf-8?B?ZEgzWHFEVE5UTHkzanI4bDRaQUVITlExTE0ydHA3NXRTVVBCWnVMUWQvaU42?= =?utf-8?B?SjRnUzZ1SXB2T3J5UzN3b0lYMnJoQ2VBenBtU3Rxd1ZaUGdEczdPWkVDZS91?= =?utf-8?B?T3QvbkZXeGF2cnNZQTlmbzQvL3ZWOVNjRHZyT0llbmZSaWdoTVpya3NBdVVq?= =?utf-8?B?ODk2clZ5VHhBcTR2SXgwMGtzTWErKzlreHR5RjFVM3VuRDRNTTJkUnRWNXRT?= =?utf-8?B?ZFN4RGJTRittSU9nMDd0dEdvdUs3MWZwbENMdXZmNmpBYTdTbVdtTVVPTldv?= =?utf-8?B?OCttRG9pd2FBblZzZG5TUXlTeTZYRlJCVzlZNVZuM3g5QkRIamQwTUNWck1l?= =?utf-8?B?ZVp5Q3VkUjdJNnZ2MFUvWC9LVG1lNGpMWDd1Yk1zMTgwS2tlZXpJOTZLRWVV?= =?utf-8?B?TXk0QlVPWENvYUxhM05TYitvdjV0dkFFY1VqWnQxWi9uemxPaU9uUlpPZHZW?= =?utf-8?B?SE53WThPaSsvL0lVcFkxSU9BMS9TcG85V0lDbUkxZzI1YzMrODkxK2tHMGVp?= =?utf-8?B?UVFNd29EaWlqSjMzYTVxM2s1Nk5tTFo4ZTFqOHFoUWIvQUJDLzk2TzNkVWpq?= =?utf-8?B?SVlZenlPZUQ0cWpJTWdHYXVtaHVOT3VqTndoVUI5K2FESXFKK0greWEyZUdY?= =?utf-8?B?SVdrc09HNDdlUU9mVXhLOGVnbHErMjIwcUczUllOZ0ZNNGp1ZWFQNzhJOEhI?= =?utf-8?Q?srqhOTIUyZrZg0HO1metg8g=3D?= X-OriginatorOrg: indexexchange.com X-MS-Exchange-CrossTenant-Network-Message-Id: 909fea5e-4bea-4899-4d83-08dab5e67d13 X-MS-Exchange-CrossTenant-AuthSource: YQXPR01MB4756.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Oct 2022 17:37:57.3118 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b07c0690-22b8-4366-8d8d-7b845d088e18 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6IsZXyPPcjG+xgzjSIxh2RyqqhDfXDm3ghuDEIZ7LA8GhjNjTiYe/JDZnAmg5sdCP6jTovogFJJ5HTJ6XvC8rNcQOpjl0hunDTvITYE1DVpTa1lR39ncPfPRNIfi3T5N X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB8559 Subject: Re: [Starlink] [M-Lab-Discuss] misery metrics & consequences X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Oct 2022 17:37:59 -0000 --------------lU9ijHc7INm5PtsmfxAYE3De Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: quoted-printable I read the talk as saying we should measure and report the failures rather = than the successes. Let's say download speed is the problem, and I'm Rogers, an ISP with lots = of subnets. For each I plot the number of downloads per net per hour which= were slower than X kb/s, not counting ones whose total time were less than= 2 seconds. That gives you a measurement that fairly closely models the degree of human= annoyance you're causing (;-)) And Rogers causes a lot, but it's usually on concalls, not downloads. --dave [I might use a poll to see if a claimed huge improvement worked, but I don'= t trust human estimations any more that you do (:-)] On 10/22/22 20:17, Glenn Fishbine via Starlink wrote: As a classic died in the wool empiricist, granted that you can identify "mi= sery" factors, given a population of 1,000 users, how do you propose derivi= ng a misery index for that population? We can measure download, upload, ping, jitter pretty much without user inte= rvention. For the measurements you hypothesize, how you you automatically = extract those indecies without subjective user contamination. I.e. my download speed sucks. Measure the download speed. My isp doesn't fix my problem. Measure what? How? Human survey technology is 70+ years old and it still has problems figuring= out how to correlate opinion with fact. Without an objective measurement scheme that doesn't require human interact= ion, the misery index is a cool hypothesis with no way to link to actual da= ta. What objective measurements can be made? Answer that and the index be= comes useful. Otherwise it's just consumer whining. Not trying to be combative here, in fact I like the concept you support, bu= t I'm hard pressed to see how the concept can lead to data, and the data le= ad to policy proposals. On Fri, Oct 21, 2022, 5:20 PM Dave Taht > wrote: One of the best talks I've ever seen on how to measure customer satisfaction properly just went up after the P99 Conference. It's called Misery Metrics. After going through a deep dive as to why and how we think and act on percentiles, bins, and other statistical methods as to how we use the web and internet are *so wrong* (well worth watching and thinking about if you are relying on or creating network metrics today), it then points to the real metrics that matter to users and the ultimate success of an internet business: Timeouts, retries, misses, failed queries, angry phone calls, abandoned shopping carts and loss of engagement. https://www.p99conf.io/session/misery-metrics-consequences/ The ending advice was - don't aim to make a specific percentile acceptable, aim for an acceptable % of misery. I enjoyed the p99 conference more than any conference I've attended in year= s. -- This song goes out to all the folk that thought Stadia would work: https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-69813666656= 07352320-FXtz Dave T=C3=A4ht CEO, TekLibre, LLC -- You received this message because you are subscribed to the Google Groups "= discuss" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to discuss+unsubscribe@measurementlab.net. To view this discussion on the web visit https://groups.google.com/a/measur= ementlab.net/d/msgid/discuss/CAA93jw4w27a1EO_QQG7NNkih%2BC3QQde5%3D_7OqGeS9= xy9nB6wkg%40mail.gmail.com. _______________________________________________ Starlink mailing list Starlink@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/starlink -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest dave.collier-brown@indexexchange.com | -- Mark Twain CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, including a= ny and all attachments, contains confidential information intended only for= the person(s) to whom it is addressed. Any dissemination, distribution, co= pying or disclosure is strictly prohibited and is not a waiver of confident= iality. If you have received this telecommunication in error, please notify= the sender immediately by return electronic mail and delete the message fr= om your inbox and deleted items folders. This telecommunication does not co= nstitute an express or implied agreement to conduct transactions by electro= nic means, nor does it constitute a contract offer, a contract amendment or= an acceptance of a contract offer. Contract terms contained in this teleco= mmunication are subject to legal review and the completion of formal docume= ntation and are not binding until same is confirmed in writing and has been= signed by an authorized signatory. --------------lU9ijHc7INm5PtsmfxAYE3De Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

I read the talk as saying we should measure and report the failures r= ather than the successes.

Let's say download speed is the problem,  and I'm Rogers, an ISP wi= th lots of subnets.  For each I plot the number of downloads per net p= er hour which were slower than X kb/s, not counting ones whose total time w= ere less than 2 seconds. 

That gives you a measurement that fairly closely models the degree of hu= man annoyance you're causing (;-))

And Rogers causes a lot, but it's usually on concalls, not downloads.

--dave

[I might use a poll to see if a claimed huge improvement worked, but I d= on't trust human estimations any more that you do (:-)]


On 10/22/22 20:17, Glenn Fishbine via Starli= nk wrote:
As a classic died in the wool empiricist, granted that yo= u can identify "misery" factors, given a population of 1,000 user= s, how do you propose deriving a misery index for that population?

We can measure download, upload, ping, jitter pretty much= without user intervention.  For the measurements you hypothesize, how= you you automatically extract those indecies without subjective user conta= mination.

I.e.  my download speed sucks. Measure the download = speed.

My isp doesn't fix my problem. Measure what? How?

Human survey technology is 70+ years old and it still has= problems figuring out how to correlate opinion with fact.

Without an objective measurement scheme that doesn't requ= ire human interaction, the misery index is a cool hypothesis with no way to= link to actual data.  What objective measurements can be made?  = Answer that and the index becomes useful. Otherwise it's just consumer whining.

Not trying to be combative here, in fact I like the conce= pt you support, but I'm hard pressed to see how the concept can lead to dat= a, and the data lead to policy proposals.


On Fri, Oct 21, 2022, 5:20 PM Dave Ta= ht <dave.taht@gmail.com> wrote:
One of the best talks I've ever seen on how to measure customer
satisfaction properly just went up after the P99 Conference.

It's called Misery Metrics.

After going through a deep dive as to why and how we think and act on
percentiles, bins, and other statistical methods as to how we use the
web and internet are *so wrong* (well worth watching and thinking
about if you are relying on or creating network metrics today), it
then points to the real metrics that matter to users and the ultimate
success of an internet business: Timeouts, retries, misses, failed
queries, angry phone calls, abandoned shopping carts and loss of
engagement.

https://www.p99conf.io/session/misery-metrics-co= nsequences/

The ending advice was - don't aim to make a specific percentile
acceptable, aim for an acceptable % of misery.

I enjoyed the p99 conference more than any conference I've attended in year= s.

--
This song goes out to all the folk that thought Stadia would work:
https://www.linkedi= n.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz Dave T=C3=A4ht CEO, TekLibre, LLC

--
You received this message because you are subscribed to the Google Groups &= quot;discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to discuss+unsubscribe@measurementlab.net.
To view this discussion on the web visit https://groups.google.com/a/measurementlab.net/d/msgid/discuss/CAA93jw4w27a= 1EO_QQG7NNkih%2BC3QQde5%3D_7OqGeS9xy9nB6wkg%40mail.gmail.com.

____________________________________=
___________
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink

--=20
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
dave.collier-brown@indexexchange.com |              -- =
Mark Twain

CONFIDENTIALITY NOTICE AND D= ISCLAIMER : T= his telecommunication, including any and all attachments, contains confiden= tial information intended only for the person(s) to whom it is addressed. Any dissemination, distribution, copying or discl= osure is strictly prohibited and is not a waiver of confidentiality. If you= have received this telecommunication in error, please notify the sender im= mediately by return electronic mail and delete the message from your inbox and deleted items folders. This tel= ecommunication does not constitute an express or implied agreement to condu= ct transactions by electronic means, nor does it constitute a contract offe= r, a contract amendment or an acceptance of a contract offer. Contract terms contained in this telecommunication ar= e subject to legal review and the completion of formal documentation and ar= e not binding until same is confirmed in writing and has been signed by an = authorized signatory.

--------------lU9ijHc7INm5PtsmfxAYE3De--