[Starlink] [M-Lab-Discuss] misery metrics & consequences

[Dave Collier-Brown]

I read the talk as saying we should measure and report the failures rather than the successes.

Let's say download speed is the problem,  and I'm Rogers, an ISP with lots of subnets.  For each I plot the number of downloads per net per hour which were slower than X kb/s, not counting ones whose total time were less than 2 seconds.

That gives you a measurement that fairly closely models the degree of human annoyance you're causing (;-))

And Rogers causes a lot, but it's usually on concalls, not downloads.

--dave

[I might use a poll to see if a claimed huge improvement worked, but I don't trust human estimations any more that you do (:-)]


On 10/22/22 20:17, Glenn Fishbine via Starlink wrote:
As a classic died in the wool empiricist, granted that you can identify "misery" factors, given a population of 1,000 users, how do you propose deriving a misery index for that population?

We can measure download, upload, ping, jitter pretty much without user intervention.  For the measurements you hypothesize, how you you automatically extract those indecies without subjective user contamination.

I.e.  my download speed sucks. Measure the download speed.

My isp doesn't fix my problem. Measure what? How?

Human survey technology is 70+ years old and it still has problems figuring out how to correlate opinion with fact.

Without an objective measurement scheme that doesn't require human interaction, the misery index is a cool hypothesis with no way to link to actual data.  What objective measurements can be made?  Answer that and the index becomes useful. Otherwise it's just consumer whining.

Not trying to be combative here, in fact I like the concept you support, but I'm hard pressed to see how the concept can lead to data, and the data lead to policy proposals.


On Fri, Oct 21, 2022, 5:20 PM Dave Taht <dave.taht at gmail.com<mailto:dave.taht at gmail.com>> wrote:
One of the best talks I've ever seen on how to measure customer
satisfaction properly just went up after the P99 Conference.

It's called Misery Metrics.

After going through a deep dive as to why and how we think and act on
percentiles, bins, and other statistical methods as to how we use the
web and internet are *so wrong* (well worth watching and thinking
about if you are relying on or creating network metrics today), it
then points to the real metrics that matter to users and the ultimate
success of an internet business: Timeouts, retries, misses, failed
queries, angry phone calls, abandoned shopping carts and loss of
engagement.

https://www.p99conf.io/session/misery-metrics-consequences/

The ending advice was - don't aim to make a specific percentile
acceptable, aim for an acceptable % of misery.

I enjoyed the p99 conference more than any conference I've attended in years.

--
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC

--
You received this message because you are subscribed to the Google Groups "discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe at measurementlab.net<mailto:discuss%2Bunsubscribe at measurementlab.net>.
To view this discussion on the web visit https://groups.google.com/a/measurementlab.net/d/msgid/discuss/CAA93jw4w27a1EO_QQG7NNkih%2BC3QQde5%3D_7OqGeS9xy9nB6wkg%40mail.gmail.com.



_______________________________________________
Starlink mailing list
Starlink at lists.bufferbloat.net<mailto:Starlink at lists.bufferbloat.net>
https://lists.bufferbloat.net/listinfo/starlink


--
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
dave.collier-brown at indexexchange.com<mailto:dave.collier-brown at indexexchange.com> |              -- Mark Twain


CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, including any and all attachments, contains confidential information intended only for the person(s) to whom it is addressed. Any dissemination, distribution, copying or disclosure is strictly prohibited and is not a waiver of confidentiality. If you have received this telecommunication in error, please notify the sender immediately by return electronic mail and delete the message from your inbox and deleted items folders. This telecommunication does not constitute an express or implied agreement to conduct transactions by electronic means, nor does it constitute a contract offer, a contract amendment or an acceptance of a contract offer. Contract terms contained in this telecommunication are subject to legal review and the completion of formal documentation and are not binding until same is confirmed in writing and has been signed by an authorized signatory.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/starlink/attachments/20221024/86c7fb61/attachment.html>