<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="DE" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Folks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">I support Neal and Sebastian, I’d also prefer visibility of a baseline/no load metric vs. an actual metric display, at least optional.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Ruediger<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>Von:</b> ippm <ippm-bounces@ietf.org> <b>Im Auftrag von </b>
Neal Cardwell<br>
<b>Gesendet:</b> Dienstag, 25. Oktober 2022 04:29<br>
<b>An:</b> Christoph Paasch <cpaasch=40apple.com@dmarc.ietf.org><br>
<b>Cc:</b> Sebastian Moeller <moeller0@gmx.de>; Glenn Fishbine <glenn@breakingpointsolutions.com>; Rpm <rpm@lists.bufferbloat.net>; tsvwg IETF list <tsvwg@ietf.org>; IETF IPPM WG <ippm@ietf.org>; Dave Taht via Starlink <starlink@lists.bufferbloat.net>; Measurement
Analysis and Tools Working Group <mat-wg@ripe.net>; discuss <discuss@measurementlab.net><br>
<b>Betreff:</b> Re: [ippm] [tsvwg] [Starlink] [Rpm] [M-Lab-Discuss] misery metrics & consequences<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Mon, Oct 24, 2022 at 7:44 PM Christoph Paasch <cpaasch=<a href="mailto:40apple.com@dmarc.ietf.org">40apple.com@dmarc.ietf.org</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Oct 24, 2022, at 1:57 PM, Sebastian Moeller <<a href="mailto:moeller0@gmx.de" target="_blank">moeller0@gmx.de</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">Hi Christoph<br>
<br style="font-variant-caps:normal;text-align:start;word-spacing:0px">
<br>
</span><o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">On Oct 24, 2022, at 22:08, Christoph Paasch <<a href="mailto:cpaasch@apple.com" target="_blank">cpaasch@apple.com</a>> wrote:<br>
<br>
Hello Sebastian,<br>
<br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">On Oct 23, 2022, at 4:57 AM, Sebastian Moeller via Starlink <<a href="mailto:starlink@lists.bufferbloat.net" target="_blank">starlink@lists.bufferbloat.net</a>> wrote:<br>
<br>
Hi Glenn,<br>
<br>
<br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">On Oct 23, 2022, at 02:17, Glenn Fishbine via Rpm <<a href="mailto:rpm@lists.bufferbloat.net" target="_blank">rpm@lists.bufferbloat.net</a>> wrote:<br>
<br>
As a classic died in the wool empiricist, granted that you can identify "misery" factors, given a population of 1,000 users, how do you propose deriving a misery index for that population?<br>
<br>
We can measure download, upload, ping, jitter pretty much without user intervention. For the measurements you hypothesize, how you you automatically extract those indecies without subjective user contamination.<br>
<br>
I.e. my download speed sucks. Measure the download speed.<br>
<br>
My isp doesn't fix my problem. Measure what? How?<br>
<br>
Human survey technology is 70+ years old and it still has problems figuring out how to correlate opinion with fact.<br>
<br>
Without an objective measurement scheme that doesn't require human interaction, the misery index is a cool hypothesis with no way to link to actual data. What objective measurements can be made? Answer that and the index becomes useful. Otherwise it's just
consumer whining.<br>
<br>
Not trying to be combative here, in fact I like the concept you support, but I'm hard pressed to see how the concept can lead to data, and the data lead to policy proposals.<o:p></o:p></span></p>
</blockquote>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><br>
[SM] So it seems that outside of seemingly simple to test throughput numbers*, the next most important quality number (or the most important depending on subjective ranking) is how does latency change under "load". Absolute latency is also important albeit
static high latency can be worked around within limits so the change under load seems more relevant. <br>
All of flent's RRUL test, apple's networkQuality/RPM, and iperf2's bounceback test offer methods to asses latency change under load**, as do waveforms bufferbloat tests and even to a degree Ookla's
<a href="http://speedtest.net" target="_blank">speedtest.net</a>. IMHO something like latency increase under load or apple's responsiveness measure RPM (basically the inverse of the latency under load calculated on a per minute basis, so it scales in the typical
higher numbers are better way, unlike raw latency under load numbers where smaller is better).<br>
IMHO what networkQuality is missing ATM is to measure and report the unloaded RPM as well as the loaded the first gives a measure over the static latency the second over how well things keep working if capacity gets tight. They report the base RTT which can
be converted to RPM. As an example:<br>
<br>
macbook:~ user$ networkQuality -v<br>
==== SUMMARY ==== <br>
Upload capacity: 24.341 Mbps<br>
Download capacity: 91.951 Mbps<br>
Upload flows: 20<br>
Download flows: 16<br>
Responsiveness: High (2123 RPM)<br>
Base RTT: 16<br>
Start: 10/23/22, 13:44:39<br>
End: 10/23/22, 13:44:53<br>
OS Version: Version 12.6 (Build 21G115)<o:p></o:p></span></p>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><br>
You should update to latest macOS:<br>
<br>
$ networkQuality<br>
==== SUMMARY ====<br>
Uplink capacity: 326.789 Mbps<br>
Downlink capacity: 446.359 Mbps<br>
Responsiveness: High (2195 RPM)<br>
Idle Latency: 5.833 milli-seconds<br>
<br>
;-)<o:p></o:p></span></p>
</blockquote>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><br>
<br>
[SM] I wish... just updated to the latest and greatest for this hardware (A1398):<br>
<br>
macbook-pro:DPZ smoeller$ networkQuality<br>
==== SUMMARY ==== <br>
Upload capacity: 7.478 Mbps<br>
Download capacity: 2.415 Mbps<br>
Upload flows: 16<br>
Download flows: 20<br>
Responsiveness: Low (90 RPM)<br>
macbook-pro:DPZ smoeller$ networkQuality -v<br>
==== SUMMARY ==== <br>
Upload capacity: 5.830 Mbps<br>
Download capacity: 6.077 Mbps<br>
Upload flows: 12<br>
Download flows: 20<br>
Responsiveness: Low (56 RPM)<br>
Base RTT: 134<br>
Start: 10/24/22, 22:47:48<br>
End: 10/24/22, 22:48:09<br>
OS Version: Version 12.6.1 (Build 21G217)<br>
macbook-pro:DPZ smoeller$ <br>
<br>
Still, I only see the "Base RTT" with the -v switch and I am not sure whether that is identical to your "Idle Latency".<br>
<br>
<br>
I guess I need to convince my employer to exchange that macbook (actually because the battery starts bulging and not because I am behind with networkQuality versions ;) )</span><o:p></o:p></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Yes, you would need macOS Ventura to get the latest and greatest.<o:p></o:p></p>
</div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt;font-variant-caps:normal;text-align:start;word-spacing:0px">
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">But, what I read is: You are suggesting that “Idle Latency” should be expressed in RPM as well? Or, Responsiveness expressed in millisecond ?<o:p></o:p></span></p>
</blockquote>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><br>
[SM] Yes, I am fine with either (or both) the idea is to make it really easy to see whether/how much "working conditions" deteriorate the responsiveness / increase the latency-under-load. At least in verbose mode it would be sweet if nwtworkQuality could expose
that information.</span><o:p></o:p></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I see - let me think about that…<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">+1 w/ Sebastian's point here. IMHO it would be great if the responsiveness under load and when idle were reported:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> (a) symmetrically, with the same metrics for both cases, and <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> (b) in both RPM and ms terms for both cases<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">So instead of:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Responsiveness: High (2195 RPM)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Idle Latency: 5.833 milli-seconds<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Perhaps something like:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Loaded Responsiveness: High (XXXX RPM)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Loaded Latency: X.XXX milli-seconds<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Idle Responsiveness: High (XXXX RPM)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Idle Latency: X.XXX milli-seconds <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Having both RPM and ms available for loaded and unloaded cases would seem to make it easier to compare loaded and idle performance more directly and in a more apples-to-apples way.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">best,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">neal<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>