<div dir="ltr">I think you misunderstood: The metric that I am testing can be computed in real time within the current NDT (the only wire change is to carry an additional result in the progress reports) and across our entire archive, including both web100 and tcp_info data. Furthermore it can be validated/calibrated by counting actual network rounds, reconstructed from packet captures. However it does not include the OS contribution, which can have a huge impact on the responsiveness seen by the user.<div><br><div>The MIT paper boosts another item that has been on my forever todo list: add automated diagnostic analysis to NDT. Ideally, NDT could localize the bottleneck:</div><div><ul><li>Client (e.g. insufficient CPU or buffers)</li><li>Last yard (WiFi)</li><li>Last Mile (Access provider)</li><li>Middle mile (Interconnects)</li><li>Server load</li></ul>(The original NDT did a limited version of something like this. Most of the above have well known signatures, except there are some really weird corner cases.) We could then better inform the user ("Fix your WiFi"), and label the data in the archive according to the nature of the likely bottleneck. <br></div><div><br></div><div><br><div><div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>Thanks,</div>--MM--<br>The best way to predict the future is to create it. - Alan Kay<br><br>We must not tolerate intolerance;</div><div dir="ltr"> however our response must be carefully measured: </div><div> too strong would be hypocritical and risks spiraling out of control;</div><div> too weak risks being mistaken for tacit approval.</div></div></div></div></div></div></div><br></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Aug 13, 2021 at 11:59 AM Livingood, Jason <<a href="mailto:Jason_Livingood@comcast.com">Jason_Livingood@comcast.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-US" style="overflow-wrap: break-word;">
<div class="gmail-m_-4366865055259000465WordSection1">
<p class="MsoNormal"><span style="font-size:14pt">Great news, Matt! If I may make a recommendation, IMO a standalone working latency test would have greater utility than integrating such a measurement into an existing test & so it may have the opportunity
of more widespread usage.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:14pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:14pt">JL<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:14pt"><u></u> <u></u></span></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12pt;color:black">From: </span></b><span style="font-size:12pt;color:black">Bloat <<a href="mailto:bloat-bounces@lists.bufferbloat.net" target="_blank">bloat-bounces@lists.bufferbloat.net</a>> on behalf of Matt Mathis via Bloat <<a href="mailto:bloat@lists.bufferbloat.net" target="_blank">bloat@lists.bufferbloat.net</a>><br>
<b>Reply-To: </b>Matt Mathis <<a href="mailto:mattmathis@google.com" target="_blank">mattmathis@google.com</a>><br>
<b>Date: </b>Tuesday, August 10, 2021 at 00:52<br>
<b>To: </b>"<a href="mailto:dave.collier-brown@indexexchange.com" target="_blank">dave.collier-brown@indexexchange.com</a>" <<a href="mailto:dave.collier-brown@indexexchange.com" target="_blank">dave.collier-brown@indexexchange.com</a>><br>
<b>Cc: </b>bloat <<a href="mailto:bloat@lists.bufferbloat.net" target="_blank">bloat@lists.bufferbloat.net</a>><br>
<b>Subject: </b>Re: [Bloat] [Rpm] Airbnb<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I am part of the M-Lab team.<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">For years we published a "jitter" metric that I considered to be bogus, basically max_rtt - min_rtt, (which were builtin Web100 instruments).<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">In 2019, we transitioned from web100 to "standard" linux tcp_info, which does not capture max_rtt. Since the web100 jitter was viewed as bogus, we did not attempt to reconstruct it, although we could have. Designing and implementing
a new latency metric was on my todo list from the beginning of that transition, but chronically preempted by more pressing problems.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">It finally made it to the top of my queue which is why I am suddenly not lurking here and the new rpm list. I was very happy to see the Apple responsiveness metric, and realized that M-Lab can implement a TCP version of it, that can be
computed both in real time on future tests and retroactively over archived tests collected over the last 12 years.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">This quick paper might be of interest: <a href="https://urldefense.com/v3/__https:/docs.google.com/document/d/1HkeJTgwLdbbdHe0dF9SWrpr8Vt7ngnovLpdDTScuUtk/edit*heading=h.sy6m01i6oa9a__;Iw!!CQl3mcHX2A!WTavRmT2GnSUSVpvPCPdmBuNtOJI_zDP1WOHE-jy8tMiXBE-CxDospZUjNote1PXn0DhWQ$" target="_blank"><b><span style="font-family:Arial,sans-serif">Preliminary
Longitudinal Study of Internet Responsiveness</span></b></a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">. <br clear="all">
<u></u><u></u></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">Thanks,<u></u><u></u></p>
</div>
<p class="MsoNormal">--MM--<br>
The best way to predict the future is to create it. - Alan Kay<br>
<br>
We must not tolerate intolerance;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> however our response must be carefully measured: <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> too strong would be hypocritical and risks spiraling out of control;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> too weak risks being mistaken for tacit approval.<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Mon, Aug 9, 2021 at 5:27 PM Dave Collier-Brown <<a href="mailto:dave.collier-brown@indexexchange.com" target="_blank">dave.collier-brown@indexexchange.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<p>Knowing programmers, I'd more likely assume latency when it's easiest to measure. At a guess, either<u></u><u></u></p>
<ul type="disc">
<li class="MsoNormal">
average of upload or download<u></u><u></u></li><li class="MsoNormal">
average of download (as it comes first)<u></u><u></u></li></ul>
<p>--dave (call me "Wally" (:-)) c-b<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>On 2021-08-09 4:24 p.m., Jonathan Morton wrote:<u></u><u></u></p>
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<pre>On 9 Aug, 2021, at 10:25 pm, Dave Collier-Brown <a href="mailto:dave.collier-brown@indexexchange.com" target="_blank"><dave.collier-brown@indexexchange.com></a> wrote:<u></u><u></u></pre>
<pre><u></u> <u></u></pre>
<pre>My run of it reported latency, but without any qualifiers...<u></u><u></u></pre>
</blockquote>
<pre>One would reasonable assume that's idle latency, then.<u></u><u></u></pre>
<pre><u></u> <u></u></pre>
<pre> - Jonathan Morton<u></u><u></u></pre>
<pre>_______________________________________________<u></u><u></u></pre>
<pre>Bloat mailing list<u></u><u></u></pre>
<pre><a href="mailto:Bloat@lists.bufferbloat.net" target="_blank">Bloat@lists.bufferbloat.net</a><u></u><u></u></pre>
<pre><a href="https://urldefense.com/v3/__https:/lists.bufferbloat.net/listinfo/bloat__;!!CQl3mcHX2A!WTavRmT2GnSUSVpvPCPdmBuNtOJI_zDP1WOHE-jy8tMiXBE-CxDospZUjNote1Oa4DQV1Q$" target="_blank">https://lists.bufferbloat.net/listinfo/bloat</a><u></u><u></u></pre>
</blockquote>
<pre>-- <u></u><u></u></pre>
<pre>David Collier-Brown, | Always do right. This will gratify<u></u><u></u></pre>
<pre>System Programmer and Author | some people and astonish the rest<u></u><u></u></pre>
<pre><a href="mailto:dave.collier-brown@indexexchange.com" target="_blank">dave.collier-brown@indexexchange.com</a> | -- Mark Twain<u></u><u></u></pre>
<p class="MsoNormal"><u></u> <u></u></p>
<p><em><b><span style="font-family:Calibri,sans-serif;color:rgb(33,33,33)">CONFIDENTIALITY NOTICE AND DISCLAIMER</span></b></em><em><span style="font-family:Calibri,sans-serif;color:rgb(33,33,33)"> : This telecommunication, including any and all attachments, contains
confidential information intended only for the person(s) to whom it is addressed. Any dissemination, distribution, copying or disclosure is strictly prohibited and is not a waiver of confidentiality. If you have received this telecommunication in error, please
notify the sender immediately by return electronic mail and delete the message from your inbox and deleted items folders. This telecommunication does not constitute an express or implied agreement to conduct transactions by electronic means, nor does it constitute
a contract offer, a contract amendment or an acceptance of a contract offer. Contract terms contained in this telecommunication are subject to legal review and the completion of formal documentation and are not binding until same is confirmed in writing and
has been signed by an authorized signatory.</span></em><u></u><u></u></p>
</div>
<p class="MsoNormal">_______________________________________________<br>
Bloat mailing list<br>
<a href="mailto:Bloat@lists.bufferbloat.net" target="_blank">Bloat@lists.bufferbloat.net</a><br>
<a href="https://urldefense.com/v3/__https:/lists.bufferbloat.net/listinfo/bloat__;!!CQl3mcHX2A!WTavRmT2GnSUSVpvPCPdmBuNtOJI_zDP1WOHE-jy8tMiXBE-CxDospZUjNote1Oa4DQV1Q$" target="_blank">https://lists.bufferbloat.net/listinfo/bloat</a><u></u><u></u></p>
</blockquote>
</div>
</div>
</div>
</blockquote></div>