From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 363853B29D for ; Mon, 16 Aug 2021 13:00:32 -0400 (EDT) Received: by mail-wm1-x332.google.com with SMTP id q11-20020a7bce8b0000b02902e6880d0accso15581817wmj.0 for ; Mon, 16 Aug 2021 10:00:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=A6FNk5mhV5duKCnUu18140nfs0NCUK69AKw0B1qUgr0=; b=Hz5id/Wpe1OGlMzqTNA6uE+aVB/R/ZRcLTvg9xmOvFfyutKOmegzHrNM2dzaO6B+8n wn6GqTkZLWOp8tz35vrXyr6ZhEErl+iGJe0k3YqfPHHVOAP6c9iJV+ne8mIL01ubcqVQ DUoIAA+FZG31M3PqLsNCGsA+ji3f+KUzeys6wLUytrsgW7eJi1c9FddreC/StFy3HCJb rFbeX6jDOXwNd1e5TDq4KaSrcUdos31c9F+9E2zHVZSEt5o5LkCTLL3YSCt9fBzY1CIF NaQUz1q/Qoc52XMELAEQMvZ/1YRry7vvxKuZnFNE/E27FwZhpAQ4wM94nCeXJ07lHBEn idVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=A6FNk5mhV5duKCnUu18140nfs0NCUK69AKw0B1qUgr0=; b=Wz3PuKIYBQ5r7LbcZ9aCJepcSPPEi0upu2BA7JSOQ1jRAQQzkEIVfVENTxR0pcS7SO C+4Jg/E62gpOeyEU7LUlTEHcy3hayHBOeKBQKEKlCa7hYCTINXyq0ipV/opX9600F8EZ 2eRPQYxjg8VFTJxAo0hkqCt+1Gew8n9K1A15sfHQRRy95oB+TT27IV7UtcQMwMyP4JxY jwwP8nogQ0Ihy6N8xectc0JWg/pejzo2ZhHBfBbdk0ihRZoI3XuqDYGl93K7dlZ3sO5A uN0QLDDy+birjhCJ0F7wnzCBp39oqsEKH2UNRVZMndsVSKeYkFzlSJe42YjyipLTjUUt cY7g== X-Gm-Message-State: AOAM530bdtDiVqlfiYMlpWezHZYeVwvDRn4H9mZ33IO2Ox3u3SPWzGY8 tUtGHhkD0daECvIVVMQl0NUZyv1KdOS8b/ls8y4jtA== X-Google-Smtp-Source: ABdhPJymGFyIZGHg07T07ldXQIANPZtetUzCQX+iFDwPhSfraUfz7SD5Y1ojizArZURLvPA/lx1PrsMQxEl0HJtNqJY= X-Received: by 2002:a7b:c185:: with SMTP id y5mr110437wmi.2.1629133230312; Mon, 16 Aug 2021 10:00:30 -0700 (PDT) MIME-Version: 1.0 References: <45E365CA-A298-44D1-8D4C-2E48CB4DC0B7@apple.com> <7D90BFF4-3A67-4A96-B0B0-37B41A72A4B5@cable.comcast.com> In-Reply-To: <7D90BFF4-3A67-4A96-B0B0-37B41A72A4B5@cable.comcast.com> From: Matt Mathis Date: Mon, 16 Aug 2021 10:00:17 -0700 Message-ID: To: "Livingood, Jason" Cc: bloat , "rpm@lists.bufferbloat.net" Content-Type: multipart/alternative; boundary="0000000000007ccd3e05c9b0234a" Subject: Re: [Rpm] [Bloat] Airbnb X-BeenThere: rpm@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: revolutions per minute - a new metric for measuring responsiveness List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Aug 2021 17:00:32 -0000 --0000000000007ccd3e05c9b0234a Content-Type: text/plain; charset="UTF-8" I think you misunderstood: The metric that I am testing can be computed in real time within the current NDT (the only wire change is to carry an additional result in the progress reports) and across our entire archive, including both web100 and tcp_info data. Furthermore it can be validated/calibrated by counting actual network rounds, reconstructed from packet captures. However it does not include the OS contribution, which can have a huge impact on the responsiveness seen by the user. The MIT paper boosts another item that has been on my forever todo list: add automated diagnostic analysis to NDT. Ideally, NDT could localize the bottleneck: - Client (e.g. insufficient CPU or buffers) - Last yard (WiFi) - Last Mile (Access provider) - Middle mile (Interconnects) - Server load (The original NDT did a limited version of something like this. Most of the above have well known signatures, except there are some really weird corner cases.) We could then better inform the user ("Fix your WiFi"), and label the data in the archive according to the nature of the likely bottleneck. Thanks, --MM-- The best way to predict the future is to create it. - Alan Kay We must not tolerate intolerance; however our response must be carefully measured: too strong would be hypocritical and risks spiraling out of control; too weak risks being mistaken for tacit approval. On Fri, Aug 13, 2021 at 11:59 AM Livingood, Jason < Jason_Livingood@comcast.com> wrote: > Great news, Matt! If I may make a recommendation, IMO a standalone working > latency test would have greater utility than integrating such a measurement > into an existing test & so it may have the opportunity of more widespread > usage. > > > > JL > > > > *From: *Bloat on behalf of Matt > Mathis via Bloat > *Reply-To: *Matt Mathis > *Date: *Tuesday, August 10, 2021 at 00:52 > *To: *"dave.collier-brown@indexexchange.com" < > dave.collier-brown@indexexchange.com> > *Cc: *bloat > *Subject: *Re: [Bloat] [Rpm] Airbnb > > > > I am part of the M-Lab team. > > > > For years we published a "jitter" metric that I considered to be bogus, > basically max_rtt - min_rtt, (which were builtin Web100 instruments). > > > > In 2019, we transitioned from web100 to "standard" linux tcp_info, which > does not capture max_rtt. Since the web100 jitter was viewed as bogus, we > did not attempt to reconstruct it, although we could have. Designing and > implementing a new latency metric was on my todo list from the beginning of > that transition, but chronically preempted by more pressing problems. > > > > It finally made it to the top of my queue which is why I am suddenly not > lurking here and the new rpm list. I was very happy to see the Apple > responsiveness metric, and realized that M-Lab can implement a TCP > version of it, that can be computed both in real time on future tests and > retroactively over archived tests collected over the last 12 years. > > > > This quick paper might be of interest: *Preliminary Longitudinal Study of > Internet Responsiveness* > > > . > > Thanks, > > --MM-- > The best way to predict the future is to create it. - Alan Kay > > We must not tolerate intolerance; > > however our response must be carefully measured: > > too strong would be hypocritical and risks spiraling out of > control; > > too weak risks being mistaken for tacit approval. > > > > > > On Mon, Aug 9, 2021 at 5:27 PM Dave Collier-Brown < > dave.collier-brown@indexexchange.com> wrote: > > Knowing programmers, I'd more likely assume latency when it's easiest to > measure. At a guess, either > > - average of upload or download > - average of download (as it comes first) > > --dave (call me "Wally" (:-)) c-b > > > > On 2021-08-09 4:24 p.m., Jonathan Morton wrote: > > On 9 Aug, 2021, at 10:25 pm, Dave Collier-Brown wrote: > > > > My run of it reported latency, but without any qualifiers... > > One would reasonable assume that's idle latency, then. > > > > - Jonathan Morton > > _______________________________________________ > > Bloat mailing list > > Bloat@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/bloat > > -- > > David Collier-Brown, | Always do right. This will gratify > > System Programmer and Author | some people and astonish the rest > > dave.collier-brown@indexexchange.com | -- Mark Twain > > > > *CONFIDENTIALITY NOTICE AND DISCLAIMER** : This telecommunication, > including any and all attachments, contains confidential information > intended only for the person(s) to whom it is addressed. Any dissemination, > distribution, copying or disclosure is strictly prohibited and is not a > waiver of confidentiality. If you have received this telecommunication in > error, please notify the sender immediately by return electronic mail and > delete the message from your inbox and deleted items folders. This > telecommunication does not constitute an express or implied agreement to > conduct transactions by electronic means, nor does it constitute a contract > offer, a contract amendment or an acceptance of a contract offer. Contract > terms contained in this telecommunication are subject to legal review and > the completion of formal documentation and are not binding until same is > confirmed in writing and has been signed by an authorized signatory.* > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > > > --0000000000007ccd3e05c9b0234a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I think you misunderstood:=C2=A0 The metric that I am test= ing can be computed in real time within the current NDT (the only wire chan= ge is to carry an additional result in the progress reports) and across our= entire archive, including both web100 and tcp_info data.=C2=A0 Furthermore= it can be validated/calibrated by counting=C2=A0actual network rounds,=C2= =A0reconstructed=C2=A0from packet captures.=C2=A0 However it does not inclu= de the OS contribution, which can have a huge impact on the responsiveness = seen by the user.

The MIT paper boosts another item that has b= een on my forever todo list: add automated diagnostic analysis=C2=A0to NDT.= =C2=A0 Ideally, NDT could localize the bottleneck:
  • Client= (e.g. insufficient=C2=A0CPU or buffers)
  • Last yard (WiFi)
  • L= ast=C2=A0 Mile (Access provider)
  • Middle mile (Interconnects)
    • Server load
(The original=C2=A0NDT did a limited version of some= thing like this.=C2=A0 Most of the above have well known signatures,=C2=A0e= xcept there are some really weird=C2=A0corner cases.)=C2=A0 =C2=A0We could = then better inform the user ("Fix your WiFi"), and label=C2=A0the= data in the archive according to the nature of the likely bottleneck.=C2= =A0


Thanks,
--MM--
The b= est way to predict the future is to create it. =C2=A0- Alan Kay

We m= ust not tolerate intolerance;
=C2=A0 =C2=A0 =C2=A0 = =C2=A0however our response must be carefully measured:=C2=A0
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 too strong would be hypocritical and= risks spiraling out of control;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 too weak risks being mistaken for tacit approval.


On Fri, Aug 13, 2021= at 11:59 AM Livingood, Jason <Jason_Livingood@comcast.com> wrote:

Great news, Matt! If = I may make a recommendation, IMO a standalone working latency test would ha= ve greater utility than integrating such a measurement into an existing tes= t & so it may have the opportunity of more widespread usage.

=C2=A0<= /span>

JL

=C2=A0<= /span>

From: = Bloat <bloat-bounces= @lists.bufferbloat.net> on behalf of Matt Mathis via Bloat <bloat@lists.buf= ferbloat.net>
Reply-To: Matt Mathis <mattmathis@google.com>
Date: Tuesday, August 10, 2021 at 00:52
To: "dave.collier-brown@indexexchange.com" <dave.col= lier-brown@indexexchange.com>
Cc: bloat <bloat@lists.bufferbloat.net>
Subject: Re: [Bloat] [Rpm] Airbnb

=C2=A0

I am part of the M-Lab team.

=C2=A0

For years we published a "jitter" metric t= hat I considered to be bogus, basically max_rtt - min_rtt, (which were buil= tin Web100 instruments).

=C2=A0

In 2019, we transitioned=C2=A0from web100 to "s= tandard" linux tcp_info, which does not capture max_rtt.=C2=A0 =C2=A0S= ince the web100 jitter was viewed as bogus, we did not attempt to reconstru= ct it, although we could have.=C2=A0 =C2=A0Designing and implementing a new latency=C2=A0metric was on my todo list from the beginning of that t= ransition, but chronically preempted by more pressing problems.

=C2=A0

It finally made it to the top of my queue which is w= hy I am suddenly not lurking here and the new rpm list.=C2=A0 I was very ha= ppy to see the Apple responsiveness metric, and realized that M-Lab can imp= lement a TCP version=C2=A0of it, that can be computed both in real time on future tests and retroactively over archived= tests collected over the last 12 years.

=C2=A0

This=C2=A0quick paper might be of interest:=C2=A0Preliminary Longitudinal Study of Internet Responsiveness=

.=C2=A0=C2=A0

Thanks,

--MM--
The best way to predict the future is to create it. =C2=A0- Alan Kay

We must not tolerate intolerance;

=C2=A0 =C2=A0 =C2=A0 =C2=A0however our response must= be carefully measured:=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 too strong= would be hypocritical and risks spiraling out of control;

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 too weak r= isks being mistaken for tacit approval.

=C2=A0

=C2=A0

On Mon, Aug 9, 2021 at 5:27 PM Dave Collier-Brown &l= t;dave.collier-brown@indexexchange.com> wrote:

Knowing programmers, I'd more likely assume latency when it's ea= siest to measure. At a guess, either

  • average of upload or download
  • average of download (as it comes first)

--dave (call me "Wally" (:-)) c-b

=C2=A0

On 2021-08-09 4:24 p.m., Jonathan Morton wrote:

On 9 Aug, 2021, at 10:25 pm, Dave Collier-Brown <dave.collier-brown@=
indexexchange.com> wrote:
=C2=A0
My run of it reported latency, but without any qualifiers...=

One would reasonable assume that's idle latency, then.
=C2=A0
 - Jonathan Morton
_______________________________________________
Bloat mailing list
Bloat=
@lists.bufferbloat.net
https://lists.bufferbloat.net/l=
istinfo/bloat
-- 
David Collier-Brown,=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=
 Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
dave.collier-brown@indexexchange.com |=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 -- Mark Twain=

=C2=A0

CONFIDENTIALITY NOTICE AND DISCLAIMER=C2=A0: This telecommuni= cation, including any and all attachments, contains confidential information intended only for the person(s) to whom it is add= ressed. Any dissemination, distribution, copying or disclosure is strictly = prohibited and is not a waiver of confidentiality. If you have received thi= s telecommunication in error, please notify the sender immediately by return electronic mail and delete the mes= sage from your inbox and deleted items folders. This telecommunication does= not constitute an express or implied agreement to conduct transactions by = electronic means, nor does it constitute a contract offer, a contract amendment or an acceptance of a contract offe= r. Contract terms contained in this telecommunication are subject to legal = review and the completion of formal documentation and are not binding until= same is confirmed in writing and has been signed by an authorized signatory.

_______________________________________________
Bloat mailing list
Bloat@list= s.bufferbloat.net
https://lists.bufferbloat.net/listin= fo/bloat

--0000000000007ccd3e05c9b0234a--