Hi, I'd like to thank you for working on a nice I-D describing an interesting and IMHO useful network measurement metric. Since feedback was asked for, I'd like to try and provide constructive feedback. In general, I like the idea of "Round-trips per Minute" (RPM) as a metric used to characterize (one aspect of) a network. I do think that introducing this would improve the status quo. Since this RPM definition comprises a specific way of adding load to the network and measuring a complex metric, I think it is useful to "standardize" it. I do not think RPM can replace all other metrics. This is, in a way, mentioned in the introduction, where it is suggested to add RPM to existing measurement platforms. As such I just want to point this out more explicitely, but do not intend to diminish the RPM idea by this. In short, I'd say it's complicated. Bandwidth matters for bulk data transfer, e.g., downloading a huge update required for playing a multiplayer game online. Minimum latency matters for the feasibility of interactive applications, e.g., controlling a toy car in your room vs. a robotic arm on the ISS from Earth vs. orbital insertion around Mars from Earth. For a more mundane use case consider a voice conference. (A good decade ago I experienced a voice conferencing system running over IP that introduced over one second of (minimum) latency and therefore was awkward to use.) Expressing 'bufferbloat as a measure of "Round-trips per Minute" (RPM)' exhibits (at least) two problems: 1. A high RPM value is associated with little bufferbloat problems. 2. A low RPM value may be caused by high minimum delay instead of bufferbloat. I think that RPM (i.e., under working conditions) measures a network's usefulness for interactive applications, but not necessarily bufferbloat. I do think that RPM is in itself more generally useful than minimum latency or bandwidth. A combination of low minimum latency with low RPM value strongly hints at bufferbloat. Other combinations are less easily characterized. Bufferbloat can still lie in hiding, e.g., when a link with bufferbloat is not yet the bottleneck, or if the communications end-points are not yet able to saturate the network inbetween. Thus high bandwidth can result in high RPM values despite (hidden) bufferbloat. The "Measuring is Hard" section mentions additional complications. All in all, I do think that "measuring bufferbloat" and "measuring RPM" should not be used synonymously. The I-D title clearly shows this: RPM is measuring "Responsiveness under Working Conditions" which may be affected by bufferbloat, among other potential factors, but is not in itself bufferbloat. Under the assumption that only a single value (performance score) is considered, I do think that RPM is more generally useful than bandwidth or idle latency. On a meta-level, I think that the word "bufferbloat" is not used according to a single self-consistent definition in the I-D. Additionally, I think that the I-D should reference DNS, HTTP/2, and TLS 1.3, since these protocols are required for implementing the RPM measurement. The same for JSON, I think. Possibly URL. Using "rpm.example" instead of "example.apple.com" would result in shorter lines for the example JSON. "host123.cdn.example" instead of "hostname123.cdnprovider.com" might be a more appropriate example DNS name. Adding an informative reference to RFC 2606 / BCP 32 might raise awareness of the existence of a BCP on example DNS names. Please find both a unified diff against the text rendering of the I-D, and a word diff produced from the unified diff, attached to this email in order to suggest editorial changes that are intended to improve the reading experience. They are intended for reading and (possibly partial) manual application, since the text rendering of an I-D is usually not the preferred form of editing it. Thanks, Erik -- Always use the right tool for the job. -- Rob Pike On Fri, Aug 13, 2021 at 02:41:05PM -0700, Christoph Paasch via Bloat wrote: > I already posted this to the RPM-list, but the audience here on bloat should > be interested as well. > > > This is the specification of Apple's responsiveness/RPM test. We believe that it > would be good for the bufferbloat-effort to have a specification of how to > quantify the extend of bufferbloat from a user's perspective. Our > Internet-draft is a first step in that direction and we hope that it will > kick off some collaboration. > > > Feedback is very welcome! > > > Cheers, > Christoph > > > ----- Forwarded message from internet-drafts@ietf.org ----- > > From: internet-drafts@ietf.org > To: Christoph Paasch , Omer Shapira , Randall Meyer , Stuart Cheshire > > Date: Fri, 13 Aug 2021 09:43:40 -0700 > Subject: New Version Notification for draft-cpaasch-ippm-responsiveness-00.txt > > > A new version of I-D, draft-cpaasch-ippm-responsiveness-00.txt > has been successfully submitted by Christoph Paasch and posted to the > IETF repository. > > Name: draft-cpaasch-ippm-responsiveness > Revision: 00 > Title: Responsiveness under Working Conditions > Document date: 2021-08-13 > Group: Individual Submission > Pages: 12 > URL: https://www.ietf.org/archive/id/draft-cpaasch-ippm-responsiveness-00.txt > Status: https://datatracker.ietf.org/doc/draft-cpaasch-ippm-responsiveness/ > Htmlized: https://datatracker.ietf.org/doc/html/draft-cpaasch-ippm-responsiveness > > > Abstract: > Bufferbloat has been a long-standing problem on the Internet with > more than a decade of work on standardizing technical solutions, > implementations and testing. However, to this date, bufferbloat is > still a very common problem for the end-users. Everyone "knows" that > it is "normal" for a video conference to have problems when somebody > else on the same home-network is watching a 4K movie. > > The reason for this problem is not the lack of technical solutions, > but rather a lack of awareness of the problem-space, and a lack of > tooling to accurately measure the problem. We believe that exposing > the problem of bufferbloat to the end-user by measuring the end- > users' experience at a high level will help to create the necessary > awareness. > > This document is a first attempt at specifying a measurement > methodology to evaluate bufferbloat the way common users are > experiencing it today, using today's most frequently used protocols > and mechanisms to accurately measure the user-experience. We also > provide a way to express the bufferbloat as a measure of "Round-trips > per minute" (RPM) to have a more intuitive way for the users to > understand the notion of bufferbloat. > > > > > The IETF Secretariat > > > > ----- End forwarded message ----- > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat