From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ma1-aaemail-dr-lapp01.apple.com (ma1-aaemail-dr-lapp01.apple.com [17.171.2.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id B0E583CB43 for ; Fri, 22 Oct 2021 19:19:43 -0400 (EDT) Received: from pps.filterd (ma1-aaemail-dr-lapp01.apple.com [127.0.0.1]) by ma1-aaemail-dr-lapp01.apple.com (8.16.0.42/8.16.0.42) with SMTP id 19MNJ45E026232; Fri, 22 Oct 2021 16:19:33 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=content-type : mime-version : subject : from : in-reply-to : date : cc : content-transfer-encoding : message-id : references : to; s=20180706; bh=MJN1kfArMBusHaplLHvA8OwvGraRAIK5d0uskiCdiFs=; b=UkU7mWCr9C3e/7unzyS+qaGNDaVdgzzybu0QGCIw0PcMLMtTed+Z7iNPoQ81o02K04p8 jAb1BaYKlKMXOft5sH9GyQ3yHsF03CyHo8BCfyiEcV31E/m036j3PeReyTqCSvfIZOX1 1lSNrWnYto6L2DLMteIvTm0HUXHltaPyAJ3wHTsCwuY0b4Zk84SjqgUDBGr1mKKPQJpl KAvfjBPdutmosuVVDV/Qes8+XGrxqI1VIixX6cUPcj5PCcY92SxijZRdy6PgD6O/mL0W mTYKlh3Bcfg/mLiDUN0cJ7fsLT/09NNk72XvpQE+ve8nC2smg2/LQ1jPKUx0JtyDAuM9 HA== Received: from rn-mailsvcp-mta-lapp01.rno.apple.com (rn-mailsvcp-mta-lapp01.rno.apple.com [10.225.203.149]) by ma1-aaemail-dr-lapp01.apple.com with ESMTP id 3bqwd6bave-9 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Fri, 22 Oct 2021 16:19:33 -0700 Received: from rn-mailsvcp-mmp-lapp04.rno.apple.com (rn-mailsvcp-mmp-lapp04.rno.apple.com [17.179.253.17]) by rn-mailsvcp-mta-lapp01.rno.apple.com (Oracle Communications Messaging Server 8.1.0.12.20210903 64bit (built Sep 3 2021)) with ESMTPS id <0R1E00DZCJGK9K80@rn-mailsvcp-mta-lapp01.rno.apple.com>; Fri, 22 Oct 2021 16:19:32 -0700 (PDT) Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp04.rno.apple.com by rn-mailsvcp-mmp-lapp04.rno.apple.com (Oracle Communications Messaging Server 8.1.0.12.20210903 64bit (built Sep 3 2021)) id <0R1E00R00IUZ5900@rn-mailsvcp-mmp-lapp04.rno.apple.com>; Fri, 22 Oct 2021 16:19:32 -0700 (PDT) X-Va-A: X-Va-T-CD: f531f8d8b46e434c6cc260b03cd4b544 X-Va-E-CD: 80ebf395e5bfaf9dd37d2e1677896837 X-Va-R-CD: 34a054e2b971ae162a64f30ab9c21b58 X-Va-CD: 0 X-Va-ID: c2c5686e-5e9d-4819-8986-b522256e6881 X-V-A: X-V-T-CD: f531f8d8b46e434c6cc260b03cd4b544 X-V-E-CD: 80ebf395e5bfaf9dd37d2e1677896837 X-V-R-CD: 34a054e2b971ae162a64f30ab9c21b58 X-V-CD: 0 X-V-ID: d492810b-ccf9-4761-b0f6-853426e1f1de X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.790 definitions=2021-10-22_05:2021-10-22, 2021-10-22 signatures=0 Received: from smtpclient.apple ([17.11.46.46]) by rn-mailsvcp-mmp-lapp04.rno.apple.com (Oracle Communications Messaging Server 8.1.0.12.20210903 64bit (built Sep 3 2021)) with ESMTPSA id <0R1E00NCSJGG5T00@rn-mailsvcp-mmp-lapp04.rno.apple.com>; Fri, 22 Oct 2021 16:19:32 -0700 (PDT) Content-type: text/plain; charset=us-ascii MIME-version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\)) From: Christoph Paasch In-reply-to: Date: Fri, 22 Oct 2021 16:19:28 -0700 Cc: Erik Auerswald , ippm@ietf.org, draft-cpaasch-ippm-responsiveness@ietf.org, bloat@lists.bufferbloat.net Content-transfer-encoding: quoted-printable Message-id: References: <20210815133922.GA18118@unix-ag.uni-kl.de> To: Toerless Eckert X-Mailer: Apple Mail (2.3693.20.0.1.32) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.790 definitions=2021-10-22_05:2021-10-22, 2021-10-22 signatures=0 Subject: Re: [Bloat] [ippm] Fwd: New Version Notification for draft-cpaasch-ippm-responsiveness-00.txt X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Oct 2021 23:19:43 -0000 Hello Toerless, thanks for your feedback! Please see inline: > On Sep 21, 2021, at 1:50 PM, Toerless Eckert wrote: >=20 > Dear authors >=20 > Thanks for the draft >=20 > a) Can you please update naming of the draft so people remembering RPM = will find the draft ? > Something like: >=20 > draft-cpaasch-ippm-rpm-bufferbloat-metric-00 > Round-trips Per Minute (RPM) under load - a Metric for bufferbloat. That's a good point! How does draft-cpaasch-ippm-responsiveness-rpm-00 = sound? I prefer not to have bufferbloat in the name as it is a loaded term = (there are different interpretations of it) and easily misunderstood. = Some consider bufferbloat to only be a problem on the routers/switches, = while others consider it to be an end-to-end problem. Our test = methodology measures end-to-end responsiveness and end-to-end here goes = all the way up to the HTTP/2 implementation. > b) The draft does not mention, or at least does not have a > separate section to discuss where the server is against which the = test is run. > It should have such a section. I can hink of at least two key = options, > - the server used for the service in question (e.g.: where contents = comes from), > - a server at a wll defined location in the access network provider. I see your point. I don't think the draft needs to "mandate" where one = should put those servers. But rather a discussion on where one may want = to place the server depending on what one is measuring. A WiFi AP vendor may want to deploy the server locally in his testbed. A = content-provider wants to host the server on its content-servers. And an = ISP wants to host it probably at the border of its domain. > c) I fear that b) leads to be biggest current issue with the metric: > The longer the path is, such as full path to a server, the more = useful the > metric is for the user. But the user will effectively get a = per-service metric. > To make this more fun to the authors: Imagine the appleTV server = nodes have a worse > path to a particular user than the Netflix servers. Or vice versa. >=20 > If we just use a path to some fixed point in the access provider, > then we take away the users ability to beat up their OTT services to > improve their paths.=20 >=20 > If we use only a path toward the service, it will be harder to=20 > hit on the service provider, if the service provider is bad. >=20 > So, obviously, i would like to have all three RPM: to Netflix, = AppleTV > and a well defined server in Comcast. Then i can triangulate where > my bufferbloat problem is. Yes, the location of the server has a huge impact on the resulting = number. The goal with responsiveness is to measure user-experience. If = the user uses Netflix, AppleTV and Comcast streaming, that's what the = user is interested in. If the user also uses some remote streaming = service on the other side of the ocean, that's as well the = responsiveness number the user would be interested in. > d) Worst yet, without having seen more example numbers (a reference = pointing > to some good collected RPM numbers would be excellent), my > concern is that instead of fixing bufferbloat on paths, we would = simply > encourage OTT to co-locate servers to the access providers own = measurement > point, aka: as close to the subscriber. Deploying close to the subscriber does not yet fix the = bufferbloat-problem. If the last-mile or the HTTP-implementation has = bufferbloat issues, they are still going to present a bad RPM-number. = Sure, once we have eradicated bufferbloat from the Internet, then = closeness to the subscriber will become a more driving factor. And that = would be a good problem to have :-) >=20 > e) To solve d), maybe two ideas: >=20 > - relevant to improve bufferbloat is only (lRPM - iRPM), where > lRPM would be your current RPM, e.g.: under (l)oaded condition), > and iRPM is idle RPM. This still does not take away from the > fact that a path with more queuing hops or higher queue loads > will fare worse than the shorter physcial propagation latency = path, > but it does mke the metric significantly be focussed on queueing, > and should help a lot when we do compare service that might not > have servers in the users metro area. The difficulty here is that it is near impossible to really measure = iRPM. Because, one cannot know whether the network really is idle or = not. >=20 > - lRPM/m - RPM under load per mile (roughly). > - Measure idle RTT in units of msec (iRTT) >=20 > - Measure load RTT in units of msec (lRTT) >=20 > - Just take iRTT as a measure for the path lenth. > normalizing it absolutely is not of first order > important, we are primarily interested in relative number, > and this keeps the example calculation simple. >=20 > - the RTT increase because of queueing is (lRTT - iRTT). >=20 > - (lRTT - iRTT) / iRTT is therefore something like queuing RTT > per path stretch. I think this is th relative number we want. >=20 > - RPM =3D iRTT / (lRTT - iRTT) * 1000 turns this into some=20 > number increasing with desired non-bufferbloat performance > with enough significant in non fractionals. >=20 > - Example:=20 > idle RTT: 5msec, loaded RTT: 20 msec =3D> 333 RPM > idle RTT: 10msec, loaded RTT: 20 msec =3D> 1000 RPM > idle RTT: 15msec, loaded RTT: 20 msec =3D> 3000 RPM >=20 > This nicely shows how the RPM will go up when the physcial > path itself gets longer, but the relevant load RTT stays=20 > the same. >=20 > idle RTT: 5msec, loaded RTT: 20 msec =3D> 333 RPM > idle RTT: 10msec, loaded RTT: 40 msec =3D> 333 RPM > idle RTT: 15msec, loaded RTT: 60 msec =3D> 333 RPM >=20 > This nicely shows that we can have servers at different > physical distance and get the same RPM number, when the > bufferbloat is the same, e.g.: 15 msec worth of bufferbloat > for every 5msec propagation latency segment. >=20 > f) I can see how you do NOT want the type of metric i am > proposing, because it only focusses on the bufferbloat > factor, and you may want to stick to the full experience of > the user, where unmistakingly the propagation latency can > not be ignored, but to repeat from above: >=20 > If we do not use a metric that fairly treats paths of different > propagation latencies as the same wrt. performance, i am > quite persuaded we will continue to just see big services > win out, because hey can more easily afford to get closer > to the user with their (rented/time-shared/owned) servers. >=20 > Aka: Right now RPM is a metric that will specifically=20 > make it easier for one of the big providers of sttreaming > such as that of the authors to position themselves better > against smaller services streaming from further away. The goal of the responsiveness measurement is not for it to be a = diagnostic tool that allows to find the accurate location of = bufferbloat. It is also not a tool to make some streaming service = providers look better than others and/or make one "win" against the = other. The goal really is to have an accurate representation of responsiveness = under working conditions. If the server end-point is at a very remote = location, then indeed the RPM number will be lower. And the = responsiveness measurement should reflect that, because it tries to = assess the user-experience. With the standardization of the methodology our hope is that = content-providers (small or big) would adopt this methodology so that = they can properly assess the network's quality for their users and their = services. Cheers, Christoph