From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 97C283B2A4 for ; Sun, 16 Apr 2023 06:41:47 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1681641706; i=moeller0@gmx.de; bh=nrCdLpK22IwjvhoJUeHl5dUssENcqlCOgQKUwLhePz0=; h=X-UI-Sender-Class:From:Subject:Date:To; b=Gg+D64Tjf5ijdC3nWQTjflqUXIOhD+8A9yzV0F4et2CECEW/J46Hg16VHexiUX9Th qY+m/0oIOQQRMqNQUw/ZOAsBpYFPm/G3KwAWA3gV7+DUUqUtwiYiiAnTXVvOAo786D 1Uqkl1V7RMfeczCY/5wb7UGhdSZSGUuBQ1oXVapNvG/2MXbR+yI5pNqel5y1jAo80t Ni1Ehv9cIksTDyNuxxLULxvA7vK4cg8zktxjkaeJ3g7vrcXXf13Cehx65WebMPRFqN UNUmvMgpGGVwLRPuF3/08e2I9xo6kaI23EBdSy1i2m+ikbt47+1Xbc7epb75tekMVG O8340VccVGSQA== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from smtpclient.apple ([77.1.147.24]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MplXp-1q8Vrb17yX-00qAS5 for ; Sun, 16 Apr 2023 12:41:46 +0200 From: Sebastian Moeller Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Message-Id: <8C55CADC-AD2E-4C99-81C3-F083339B7CE8@gmx.de> Date: Sun, 16 Apr 2023 12:41:45 +0200 To: Rpm X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Provags-ID: V03:K1:Wk7qV1NSCic97ov1fU4koIXhyVR4pLRhMXWuxuo7VolzsIfb3eB 0Zm5a2GpUY8dgKlS5oqcKHVulInk8Oeq98b8Oxv2977bwJAtrkmJGTjpcY0lxJbOvyS5k5E qHeKzSm8vQ/utUIRYoYbneXxpgQpYK3e+nGeAvbWNDn5RRoqaP5frwzUtP4nvixFVqtnM5K yvHsBZGCvkVQdPA2R681w== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:aL8pcXOat9c=;28EJY/RXj+FNTHQOhl2jY/itIU1 aO8z5H+K4U4GEx5F+3vnAHlVmEy6Ig+prsKtdN3Ldb8gvcNQMwzQE2hRt9zMffTP42VXQVzuc BOpI4OZf6K28qGMXkx9FferwjWIHMn+FqbN/JZ42npBoB/4Pw41TBay9tbMASQ8EWa1OjKbaI g3Kg2LMUwQujavhE/ptnrr8YHkEV/28PUKw+TwG4UVqbO0LTtwZA133LGUSDxL/fOXKSdkmEH MPeFzuQUbwKCCla7/lVFO8Yux9jN3oYL6JiFL1sHcLWdKiB/PMhQbFQckbRDWKltXTuz+hwXl bxM5siPPixNB+hc6sVwB8cQs5j+GE9ZOkBCJbuiplD7FcdMNmFwwX+OStGX/0O4s/3nk2p8v7 9GukmagrdMrT8Cs6y/FQ47cc5zFwH7cJ5HJKjqJdZRqEi9jjPr8QwiTcYHA46rtLpAuDDCsRR WR+lHhMOILjFYo9o/63KEIPQiur8B3exOJpZsTLQtaXbKf2Bvffh8E2j2BFtAbJi7yBE4amXk rDX8IVZO4HQNfUMvLhQ6chGG+Oute5n7ICRc/K32zzp05doA5GTy4w+f6HkIm4M6U4ap93abr On9vwhudhgdsaWXl+B6mROKSzfuHJaoQpxSzAJOvfN+DcNTod9csXPuamqvzldlBG/1d5R2I7 nlImbHB5DuZeR+/e7w+c74qKcUTsYs9LDJPNRLsBZBT3SMt+k/OfCBccVRD0IJtPnkucvTPik 92zVnkqyzuL+cmc+kGZhQz4xAdc1lznJYUZsS6weYmOCazltAAQQ4DPcAeQ3o4PFNcQzuGD+T rzKq2U5d4XYfR4iHlA65IYE7VJxGPzIS/1fgawGHyUkT1DG6rGY/ShT1ZveX0XLZ7W5tpHBKX uQpcU2w6C27Oxw+f4+nQ132aGZeMrGgAH4YlMXHwWoZhmMoPLdBdq7K3dgjfy4DevwubeEy7p 045NCA== Subject: [Rpm] final farewell. X-BeenThere: rpm@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: revolutions per minute - a new metric for measuring responsiveness List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Apr 2023 10:41:47 -0000 Mmmh, so I recompiled and tested goresponsiveness after a fix for a cosmetic = issue was anounced (issue is fixed, thanks) But then I stumbled over a new line in the output: "bash-3.2$ time ./networkQuality --config mensura.cdn-apple.com --port = 443 --path /api/v1/gm/config --rpmtimeout 60 --extended-stats = --logger-filename go_networkQuality_$(date +%Y%m%d_%H%M%S) 04-16-2023 10:02:21 UTC Go Responsiveness to = mensura.cdn-apple.com:443... RPM: 39 (P90) RPM: 207 (Double-Sided 10% Trimmed Mean) Download: 80.994 Mbps ( 10.124 MBps), using 12 parallel connections. Upload: 23.125 Mbps ( 2.891 MBps), using 12 parallel connections. Extended Statistics: Maximum Segment Size: 1208 Total Bytes Retransmitted: 4268 Retransmission Ratio: 0.54% Total Bytes Reordered: 98003004 Average RTT: 33.46153846153846 real 0m12.842s user 0m1.010s sys 0m1.625s" RPM: 207 (Double-Sided 10% Trimmed Mean) ? [SM] Trimmed mean? That got my whiskers in attention position, = looking at the current draft I found the following "rationale": "Over the timeframe of these intervals a potentially large number of = samples has been collected. These may be affected by noise in the = measurements, and outliers. Thus, to aggregate these we suggest to use a = trimmed mean at the TMP (Trimmed Mean Percentage - default to 95%) = percentile, thus providing the following numbers: TM(tcp_f), TM(tls_f), = TM(http_f), TM(http_l). The responsiveness is then calculated as the weighted mean: Responsiveness =3D 60000 / (1/6*(TM(tcp_f) + TM(tls_f) + TM(http_f)) + 1/2*TM(http_s)) This responsiveness value presents round-trips per minute (RPM)." [SM] First a question TMP 95% what does that mean, the mean from 2.5% to = 97.5% or from 0% to 95% 0r fro 5% to 95%? I am sure there are clear = definitions but I think the draft should spell that out explicitly. [SM] But honestly, I am not 100% convinced this is actually a = good idea... I think I understand the motivation here, boiling down a = somewhat complex issue into a single number, but "trimmed mean" seems = exceptionally un-useful for the type of distribution we are dealing with = here: a) we have zero means to differentiate "noise in the measurements, and = outliers (which too me are more or less the same, any "outlier" not = assigned to a random noise process is data)" from veridical measurements b) RTT/OWD distributions are IMHO almost never symmetric (there is a = hard lower limit, but more slack on the other end) c) doing a trimmed mean of a skewed distribution will likely result in a = higher RPM estimate than using a non-trimmed mean, which given a) seems = problematic The core of the issue is the desire for a single number... but I believe = that is non-negotiable by now (similar the use of the inverse of the = actually relevant delay data, few folks actually intuitively grasps = inverse values and how to operate on them) [SM] Also this: "The more probes that are sent, the more data available for calculation. = In order to generate as much data as possible, the Responsiveness Test = specifies that a client issue these probes regularly. There is, however, = a risk that on low-capacity networks the responsiveness probes = themselves will consume a significant amount of the capacity. Because = the test mandates first saturating capacity before probing for = responsiveness, we are able to accurately estimate how much of the = capacity the responsiveness probes will consume and never send more = probes than the network can handle." [SM] seems convincing, but in my case the reported working load = stayed well below the actual capacity, so the "we are able to accurately = estimate how much of the capacity the responsiveness probes will = consume" seems a bit optimistic... I understand posting here will not change anything, but I thought is = appropriate to make my last post on-topic, for a change ;) Regards Sebastian