From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from rn-mailsvcp-mx-lapp02.apple.com (rn-mailsvcp-mx-lapp02.apple.com [17.179.253.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 8EE103B29E for ; Mon, 17 Apr 2023 12:57:51 -0400 (EDT) Received: from rn-mailsvcp-mta-lapp04.rno.apple.com (rn-mailsvcp-mta-lapp04.rno.apple.com [10.225.203.152]) by rn-mailsvcp-mx-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.22.20230228 64bit (built Feb 28 2023)) with ESMTPS id <0RT900SIIR402620@rn-mailsvcp-mx-lapp02.rno.apple.com> for rpm@lists.bufferbloat.net; Mon, 17 Apr 2023 09:57:50 -0700 (PDT) X-Proofpoint-ORIG-GUID: tizQROuDSuOq71hUUvUw97TQnVkssNQ7 X-Proofpoint-GUID: tizQROuDSuOq71hUUvUw97TQnVkssNQ7 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.573, 18.0.942 definitions=2023-04-17_11:2023-04-17, 2023-04-17 signatures=0 X-Proofpoint-Spam-Details: rule=interactive_user_notspam policy=interactive_user score=0 adultscore=0 phishscore=0 malwarescore=0 suspectscore=0 bulkscore=0 mlxscore=0 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304170149 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=content-type : mime-version : subject : from : in-reply-to : date : cc : content-transfer-encoding : message-id : references : to; s=20180706; bh=j4MODPAvCEWad7xed8o/TrO8PjuEstah+WzWAVA0fjc=; b=T4YYRbBGwoYpOms1WsRvzwqMN6DbD3PrZPD6GTaoHHiggG8V9E56EbzABaiUGv447WEe M8orU9tTCGJEcxG47G4vdt6vxUtyHA4zSDMctv/VyrG2bTOCAmSAFFeHGnorYcy6bZXG Lcg3dIEOiC+T16ESICoSF86w4Ic45qZvlE30aHgpQHIY6odZU8B+veYFgqcnySFyl/n9 VsLxodqAvj75vqMNU5MjyBYw5GWFK7Q3l4ynNlXOSr/g/wkfRPTfGu1TCryOIMjd+wql nMJNKVm3ZVG0NfeYyRhUvdg9qnrqRKwHOwXOgZkvb3FOPj0Xtl7+HXqW5cE/ft/JQSSh 8Q== Received: from rn-mailsvcp-mmp-lapp02.rno.apple.com (rn-mailsvcp-mmp-lapp02.rno.apple.com [17.179.253.15]) by rn-mailsvcp-mta-lapp04.rno.apple.com (Oracle Communications Messaging Server 8.1.0.22.20230228 64bit (built Feb 28 2023)) with ESMTPS id <0RT900V4VR4CT0O0@rn-mailsvcp-mta-lapp04.rno.apple.com>; Mon, 17 Apr 2023 09:57:48 -0700 (PDT) Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp02.rno.apple.com by rn-mailsvcp-mmp-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.22.20230228 64bit (built Feb 28 2023)) id <0RT900K00QT25C00@rn-mailsvcp-mmp-lapp02.rno.apple.com>; Mon, 17 Apr 2023 09:57:48 -0700 (PDT) X-Va-A: X-Va-T-CD: 0af778c0afa90afa8c4c05937d25c782 X-Va-E-CD: 57ac8101054c88becc91b867e8f3e2e6 X-Va-R-CD: e8e2fd97875822dc5d5b5a62e68547b3 X-Va-ID: 5f74e7d0-1197-4654-9776-56c7a648cc1e X-Va-CD: 0 X-V-A: X-V-T-CD: 0af778c0afa90afa8c4c05937d25c782 X-V-E-CD: 57ac8101054c88becc91b867e8f3e2e6 X-V-R-CD: e8e2fd97875822dc5d5b5a62e68547b3 X-V-ID: c4f46659-fde5-4af1-8a81-928cc0690cae X-V-CD: 0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.573, 18.0.942 definitions=2023-04-17_11:2023-04-17, 2023-04-17 signatures=0 Received: from smtpclient.apple (unknown [17.230.192.223]) by rn-mailsvcp-mmp-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.22.20230228 64bit (built Feb 28 2023)) with ESMTPSA id <0RT900U6IR4CK900@rn-mailsvcp-mmp-lapp02.rno.apple.com>; Mon, 17 Apr 2023 09:57:48 -0700 (PDT) Content-type: text/plain; charset=us-ascii MIME-version: 1.0 (Mac OS X Mail 16.0 \(3731.500.221\)) From: Christoph Paasch In-reply-to: <8C55CADC-AD2E-4C99-81C3-F083339B7CE8@gmx.de> Date: Mon, 17 Apr 2023 09:57:37 -0700 Cc: Rpm Content-transfer-encoding: quoted-printable Message-id: <57F4EB11-DE88-4D76-983D-B8BB4491A345@apple.com> References: <8C55CADC-AD2E-4C99-81C3-F083339B7CE8@gmx.de> To: Sebastian Moeller X-Mailer: Apple Mail (2.3731.500.221) Subject: Re: [Rpm] final farewell. X-BeenThere: rpm@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: revolutions per minute - a new metric for measuring responsiveness List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Apr 2023 16:57:51 -0000 Hello Sebastian, thanks for the feedback. I replied to the github issue you created. We = need to indeed spell out a bit better which part is trimmed. You mention that the reported capacity is well below the actual = capacity? Can you give more detail? (I also invite you to join the slack workspace as we have a dedicated = goresponsiveness channel there) Thanks, Christoph=20 > On Apr 16, 2023, at 3:41 AM, Sebastian Moeller via Rpm = wrote: >=20 > Mmmh, >=20 > so I recompiled and tested goresponsiveness after a fix for a cosmetic = issue was anounced (issue is fixed, thanks) >=20 > But then I stumbled over a new line in the output: >=20 > "bash-3.2$ time ./networkQuality --config mensura.cdn-apple.com --port = 443 --path /api/v1/gm/config --rpmtimeout 60 --extended-stats = --logger-filename go_networkQuality_$(date +%Y%m%d_%H%M%S) > 04-16-2023 10:02:21 UTC Go Responsiveness to = mensura.cdn-apple.com:443... > RPM: 39 (P90) > RPM: 207 (Double-Sided 10% Trimmed Mean) > Download: 80.994 Mbps ( 10.124 MBps), using 12 parallel connections. > Upload: 23.125 Mbps ( 2.891 MBps), using 12 parallel connections. > Extended Statistics: > Maximum Segment Size: 1208 > Total Bytes Retransmitted: 4268 > Retransmission Ratio: 0.54% > Total Bytes Reordered: 98003004 > Average RTT: 33.46153846153846 >=20 >=20 > real 0m12.842s > user 0m1.010s > sys 0m1.625s" >=20 >=20 > RPM: 207 (Double-Sided 10% Trimmed Mean) ? >=20 >=20 > [SM] Trimmed mean? That got my whiskers in attention position, = looking at the current draft I found the following "rationale": >=20 >=20 > "Over the timeframe of these intervals a potentially large number of = samples has been collected. These may be affected by noise in the = measurements, and outliers. Thus, to aggregate these we suggest to use a = trimmed mean at the TMP (Trimmed Mean Percentage - default to 95%) = percentile, thus providing the following numbers: TM(tcp_f), TM(tls_f), = TM(http_f), TM(http_l). >=20 > The responsiveness is then calculated as the weighted mean: >=20 > Responsiveness =3D 60000 / > (1/6*(TM(tcp_f) + TM(tls_f) + TM(http_f)) + 1/2*TM(http_s)) >=20 > This responsiveness value presents round-trips per minute (RPM)." >=20 >=20 > [SM] First a question TMP 95% what does that mean, the mean from 2.5% = to 97.5% or from 0% to 95% 0r fro 5% to 95%? I am sure there are clear = definitions but I think the draft should spell that out explicitly. >=20 >=20 > [SM] But honestly, I am not 100% convinced this is actually a = good idea... I think I understand the motivation here, boiling down a = somewhat complex issue into a single number, but "trimmed mean" seems = exceptionally un-useful for the type of distribution we are dealing with = here: > a) we have zero means to differentiate "noise in the measurements, and = outliers (which too me are more or less the same, any "outlier" not = assigned to a random noise process is data)" from veridical measurements > b) RTT/OWD distributions are IMHO almost never symmetric (there is a = hard lower limit, but more slack on the other end) > c) doing a trimmed mean of a skewed distribution will likely result in = a higher RPM estimate than using a non-trimmed mean, which given a) = seems problematic >=20 >=20 > The core of the issue is the desire for a single number... but I = believe that is non-negotiable by now (similar the use of the inverse of = the actually relevant delay data, few folks actually intuitively grasps = inverse values and how to operate on them) >=20 > [SM] Also this: >=20 > "The more probes that are sent, the more data available for = calculation. In order to generate as much data as possible, the = Responsiveness Test specifies that a client issue these probes = regularly. There is, however, a risk that on low-capacity networks the = responsiveness probes themselves will consume a significant amount of = the capacity. Because the test mandates first saturating capacity before = probing for responsiveness, we are able to accurately estimate how much = of the capacity the responsiveness probes will consume and never send = more probes than the network can handle." >=20 >=20 > [SM] seems convincing, but in my case the reported working load = stayed well below the actual capacity, so the "we are able to accurately = estimate how much of the capacity the responsiveness probes will = consume" seems a bit optimistic... >=20 >=20 > I understand posting here will not change anything, but I thought is = appropriate to make my last post on-topic, for a change ;) >=20 > Regards > Sebastian >=20 >=20 > _______________________________________________ > Rpm mailing list > Rpm@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/rpm