From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 72FD23B29D; Mon, 13 Mar 2023 06:02:24 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1678701736; i=moeller0@gmx.de; bh=bN7WUQynfqKAKc5FTJpZnCW2oiF6RWvnRBoRgk35sys=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=kEp+qNorelh9e1wnQhp7pH0pkzQDVCCSSB4Lxu/zYoEK3R3UVdBLsm9pDVlRmHlq0 MfU9tzRtExwpkTGSX6FCxzyJxLm6tMPtO9Xe/hP3sXfVncKMmlFzuQjqu8C3eK1KnQ rpPWFFbEYAo8VNpqBORZ6VyO7zvQuBmUQ8aRlYSTi+YaqL1D2OR5RxeH2Ao4qHOkey aADUq1RtmZ+YTqXwfg7qs7B6DB8VRilGcQwoalKnLURQP1aZChlyNRdKEI/hc8vI7I 3ScON7ZerN/x77l9LR3Bc9f3KE6d2UbMTMKMwXdtrVvgdAlLrdXnl/bIapI0WqAT2M 1zA2/z2+VrQmQ== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from smtpclient.apple ([77.8.69.99]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1Mzyyk-1qUytC1tSw-00x1z6; Mon, 13 Mar 2023 11:02:16 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) From: Sebastian Moeller In-Reply-To: Date: Mon, 13 Mar 2023 11:02:15 +0100 Cc: rjmcmahon , Dave Taht via Starlink , Rpm , "Livingood, Jason" , libreqos , bloat Content-Transfer-Encoding: quoted-printable Message-Id: References: <1672786712.106922180@apps.rackspace.com> <77CCAD19-07E0-4F9E-88C1-D207CF7BF376@cable.comcast.com> <83ffc0dad19e3343e49271889369cefc@rjmcmahon.com> To: dan X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Provags-ID: V03:K1:bqtQGmhevgSc31CHGoHQMrlGPCXCZwCiO2TkPwPC5N00nkDCxBZ a2qWfiiy3XG6SelVyA7mPf4JoDCmUYr72UjY6nvrTaxznuVdcxueKDLa/ghy474ary9vDU4 3DBix7R1dit0hBKAqdrjPFR2OeLRNQVZSthtC8AGZJO2gzFxtuh10PrKOqofMuntqUjUZ/Q ZcvJRXjAmByEhy0HaEomg== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:8adNoTMqkL8=;VX8bZ4rRNJMVjHsIVthj3zhecEK NkuFCvdfbsDYMZftHiPkBlYUHqJJnvdQG1qbTQep1XsriI/4rqjcGZ6/c8oGnfrHDrneBNLlP B0JCBl5Mf53M8lU0rGsoMx+/56uu4C7duGpK/Sg7RhJGJ5qyXD8zFpwJDSHw3I4GIYTR93WQ7 gJR2t1029xqUPHj0cnQ7xeBs5g7KwH/rPPUwLv8e0643JrL+pYLK2VcSjYHW99fuTubH2CTIO vh1JUVotNnpekT6yZdXrsmakgCVfEh/InWqmWfsABXcBeOg/714NEDvscK22eepDXEHGRyrNF Lig2igRFv0INvFlAQriGBn8RR1l/VXCCRJFEz5GoyPhRkPKqpydWeoAnWqbCWaPeYF28QkchK DiSX1nhJ0ztsQR/pgSy189OpTjNcaATfyX4TOSQvBxBYXrPp+GP0BQgmvbfHRzGYGK2/6oMSP lpFqqwYomZgq3R+zdH7/XOzLfjJ4muk4vMq4SAemQWRQnQMB4sJnn0CEhgX0jWbQUBwX2DqpW EJXw4hZPV6BzVGWWWIf5yoRvnEAFUUGBIwt8JcfQPsSgakSJ0a2DxfBc+HvVWmMxX/Cui0fV/ Nt505WT6ljHBRNHX6jk5e9G9OM4L4t8V+1ESiHJ17WJNMhW1mJ1ZKh+ASnBxro/R3B2yktz9w YzfZKGHn1xnOOuC4Ic0Gpd7iYAotCwpceagrem70tLR18wHKkdA3M2FsOcz+w+1a97FtKedCW mJVkpUfKZHa0fAYek5trwDzjA+xsDSumqUkQoNf25br6EIoOkS3gTz1LsJ1cbVJkPCVmHltVK 7MI/3U7EAyq0/p7hRn671udfAqC9dQ5KIxPnd4UdAgzkhyTUaQ2LyFR00zZ3G5+uRx5ktgpXt KHAgd0qwWiNFiuVwAx/xQkMpbv7GDeZM5sU1gPH4zHkp0enYnUFId4yFSZHNLL1iz8pkocrvK lhcg3udFCMKtqMWhgKhDWspuK/M= Subject: Re: [Bloat] [Rpm] [LibreQoS] [EXTERNAL] Re: [Starlink] Researchers Seeking Probe Volunteers in USA X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Mar 2023 10:02:24 -0000 Hi Dan, > On Jan 9, 2023, at 20:56, dan via Rpm = wrote: >=20 > I'm not offering a complete solution here.... I'm not so keen on > speed tests. It's akin to testing your car's performance by flooring > it til you hit the governor and hard breaking til you stop *while in > traffic*. That doesn't demonstrate the utility of the car. >=20 > Data is already being transferred, let's measure that. =20 [SM] For a home link that means you need to measure on the = router, as end-hosts will only ever see the fraction of traffic they = sink/source themselves... > Doing some > routine simple tests intentionally during low, mid, high congestion > periods to see how the service is actually performing for the end > user. [SM] No ISP I know of publishes which periods are low, mid, high = congestion so end-users will need to make some assumptions here (e.g. by = looking at per day load graphs of big traffic exchanges like DE-CIX here = https://www.de-cix.net/en/locations/frankfurt/statistics ) > You don't need to generate the traffic on a link to measure how > much traffic a link can handle. [SM] OK, I will bite, how do you measure achievable throughput = without actually generating it? Packet-pair techniques are notoriously = imprecise and have funny failure modes. > And determining congestion on a > service in a fairly rudimentary way would be frequent latency tests to > 'known good' service ie high capacity services that are unlikely to > experience congestion. [SM] Yes, that sort of works, see e.g. = https://github.com/lynxthecat/cake-autorate for a home-made approach by = non-networking people to estimate whether the immediate load is at = capacity or not, and using that information to control a traffic shaper = to "bound" latency under load. >=20 > There are few use cases that matche a 2 minute speed test outside of > 'wonder what my internet connection can do'. [SM] I would have agreed some months ago, but ever since the = kids started to play more modern games than tetris/minecraft long = duration multi-flow downloads have become a staple in our networking. = OK, noone really cares about the intra-flow latency of these download = flows, but we do care that the rest of our traffic stays responsive. > And in those few use > cases such as a big file download, a routine latency test is a really > great measure of the quality of a service. Sure, troubleshooting by > the ISP might include a full bore multi-minute speed test but that's > really not useful for the consumer. [SM] I mildly disagree, if it is informative for the ISP's = technicians it is also informative for the end-customers; not all ISPs = are so enlightened that they pro-actively solve issues for their = customers (but some are!) so occasionally it helps to be able to do such = diagnostic measurements one-self.=20 >=20 > Further, exposing this data to the end users, IMO, is likely better as > a chart of congestion and flow durations and some scoring. ie, slice > out 7-8pm, during this segment you were able to pull 427Mbps without > congestion, netflix or streaming service use approximately 6% of > capacity. Your service was busy for 100% of this time ( likely > measuring buffer bloat ). Expressed as a pretty chart with consumer > friendly language. [SM] Sounds nice. > When you guys are talking about per segment latency testing, you're > really talking about metrics for operators to be concerned with, not > end users. It's useless information for them. [SM] Well is it really useless? If I know the to be expected = latency-under-load increase I can eye-ball e.h. how far away a server I = can still interact with in a "snappy" way.=20 > I had a woman about 2 > months ago complain about her frame rates because her internet > connection was 15 emm ess's and that was terrible and I needed to fix > it. (slow computer was the problem, obviously) but that data from > speedtest.net didn't actually help her at all, it just confused her. [SM The solution to lack of knowledge, IMHO should be to teach = people what they need to know, not hide information that could be = mis-interpreted (because that applies to all information). >=20 > Running timed speed tests at 3am (Eero, I'm looking at you) is pretty > pointless. =20 [SM] I would argue that this is likely a decent period to = establish baseline values for uncongested conditions (that is = uncongested by other traffic sources than the measuring network). > Running speed tests during busy hours is a little bit > harmful overall considering it's pushing into oversells on every ISP. [SM] Oversell, or under-provisioning, IMHO is a viable technique = to reduce costs, but it is not an excuse for short-shifting one's = customers; if an ISP advertised and sells X Mbps, he/she needs to be = willing to actually deliver independent on how "active" a given shared = segment is. By this I do NOT mean that the contracted speed needs to be = available 100% at all times, but that there is a reasonably high chance = of getting close to the contracted rates. If that means either = increasing prices to match cost targets or reduce maximally advertised = contracted rates, or going to completely different kind of contracts = (say, 1/Nth of a Gigabit link with equitable sharing among all N users = on the link). Under-provisioning is fine as an optimization method to = increase profitability, but IMHO no excuse on not delivering on one's = contract. > I could talk endlessly about how useless speed tests are to end user = experience. [SM] My take on this is a satisfied customer is unlikely to make = a big fuss. And delivering great responsiveness is a great way for an = ISP to make end-customers care less about achievable throughput. Yes, = some will, e.g. gamers that insist on loading multi-gigabit updates just = before playing instead of over-night (a strategy I have some sympathy = for, shutting down power consumers fully over night instead of wasting = watts on "stand-by" of some sort is a more reliable way to save = power/cost). Regards Sebastian >=20 >=20 > On Mon, Jan 9, 2023 at 12:20 PM rjmcmahon via LibreQoS > wrote: >>=20 >> User based, long duration tests seem fundamentally flawed. QoE for = users >> is driven by user expectations. And if a user won't wait on a long = test >> they for sure aren't going to wait minutes for a web page download. = If >> it's a long duration use case, e.g. a file download, then latency = isn't >> typically driving QoE. >>=20 >> Not: Even for internal tests, we try to keep our automated tests down = to >> 2 seconds. There are reasons to test for minutes (things like phy = cals >> in our chips) but it's more of the exception than the rule. >>=20 >> Bob >>>> 0) None of the tests last long enough. >>>=20 >>> The user-initiated ones tend to be shorter - likely because the >>> average user does not want to wait several minutes for a test to >>> complete. But IMO this is where a test platform like SamKnows, = Ookla's >>> embedded client, NetMicroscope, and others can come in - since they >>> run in the background on some randomized schedule w/o user >>> intervention. Thus, the user's time-sensitivity is no longer a = factor >>> and a longer duration test can be performed. >>>=20 >>>> 1) Not testing up + down + ping at the same time >>>=20 >>> You should consider publishing a LUL BCP I-D in the IRTF/IETF - like = in >>> IPPM... >>>=20 >>> JL >>>=20 >>> _______________________________________________ >>> Rpm mailing list >>> Rpm@lists.bufferbloat.net >>> https://lists.bufferbloat.net/listinfo/rpm >> _______________________________________________ >> LibreQoS mailing list >> LibreQoS@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/libreqos > _______________________________________________ > Rpm mailing list > Rpm@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/rpm