From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yw1-x1130.google.com (mail-yw1-x1130.google.com [IPv6:2607:f8b0:4864:20::1130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 1C03A3B2A4; Mon, 9 Jan 2023 14:56:30 -0500 (EST) Received: by mail-yw1-x1130.google.com with SMTP id 00721157ae682-4c131bede4bso127499567b3.5; Mon, 09 Jan 2023 11:56:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=sA8pAo25fzNLrG/07a8kprp3nGZF48OoAftyI8a4luE=; b=gIN2A7JeBHnuaxDo8ZNhpeu8nWl1G9A9og66MKu7+1hH6WTk72EZviTCf6R8obetJi rZOUiSjuMRyOemN1E1excKT8wG3A7LBZCOT4U6h2R4Vqmksb8p8ZYI9ktszL5E0Vz1G9 b+l1yKvy5s+ijCZ4s+dwL+MWy7NvUS/G7FKTGFBVO6dXhrJA+gKEZ10tEMZ9BsWO2RPY IYjeiFpgqQYyfmqlDWNpL0ztkvpeS4t8pC3RqJk4dp325OJ3QZaS7sBSrPdHadJOoFwd yTkpRiaspE5R4V4v2XfGbJ6dku2awVTjJ0GN9oU8h6EGNBPa/fKmVIuVs1/0paccEDlg Nw/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=sA8pAo25fzNLrG/07a8kprp3nGZF48OoAftyI8a4luE=; b=GoV1RjjPU7RlEAUFU7wbLgwCsC5zcYzQpayzFaR9kl/iCcic/P9gZ/pJIdQLv7yepo C7W2SAWUenEO+9J2VKMlP/cB79NP9Nf9RStU0z2RB+Gm5LtZ3pN/5gXH1Bg1GSCq2NgJ 7zstChUFab33FevFPSEKXWAo7M9hwYgpE3un+IZRLKNjNkNJYkse9UjtEl4wFadOVJJA 4JGcg3i24QfOczofT8bPa7D2mbFvsAaNodVakwMd/KRFdTGsULu2n6ecRKNH7zouYimw 5V+DHibprDokvpp86F1rQ6ZscrG4mjKBOeCz9w1arr5aRMCEfMq4LpAoShOvMofw64GN d8aA== X-Gm-Message-State: AFqh2krG8ItAhwMODsw4yuhUx0bpYNGVlPb1jjQRAF6KR/h2ZGI65iTA s9dS7sUFfSa2CP0AYrIWHqgFzFTXb+FojmPh3pzyFZUlEVI= X-Google-Smtp-Source: AMrXdXsjMimCjToPue6XdvJRxWQ6h+8XpuWH5I0Pr3RXZ69DuUAy/CcFBvtNPTbJzv/EoL7Agk05WNHPIoonRCvjGVY= X-Received: by 2002:a81:f0b:0:b0:473:4599:adba with SMTP id 11-20020a810f0b000000b004734599adbamr567703ywp.239.1673294189344; Mon, 09 Jan 2023 11:56:29 -0800 (PST) MIME-Version: 1.0 References: <1672786712.106922180@apps.rackspace.com> <77CCAD19-07E0-4F9E-88C1-D207CF7BF376@cable.comcast.com> <83ffc0dad19e3343e49271889369cefc@rjmcmahon.com> In-Reply-To: <83ffc0dad19e3343e49271889369cefc@rjmcmahon.com> From: dan Date: Mon, 9 Jan 2023 12:56:18 -0700 Message-ID: To: rjmcmahon Cc: "Livingood, Jason" , starlink@lists.bufferbloat.net, Rpm , bloat , libreqos Content-Type: text/plain; charset="UTF-8" Subject: Re: [Bloat] [LibreQoS] [Rpm] [EXTERNAL] Re: [Starlink] Researchers Seeking Probe Volunteers in USA X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jan 2023 19:56:30 -0000 I'm not offering a complete solution here.... I'm not so keen on speed tests. It's akin to testing your car's performance by flooring it til you hit the governor and hard breaking til you stop *while in traffic*. That doesn't demonstrate the utility of the car. Data is already being transferred, let's measure that. Doing some routine simple tests intentionally during low, mid, high congestion periods to see how the service is actually performing for the end user. You don't need to generate the traffic on a link to measure how much traffic a link can handle. And determining congestion on a service in a fairly rudimentary way would be frequent latency tests to 'known good' service ie high capacity services that are unlikely to experience congestion. There are few use cases that matche a 2 minute speed test outside of 'wonder what my internet connection can do'. And in those few use cases such as a big file download, a routine latency test is a really great measure of the quality of a service. Sure, troubleshooting by the ISP might include a full bore multi-minute speed test but that's really not useful for the consumer. Further, exposing this data to the end users, IMO, is likely better as a chart of congestion and flow durations and some scoring. ie, slice out 7-8pm, during this segment you were able to pull 427Mbps without congestion, netflix or streaming service use approximately 6% of capacity. Your service was busy for 100% of this time ( likely measuring buffer bloat ). Expressed as a pretty chart with consumer friendly language. When you guys are talking about per segment latency testing, you're really talking about metrics for operators to be concerned with, not end users. It's useless information for them. I had a woman about 2 months ago complain about her frame rates because her internet connection was 15 emm ess's and that was terrible and I needed to fix it. (slow computer was the problem, obviously) but that data from speedtest.net didn't actually help her at all, it just confused her. Running timed speed tests at 3am (Eero, I'm looking at you) is pretty pointless. Running speed tests during busy hours is a little bit harmful overall considering it's pushing into oversells on every ISP. I could talk endlessly about how useless speed tests are to end user experience. On Mon, Jan 9, 2023 at 12:20 PM rjmcmahon via LibreQoS wrote: > > User based, long duration tests seem fundamentally flawed. QoE for users > is driven by user expectations. And if a user won't wait on a long test > they for sure aren't going to wait minutes for a web page download. If > it's a long duration use case, e.g. a file download, then latency isn't > typically driving QoE. > > Not: Even for internal tests, we try to keep our automated tests down to > 2 seconds. There are reasons to test for minutes (things like phy cals > in our chips) but it's more of the exception than the rule. > > Bob > >> 0) None of the tests last long enough. > > > > The user-initiated ones tend to be shorter - likely because the > > average user does not want to wait several minutes for a test to > > complete. But IMO this is where a test platform like SamKnows, Ookla's > > embedded client, NetMicroscope, and others can come in - since they > > run in the background on some randomized schedule w/o user > > intervention. Thus, the user's time-sensitivity is no longer a factor > > and a longer duration test can be performed. > > > >> 1) Not testing up + down + ping at the same time > > > > You should consider publishing a LUL BCP I-D in the IRTF/IETF - like in > > IPPM... > > > > JL > > > > _______________________________________________ > > Rpm mailing list > > Rpm@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/rpm > _______________________________________________ > LibreQoS mailing list > LibreQoS@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/libreqos