[Starlink] [EXTERNAL] Re: ookla's latest speedtest app does latency under load

Thu May 19 10:06:16 EDT 2022

From: Sebastian Moeller <moeller0 at gmx.de>
Date: Friday, May 13, 2022 at 10:50
To: Dave Taht <dave.taht at gmail.com>
Cc: "Livingood, Jason" <Jason_Livingood at cable.comcast.com>, "starlink at lists.bufferbloat.net" <starlink at lists.bufferbloat.net>, Brennen Smith <brennen at ookla.com>
Subject: [EXTERNAL] Re: [Starlink] ookla's latest speedtest app does latency under load

Hi Dave,

On May 13, 2022, at 16:15, Dave Taht <dave.taht at gmail.com<mailto:dave.taht at gmail.com>> wrote:

"Was not the miracle of the juniper berries enough?" - Monty Python, Life of brian  -

I've had a whole day of warm mellow glow (and I did a little SEO on slashdot, reddit and elsewhere) and have a few comments here and there also.

I love the symbols for idle, up and down, and especially, jitter,

Nitpick, the perfect sine wave would have zero jitter ;)

and like that the colors of those match between the references to them. Is there a unicode equivalent for these? creating a unified symbolic language for these concepts would be great.

on the other hand, indicating that the latency numbers are meaningful in some respect, rather than using white for them, might be useful. +30m sover baseline = green, +60ms over baseline = yellow +150ms = red, and numbers like 400+ms should turn the entire application into a mutant morphed dali painting and emit a raspberry sound. This color scheme is similar to http://www.dslreports.com/speedtest/results/bufferbloat?up=1<https://urldefense.com/v3/__http:/www.dslreports.com/speedtest/results/bufferbloat?up=1__;!!CQl3mcHX2A!BL0WciQlv8iyTBeG4H22uqWAAom4L60HHLkAcogzp3dywlgXp1Xne8ZwDjY9X0aDhPyPWo2IP9OgDyPRE876pUge$> - being an extremist, I felt that "red" at 1s was kind of late, especially because the test didn't run long enough to find delays much past 4s (which certainly exist). 150ms is at the outer limit for quality voip.

IIRC, according to the ITU these 150ms are one-way mouth-to-ear delay, which would translate into an RTT of up to 300ms... That said, I am not sure that acceptable standards today are the same as when this number was deduced....
Side note, the national regulator in Germany finally published its first recommendation on the minimum service guarantee people living in Germany have a right to, IIRC >= 10/1.3 Mbps and RTT <= 150ms (these values are probably always in reference the the regulator's official speedtest platform breitbandmessung.de<https://urldefense.com/v3/__http:/breitbandmessung.de__;!!CQl3mcHX2A!BL0WciQlv8iyTBeG4H22uqWAAom4L60HHLkAcogzp3dywlgXp1Xne8ZwDjY9X0aDhPyPWo2IP9OgDyPRE3diXt2f$>). This is probably based on the same ITU numbers, the only good it does essentially is moving GEO satellites out of the ring of admissible access technologies of last resort, but LEO clearly can live comfortably within that generous 150ms limit...

semantically, I love the word "responsiveness", and would like to be using that in a vague sense as here. Elsewhere, as part of the rpm concept it's getting conflated....

In the explanation of results there is some language regarding "ping" that I would soften and explain that if you are getting high latency and have no packet loss, that's probably the root of the problem,andr if you are getting some packet loss but low latency, that it's good.

On Fri, May 13, 2022 at 6:47 AM Livingood, Jason <Jason_Livingood at comcast.com<mailto:Jason_Livingood at comcast.com>> wrote:
Hi Brennen – on #3, do you mean that (if we look for example at https://www.speedtest.net/result/i/5112435305<https://urldefense.com/v3/__https:/www.speedtest.net/result/i/5112435305__;!!CQl3mcHX2A!BL0WciQlv8iyTBeG4H22uqWAAom4L60HHLkAcogzp3dywlgXp1Xne8ZwDjY9X0aDhPyPWo2IP9OgDyPRE0H7K-fR$>) that 419ms is the mean/average? And when we see a result page like https://www.speedtest.net/result/i/5112435305 how<https://urldefense.com/v3/__https:/www.speedtest.net/result/i/5112435305**Ahow__;wqA!!CQl3mcHX2A!BL0WciQlv8iyTBeG4H22uqWAAom4L60HHLkAcogzp3dywlgXp1Xne8ZwDjY9X0aDhPyPWo2IP9OgDyPRE2nK1chv$> do we expand/see detailed results that have low & high*? And is high max or p99/p98/p95?

And will this be added to the MacOS client soon?

Thanks for Ookla doing this BTW!

Jason

* If that is not possible in the current version, please suggest it as a feature request for your team’s next code sprint.

I have personally managed to avoid the concept of sprinting in favor of a long, consistent slog. Judging from brennen's first (excellent) post here, I am not sure if they have adopted agile.

https://www.ookla.com/articles/maximized-speed-non-gigabit-internet-connection<https://urldefense.com/v3/__https:/www.ookla.com/articles/maximized-speed-non-gigabit-internet-connection__;!!CQl3mcHX2A!BL0WciQlv8iyTBeG4H22uqWAAom4L60HHLkAcogzp3dywlgXp1Xne8ZwDjY9X0aDhPyPWo2IP9OgDyPRE5_CUdq6$>

From: Starlink <starlink-bounces at lists.bufferbloat.net> on behalf of Brennen Smith <brennen at ookla.com>
Date: Thursday, May 12, 2022 at 19:49
To: Dave Taht <dave.taht at gmail.com>
Cc: "starlink at lists.bufferbloat.net" <starlink at lists.bufferbloat.net>
Subject: Re: [Starlink] ookla's latest speedtest app does latency under load

Hey all,

Here's some details about the final figures:

1. Loaded latency is simply an extension of the latency stage - same methodology and medium

2. We use a "warmed up" TCP connection for latency

3. We use the interquartile mean for the "displayed value"

Awesome seeing those numbers in the wild and looking forward to bringing more awareness to this issue.

Cheers,

Brennen Smith VP Technology

(206) 739-0807 | brennen at ookla.com
linkedin.com/in/brennensmith

On Thu, May 12, 2022 at 4:18 PM Dave Taht <dave.taht at gmail.com> wrote:

Yea. peak 2500ms latency on download, 600ms on upload, that's about right. :/ The upload figure appears to be a bit lower (better) than what I measured 1 year ago, here (for the 30 or so new subscribers on this list, this was why I started it): https://docs.google.com/document/d/1puRjUVxJ6cCv-rgQ_zn-jWZU9ae0jZbFATLf4PQKblM/edit<https://urldefense.com/v3/__https:/docs.google.com/document/d/1puRjUVxJ6cCv-rgQ_zn-jWZU9ae0jZbFATLf4PQKblM/edit__;!!CQl3mcHX2A!BL0WciQlv8iyTBeG4H22uqWAAom4L60HHLkAcogzp3dywlgXp1Xne8ZwDjY9X0aDhPyPWo2IP9OgDyPRE9eEdjYk$>

I don't know if ookla are throwing out arp related stuff, or using the 99th percentile to calculate the final figure. I hope to learn more about their calculations in the coming weeks.

I do keep hoping we can get more updated flent data over longer intervals than 20s. With starlink changing their allocation scheme every 15s, what number do you pick?

And like I said, some packet caps of this app would be good, too.

On Thu, May 12, 2022 at 4:07 PM Nathan Owens <nathan at nathan.io> wrote:

Here's a test I did today: https://www.speedtest.net/my-result/i/5112435305

[cid:image001.jpg at 01D86B68.13E11AC0]

On Thu, May 12, 2022 at 4:04 PM Dave Taht <dave.taht at gmail.com> wrote:

Ookla has just put out an ios and android app that also continuously
measures latency under load.

I'm interested in calibrating the results of that vs a vs flent
benchmarks verses starlink. Can someone here give this a shot? More
details here:

https://www.ookla.com/articles/introducing-loaded-latency<https://urldefense.com/v3/__https:/www.ookla.com/articles/introducing-loaded-latency__;!!CQl3mcHX2A!BL0WciQlv8iyTBeG4H22uqWAAom4L60HHLkAcogzp3dywlgXp1Xne8ZwDjY9X0aDhPyPWo2IP9OgDyPRE5yWysjn$>

IDEALLY, this would be over wifi -> starlink , with a packet capture
in the middle. But I'd settle for a few test results from starlink
folk of this new app, first.

--
FQ World Domination pending: https://blog.cerowrt.org/post/state_of_fq_codel/
Dave Täht CEO, TekLibre, LLC
_______________________________________________
Starlink mailing list
Starlink at lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink<https://urldefense.com/v3/__https:/lists.bufferbloat.net/listinfo/starlink__;!!CQl3mcHX2A!BL0WciQlv8iyTBeG4H22uqWAAom4L60HHLkAcogzp3dywlgXp1Xne8ZwDjY9X0aDhPyPWo2IP9OgDyPREzG1gv_N$>

--

FQ World Domination pending: https://blog.cerowrt.org/post/state_of_fq_codel/

Dave Täht CEO, TekLibre, LLC

This email, its contents and attachments contain information from Ziff Davis, Inc. and/or its affiliates which may be privileged, confidential or otherwise protected from disclosure. The information is intended to be for the addressee(s) only. If you are not an addressee, any disclosure, copy, distribution or use of the contents of this message is prohibited. If you have received this email in error, please notify the sender by reply email and delete the original message and any copies.

--
FQ World Domination pending: https://blog.cerowrt.org/post/state_of_fq_codel/
Dave Täht CEO, TekLibre, LLC
_______________________________________________
Starlink mailing list
Starlink at lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink<https://urldefense.com/v3/__https:/lists.bufferbloat.net/listinfo/starlink__;!!CQl3mcHX2A!BL0WciQlv8iyTBeG4H22uqWAAom4L60HHLkAcogzp3dywlgXp1Xne8ZwDjY9X0aDhPyPWo2IP9OgDyPREzG1gv_N$>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/private/starlink/attachments/20220519/3b3774fc/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 82379 bytes
Desc: image001.jpg
URL: <https://lists.bufferbloat.net/private/starlink/attachments/20220519/3b3774fc/attachment-0001.jpg>