Starlink has bufferbloat. Bad.
 help / color / mirror / Atom feed
From: Nathan Owens <nathan@nathan.io>
To: Jonathan Bennett <jonathanbennett@hackaday.com>
Cc: Dave Taht <dave.taht@gmail.com>, starlink@lists.bufferbloat.net
Subject: Re: [Starlink] insanely great waveform result for starlink
Date: Fri, 13 Jan 2023 12:30:32 -0800	[thread overview]
Message-ID: <CALjsLJswwT=ZtyQD34RCTYH=udqqCcb_s3gK-h7Onr4bjj9Fcg@mail.gmail.com> (raw)
In-Reply-To: <CALjsLJuRK8F7SbCanYR3d3FFi=1j=opXrYXsj8sBxLui_A1_HA@mail.gmail.com>


[-- Attachment #1.1.1: Type: text/plain, Size: 10110 bytes --]

Here's the data visualization for Johnathan's Data

[image: Screenshot 2023-01-13 at 12.29.15 PM.png]

You can see the path change at :12, :27, :42, :57 after the minute. Some
paths are clearly busier than others with increased loss, latency, and
jitter.

On Fri, Jan 13, 2023 at 10:09 AM Nathan Owens <nathan@nathan.io> wrote:

> I’ll run my visualization code on this result this afternoon and report
> back!
>
> On Fri, Jan 13, 2023 at 9:41 AM Jonathan Bennett via Starlink <
> starlink@lists.bufferbloat.net> wrote:
>
>> The irtt command, run with normal, light usage:
>> https://drive.google.com/file/d/1SiVCiUYnx7nDTxIVOY5w-z20S2O059rA/view?usp=share_link
>>
>> Jonathan Bennett
>> Hackaday.com
>>
>>
>> On Fri, Jan 13, 2023 at 11:26 AM Dave Taht <dave.taht@gmail.com> wrote:
>>
>>> packet caps would be nice... all this is very exciting news.
>>>
>>> I'd so love for one or more of y'all reporting such great uplink
>>> results nowadays to duplicate and re-plot the original irtt tests we
>>> did:
>>>
>>> irtt client -i3ms -d300s myclosestservertoyou.starlink.taht.net -o
>>> whatever.json
>>>
>>> They MUST have changed their scheduling to get such amazing uplink
>>> results, in addition to better queue management.
>>>
>>> (for the record, my servers are de, london, fremont, sydney, dallas,
>>> newark, atlanta, singapore, mumbai)
>>>
>>> There's an R and gnuplot script for plotting that output around here
>>> somewhere (I have largely personally put down the starlink project,
>>> loaning out mine) - that went by on this list... I should have written
>>> a blog entry so I can find that stuff again.
>>>
>>> On Fri, Jan 13, 2023 at 9:02 AM Jonathan Bennett via Starlink
>>> <starlink@lists.bufferbloat.net> wrote:
>>> >
>>> >
>>> > On Fri, Jan 13, 2023 at 6:28 AM Ulrich Speidel via Starlink <
>>> starlink@lists.bufferbloat.net> wrote:
>>> >>
>>> >> On 13/01/2023 6:13 pm, Ulrich Speidel wrote:
>>> >> >
>>> >> > From Auckland, New Zealand, using a roaming subscription, it puts me
>>> >> > in touch with a server 2000 km away. OK then:
>>> >> >
>>> >> >
>>> >> > IP address: nix six.
>>> >> >
>>> >> > My thoughts shall follow later.
>>> >>
>>> >> OK, so here we go.
>>> >>
>>> >> I'm always a bit skeptical when it comes to speed tests - they're
>>> really
>>> >> laden with so many caveats that it's not funny. I took our new work
>>> >> Starlink kit home in December to give it a try and the other day
>>> finally
>>> >> got around to set it up. It's on a roaming subscription because our
>>> >> badly built-up campus really isn't ideal in terms of a clear view of
>>> the
>>> >> sky. Oh - and did I mention that I used the Starlink Ethernet adapter,
>>> >> not the WiFi?
>>> >>
>>> >> Caveat 1: Location, location. I live in a place where the best
>>> Starlink
>>> >> promises is about 1/3 in terms of data rate you can actually get from
>>> >> fibre to the home at under half of Starlink's price. Read: There are
>>> few
>>> >> Starlink users around. I might be the only one in my suburb.
>>> >>
>>> >> Caveat 2: Auckland has three Starlink gateways close by: Clevedon
>>> (which
>>> >> is at a stretch daytrip cycling distance from here), Te Hana and
>>> Puwera,
>>> >> the most distant of the three and about 130 km away from me as the
>>> crow
>>> >> flies. Read: My dishy can use any satellite that any of these three
>>> can
>>> >> see, and then depending on where I put it and how much of the southern
>>> >> sky it can see, maybe also the one in Hinds, 840 km away, although
>>> that
>>> >> is obviously stretching it a bit. Either way, that's plenty of options
>>> >> for my bits to travel without needing a lot of handovers. Why? Easy:
>>> If
>>> >> your nearest teleport is close by, then the set of satellites that the
>>> >> teleport can see and the set that you can see is almost the same, so
>>> you
>>> >> can essentially stick with the same satellite while it's in view for
>>> you
>>> >> because it'll also be in view for the teleport. Pretty much any bird
>>> >> above you will do.
>>> >>
>>> >> And because I don't get a lot of competition from other users in my
>>> area
>>> >> vying for one of the few available satellites that can see both us and
>>> >> the teleport, this is about as good as it gets at 37S latitude. If I'd
>>> >> want it any better, I'd have to move a lot further south.
>>> >>
>>> >> It'd be interesting to hear from Jonathan what the availability of
>>> home
>>> >> broadband is like in the Dallas area. I note that it's at a lower
>>> >> latitude (33N) than Auckland, but the difference isn't huge. I notice
>>> >> two teleports each about 160 km away, which is also not too bad. I
>>> also
>>> >> note Starlink availability in the area is restricted at the moment -
>>> >> oversubscribed? But if Jonathan gets good data rates, then that means
>>> >> that competition for bird capacity can't be too bad - for whatever
>>> reason.
>>> >
>>> > I'm in Southwest Oklahoma, but Dallas is the nearby Starlink gateway.
>>> In cities, like Dallas, and Lawton where I live, there are good broadband
>>> options. But there are also many people that live outside cities, and the
>>> options are much worse. The low density userbase in rural Oklahoma and
>>> Texas is probably ideal conditions for Starlink.
>>> >>
>>> >>
>>> >> Caveat 3: Backhaul. There isn't just one queue between me and
>>> whatever I
>>> >> talk to in terms of my communications. Traceroute shows about 10 hops
>>> >> between me and the University of Auckland via Starlink. That's 10
>>> >> queues, not one. Many of them will have cross traffic. So it's a bit
>>> >> hard to tell where our packets really get to wait or where they get
>>> >> dropped. The insidious bit here is that a lot of them will be between
>>> 1
>>> >> Gb/s and 10 Gb/s links, and with a bit of cross traffic, they can all
>>> >> turn into bottlenecks. This isn't like a narrowband GEO link of a few
>>> >> Mb/s where it's obvious where the dominant long latency bottleneck in
>>> >> your TCP connection's path is. Read: It's pretty hard to tell whether
>>> a
>>> >> drop in "speed" is due to a performance issue in the Starlink system
>>> or
>>> >> somewhere between Starlink's systems and the target system.
>>> >>
>>> >> I see RTTs here between 20 ms and 250 ms, where the physical latency
>>> >> should be under 15 ms. So there's clearly a bit of buffer here along
>>> the
>>> >> chain that occasionally fills up.
>>> >>
>>> >> Caveat 4: Handovers. Handover between birds and teleports is
>>> inevitably
>>> >> associated with a change in RTT and in most cases also available
>>> >> bandwidth. Plus your packets now arrive at a new queue on a new
>>> >> satellite while your TCP is still trying to respond to whatever it
>>> >> thought the queue on the previous bird was doing. Read: Whatever your
>>> >> cwnd is immediately after a handover, it's probably not what it
>>> should be.
>>> >>
>>> >> I ran a somewhat hamstrung (sky view restricted) set of four Ookla
>>> >> speedtest.net tests each to five local servers. Average upload rate
>>> was
>>> >> 13 Mb/s, average down 75.5 Mb/s. Upload to the server of the ISP that
>>> >> Starlink seems to be buying its local connectivity from (Vocus Group)
>>> >> varied between 3.04 and 14.38 Mb/s, download between 23.33 and 52.22
>>> >> Mb/s, with RTTs between 37 and 56 ms not correlating well to rates
>>> >> observed. In fact, they were the ISP with consistently the worst
>>> rates.
>>> >>
>>> >> Another ISP (MyRepublic) scored between 11.81 and 21.81 Mb/s up and
>>> >> between 106.5 and 183.8 Mb/s down, again with RTTs badly correlating
>>> >> with rates. Average RTT was the same as for Vocus.
>>> >>
>>> >> Note the variation though: More or less a factor of two between
>>> highest
>>> >> and lowest rates for each ISP. Did MyRepublic just get lucky in my
>>> >> tests? Or is there something systematic behind this? Way too few tests
>>> >> to tell.
>>> >>
>>> >> What these tests do is establish a ballpark.
>>> >>
>>> >> I'm currently repeating tests with dish placed on a trestle closer to
>>> >> the heavens. This seems to have translated into fewer outages / ping
>>> >> losses (around 1/4 of what I had yesterday with dishy on the ground on
>>> >> my deck). Still good enough for a lengthy video Skype call with my
>>> folks
>>> >> in Germany, although they did comment about reduced video quality. But
>>> >> maybe that was the lighting or the different background as I wasn't in
>>> >> my usual spot with my laptop when I called them.
>>> >
>>> > Clear view of the sky is king for Starlink reliability. I've got my
>>> dishy mounted on the back fence, looking up over an empty field, so it's
>>> pretty much best-case scenario here.
>>> >>
>>> >>
>>> >> --
>>> >>
>>> >> ****************************************************************
>>> >> Dr. Ulrich Speidel
>>> >>
>>> >> School of Computer Science
>>> >>
>>> >> Room 303S.594 (City Campus)
>>> >>
>>> >> The University of Auckland
>>> >> u.speidel@auckland.ac.nz
>>> >> http://www.cs.auckland.ac.nz/~ulrich/
>>> >> ****************************************************************
>>> >>
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> Starlink mailing list
>>> >> Starlink@lists.bufferbloat.net
>>> >> https://lists.bufferbloat.net/listinfo/starlink
>>> >
>>> > _______________________________________________
>>> > Starlink mailing list
>>> > Starlink@lists.bufferbloat.net
>>> > https://lists.bufferbloat.net/listinfo/starlink
>>>
>>>
>>>
>>> --
>>> This song goes out to all the folk that thought Stadia would work:
>>>
>>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>>> Dave Täht CEO, TekLibre, LLC
>>>
>> _______________________________________________
>> Starlink mailing list
>> Starlink@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/starlink
>>
>

[-- Attachment #1.1.2: Type: text/html, Size: 13177 bytes --]

[-- Attachment #1.2: Screenshot 2023-01-13 at 12.29.15 PM.png --]
[-- Type: image/png, Size: 1380606 bytes --]

[-- Attachment #2: jp_starlink_irtt.pdf --]
[-- Type: application/pdf, Size: 697557 bytes --]

  reply	other threads:[~2023-01-13 20:30 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-01-04 17:26 [Starlink] the grinch meets cloudflare's christmas present Dave Taht
2023-01-04 19:20 ` [Starlink] [Rpm] " jf
2023-01-04 20:02   ` rjmcmahon
2023-01-04 21:16     ` Ulrich Speidel
2023-01-04 23:54       ` Bruce Perens
2023-01-05  2:48         ` Dave Collier-Brown
2023-01-05  3:11           ` Dick Roy
2023-01-05 11:25             ` Sebastian Moeller
2023-01-06  0:01               ` Dick Roy
2023-01-06  9:43                 ` Sebastian Moeller
2023-01-05  6:11         ` rjmcmahon
2023-01-05 11:11     ` [Starlink] [Bloat] " Sebastian Moeller
2023-01-06 16:38 ` [Starlink] [LibreQoS] " MORTON JR., AL
2023-01-06 20:38   ` [Starlink] [Rpm] " rjmcmahon
2023-01-06 20:47     ` rjmcmahon
     [not found]     ` <89D796E75967416B9723211C183A8396@SRA6>
     [not found]       ` <a082b2436e6ba7892d2de8e0dfcc5acd@rjmcmahon.com>
     [not found]         ` <3696AEA5409D4303ABCBC439727A5E40@SRA6>
     [not found]           ` <CAKJdXWBb0VxSSoGAQTe3BXFLXCHd6NSspRnXd1frK2f66SLiUw@mail.gmail.com>
     [not found]             ` <CAA93jw6B_9-WE9EEFuac+FAH-2dcULk=_3i_HfhCSVSOxyM7Eg@mail.gmail.com>
     [not found]               ` <CA+Ld8r8hR8KF35Yv7A3hb1QvC9v9ka2Nh2J=HEm0XhPfvAAcag@mail.gmail.com>
     [not found]                 ` <CAKJdXWC+aEy1b3vB-FFd+tnfT+Ni5O9bZ+p4kkhj-FzMPVGGcQ@mail.gmail.com>
     [not found]                   ` <CAA93jw4DcBhA8CevRQoMbzjO-3Jt+emr+xvnJ-hUGkT+n0KJzg@mail.gmail.com>
     [not found]                     ` <CH0PR02MB79800FF2E40CE037D6802D71D3FD9@CH0PR02MB7980.namprd02.prod.outlook.com>
     [not found]                       ` <CAKJdXWDOFbzsam2C_24e9DLkc18ed4uhV51hOKVjDipk1Uhc2g@mail.gmail.com>
2023-01-13  4:08                         ` [Starlink] insanely great waveform result for starlink Dave Taht
2023-01-13  4:26                           ` Jonathan Bennett
2023-01-13  5:13                             ` Ulrich Speidel
2023-01-13  5:25                               ` Dave Taht
2023-01-13 12:27                               ` Ulrich Speidel
2023-01-13 17:02                                 ` Jonathan Bennett
2023-01-13 17:26                                   ` Dave Taht
2023-01-13 17:41                                     ` Jonathan Bennett
2023-01-13 18:09                                       ` Nathan Owens
2023-01-13 20:30                                         ` Nathan Owens [this message]
2023-01-13 20:37                                           ` Dave Taht
2023-01-13 21:24                                             ` Nathan Owens
2023-01-13 20:49                                           ` Dave Taht
2023-01-13 21:25                                             ` Luis A. Cornejo
2023-01-13 21:30                                               ` Nathan Owens
2023-01-13 22:09                                                 ` Jonathan Bennett
2023-01-13 22:30                                                   ` Luis A. Cornejo
2023-01-13 22:32                                                   ` Dave Taht
2023-01-13 22:36                                                     ` Luis A. Cornejo
2023-01-13 22:42                                                     ` Jonathan Bennett
2023-01-13 22:49                                                       ` Dave Taht
2023-01-13 23:44                                                   ` Dave Taht
     [not found]                                                     ` <CALjsLJv5cbfHfkxqHnbjxoVHczspYvxc_jrshzs1CpHLEDWyew@mail.gmail.com>
2023-01-14 14:20                                                       ` Nathan Owens
2023-01-14 15:53                                                         ` Dave Taht
2023-01-14 16:33                                                           ` Dave Taht
2023-01-14 15:52                                                       ` Dave Taht
2023-01-13 20:25                                     ` Pat Jensen
2023-01-13 20:40                                       ` Dave Taht
2023-01-13 22:51                                   ` Ulrich Speidel

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.bufferbloat.net/postorius/lists/starlink.lists.bufferbloat.net/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CALjsLJswwT=ZtyQD34RCTYH=udqqCcb_s3gK-h7Onr4bjj9Fcg@mail.gmail.com' \
    --to=nathan@nathan.io \
    --cc=dave.taht@gmail.com \
    --cc=jonathanbennett@hackaday.com \
    --cc=starlink@lists.bufferbloat.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox