Here's a bit of a question to you all. See what you make of it.
I've been thinking a bit about the latencies we see in the Starlink
network. This is why this list exist (right, Dave?). So what do we know?
1) We know that RTTs can be in the 100's of ms even in what appear to be
bent-pipe scenarios where the physical one-way path should be well under
3000 km, with physical RTT under 20 ms.
2) We know from plenty of traceroutes that these RTTs accrue in the
Starlink network, not between the Starlink handover point (POP) to the
Internet.
3) We know that they aren't an artifact of the Starlink WiFi router (our
traceroutes were done through their Ethernet adaptor, which bypasses the
router), so they must be delays on the satellites or the teleports.
4) We know that processing delay isn't a huge factor because we also see
RTTs well under 30 ms.
5) That leaves queuing delays.
This issue has been known for a while now. Starlink have been innovating
their heart out around pretty much everything here - and yet, this
bufferbloat issue hasn't changed, despite Dave proposing what appears to
be an easy fix compared to a lot of other things they have done. So what
are we possibly missing here?
Going back to first principles: The purpose of a buffer on a network
device is to act as a shock absorber against sudden traffic bursts. If I
want to size that buffer correctly, I need to know at the very least
(paraphrasing queueing theory here) something about my packet arrival
process.
If I look at conventional routers, then that arrival process involves
traffic generated by a user population that changes relatively slowly:
WiFi users come and go. One at a time. Computers in a company get turned
on and off and rebooted, but there are no instantaneous jumps in load -
you don't suddenly have a hundred users in the middle of watching
Netflix turning up that weren't there a second ago. Most of what we know
about Internet traffic behaviour is based on this sort of network, and
this is what we've designed our queuing systems around, right?
Observation: Starlink potentially breaks that paradigm. Why? Imagine a
satellite X handling N users that are located closely together in a
fibre-less rural town watching a range of movies. Assume that N is
relatively large. Say these users are currently handled through ground
station teleport A some distance away to the west (bent pipe with
switching or basic routing on the satellite). X is in view of both A and
the N users, but with X being a LEO satellite, that bliss doesn't last.
Say X is moving to the (south- or north-)east and out of A's range.
Before connection is lost, the N users migrate simultaneously to a new
satellite Y that has moved into view of both A and themselves. Y is
doing so from the west and is also catering to whatever users it can see
there, and let's suppose has been using A for a while already.
The point is that the user load on X and Y from users other than our N
friends could be quite different. E.g., one of them could be over the
ocean with few users, the other over countryside with a lot of
customers. The TCP stacks of our N friends are (hopefully) somewhat
adapted to the congestion situation on X with their cwnds open to
reasonable sizes, but they are now thrown onto a completely different
congestion scenario on Y. Similarly, say that Y had less than N users
before the handover. For existing users on Y, there is now a huge surge
of competing traffic that wasn't there a second ago - surging far faster
than we would expect this to happen in a conventional network because
there is no slow start involved.
This seems to explain the huge jumps you see on Starlink in TCP goodput
over time.
But could this be throwing a few spanners into the works in terms of
queuing? Does it invalidate what we know about queues and queue
management? Would surges like these justify larger buffers?
--
****************************************************************
Dr. Ulrich Speidel
School of Computer Science
Room 303S.594 (City Campus)
The University of Auckland
u.speidel@auckland.ac.nz
http://www.cs.auckland.ac.nz/~ulrich/
****************************************************************
_______________________________________________
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink