[Starlink] Starlink and bufferbloat status?

Sat Jul 17 14:36:35 EDT 2021

Just to clarify, Starlink does NOT do inter-satellite data transfer. Each satellite is a "bent-pipe" which just reflects data transmitted to it back down.

NY-Tokyo isn't line of sight, even at LEO. This is why GEO satellites are used.

Iridium had the ability to inter-satellite routing. However, still, multiple hops are needed, and tracking all the other satellites in view is not easy, and stable routing management would be non-trivial indeed. (not counting the much lower power/bit available between battery powered satellites that aren't in the sun half the time. That was Iridium's basic technical issue - battery storage. 60 satellites were barely tractable for routing.

There seems to be a lot of imagination by the credulous technical community going into dreaming about what Starlink actually can achieve. Those who have actually worked with satellite systems know there is no magical genius that Musk has. He just launches lots of simple orbiting "mirrors" (active repeaters) close to the Earth's surface. Pragmatic engineering, exploiting better digital signal processing, lower power digital systems, better antenna systems that use MIMO (or phased array, or whatever you want to call it).

The reason it is all so hush-hush, to this old engineer, anyway, is the usual attempt to exploit the Wizard of Oz effect. (Trade secrets don't work well, so Musk would have filed patents worldwide if he had any novel special engineering magic). Musk is great at exploiting publicity to get people to think there's an all-powerful wizard behind the scenes. There isn't. This is like the belief that there is lots of highly classified technology that scientists don't understand unless cleared. There really isn't that much - the classification is about hiding all the weaknesses in military systems. A good thing, but not proof of mysterious magical science beyond commercial practice. Star Wars was this kind of exploitation of Magical Thinking in the public. A marketing blitz to con people who projected their imagination on a fairly mundane engineering project with lots of weaknesses.

Now I'm happy to see Musk lose money hand over fist to show us how bent-pipe LEO systems can work somewhat reliably. What else should a billionaire do to make his money useful? We learn stuff that way. And issues like bufferbloat get rediscovered, and fixed, if his team pays attention. This is good in the long run.

But customers who bought Tesla's expecting Autopilot to become self-driving? Or people who bought Tesla stock thinking no other companies could compete? They are waiting for something that is not real.

On Friday, July 16, 2021 1:35pm, "Nathan Owens" <nathan at nathan.io> said:

> The other case where they could provide benefit is very long distance paths
> --- NY to Tokyo, Johannesburg to London, etc... but presumably at high
> cost, as the capacity will likely be much lower than submarine cables.
> 
> On Fri, Jul 16, 2021 at 10:31 AM Mike Puchol <mike at starlink.sx> wrote:
> 
>> Satellite optical links are useful to extend coverage to areas where you
>> don’t have gateways - thus, they will introduce additional latency
>> compared
>> to two space segment hops (terminal to satellite -> satellite to gateway).
>> If you have terminal to satellite, two optical hops, then final satellite
>> to gateway, you will have more latency, not less.
>>
>> We are being “sold” optical links for what they are not IMHO.
>>
>> Best,
>>
>> Mike
>> On Jul 16, 2021, 19:29 +0200, Nathan Owens <nathan at nathan.io>, wrote:
>>
>> > As there are more satellites, the up down time will get closer to 4-5ms
>> rather then the ~7ms you list
>>
>> Possibly, if you do steering to always jump to the lowest latency
>> satellite.
>>
>> > with laser relays in orbit, and terminal to terminal routing in orbit,
>> there is the potential for the theoretical minimum to tend lower
>> Maybe for certain users really in the middle of nowhere, but I did the
>> best-case math for "bent pipe" in Seattle area, which is as good as it gets.
>>
>> On Fri, Jul 16, 2021 at 10:24 AM David Lang <david at lang.hm> wrote:
>>
>>> hey, it's a good attitude to have :-)
>>>
>>> Elon tends to set 'impossible' goals, miss the timeline a bit, and come
>>> very
>>> close to the goal, if not exceed it.
>>>
>>> As there are more staellites, the up down time will get closer to 4-5ms
>>> rather
>>> then the ~7ms you list, and with laser relays in orbit, and terminal to
>>> terminal
>>> routing in orbit, there is the potential for the theoretical minimum to
>>> tend
>>> lower, giving some headroom for other overhead but still being in the 20ms
>>> range.
>>>
>>> David Lang
>>>
>>>   On Fri, 16 Jul 2021, Nathan Owens wrote:
>>>
>>> > Elon said "foolish packet routing" for things over 20ms! Which seems
>>> crazy
>>> > if you do some basic math:
>>> >
>>> >   - Sat to User Terminal distance: 550-950km air/vacuum: 1.9 - 3.3ms
>>> >   - Sat to GW distance: 550-950km air/vacuum: 1.9 - 3.3ms
>>> >   - GW to PoP Distance: 50-800km fiber: 0.25 - 4ms
>>> >   - PoP to Internet Distance: 50km fiber: 0.25 - 0.5ms
>>> >   - Total one-way delay: 4.3 - 11.1ms
>>> >   - Theoretical minimum RTT: 8.6ms - 22.2ms, call it 15.4ms
>>> >
>>> > This includes no transmission delay, queuing delay,
>>> > processing/fragmentation/reassembly/etc, and no time-division
>>> multiplexing.
>>> >
>>> > On Fri, Jul 16, 2021 at 10:09 AM David Lang <david at lang.hm> wrote:
>>> >
>>> >> I think it depends on if you are looking at datacenter-to-datacenter
>>> >> latency of
>>> >> home to remote datacenter latency :-)
>>> >>
>>> >> my rule of thumb for cross US ping time has been 80-100ms latency (but
>>> >> it's been
>>> >> a few years since I tested it).
>>> >>
>>> >> I note that an article I saw today said that Elon is saying that
>>> latency
>>> >> will
>>> >> improve significantly in the near future, that up/down latency is ~20ms
>>> >> and the
>>> >> additional delays pushing it to the 80ms range are 'stupid packet
>>> routing'
>>> >> problems that they are working on.
>>> >>
>>> >> If they are still in that level of optimization, it doesn't surprise me
>>> >> that
>>> >> they haven't really focused on the bufferbloat issue, they have more
>>> >> obvious
>>> >> stuff to fix first.
>>> >>
>>> >> David Lang
>>> >>
>>> >>
>>> >>   On Fri, 16 Jul 2021, Wheelock, Ian wrote:
>>> >>
>>> >>> Date: Fri, 16 Jul 2021 10:21:52 +0000
>>> >>> From: "Wheelock, Ian" <ian.wheelock at commscope.com>
>>> >>> To: David Lang <david at lang.hm>, David P. Reed <dpreed at deepplum.com>
>>> >>> Cc: "starlink at lists.bufferbloat.net" <starlink at lists.bufferbloat.net>
>>> >>> Subject: Re: [Starlink] Starlink and bufferbloat status?
>>> >>>
>>> >>> Hi David
>>> >>> In terms of the Latency that David (Reed) mentioned for California to
>>> >> Massachusetts of about 17ms over the public internet, it seems a bit
>>> faster
>>> >> than what I would expect. My own traceroute via my VDSL link shows 14ms
>>> >> just to get out of the operator network.
>>> >>>
>>> >>> https://www.wondernetwork.com  is a handy tool for checking
>>> geographic
>>> >> ping perf between cities, and it shows a min of about 66ms for pings
>>> >> between Boston and San Diego
>>> >> https://wondernetwork.com/pings/boston/San%20Diego (so about 33ms for
>>> >> 1-way transfer).
>>> >>>
>>> >>> Distance wise this is about 4,100 KM (2,500 M), and @2/3 speed of
>>> light
>>> >> (through a pure fibre link of that distance) the propagation time is
>>> just
>>> >> over 20ms. If the network equipment between the Boston and San Diego is
>>> >> factored in, with some buffering along the way, 33ms does seem quite
>>> >> reasonable over the 20ms for speed of light in fibre for that 1-way
>>> transfer
>>> >>>
>>> >>> -Ian Wheelock
>>> >>>
>>> >>> From: Starlink <starlink-bounces at lists.bufferbloat.net> on behalf of
>>> >> David Lang <david at lang.hm>
>>> >>> Date: Friday 9 July 2021 at 23:59
>>> >>> To: "David P. Reed" <dpreed at deepplum.com>
>>> >>> Cc: "starlink at lists.bufferbloat.net" <starlink at lists.bufferbloat.net>
>>> >>> Subject: Re: [Starlink] Starlink and bufferbloat status?
>>> >>>
>>> >>> IIRC, the definition of 'low latency' for the FCC was something like
>>> >> 100ms, and Musk was predicting <40ms. roughly competitive with
>>> landlines,
>>> >> and worlds better than geostationary satellite (and many
>>> >>> External (mailto:david at lang.hm)
>>> >>>
>>> >>
>>> https://shared.outlook.inky.com/report?id=Y29tbXNjb3BlL2lhbi53aGVlbG9ja0Bjb21tc2NvcGUuY29tL2I1MzFjZDA4OTZmMWI0Yzc5NzdiOTIzNmY3MTAzM2MxLzE2MjU4NzE1NDkuNjU=#key=19e8545676e28e577c813de83a4cf1dc
>>> >>  https://www.inky.com/banner-faq/  https://www.inky.com
>>> >>>
>>> >>> IIRC, the definition of 'low latency' for the FCC was something like
>>> >> 100ms, and
>>> >>> Musk was predicting <40ms.
>>> >>>
>>> >>> roughly competitive with landlines, and worlds better than
>>> geostationary
>>> >>> satellite (and many wireless ISPs)
>>> >>>
>>> >>> but when doing any serious testing of latency, you need to be wired to
>>> >> the
>>> >>> router, wifi introduces so much variability that it swamps the signal.
>>> >>>
>>> >>> David Lang
>>> >>>
>>> >>> On Fri, 9 Jul 2021, David P. Reed wrote:
>>> >>>
>>> >>>> Date: Fri, 9 Jul 2021 14:40:01 -0400 (EDT)
>>> >>>> From: David P. Reed <dpreed at deepplum.com>
>>> >>>> To: starlink at lists.bufferbloat.net
>>> >>>> Subject: [Starlink] Starlink and bufferbloat status?
>>> >>>>
>>> >>>>
>>> >>>> Early measurements of performance of Starlink have shown significant
>>> >> bufferbloat, as Dave Taht has shown.
>>> >>>>
>>> >>>> But...  Starlink is a moving target. The bufferbloat isn't a hardware
>>> >> issue, it should be completely manageable, starting by simple firmware
>>> >> changes inside the Starlink system itself. For example, implementing
>>> >> fq_codel so that bottleneck links just drop packets according to the
>>> Best
>>> >> Practices RFC,
>>> >>>>
>>> >>>> So I'm hoping this has improved since Dave's measurements. How much
>>> has
>>> >> it improved? What's the current maximum packet latency under full
>>> >> load,  Ive heard anecdotally that a friend of a friend gets 84 msec.
>>> *ping
>>> >> times under full load*, but he wasn't using flent or some other
>>> measurement
>>> >> tool of good quality that gives a true number.
>>> >>>>
>>> >>>> 84 msec is not great - it's marginal for Zoom quality experience (you
>>> >> want latencies significantly less than 100 msec. as a rule of thumb for
>>> >> teleconferencing quality). But it is better than Dave's measurements
>>> showed.
>>> >>>>
>>> >>>> Now Musk bragged that his network was "low latency" unlike other high
>>> >> speed services, which means low end-to-end latency.  That got him
>>> >> permission from the FCC to operate Starlink at all. His number was, I
>>> >> think, < 5 msec. 84 is a lot more than 5. (I didn't believe 5, because
>>> he
>>> >> probably meant just the time from the ground station to the terminal
>>> >> through the satellite. But I regularly get 17 msec. between California
>>> and
>>> >> Massachusetts over the public Internet)
>>> >>>>
>>> >>>> So 84 might be the current status. That would mean that someone at
>>> >> Srarlink might be paying some attention, but it is a long way from what
>>> >> Musk implied.
>>> >>>>
>>> >>>>
>>> >>>> PS: I forget the number of the RFC, but the number of packets queued
>>> on
>>> >> an egress link should be chosen by taking the hardware bottleneck
>>> >> throughput of any path, combined with an end-to-end Internet underlying
>>> >> delay of about 10 msec. to account for hops between source and
>>> destination.
>>> >> Lets say Starlink allocates 50 Mb/sec to each customer, packets are
>>> limited
>>> >> to 10,000 bits (1500 * 8), so the outbound queues should be limited to
>>> >> about 0.01 * 50,000,000 / 10,000, which comes out to about 250 packets
>>> from
>>> >> each terminal of buffering, total, in the path from terminal to public
>>> >> Internet, assuming the connection to the public Internet is not a
>>> problem.
>>> >>> _______________________________________________
>>> >>> Starlink mailing list
>>> >>> Starlink at lists.bufferbloat.net
>>> >>>
>>> >>
>>> https://secure-web.cisco.com/1sNc_-1HhGCW7xdirt_lAoAy5Nn5T6UA85Scjn5BR7QHXtumhrf6RKn78SuRJG7DUKI3duggU9g6hJKW-Ze07HTczYqB9mBpIeALqk5drQ7nMvM8K7JbWfUbPR7JSNrI75UjiNXQk0wslBfoOTvkMlRj5eMOZhps7DMGBRQTVAeTd5vwXoQtDgS6zLCcJkrcO2S9MRSCC4f1I17SzgQJIwqo3LEwuN6lD-pkX0MFLqGr2zzsHw5eapd-VBlHu5reC4-OEn2zHkb7HNzS1pcueF6tsUE1vFRsWs2SIOwU5MvbKe3J3Q6NRQ40cHI1AGd-i/https://lists.bufferbloat.net/listinfo/starlink
>>> >>>
>>> >>> _______________________________________________
>>> >> Starlink mailing list
>>> >> Starlink at lists.bufferbloat.net
>>> >> https://lists.bufferbloat.net/listinfo/starlink
>>> >>
>>> >
>>>
>> _______________________________________________
>> Starlink mailing list
>> Starlink at lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/starlink
>>
>>
> 