From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.ece.northwestern.edu (mail.ece.northwestern.edu [129.105.8.4]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 10F593B2A4 for ; Sat, 17 Jul 2021 11:55:08 -0400 (EDT) Received: from [172.20.10.4] (mobile-166-175-185-26.mycingular.net [166.175.185.26]) by mail.ece.northwestern.edu (Postfix) with ESMTP id 18A513C0A1; Sat, 17 Jul 2021 10:55:07 -0500 (CDT) Date: Sat, 17 Jul 2021 10:55:00 -0500 From: "Fabian E. Bustamante" To: Mike Puchol , David Lang Cc: "=?utf-8?Q?starlink=40lists.bufferbloat.net?=" , "David P. Reed" Message-ID: <6d5c65c8-00a9-49f8-80ce-4b16094198a1@Spark> In-Reply-To: References: <1625856001.74681750@apps.rackspace.com> <33ae5470-a05a-484e-adc6-4baca6ede9ad@Spark> X-Readdle-Message-ID: 6d5c65c8-00a9-49f8-80ce-4b16094198a1@Spark MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="60f2fd59_71f32454_1c0" Subject: Re: [Starlink] Starlink and bufferbloat status? X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jul 2021 15:55:08 -0000 --60f2fd59_71f32454_1c0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Maybe already mentioned but just in case, there are two HotNets papers on= 2019 and 2020 looking at the idea of bent-pipe connectivity (up and down= from sat to grind) as an alternative (third=3F) solution. The 2019 paper= by Handley=C2=A0=C2=A0suggest that with a dense-enough deployment of GTs= you could achieve latencies comparable to constellations with ISL. The 2= 020 paper, with additional details in their analysis, shows this would co= me with higher, more variable latencies due to obvious things like weathe= r, the need for GTs in unfriendly locations, etc. fabian --- =7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E= =7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E= =7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E =46abian E. Bustamante, Professor Department of Computer Science =7C=C2=A0Northwestern U. Lead Scientist =40 PhenixRTS http://www.aqualab.cs.northwestern.edu twitter: =40bustamantefe On Jul 16, 2021, 8:13 PM -0500, David Lang , wrote: > If you limit your ground stations to areas with little cloud cover, you= then > have to use regular Internet to get to the servers, which will add far = more > latency. > > Starlink is based on the idea of lots of cheap devices at close range r= ather > than a few expensive devices that (because they are few) are at longer = range > > satellites in one shell can be far futher apart than the distance from = one shell > to another, and each shell is a sphere, so the angle from one point on = the > sphere to another is going to vary in both dimensions (remember, the sa= tellites > will do collision avoidance) so you cannot count on them being at the p= erfect > angle in one dimension. > > If you are needing to talk from the antartica to a system in an AWS dat= acenter > in Virginia, your first hop is going to be a satellite in one of the po= lar > shells, from there it can send the signal via laser to a satellite at a= > different altitude in one of the main shells (which may need to send it= to > another satellite, which may be at a different altitude/different shell= ...) > which then can send the signal down to a ground station on the roof of = the AWS > datacenter. > > If they do have shells at higher altitudes, the satellites at higher al= titudes > can send the signal further without having the laser go through air, so= a hop > from a lo altitude shell to a higher altitude shell can save several ho= ps > through the low altitude shells (more significant as load goes up) > > the wikipedia page lists the details of the different shells that are p= lanned > > https://urldefense.com/v3/=5F=5Fhttps://en.wikipedia.org/wiki/Starlink=5F= =5F;=21=21Dq0X2Dk=46hy=4693HkjWTBQKhk=21Cyoz1r9ywvQvHilpxb1Yt4qmJh0Y0-rUJ= hH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7D9SDmTDw=24 > > a shell is all at roughly the same altitude and inclination > > they have populated the first shell, and started on a second shell, in = phase 2 > they are scheduled to start populating shells at lower altitudes (lower= latency, > better handling of dense areas, but the satellites won't last as long a= nd won't > have as long a horizon so more hops would be needed) > > David Lang > > > On =46ri, 16 Jul 2021, Mike Puchol wrote: > > > If we understand shell as =E2=80=9Cgroup of satellites at a certain a= ltitude range=E2=80=9D, there is not much point in linking between shells= if you can link within one shell and orbital plane, and that plane has a= t least one satellite within range of a gateway. I could be proven wrong,= but IMHO the first generation of links are meant of intra-plane, and may= be at a stretch cross-plane to the next plane East or West. > > > > The only way to eventually go is optical links to the ground too, as = R=46 will only get you so far. At that stage, every shell will have its o= wn optical links to the ground, with gateways placed in areas with little= average cloud cover. > > > > Best, > > > > Mike > > On Jul 16, 2021, 23:30 +0200, David Lang , wrote: > > > at satellite distances, you need to adjust your vertical direction = depending on > > > how far away the satellite you are talking to is, even if it's at t= he same > > > altitude > > > > > > the difference between shells that are only a few KM apart is less = than the > > > angles that you could need to satellites in the same shell further = away. > > > > > > David Lang > > > > > > On =46ri, 16 Jul 2021, Mike Puchol wrote: > > > > > > > Date: =46ri, 16 Jul 2021 22:57:14 +0200 > > > > =46rom: Mike Puchol > > > > To: David Lang > > > > Cc: Nathan Owens , > > > > =22starlink=40lists.bufferbloat.net=22 , > > > > David P. Reed > > > > Subject: Re: =5BStarlink=5D Starlink and bufferbloat status=3F > > > > > > > > Correct. A mirror tracking head that turns around the perpendicul= ar to the satellite path allows you to track satellites in the same plane= , in front or behind, when they change altitude by a few kilometers as pa= rt of orbital adjustments or collision avoidance. To have a fully gimbale= d head that can track any satellite in any direction (and at any relative= velocity=21) is a totally different problem. I could see satellites link= ed to the next longitudinal plane apart from those on the same plane, but= cross-plane when one is ascending and the other descending is way harder= . The next shells will be at lower altitudes, around 300-350km, and they = have also stated they want to go for higher shells at 1000+ km. > > > > > > > > Best, > > > > > > > > Mike > > > > On Jul 16, 2021, 20:48 +0200, David Lang , wrote= : > > > > > I expect the lasers to have 2d gimbles, which lets them track m= ost things in > > > > > their field of view. Remember that Starlink has compressed thei= r orbital planes, > > > > > they are going to be running almost everything in the 550km ran= ge (500-600km > > > > > IIRC) and have almost entirely eliminated the =7E1000km planes > > > > > > > > > > David Lang > > > > > > > > > > On =46ri, 16 Jul 2021, > > > > > Mike Puchol wrote: > > > > > > > > > > > Date: =46ri, 16 Jul 2021 19:42:55 +0200 > > > > > > =46rom: Mike Puchol > > > > > > To: David Lang > > > > > > Cc: Nathan Owens , > > > > > > =22starlink=40lists.bufferbloat.net=22 , > > > > > > David P. Reed > > > > > > Subject: Re: =5BStarlink=5D Starlink and bufferbloat status=3F= > > > > > > > > > > > > True, but we are then assuming that the optical links are a m= esh between satellites in the same plane, plus between planes. =46rom an = engineering problem point of view, keeping optical links in-plane only ma= kes the system extremely simpler (no full =46OV gimbals with the optical = train in them, for example), and it solves the issue, as it is highly lik= ely that at least one satellite in any given plane will be within reach o= f a gateway. > > > > > > > > > > > > Routing to an arbitrary gateway may involve passing via inter= mediate gateways, ground segments, and even using terminals as a hopping = point. > > > > > > > > > > > > Best, > > > > > > > > > > > > Mike > > > > > > On Jul 16, 2021, 19:38 +0200, David Lang , w= rote: > > > > > > > the speed of light in a vaccum is significantly better than= the speed of light > > > > > > > in fiber, so if you are doing a cross country hop, terminal= -> sat -> sat -> sat > > > > > > > -> ground station (especially if the ground station is in t= he target datacenter) > > > > > > > can be faster than terminal -> sat -> ground station -> cro= ss-country fiber, > > > > > > > even accounting for the longer distance at 550km altitude t= han at ground level. > > > > > > > > > > > > > > This has interesting implications for supplementing/replaci= ng undersea cables as > > > > > > > the sats over the ocean are not going to be heavily used, d= edicated ground > > > > > > > stations could be setup that use sats further offshore than= normal (and are > > > > > > > shielded from sats over land) to leverage the system withou= t interfering > > > > > > > significantly with more 'traditional' uses > > > > > > > > > > > > > > David Lang > > > > > > > > > > > > > > On =46ri, 16 Jul 2021, Mike Puchol wrote: > > > > > > > > > > > > > > > Date: =46ri, 16 Jul 2021 19:31:37 +0200 > > > > > > > > =46rom: Mike Puchol > > > > > > > > To: David Lang , Nathan Owens > > > > > > > > Cc: =22starlink=40lists.bufferbloat.net=22 , > > > > > > > > David P. Reed > > > > > > > > Subject: Re: =5BStarlink=5D Starlink and bufferbloat stat= us=3F > > > > > > > > > > > > > > > > Satellite optical links are useful to extend coverage to = areas where you don=E2=80=99t have gateways - thus, they will introduce a= dditional latency compared to two space segment hops (terminal to satelli= te -> satellite to gateway). If you have terminal to satellite, two optic= al hops, then final satellite to gateway, you will have more latency, not= less. > > > > > > > > > > > > > > > > We are being =E2=80=9Csold=E2=80=9D optical links for wha= t they are not IMHO. > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > Mike > > > > > > > > On Jul 16, 2021, 19:29 +0200, Nathan Owens , wrote: > > > > > > > > > > As there are more satellites, the up down time will g= et closer to 4-5ms rather then the =7E7ms you list > > > > > > > > > > > > > > > > > > Possibly, if you do steering to always jump to the lowe= st latency satellite. > > > > > > > > > > > > > > > > > > > with laser relays in orbit, and terminal to terminal = routing in orbit, there is the potential for the theoretical minimum to t= end lower > > > > > > > > > Maybe for certain users really in the middle of nowhere= , but I did the best-case math for =22bent pipe=22 in Seattle area, which= is as good as it gets. > > > > > > > > > > > > > > > > > > > On =46ri, Jul 16, 2021 at 10:24 AM David Lang wrote: > > > > > > > > > > > hey, it's a good attitude to have :-) > > > > > > > > > > > > > > > > > > > > > > Elon tends to set 'impossible' goals, miss the time= line a bit, and come very > > > > > > > > > > > close to the goal, if not exceed it. > > > > > > > > > > > > > > > > > > > > > > As there are more staellites, the up down time will= get closer to 4-5ms rather > > > > > > > > > > > then the =7E7ms you list, and with laser relays in = orbit, and terminal to terminal > > > > > > > > > > > routing in orbit, there is the potential for the th= eoretical minimum to tend > > > > > > > > > > > lower, giving some headroom for other overhead but = still being in the 20ms > > > > > > > > > > > range. > > > > > > > > > > > > > > > > > > > > > > David Lang > > > > > > > > > > > > > > > > > > > > > > =C2=A0 On =46ri, 16 Jul 2021, Nathan Owens wrote: > > > > > > > > > > > > > > > > > > > > > > > Elon said =22foolish packet routing=22 for things= over 20ms=21 Which seems crazy > > > > > > > > > > > > if you do some basic math: > > > > > > > > > > > > > > > > > > > > > > > > =C2=A0 =C2=A0- Sat to User Terminal distance: 550= -950km air/vacuum: 1.9 - 3.3ms > > > > > > > > > > > > =C2=A0 =C2=A0- Sat to GW distance: 550-950km air/= vacuum: 1.9 - 3.3ms > > > > > > > > > > > > =C2=A0 =C2=A0- GW to PoP Distance: 50-800km fiber= : 0.25 - 4ms > > > > > > > > > > > > =C2=A0 =C2=A0- PoP to Internet Distance: 50km fib= er: 0.25 - 0.5ms > > > > > > > > > > > > =C2=A0 =C2=A0- Total one-way delay: 4.3 - 11.1ms > > > > > > > > > > > > =C2=A0 =C2=A0- Theoretical minimum RTT: 8.6ms - 2= 2.2ms, call it 15.4ms > > > > > > > > > > > > > > > > > > > > > > > > This includes no transmission delay, queuing dela= y, > > > > > > > > > > > > processing/fragmentation/reassembly/etc, and no t= ime-division multiplexing. > > > > > > > > > > > > > > > > > > > > > > > > On =46ri, Jul 16, 2021 at 10:09 AM David Lang wrote: > > > > > > > > > > > > > > > > > > > > > > > > > I think it depends on if you are looking at dat= acenter-to-datacenter > > > > > > > > > > > > > latency of > > > > > > > > > > > > > home to remote datacenter latency :-) > > > > > > > > > > > > > > > > > > > > > > > > > > my rule of thumb for cross US ping time has bee= n 80-100ms latency (but > > > > > > > > > > > > > it's been > > > > > > > > > > > > > a few years since I tested it). > > > > > > > > > > > > > > > > > > > > > > > > > > I note that an article I saw today said that El= on is saying that latency > > > > > > > > > > > > > will > > > > > > > > > > > > > improve significantly in the near future, that = up/down latency is =7E20ms > > > > > > > > > > > > > and the > > > > > > > > > > > > > additional delays pushing it to the 80ms range = are 'stupid packet routing' > > > > > > > > > > > > > problems that they are working on. > > > > > > > > > > > > > > > > > > > > > > > > > > If they are still in that level of optimization= , it doesn't surprise me > > > > > > > > > > > > > that > > > > > > > > > > > > > they haven't really focused on the bufferbloat = issue, they have more > > > > > > > > > > > > > obvious > > > > > > > > > > > > > stuff to fix first. > > > > > > > > > > > > > > > > > > > > > > > > > > David Lang > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > =C2=A0 =C2=A0On =46ri, 16 Jul 2021, Wheelock, I= an wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Date: =46ri, 16 Jul 2021 10:21:52 +0000 > > > > > > > > > > > > > > =46rom: =22Wheelock, Ian=22 > > > > > > > > > > > > > > To: David Lang , David P. Re= ed > > > > > > > > > > > > > > Cc: =22starlink=40lists.bufferbloat.net=22 > > > > > > > > > > > > > > Subject: Re: =5BStarlink=5D Starlink and buff= erbloat status=3F > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi David > > > > > > > > > > > > > > In terms of the Latency that David (Reed) men= tioned for California to > > > > > > > > > > > > > Massachusetts of about 17ms over the public int= ernet, it seems a bit faster > > > > > > > > > > > > > than what I would expect. My own traceroute via= my VDSL link shows 14ms > > > > > > > > > > > > > just to get out of the operator network. > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/=5F=5Fhttps://www.w= ondernetwork.com=5F=5F;=21=21Dq0X2Dk=46hy=4693HkjWTBQKhk=21Cyoz1r9ywvQvHi= lpxb1Yt4qmJh0Y0-rUJhH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7BGSmtMvg=24 =C2=A0 is= a handy tool for checking geographic > > > > > > > > > > > > > ping perf between cities, and it shows a min of= about 66ms for pings > > > > > > > > > > > > > between Boston and San Diego > > > > > > > > > > > > > https://urldefense.com/v3/=5F=5Fhttps://wondern= etwork.com/pings/boston/San*20Diego=5F=5F;JQ=21=21Dq0X2Dk=46hy=4693HkjWTB= QKhk=21Cyoz1r9ywvQvHilpxb1Yt4qmJh0Y0-rUJhH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7= Bf=46fHabQ=24 (so about 33ms for > > > > > > > > > > > > > 1-way transfer). > > > > > > > > > > > > > > > > > > > > > > > > > > > > Distance wise this is about 4,100 KM (2,500 M= ), and =402/3 speed of light > > > > > > > > > > > > > (through a pure fibre link of that distance) th= e propagation time is just > > > > > > > > > > > > > over 20ms. If the network equipment between the= Boston and San Diego is > > > > > > > > > > > > > factored in, with some buffering along the way,= 33ms does seem quite > > > > > > > > > > > > > reasonable over the 20ms for speed of light in = fibre for that 1-way transfer > > > > > > > > > > > > > > > > > > > > > > > > > > > > -Ian Wheelock > > > > > > > > > > > > > > > > > > > > > > > > > > > > =46rom: Starlink on behalf of > > > > > > > > > > > > > David Lang > > > > > > > > > > > > > > Date: =46riday 9 July 2021 at 23:59 > > > > > > > > > > > > > > To: =22David P. Reed=22 > > > > > > > > > > > > > > Cc: =22starlink=40lists.bufferbloat.net=22 > > > > > > > > > > > > > > Subject: Re: =5BStarlink=5D Starlink and buff= erbloat status=3F > > > > > > > > > > > > > > > > > > > > > > > > > > > > IIRC, the definition of 'low latency' for the= =46CC was something like > > > > > > > > > > > > > 100ms, and Musk was predicting <40ms. roughly c= ompetitive with landlines, > > > > > > > > > > > > > and worlds better than geostationary satellite = (and many > > > > > > > > > > > > > > External (mailto:david=40lang.hm) > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/=5F=5Fhttps://shared.= outlook.inky.com/report=3Fid=3DY29tbXNjb3BlL2lhbi53aGVlbG9ja0Bjb21tc2NvcG= UuY29tL2I1Mz=46jZDA4OTZmMWI0Yzc5NzdiOTIzNmY3MTAzM2MxLzE2MjU4NzE1NDkuNjU=3D= *key=3D19e8545676e28e577c813de83a4cf1dc=5F=5F;Iw=21=21Dq0X2Dk=46hy=4693Hk= jWTBQKhk=21Cyoz1r9ywvQvHilpxb1Yt4qmJh0Y0-rUJhH-QUV0z9jqTWXnL0EC=5FBY=5FwX= 65z7AbUggbLA=24 > > > > > > > > > > > > > =C2=A0 https://urldefense.com/v3/=5F=5Fhttps://= www.inky.com/banner-faq/=5F=5F;=21=21Dq0X2Dk=46hy=4693HkjWTBQKhk=21Cyoz1r= 9ywvQvHilpxb1Yt4qmJh0Y0-rUJhH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7Dcy2JKhA=24 =C2= =A0 https://urldefense.com/v3/=5F=5Fhttps://www.inky.com=5F=5F;=21=21Dq0X= 2Dk=46hy=4693HkjWTBQKhk=21Cyoz1r9ywvQvHilpxb1Yt4qmJh0Y0-rUJhH-QUV0z9jqTWX= nL0EC=5FBY=5FwX65z7A=463-h2-w=24 > > > > > > > > > > > > > > > > > > > > > > > > > > > > IIRC, the definition of 'low latency' for the= =46CC was something like > > > > > > > > > > > > > 100ms, and > > > > > > > > > > > > > > Musk was predicting <40ms. > > > > > > > > > > > > > > > > > > > > > > > > > > > > roughly competitive with landlines, and world= s better than geostationary > > > > > > > > > > > > > > satellite (and many wireless ISPs) > > > > > > > > > > > > > > > > > > > > > > > > > > > > but when doing any serious testing of latency= , you need to be wired to > > > > > > > > > > > > > the > > > > > > > > > > > > > > router, wifi introduces so much variability t= hat it swamps the signal. > > > > > > > > > > > > > > > > > > > > > > > > > > > > David Lang > > > > > > > > > > > > > > > > > > > > > > > > > > > > On =46ri, 9 Jul 2021, David P. Reed wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Date: =46ri, 9 Jul 2021 14:40:01 -0400 (EDT= ) > > > > > > > > > > > > > > > =46rom: David P. Reed > > > > > > > > > > > > > > > To: starlink=40lists.bufferbloat.net > > > > > > > > > > > > > > > Subject: =5BStarlink=5D Starlink and buffer= bloat status=3F > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Early measurements of performance of Starli= nk have shown significant > > > > > > > > > > > > > bufferbloat, as Dave Taht has shown. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > But...=C2=A0 Starlink is a moving target. T= he bufferbloat isn't a hardware > > > > > > > > > > > > > issue, it should be completely manageable, star= ting by simple firmware > > > > > > > > > > > > > changes inside the Starlink system itself. =46o= r example, implementing > > > > > > > > > > > > > fq=5Fcodel so that bottleneck links just drop p= ackets according to the Best > > > > > > > > > > > > > Practices R=46C, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So I'm hoping this has improved since Dave'= s measurements. How much has > > > > > > > > > > > > > it improved=3F What's the current maximum packe= t latency under full > > > > > > > > > > > > > load,=C2=A0 Ive heard anecdotally that a friend= of a friend gets 84 msec. *ping > > > > > > > > > > > > > times under full load*, but he wasn't using fle= nt or some other measurement > > > > > > > > > > > > > tool of good quality that gives a true number. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 84 msec is not great - it's marginal for Zo= om quality experience (you > > > > > > > > > > > > > want latencies significantly less than 100 msec= . as a rule of thumb for > > > > > > > > > > > > > teleconferencing quality). But it is better tha= n Dave's measurements showed. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Now Musk bragged that his network was =22lo= w latency=22 unlike other high > > > > > > > > > > > > > speed services, which means low end-to-end late= ncy.=C2=A0 That got him > > > > > > > > > > > > > permission from the =46CC to operate Starlink a= t all. His number was, I > > > > > > > > > > > > > think, < 5 msec. 84 is a lot more than 5. (I di= dn't believe 5, because he > > > > > > > > > > > > > probably meant just the time from the ground st= ation to the terminal > > > > > > > > > > > > > through the satellite. But I regularly get 17 m= sec. between California and > > > > > > > > > > > > > Massachusetts over the public Internet) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So 84 might be the current status. That wou= ld mean that someone at > > > > > > > > > > > > > Srarlink might be paying some attention, but it= is a long way from what > > > > > > > > > > > > > Musk implied. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > PS: I forget the number of the R=46C, but t= he number of packets queued on > > > > > > > > > > > > > an egress link should be chosen by taking the h= ardware bottleneck > > > > > > > > > > > > > throughput of any path, combined with an end-to= -end Internet underlying > > > > > > > > > > > > > delay of about 10 msec. to account for hops bet= ween source and destination. > > > > > > > > > > > > > Lets say Starlink allocates 50 Mb/sec to each c= ustomer, packets are limited > > > > > > > > > > > > > to 10,000 bits (1500 * 8), so the outbound queu= es should be limited to > > > > > > > > > > > > > about 0.01 * 50,000,000 / 10,000, which comes o= ut to about 250 packets from > > > > > > > > > > > > > each terminal of buffering, total, in the path = from terminal to public > > > > > > > > > > > > > Internet, assuming the connection to the public= Internet is not a problem. > > > > > > > > > > > > > > =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F > > > > > > > > > > > > > > Starlink mailing list > > > > > > > > > > > > > > Starlink=40lists.bufferbloat.net > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/=5F=5Fhttps://secure-= web.cisco.com/1sNc=5F-1HhGCW7xdirt=5FlAoAy5Nn5T6UA85Scjn5BR7QHXtumhrf6RKn= 78SuRJG7DUKI3duggU9g6hJKW-Ze07HTczYqB9mBpIeALqk5drQ7nMvM8K7JbWfUbPR7JSNrI= 75UjiNXQk0wslBfoOTvkMlRj5eMOZhps7DMGBRQTVAeTd5vwXoQtDgS6zLCcJkrcO2S9MRSCC= 4f1I17SzgQJIwqo3LEwuN6lD-pkX0M=46LqGr2zzsHw5eapd-VBlHu5reC4-OEn2zHkb7HNzS= 1pcue=466tsUE1v=46RsWs2SIOwU5MvbKe3J3Q6NRQ40cHI1AGd-i/https:/*lists.buffe= rbloat.net/listinfo/starlink=5F=5F;Lw=21=21Dq0X2Dk=46hy=4693HkjWTBQKhk=21= Cyoz1r9ywvQvHilpxb1Yt4qmJh0Y0-rUJhH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7C1Ug-U=46= w=24 > > > > > > > > > > > > > > > > > > > > > > > > > > > > =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F > > > > > > > > > > > > > Starlink mailing list > > > > > > > > > > > > > Starlink=40lists.bufferbloat.net > > > > > > > > > > > > > https://urldefense.com/v3/=5F=5Fhttps://lists.b= ufferbloat.net/listinfo/starlink=5F=5F;=21=21Dq0X2Dk=46hy=4693HkjWTBQKhk=21= Cyoz1r9ywvQvHilpxb1Yt4qmJh0Y0-rUJhH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7CR8V3Ec= g=24 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F > > > > > > > > > Starlink mailing list > > > > > > > > > Starlink=40lists.bufferbloat.net > > > > > > > > > https://urldefense.com/v3/=5F=5Fhttps://lists.bufferblo= at.net/listinfo/starlink=5F=5F;=21=21Dq0X2Dk=46hy=4693HkjWTBQKhk=21Cyoz1r= 9ywvQvHilpxb1Yt4qmJh0Y0-rUJhH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7CR8V3Ecg=24 --60f2fd59_71f32454_1c0 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
Maybe already mentioned but just in case, there are= two HotNets papers on 2019 and 2020 looking at the idea of bent-pipe con= nectivity (up and down from sat to grind) as an alternative (third=3F) so= lution. The 2019 paper by Handley&=23160;&=23160;suggest that with a dens= e-enough deployment of GTs you could achieve latencies comparable to cons= tellations with ISL. The 2020 paper, with additional details in their ana= lysis, shows this would come with higher, more variable latencies due to = obvious things like weather, the need for GTs in unfriendly locations, et= c.

fabian

---
= =7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E= =7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E= =7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E=7E
=46ab= ian E. Bustamante, Professor&=23160;
Department of Computer Science =7C&=23160;Northwestern U.
Lead Scientist =40 PhenixRTS
http://www.aqualab.cs.northwestern.edu
twitter: =40bustamantefe
On Jul 16, 2021, 8:13 PM -0500, Dav= id Lang <david=40lang.hm>, wrote:
If you limit your ground stations to areas with little cloud co= ver, you then
have to use regular Internet to get to the servers, which will add far mo= re
latency.

Starlink is based on the idea of lots of cheap devices at close range rat= her
than a few expensive devices that (because they are few) are at longer ra= nge

satellites in one shell can be far futher apart than the distance from on= e shell
to another, and each shell is a sphere, so the angle from one point on th= e
sphere to another is going to vary in both dimensions (remember, the sate= llites
will do collision avoidance) so you cannot count on them being at the per= fect
angle in one dimension.

If you are needing to talk from the antartica to a system in an AWS datac= enter
in Virginia, your first hop is going to be a satellite in one of the pola= r
shells, from there it can send the signal via laser to a satellite at a different altitude in one of the main shells (which may need to send it t= o
another satellite, which may be at a different altitude/different shell .= ..)
which then can send the signal down to a ground station on the roof of th= e AWS
datacenter.

If they do have shells at higher altitudes, the satellites at higher alti= tudes
can send the signal further without having the laser go through air, so a= hop
from a lo altitude shell to a higher altitude shell can save several hops=
through the low altitude shells (more significant as load goes up)

the wikipedia page lists the details of the different shells that are pla= nned

https://urldefense.com/v3/=5F=5Fhttps://en.wikipedia.org/wiki/Starlink=5F= =5F;=21=21Dq0X2Dk=46hy=4693HkjWTBQKhk=21Cyoz1r9ywvQvHilpxb1Yt4qmJh0Y0-rUJ= hH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7D9SDmTDw=24

a shell is all at roughly the same altitude and inclination

they have populated the first shell, and started on a second shell, in ph= ase 2
they are scheduled to start populating shells at lower altitudes (lower l= atency,
better handling of dense areas, but the satellites won't last as long and= won't
have as long a horizon so more hops would be needed)

David Lang


On =46ri, 16 Jul 2021, Mike Puchol wrote:

If we understand shell as =E2=80=9Cgroup of= satellites at a certain altitude range=E2=80=9D, there is not much point= in linking between shells if you can link within one shell and orbital p= lane, and that plane has at least one satellite within range of a gateway= . I could be proven wrong, but IMHO the first generation of links are mea= nt of intra-plane, and maybe at a stretch cross-plane to the next plane E= ast or West.

The only way to eventually go is optical links to the ground too, as R=46= will only get you so far. At that stage, every shell will have its own o= ptical links to the ground, with gateways placed in areas with little ave= rage cloud cover.

Best,

Mike
On Jul 16, 2021, 23:30 +0200, David Lang <david=40lang.hm>, wrote:<= br />
at satellite distances, you need to adjust = your vertical direction depending on
how far away the satellite you are talking to is, even if it's at the sam= e
altitude

the difference between shells that are only a few KM apart is less than t= he
angles that you could need to satellites in the same shell further away.<= br />
David Lang

On =46ri, 16 Jul 2021, Mike Puchol wrote:

Date: =46ri, 16 Jul 2021 22:57:14 +0200
=46rom: Mike Puchol <mike=40starlink.sx>
To: David Lang <david=40lang.hm>
Cc: Nathan Owens <nathan=40nathan.io>,
=22starlink=40lists.bufferbloat.net=22 <starlink=40lists.bufferbloat.n= et>,
David P. Reed <dpreed=40deepplum.com>
Subject: Re: =5BStarlink=5D Starlink and bufferbloat status=3F

Correct. A mirror tracking head that turns around the perpendicular to th= e satellite path allows you to track satellites in the same plane, in fro= nt or behind, when they change altitude by a few kilometers as part of or= bital adjustments or collision avoidance. To have a fully gimbaled head t= hat can track any satellite in any direction (and at any relative velocit= y=21) is a totally different problem. I could see satellites linked to th= e next longitudinal plane apart from those on the same plane, but cross-p= lane when one is ascending and the other descending is way harder. The ne= xt shells will be at lower altitudes, around 300-350km, and they have als= o stated they want to go for higher shells at 1000+ km.

Best,

Mike
On Jul 16, 2021, 20:48 +0200, David Lang <david=40lang.hm>, wrote:<= br />
I expect the lasers to have 2d gimbles, whi= ch lets them track most things in
their field of view. Remember that Starlink has compressed their orbital = planes,
they are going to be running almost everything in the 550km range (500-60= 0km
IIRC) and have almost entirely eliminated the =7E1000km planes

David Lang

On =46ri, 16 Jul 2021,
Mike Puchol wrote:

Date: =46ri, 16 Jul 2021 19:42:55 +0200
=46rom: Mike Puchol <mike=40starlink.sx>
To: David Lang <david=40lang.hm>
Cc: Nathan Owens <nathan=40nathan.io>,
=22starlink=40lists.bufferbloat.net=22 <starlink=40lists.bufferbloat.n= et>,
David P. Reed <dpreed=40deepplum.com>
Subject: Re: =5BStarlink=5D Starlink and bufferbloat status=3F

True, but we are then assuming that the optical links are a mesh between = satellites in the same plane, plus between planes. =46rom an engineering = problem point of view, keeping optical links in-plane only makes the syst= em extremely simpler (no full =46OV gimbals with the optical train in the= m, for example), and it solves the issue, as it is highly likely that at = least one satellite in any given plane will be within reach of a gateway.=

Routing to an arbitrary gateway may involve passing via intermediate gate= ways, ground segments, and even using terminals as a hopping point.
=
Best,

Mike
On Jul 16, 2021, 19:38 +0200, David Lang <david=40lang.hm>, wrote:<= br />
the speed of light in a vaccum is significa= ntly better than the speed of light
in fiber, so if you are doing a cross country hop, terminal -> sat -&g= t; sat -> sat
-> ground station (especially if the ground station is in the target d= atacenter)
can be faster than terminal -> sat -> ground station -> cross-co= untry fiber,
even accounting for the longer distance at 550km altitude than at ground = level.

This has interesting implications for supplementing/replacing undersea ca= bles as
the sats over the ocean are not going to be heavily used, dedicated groun= d
stations could be setup that use sats further offshore than normal (and a= re
shielded from sats over land) to leverage the system without interfering<= br /> significantly with more 'traditional' uses

David Lang

On =46ri, 16 Jul 2021, Mike Puchol wrote:

Date: =46ri, 16 Jul 2021 19:31:37 +0200
=46rom: Mike Puchol <mike=40starlink.sx>
To: David Lang <david=40lang.hm>, Nathan Owens <nathan=40nathan.= io>
Cc: =22starlink=40lists.bufferbloat.net=22 <starlink=40lists.bufferblo= at.net>,
David P. Reed <dpreed=40deepplum.com>
Subject: Re: =5BStarlink=5D Starlink and bufferbloat status=3F

Satellite optical links are useful to extend coverage to areas where you = don=E2=80=99t have gateways - thus, they will introduce additional latenc= y compared to two space segment hops (terminal to satellite -> satelli= te to gateway). If you have terminal to satellite, two optical hops, then= final satellite to gateway, you will have more latency, not less.

We are being =E2=80=9Csold=E2=80=9D optical links for what they are not I= MHO.

Best,

Mike
On Jul 16, 2021, 19:29 +0200, Nathan Owens <nathan=40nathan.io>, wr= ote:
As there are more satellites, the up down t= ime will get closer to 4-5ms rather then the =7E7ms you list

Possibly, if you do steering to always jump to the lowest latency satelli= te.

with laser relays in orbit, and terminal to= terminal routing in orbit, there is the potential for the theoretical mi= nimum to tend lower
Maybe for certain users really in the middle of nowhere, but I did the be= st-case math for =22bent pipe=22 in Seattle area, which is as good as it = gets.

On =46ri, Jul 16, 2021 at 10:24 AM David La= ng <david=40lang.hm> wrote:
hey, it's a good attitude to have :-)
=
Elon tends to set 'impossible' goals, miss the timeline a bit, and come v= ery
close to the goal, if not exceed it.

As there are more staellites, the up down time will get closer to 4-5ms r= ather
then the =7E7ms you list, and with laser relays in orbit, and terminal to= terminal
routing in orbit, there is the potential for the theoretical minimum to t= end
lower, giving some headroom for other overhead but still being in the 20m= s
range.

David Lang

&=23160; On =46ri, 16 Jul 2021, Nathan Owens wrote:

Elon said =22foolish packet routing=22 for = things over 20ms=21 Which seems crazy
if you do some basic math:

&=23160; &=23160;- Sat to User Terminal distance: 550-950km air/vacuum: 1= .9 - 3.3ms
&=23160; &=23160;- Sat to GW distance: 550-950km air/vacuum: 1.9 - 3.3ms<= br /> &=23160; &=23160;- GW to PoP Distance: 50-800km fiber: 0.25 - 4ms
&=23160; &=23160;- PoP to Internet Distance: 50km fiber: 0.25 - 0.5ms
&=23160; &=23160;- Total one-way delay: 4.3 - 11.1ms
&=23160; &=23160;- Theoretical minimum RTT: 8.6ms - 22.2ms, call it 15.4m= s

This includes no transmission delay, queuing delay,
processing/fragmentation/reassembly/etc, and no time-division multiplexin= g.

On =46ri, Jul 16, 2021 at 10:09 AM David Lang <david=40lang.hm> wro= te:

I think it depends on if you are looking at= datacenter-to-datacenter
latency of
home to remote datacenter latency :-)

my rule of thumb for cross US ping time has been 80-100ms latency (but it's been
a few years since I tested it).

I note that an article I saw today said that Elon is saying that latency<= br /> will
improve significantly in the near future, that up/down latency is =7E20ms=
and the
additional delays pushing it to the 80ms range are 'stupid packet routing= '
problems that they are working on.

If they are still in that level of optimization, it doesn't surprise me that
they haven't really focused on the bufferbloat issue, they have more
obvious
stuff to fix first.

David Lang


&=23160; &=23160;On =46ri, 16 Jul 2021, Wheelock, Ian wrote:

Date: =46ri, 16 Jul 2021 10:21:52 +0000
=46rom: =22Wheelock, Ian=22 <ian.wheelock=40commscope.com>
To: David Lang <david=40lang.hm>, David P. Reed <dpreed=40deeppl= um.com>
Cc: =22starlink=40lists.bufferbloat.net=22 <starlink=40lists.bufferblo= at.net>
Subject: Re: =5BStarlink=5D Starlink and bufferbloat status=3F

Hi David
In terms of the Latency that David (Reed) mentioned for California to
Massachusetts of about 17ms over the public internet, it seems a bit fast= er
than what I would expect. My own traceroute via my VDSL link shows 14ms just to get out of the operator network.

https://urldefense.com/v3/=5F=5Fhttps://www.wondernetwork.com=5F=5F;=21=21= Dq0X2Dk=46hy=4693HkjWTBQKhk=21Cyoz1r9ywvQvHilpxb1Yt4qmJh0Y0-rUJhH-QUV0z9j= qTWXnL0EC=5FBY=5FwX65z7BGSmtMvg=24 &=23160; is a handy tool for checking = geographic
ping perf between cities, and it shows a min of about 66ms for pings
between Boston and San Diego
https://urldefense.com/v3/=5F=5Fhttps://wondernetwork.com/pings/boston/Sa= n*20Diego=5F=5F;JQ=21=21Dq0X2Dk=46hy=4693HkjWTBQKhk=21Cyoz1r9ywvQvHilpxb1= Yt4qmJh0Y0-rUJhH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7Bf=46fHabQ=24 (so about 33= ms for
1-way transfer).

Distance wise this is about 4,100 KM (2,500 M), and =402/3 speed of light=
(through a pure fibre link of that distance) the propagation time is just=
over 20ms. If the network equipment between the Boston and San Diego is factored in, with some buffering along the way, 33ms does seem quite
reasonable over the 20ms for speed of light in fibre for that 1-way trans= fer

-Ian Wheelock

=46rom: Starlink <starlink-bounces=40lists.bufferbloat.net> on beha= lf of
David Lang <david=40lang.hm>
Date: =46riday 9 July 2021 at 23:59
To: =22David P. Reed=22 <dpreed=40deepplum.com>
Cc: =22starlink=40lists.bufferbloat.net=22 <starlink=40lists.bufferblo= at.net>
Subject: Re: =5BStarlink=5D Starlink and bufferbloat status=3F

IIRC, the definition of 'low latency' for the =46CC was something like
100ms, and Musk was predicting <40ms. roughly competitive with landlin= es,
and worlds better than geostationary satellite (and many
External (mailto:david=40lang.hm)

https://urldefense.com/v3/=5F=5Fhttps://shared.outlook.inky.com/report=3F= id=3DY29tbXNjb3BlL2lhbi53aGVlbG9ja0Bjb21tc2NvcGUuY29tL2I1Mz=46jZDA4OTZmMW= I0Yzc5NzdiOTIzNmY3MTAzM2MxLzE2MjU4NzE1NDkuNjU=3D*key=3D19e8545676e28e577c= 813de83a4cf1dc=5F=5F;Iw=21=21Dq0X2Dk=46hy=4693HkjWTBQKhk=21Cyoz1r9ywvQvHi= lpxb1Yt4qmJh0Y0-rUJhH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7AbUggbLA=24
&=23160; https://urldefense.com/v3/=5F=5Fhttps://www.inky.com/banner-faq/= =5F=5F;=21=21Dq0X2Dk=46hy=4693HkjWTBQKhk=21Cyoz1r9ywvQvHilpxb1Yt4qmJh0Y0-= rUJhH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7Dcy2JKhA=24 &=23160; https://urldefen= se.com/v3/=5F=5Fhttps://www.inky.com=5F=5F;=21=21Dq0X2Dk=46hy=4693HkjWTBQ= Khk=21Cyoz1r9ywvQvHilpxb1Yt4qmJh0Y0-rUJhH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7A= =463-h2-w=24

IIRC, the definition of 'low latency' for the =46CC was something like
100ms, and
Musk was predicting <40ms.

roughly competitive with landlines, and worlds better than geostationary<= br /> satellite (and many wireless ISPs)

but when doing any serious testing of latency, you need to be wired to
the
router, wifi introduces so much variability= that it swamps the signal.

David Lang

On =46ri, 9 Jul 2021, David P. Reed wrote:

Date: =46ri, 9 Jul 2021 14:40:01 -0400 (EDT= )
=46rom: David P. Reed <dpreed=40deepplum.com>
To: starlink=40lists.bufferbloat.net
Subject: =5BStarlink=5D Starlink and bufferbloat status=3F


Early measurements of performance of Starlink have shown significant
bufferbloat, as Dave Taht has shown.

But...&=23160; Starlink is a moving target. The bufferbloat isn't a hardw= are
issue, it should be completely manageable, starting by simple firmware changes inside the Starlink system itself. =46or example, implementing fq=5Fcodel so that bottleneck links just drop packets according to the Be= st
Practices R=46C,

So I'm hoping this has improved since Dave's measurements. How much has
it improved=3F What's the current maximum packet latency under full
= load,&=23160; Ive heard anecdotally that a friend of a friend gets 84 mse= c. *ping
times under full load*, but he wasn't using flent or some other measureme= nt
tool of good quality that gives a true number.

84 msec is not great - it's marginal for Zoom quality experience (you
want latencies significantly less than 100 msec. as a rule of thumb for teleconferencing quality). But it is better than Dave's measurements show= ed.

Now Musk bragged that his network was =22low latency=22 unlike other high=
speed services, which means low end-to-end latency.&=23160; That got him<= br /> permission from the =46CC to operate Starlink at all. His number was, I think, < 5 msec. 84 is a lot more than 5. (I didn't believe 5, because= he
probably meant just the time from the ground station to the terminal
through the satellite. But I regularly get 17 msec. between California an= d
Massachusetts over the public Internet)

So 84 might be the current status. That would mean that someone at
<= /blockquote>
Srarlink might be paying some attention, but it is a long way from what Musk implied.


PS: I forget the number of the R=46C, but the number of packets queued on=
an egress link should be chosen by taking the hardware bottleneck
throughput of any path, combined with an end-to-end Internet underlying delay of about 10 msec. to account for hops between source and destinatio= n.
Lets say Starlink allocates 50 Mb/sec to each customer, packets are limit= ed
to 10,000 bits (1500 * 8), so the outbound queues should be limited to about 0.01 * 50,000,000 / 10,000, which comes out to about 250 packets fr= om
each terminal of buffering, total, in the path from terminal to public Internet, assuming the connection to the public Internet is not a problem= .
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F
Starlink mailing list
Starlink=40lists.bufferbloat.net

https://urldefense.com/v3/=5F=5Fhttps://secure-web.cisco.com/1sNc=5F-1HhG= CW7xdirt=5FlAoAy5Nn5T6UA85Scjn5BR7QHXtumhrf6RKn78SuRJG7DUKI3duggU9g6hJKW-= Ze07HTczYqB9mBpIeALqk5drQ7nMvM8K7JbWfUbPR7JSNrI75UjiNXQk0wslBfoOTvkMlRj5e= MOZhps7DMGBRQTVAeTd5vwXoQtDgS6zLCcJkrcO2S9MRSCC4f1I17SzgQJIwqo3LEwuN6lD-p= kX0M=46LqGr2zzsHw5eapd-VBlHu5reC4-OEn2zHkb7HNzS1pcue=466tsUE1v=46RsWs2SIO= wU5MvbKe3J3Q6NRQ40cHI1AGd-i/https:/*lists.bufferbloat.net/listinfo/starli= nk=5F=5F;Lw=21=21Dq0X2Dk=46hy=4693HkjWTBQKhk=21Cyoz1r9ywvQvHilpxb1Yt4qmJh= 0Y0-rUJhH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7C1Ug-U=46w=24

=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
<= /blockquote> Starlink mailing list
Starlink=40lists.bufferbloat.net
https://urldefense.com/v3/=5F=5Fhttps://lists.bufferbloat.net/listinfo/st= arlink=5F=5F;=21=21Dq0X2Dk=46hy=4693HkjWTBQKhk=21Cyoz1r9ywvQvHilpxb1Yt4qm= Jh0Y0-rUJhH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7CR8V3Ecg=24


=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Starlink mailing list
Starlink=40lists.bufferbloat.net
https://urldefense.com/v3/=5F=5Fhttps://lists.bufferbloat.net/listinfo/st= arlink=5F=5F;=21=21Dq0X2Dk=46hy=4693HkjWTBQKhk=21Cyoz1r9ywvQvHilpxb1Yt4qm= Jh0Y0-rUJhH-QUV0z9jqTWXnL0EC=5FBY=5FwX65z7CR8V3Ecg=24
--60f2fd59_71f32454_1c0--