<div><div dir="auto">No requirement for layer3 for that. I’d bet money they’ll keep L3 out of space. </div><div dir="auto"><br></div><div dir="auto">nb</div></div><div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Jul 18, 2021 at 8:20 PM David Lang <<a href="mailto:david@lang.hm" target="_blank">david@lang.hm</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204)">Elon is talking about a viable path in the future being dishy - sat - sat - <br>
dishy<br>
<br>
They aren't there yet, but they are sure planning on it<br>
<br>
David Lang<br>
<br>
On Sun, 18 Jul 2021, Nick Buraglio wrote:<br>
<br>
> We keep saying “route”. What do we actually mean from a network stack<br>
> perspective? Are we talking about relaying light / frames / electric or do<br>
> we mean actual packet routing, because there are obviously a lot of<br>
> important distinctions there.<br>
> I’m willing to bet that there is no routing (as in layer 3 packet routing)<br>
> at all except the Dish NAT all the way into their peering data center. The<br>
> ground stations are very likely RF to fiber wave division back to a carrier<br>
> hotel with no L3 buffering at all. That keeps latency very low (think O-E-O<br>
> and E-O transitions) and moves L3 buffering to two locations and keeps the<br>
> terrestrial network very easy to make redundant (optical protection, etc.).<br>
><br>
> nb<br>
><br>
> On Fri, Jul 16, 2021 at 12:39 PM Jonathan Bennett <<br>
> <a href="mailto:jonathanbennett@hackaday.com" target="_blank">jonathanbennett@hackaday.com</a>> wrote:<br>
><br>
>><br>
>><br>
>> On Fri, Jul 16, 2021, 12:35 PM Nathan Owens <<a href="mailto:nathan@nathan.io" target="_blank">nathan@nathan.io</a>> wrote:<br>
>><br>
>>> The other case where they could provide benefit is very long distance<br>
>>> paths --- NY to Tokyo, Johannesburg to London, etc... but presumably at<br>
>>> high cost, as the capacity will likely be much lower than submarine cables.<br>
>>><br>
>>>><br>
>> Or traffic between Starlink customers. A video call between me and someone<br>
>> else on the Starlink network is going to be drastically better if it can<br>
>> route over the sats.<br>
>><br>
>>><br>
>>>> On Fri, Jul 16, 2021 at 10:31 AM Mike Puchol <<a href="mailto:mike@starlink.sx" target="_blank">mike@starlink.sx</a>> wrote:<br>
>>><br>
>>>> Satellite optical links are useful to extend coverage to areas where you<br>
>>>> don’t have gateways - thus, they will introduce additional latency compared<br>
>>>> to two space segment hops (terminal to satellite -> satellite to gateway).<br>
>>>> If you have terminal to satellite, two optical hops, then final satellite<br>
>>>> to gateway, you will have more latency, not less.<br>
>>>><br>
>>>> We are being “sold” optical links for what they are not IMHO.<br>
>>>><br>
>>>> Best,<br>
>>>><br>
>>>> Mike<br>
>>>> On Jul 16, 2021, 19:29 +0200, Nathan Owens <<a href="mailto:nathan@nathan.io" target="_blank">nathan@nathan.io</a>>, wrote:<br>
>>>><br>
>>>>> As there are more satellites, the up down time will get closer to<br>
>>>> 4-5ms rather then the ~7ms you list<br>
>>>><br>
>>>> Possibly, if you do steering to always jump to the lowest latency<br>
>>>> satellite.<br>
>>>><br>
>>>>> with laser relays in orbit, and terminal to terminal routing in orbit,<br>
>>>> there is the potential for the theoretical minimum to tend lower<br>
>>>> Maybe for certain users really in the middle of nowhere, but I did the<br>
>>>> best-case math for "bent pipe" in Seattle area, which is as good as it gets.<br>
>>>><br>
>>>> On Fri, Jul 16, 2021 at 10:24 AM David Lang <<a href="mailto:david@lang.hm" target="_blank">david@lang.hm</a>> wrote:<br>
>>>><br>
>>>>> hey, it's a good attitude to have :-)<br>
>>>>><br>
>>>>> Elon tends to set 'impossible' goals, miss the timeline a bit, and come<br>
>>>>> very<br>
>>>>> close to the goal, if not exceed it.<br>
>>>>><br>
>>>>> As there are more staellites, the up down time will get closer to 4-5ms<br>
>>>>> rather<br>
>>>>> then the ~7ms you list, and with laser relays in orbit, and terminal to<br>
>>>>> terminal<br>
>>>>> routing in orbit, there is the potential for the theoretical minimum to<br>
>>>>> tend<br>
>>>>> lower, giving some headroom for other overhead but still being in the<br>
>>>>> 20ms<br>
>>>>> range.<br>
>>>>><br>
>>>>> David Lang<br>
>>>>><br>
>>>>> On Fri, 16 Jul 2021, Nathan Owens wrote:<br>
>>>>><br>
>>>>>> Elon said "foolish packet routing" for things over 20ms! Which seems<br>
>>>>> crazy<br>
>>>>>> if you do some basic math:<br>
>>>>>><br>
>>>>>> - Sat to User Terminal distance: 550-950km air/vacuum: 1.9 - 3.3ms<br>
>>>>>> - Sat to GW distance: 550-950km air/vacuum: 1.9 - 3.3ms<br>
>>>>>> - GW to PoP Distance: 50-800km fiber: 0.25 - 4ms<br>
>>>>>> - PoP to Internet Distance: 50km fiber: 0.25 - 0.5ms<br>
>>>>>> - Total one-way delay: 4.3 - 11.1ms<br>
>>>>>> - Theoretical minimum RTT: 8.6ms - 22.2ms, call it 15.4ms<br>
>>>>>><br>
>>>>>> This includes no transmission delay, queuing delay,<br>
>>>>>> processing/fragmentation/reassembly/etc, and no time-division<br>
>>>>> multiplexing.<br>
>>>>>><br>
>>>>>> On Fri, Jul 16, 2021 at 10:09 AM David Lang <<a href="mailto:david@lang.hm" target="_blank">david@lang.hm</a>> wrote:<br>
>>>>>><br>
>>>>>>> I think it depends on if you are looking at datacenter-to-datacenter<br>
>>>>>>> latency of<br>
>>>>>>> home to remote datacenter latency :-)<br>
>>>>>>><br>
>>>>>>> my rule of thumb for cross US ping time has been 80-100ms latency<br>
>>>>> (but<br>
>>>>>>> it's been<br>
>>>>>>> a few years since I tested it).<br>
>>>>>>><br>
>>>>>>> I note that an article I saw today said that Elon is saying that<br>
>>>>> latency<br>
>>>>>>> will<br>
>>>>>>> improve significantly in the near future, that up/down latency is<br>
>>>>> ~20ms<br>
>>>>>>> and the<br>
>>>>>>> additional delays pushing it to the 80ms range are 'stupid packet<br>
>>>>> routing'<br>
>>>>>>> problems that they are working on.<br>
>>>>>>><br>
>>>>>>> If they are still in that level of optimization, it doesn't surprise<br>
>>>>> me<br>
>>>>>>> that<br>
>>>>>>> they haven't really focused on the bufferbloat issue, they have more<br>
>>>>>>> obvious<br>
>>>>>>> stuff to fix first.<br>
>>>>>>><br>
>>>>>>> David Lang<br>
>>>>>>><br>
>>>>>>><br>
>>>>>>> On Fri, 16 Jul 2021, Wheelock, Ian wrote:<br>
>>>>>>><br>
>>>>>>>> Date: Fri, 16 Jul 2021 10:21:52 +0000<br>
>>>>>>>> From: "Wheelock, Ian" <<a href="mailto:ian.wheelock@commscope.com" target="_blank">ian.wheelock@commscope.com</a>><br>
>>>>>>>> To: David Lang <<a href="mailto:david@lang.hm" target="_blank">david@lang.hm</a>>, David P. Reed <<a href="mailto:dpreed@deepplum.com" target="_blank">dpreed@deepplum.com</a>><br>
>>>>>>>> Cc: "<a href="mailto:starlink@lists.bufferbloat.net" target="_blank">starlink@lists.bufferbloat.net</a>" <<br>
>>>>> <a href="mailto:starlink@lists.bufferbloat.net" target="_blank">starlink@lists.bufferbloat.net</a>><br>
>>>>>>>> Subject: Re: [Starlink] Starlink and bufferbloat status?<br>
>>>>>>>><br>
>>>>>>>> Hi David<br>
>>>>>>>> In terms of the Latency that David (Reed) mentioned for California<br>
>>>>> to<br>
>>>>>>> Massachusetts of about 17ms over the public internet, it seems a bit<br>
>>>>> faster<br>
>>>>>>> than what I would expect. My own traceroute via my VDSL link shows<br>
>>>>> 14ms<br>
>>>>>>> just to get out of the operator network.<br>
>>>>>>>><br>
>>>>>>>> <a href="https://www.wondernetwork.com" rel="noreferrer" target="_blank">https://www.wondernetwork.com</a> is a handy tool for checking<br>
>>>>> geographic<br>
>>>>>>> ping perf between cities, and it shows a min of about 66ms for pings<br>
>>>>>>> between Boston and San Diego<br>
>>>>>>> <a href="https://wondernetwork.com/pings/boston/San%20Diego" rel="noreferrer" target="_blank">https://wondernetwork.com/pings/boston/San%20Diego</a> (so about 33ms<br>
>>>>> for<br>
>>>>>>> 1-way transfer).<br>
>>>>>>>><br>
>>>>>>>> Distance wise this is about 4,100 KM (2,500 M), and @2/3 speed of<br>
>>>>> light<br>
>>>>>>> (through a pure fibre link of that distance) the propagation time is<br>
>>>>> just<br>
>>>>>>> over 20ms. If the network equipment between the Boston and San Diego<br>
>>>>> is<br>
>>>>>>> factored in, with some buffering along the way, 33ms does seem quite<br>
>>>>>>> reasonable over the 20ms for speed of light in fibre for that 1-way<br>
>>>>> transfer<br>
>>>>>>>><br>
>>>>>>>> -Ian Wheelock<br>
>>>>>>>><br>
>>>>>>>> From: Starlink <<a href="mailto:starlink-bounces@lists.bufferbloat.net" target="_blank">starlink-bounces@lists.bufferbloat.net</a>> on behalf<br>
>>>>> of<br>
>>>>>>> David Lang <<a href="mailto:david@lang.hm" target="_blank">david@lang.hm</a>><br>
>>>>>>>> Date: Friday 9 July 2021 at 23:59<br>
>>>>>>>> To: "David P. Reed" <<a href="mailto:dpreed@deepplum.com" target="_blank">dpreed@deepplum.com</a>><br>
>>>>>>>> Cc: "<a href="mailto:starlink@lists.bufferbloat.net" target="_blank">starlink@lists.bufferbloat.net</a>" <<br>
>>>>> <a href="mailto:starlink@lists.bufferbloat.net" target="_blank">starlink@lists.bufferbloat.net</a>><br>
>>>>>>>> Subject: Re: [Starlink] Starlink and bufferbloat status?<br>
>>>>>>>><br>
>>>>>>>> IIRC, the definition of 'low latency' for the FCC was something like<br>
>>>>>>> 100ms, and Musk was predicting <40ms. roughly competitive with<br>
>>>>> landlines,<br>
>>>>>>> and worlds better than geostationary satellite (and many<br>
>>>>>>>> External (mailto:<a href="mailto:david@lang.hm" target="_blank">david@lang.hm</a>)<br>
>>>>>>>><br>
>>>>>>><br>
>>>>> <a href="https://shared.outlook.inky.com/report?id=Y29tbXNjb3BlL2lhbi53aGVlbG9ja0Bjb21tc2NvcGUuY29tL2I1MzFjZDA4OTZmMWI0Yzc5NzdiOTIzNmY3MTAzM2MxLzE2MjU4NzE1NDkuNjU=#key=19e8545676e28e577c813de83a4cf1dc" rel="noreferrer" target="_blank">https://shared.outlook.inky.com/report?id=Y29tbXNjb3BlL2lhbi53aGVlbG9ja0Bjb21tc2NvcGUuY29tL2I1MzFjZDA4OTZmMWI0Yzc5NzdiOTIzNmY3MTAzM2MxLzE2MjU4NzE1NDkuNjU=#key=19e8545676e28e577c813de83a4cf1dc</a><br>
>>>>>>> <a href="https://www.inky.com/banner-faq/" rel="noreferrer" target="_blank">https://www.inky.com/banner-faq/</a> <a href="https://www.inky.com" rel="noreferrer" target="_blank">https://www.inky.com</a><br>
>>>>>>>><br>
>>>>>>>> IIRC, the definition of 'low latency' for the FCC was something like<br>
>>>>>>> 100ms, and<br>
>>>>>>>> Musk was predicting <40ms.<br>
>>>>>>>><br>
>>>>>>>> roughly competitive with landlines, and worlds better than<br>
>>>>> geostationary<br>
>>>>>>>> satellite (and many wireless ISPs)<br>
>>>>>>>><br>
>>>>>>>> but when doing any serious testing of latency, you need to be wired<br>
>>>>> to<br>
>>>>>>> the<br>
>>>>>>>> router, wifi introduces so much variability that it swamps the<br>
>>>>> signal.<br>
>>>>>>>><br>
>>>>>>>> David Lang<br>
>>>>>>>><br>
>>>>>>>> On Fri, 9 Jul 2021, David P. Reed wrote:<br>
>>>>>>>><br>
>>>>>>>>> Date: Fri, 9 Jul 2021 14:40:01 -0400 (EDT)<br>
>>>>>>>>> From: David P. Reed <<a href="mailto:dpreed@deepplum.com" target="_blank">dpreed@deepplum.com</a>><br>
>>>>>>>>> To: <a href="mailto:starlink@lists.bufferbloat.net" target="_blank">starlink@lists.bufferbloat.net</a><br>
>>>>>>>>> Subject: [Starlink] Starlink and bufferbloat status?<br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>> Early measurements of performance of Starlink have shown<br>
>>>>> significant<br>
>>>>>>> bufferbloat, as Dave Taht has shown.<br>
>>>>>>>>><br>
>>>>>>>>> But... Starlink is a moving target. The bufferbloat isn't a<br>
>>>>> hardware<br>
>>>>>>> issue, it should be completely manageable, starting by simple<br>
>>>>> firmware<br>
>>>>>>> changes inside the Starlink system itself. For example, implementing<br>
>>>>>>> fq_codel so that bottleneck links just drop packets according to the<br>
>>>>> Best<br>
>>>>>>> Practices RFC,<br>
>>>>>>>>><br>
>>>>>>>>> So I'm hoping this has improved since Dave's measurements. How<br>
>>>>> much has<br>
>>>>>>> it improved? What's the current maximum packet latency under full<br>
>>>>>>> load, Ive heard anecdotally that a friend of a friend gets 84 msec.<br>
>>>>> *ping<br>
>>>>>>> times under full load*, but he wasn't using flent or some other<br>
>>>>> measurement<br>
>>>>>>> tool of good quality that gives a true number.<br>
>>>>>>>>><br>
>>>>>>>>> 84 msec is not great - it's marginal for Zoom quality experience<br>
>>>>> (you<br>
>>>>>>> want latencies significantly less than 100 msec. as a rule of thumb<br>
>>>>> for<br>
>>>>>>> teleconferencing quality). But it is better than Dave's measurements<br>
>>>>> showed.<br>
>>>>>>>>><br>
>>>>>>>>> Now Musk bragged that his network was "low latency" unlike other<br>
>>>>> high<br>
>>>>>>> speed services, which means low end-to-end latency. That got him<br>
>>>>>>> permission from the FCC to operate Starlink at all. His number was, I<br>
>>>>>>> think, < 5 msec. 84 is a lot more than 5. (I didn't believe 5,<br>
>>>>> because he<br>
>>>>>>> probably meant just the time from the ground station to the terminal<br>
>>>>>>> through the satellite. But I regularly get 17 msec. between<br>
>>>>> California and<br>
>>>>>>> Massachusetts over the public Internet)<br>
>>>>>>>>><br>
>>>>>>>>> So 84 might be the current status. That would mean that someone at<br>
>>>>>>> Srarlink might be paying some attention, but it is a long way from<br>
>>>>> what<br>
>>>>>>> Musk implied.<br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>> PS: I forget the number of the RFC, but the number of packets<br>
>>>>> queued on<br>
>>>>>>> an egress link should be chosen by taking the hardware bottleneck<br>
>>>>>>> throughput of any path, combined with an end-to-end Internet<br>
>>>>> underlying<br>
>>>>>>> delay of about 10 msec. to account for hops between source and<br>
>>>>> destination.<br>
>>>>>>> Lets say Starlink allocates 50 Mb/sec to each customer, packets are<br>
>>>>> limited<br>
>>>>>>> to 10,000 bits (1500 * 8), so the outbound queues should be limited<br>
>>>>> to<br>
>>>>>>> about 0.01 * 50,000,000 / 10,000, which comes out to about 250<br>
>>>>> packets from<br>
>>>>>>> each terminal of buffering, total, in the path from terminal to<br>
>>>>> public<br>
>>>>>>> Internet, assuming the connection to the public Internet is not a<br>
>>>>> problem.<br>
>>>>>>>> _______________________________________________<br>
>>>>>>>> Starlink mailing list<br>
>>>>>>>> <a href="mailto:Starlink@lists.bufferbloat.net" target="_blank">Starlink@lists.bufferbloat.net</a><br>
>>>>>>>><br>
>>>>>>><br>
>>>>> <a href="https://secure-web.cisco.com/1sNc_-1HhGCW7xdirt_lAoAy5Nn5T6UA85Scjn5BR7QHXtumhrf6RKn78SuRJG7DUKI3duggU9g6hJKW-Ze07HTczYqB9mBpIeALqk5drQ7nMvM8K7JbWfUbPR7JSNrI75UjiNXQk0wslBfoOTvkMlRj5eMOZhps7DMGBRQTVAeTd5vwXoQtDgS6zLCcJkrcO2S9MRSCC4f1I17SzgQJIwqo3LEwuN6lD-pkX0MFLqGr2zzsHw5eapd-VBlHu5reC4-OEn2zHkb7HNzS1pcueF6tsUE1vFRsWs2SIOwU5MvbKe3J3Q6NRQ40cHI1AGd-i/https://lists.bufferbloat.net/listinfo/starlink" rel="noreferrer" target="_blank">https://secure-web.cisco.com/1sNc_-1HhGCW7xdirt_lAoAy5Nn5T6UA85Scjn5BR7QHXtumhrf6RKn78SuRJG7DUKI3duggU9g6hJKW-Ze07HTczYqB9mBpIeALqk5drQ7nMvM8K7JbWfUbPR7JSNrI75UjiNXQk0wslBfoOTvkMlRj5eMOZhps7DMGBRQTVAeTd5vwXoQtDgS6zLCcJkrcO2S9MRSCC4f1I17SzgQJIwqo3LEwuN6lD-pkX0MFLqGr2zzsHw5eapd-VBlHu5reC4-OEn2zHkb7HNzS1pcueF6tsUE1vFRsWs2SIOwU5MvbKe3J3Q6NRQ40cHI1AGd-i/https://lists.bufferbloat.net/listinfo/starlink</a><br>
>>>>>>>><br>
>>>>>>>> _______________________________________________<br>
>>>>>>> Starlink mailing list<br>
>>>>>>> <a href="mailto:Starlink@lists.bufferbloat.net" target="_blank">Starlink@lists.bufferbloat.net</a><br>
>>>>>>> <a href="https://lists.bufferbloat.net/listinfo/starlink" rel="noreferrer" target="_blank">https://lists.bufferbloat.net/listinfo/starlink</a><br>
>>>>>>><br>
>>>>>><br>
>>>>><br>
>>>> _______________________________________________<br>
>>>> Starlink mailing list<br>
>>>> <a href="mailto:Starlink@lists.bufferbloat.net" target="_blank">Starlink@lists.bufferbloat.net</a><br>
>>>> <a href="https://lists.bufferbloat.net/listinfo/starlink" rel="noreferrer" target="_blank">https://lists.bufferbloat.net/listinfo/starlink</a><br>
>>>><br>
>>>> _______________________________________________<br>
>>> Starlink mailing list<br>
>>> <a href="mailto:Starlink@lists.bufferbloat.net" target="_blank">Starlink@lists.bufferbloat.net</a><br>
>>> <a href="https://lists.bufferbloat.net/listinfo/starlink" rel="noreferrer" target="_blank">https://lists.bufferbloat.net/listinfo/starlink</a><br>
>>><br>
>> _______________________________________________<br>
>> Starlink mailing list<br>
>> <a href="mailto:Starlink@lists.bufferbloat.net" target="_blank">Starlink@lists.bufferbloat.net</a><br>
>> <a href="https://lists.bufferbloat.net/listinfo/starlink" rel="noreferrer" target="_blank">https://lists.bufferbloat.net/listinfo/starlink</a><br>
>><br>
>_______________________________________________<br>
Starlink mailing list<br>
<a href="mailto:Starlink@lists.bufferbloat.net" target="_blank">Starlink@lists.bufferbloat.net</a><br>
<a href="https://lists.bufferbloat.net/listinfo/starlink" rel="noreferrer" target="_blank">https://lists.bufferbloat.net/listinfo/starlink</a><br>
</blockquote></div></div>
</div>