[Starlink] fiber IXPs in space

[David Lang]

On Mon, 17 Apr 2023, Ulrich Speidel wrote:

> On 17/04/2023 2:34 pm, David Lang wrote:
>> On Mon, 17 Apr 2023, Ulrich Speidel wrote:
>> 
>>> On 17/04/2023 1:04 pm, David Lang wrote:
>>>> I think it is going to be fairly common, but the beauty of the idea is 
>>>> that you don't have to risk much to try it. Long lived DNS answers (and 
>>>> especially root servers and TLD servers) can trivially be mirrored to the 
>>>> satellites, and you can experiment with caching to see what sort of hit 
>>>> rate you get. Even if you don't cache a lot of the CDN type traffic, it 
>>>> should still be a win to have the longer term stuff there.
>>> 
>>> Yes - but root servers and TLD servers also get cached a long time at the 
>>> clients. If each of your clients needs a root server and a few TLD lookups 
>>> a day, it's not a huge gain in terms of performance.
>>> 
>>> It is however a significant step up in terms of complexity. E.g., your 
>>> satellite-based DNS would have to point you at the TLD server that is 
>>> topologically closest to your Starlink gateway, or risk a potentially much 
>>> longer RTT for the lookup. So you'd need to maintain a list of TLD 
>>> instances on your satellite-based DNS and return the one that corresponds 
>>> to what your gateway-based DNS would get. Sure possible but more complex 
>>> than a bog standard DNS server in a fixed network.
>> 
>> really? I don't think the root and TLD servers do any geolocation, I think 
>> they have fixed answers and rely on anycast addresses to find the closest 
>> one. Adding those anycast addresses to the satellites would be transpoarent 
>> to all users (assuming the satellites are operating at the IP layer, the 
>> old bent-pipe approach did not, but once you have routing in space via the 
>> laser links...)
>
> So what you're suggesting would be a generic anycast-based set of DNS records 
> to be kept on all satellites?
>
> Hm. My terrestrial RTT to the closest 8.8.8.8 and 8.8.4.4 anycast DNS servers 
> is about 28 ms, indicating that they're over in Australia. No big surprise 
> here, but that RTT isn't all too different from what I'd expect from a 
> Starlink connection. That compares to < 1 ms to my local resolver.
>
> Would an anycast-based generic DNS give us better RTT than a gateway-based 
> DNS with full ability to cache local information?

are you going to replicate the root server data to your gateway? what's a fairly 
trivial amount of data for a server (including one in space) can be a 
significant amount of data to store on an AP/gateway. If you have your gateway 
caching it, you probably won't hit the space based servers in any case.

in part it would depend on how much the gateway gets turned off. (some people do 
it for security, people who are mobile power things down to move)

> Even with the laser links, it's worth repeating that most of Starlink's 
> operation is still bent pipe, and likely to continue to be so in future. 
> There is little evidence that the lasers are currently being used for 
> anything but remote location access.
>
> That said, bent pipe isn't the same as bent pipe. Analog telecom GEO sats 
> used to do physical layer bent pipe with the satellite incapable of IP. If 
> you have a satellite capable of IP, you can still call it a bent pipe as long 
> as it downlinks traffic that has come in on the uplink. You could even run an 
> IP-based tunnel between dishy and gateway that makes it appear like a single 
> link for the next network layer up the stack. And do the same for lasers. 
> There'd be a number of advantages in this.

and bent pipe for normal operations doesn't mean that you can't watch for an 
anycast address to serve locally.

>>>>> Practically speaking, we know from various sources that each Starlink 
>>>>> satellite provides - ballpark - a couple of dozen Gb/s in capacity, and 
>>>>> that active users on a "busy" satellite see a couple of dozen Mb/s of 
>>>>> that. "Busy" means most active users, and so we can conclude that the 
>>>>> number of users per satellite who use any site is at most around 1000. 
>>>>> The subset of users navigating to new sites among them is probably in 
>>>>> the low 100's at best. If we're excluding new sites that aren't dynamic, 
>>>>> we're probably down to a couple of dozen new static sites being queried 
>>>>> per satellite pass. How many of these queries will be duplicates? Not a 
>>>>> lot. If we're including sites that are dynamic, we're still not getting 
>>>>> a huge probability of cache entry re-use.
>>>> 
>>>> I think that each user is typically going to use a lot less than the 
>>>> 'couple dozen MB' that is the limits that we see, so the number of users 
>>>> in a cell would be much higher.
>>> Yes, but these users aren't "active" in the sense that they will be firing 
>>> off DNS queries during the pass...
>> 
>> even users who are using the connection aren't going to be using dozens of 
>> MB of bandwidth.
> Yes - but that should reflect as a lower level of DNS usage in most cases, 
> shouldn't it?

not necessarily, if they are browsing pages that are made of small pieces, they 
can generate lots of connections and lots of DNS lookups, but not consume a lot 
of bandwidth. You don't get to the dozens of MB range without some bulk download 
traffic in there.

David Lang