[Starlink] thinking about the laser links again

[David Lang]

On Wed, 27 Oct 2021, Ulrich Speidel wrote:

> Things that change for you if you are a Starlink satellite:
>
> - Cells you can service (and therefore hosts attached to Dishys you can 
> reach). You may be over Brazil now, the Atlantic next, then over the UK, and 
> then over Germany, and so on... So your on-board routing table would need to 
> reflect that. You never spend more than a minutes or so over the same cell.
> - Ground station gateways you can attach to - similar timing here.

however, you know where you are, so the territory you cover is predictable, it 
doesn't have to be discovered.

> - Satellites on neighbouring orbital planes in the same shell that are in the 
> northbound part of their orbit while you're heading south (and vice versa) 
> come within range and disappear again.
> - Neighbouring satellites in different shells.

either by altitude or inclination

> Things that don't change:
>
> - The satellite in front and behind you in the same orbit.
> - Neighbouring satellites orbiting in the same direction in adjacent orbital 
> planes. These require tracking but are always in range.
>
> So, if we're a satellite, part of our routing table changes every few seconds 
> because some node that was a neighbour no longer is, and a node that wasn't 
> now becomes a neighbour. And because nothing in our constellation-spawned 
> network resembles a tree structure, we'd need something like BGP to propagate 
> that routing information through the network. Because we may be the only 
> satellite that can currently serve a particular destination host, that 
> information needs to propagate through the network, so other nodes in the 
> network need to know that they need to send traffic to that destination to 
> us. I say "something like BGP" here as the network is proprietary, so doesn't 
> have to stick to any open standards.

this is where I lean into something like allocating ipv6 blocks based on 
lat/long and you route based on your location to satellites 'in the right 
direction' to reduce this problem from needing global knowledge to only needing 
nearby knowledge.

> Now our neighbours are the first that need to know that we're the new kid on 
> the block to talk directly to Dave's Dishy, and sending them an update every 
> few seconds wouldn't be too onerous. But the problem is that our neighbours, 
> too, are on the move, so we're basically forced to propagate our update (and 
> everyone else's) throughout the entire network of thousands of satellites. 
> Moreover, if we consider the fact that along a forwarding path of a dozen 
> satellites or more between, say Tokyo and New York, all satellites are on the 
> move, then losing only one link on that path can render a route completely 
> infeasible. And while our individual satellite's neighbourhood changes every 
> few seconds, a path like this might change on a sub-second scale.

this is why I don't think trying to craft routes to specific nodes is a good 
idea. head in the right direction until you reach a satellite that can service 
the endpoint.

endpoints need to register themselves with a satellite, that location determines 
their IPv6 block, when they move, they register the change (thus my comment 
about the mobile IP problem)

> So suppose we ignore anything we know about the constellation geometry and we 
> have N satellites. Say we want to use IP routing still with BGP. Then each 
> satellite needs to be its own Autonomous System. That boils down to telling 
> each satellite in the constellation about the connectivity state of 
> essentially every other satellite (or plane). That's an O(N^2) problem, so 
> not very scaleable, especially if we need to do this every few milliseconds 
> (with the required intervals also decreasing with N). So conventional IP 
> routing doesn't work here, basically.

route consolodation (or in this case, lat/long direction) changes this from a 
O(N^2) problem to a 'head in the right direction' problem. you (as the 
satellite) know where you are, know where nearby satellites are and how they are 
moving (you have to know this to aim your laser at them anyway), this should be 
enough to head the traffic in the right direction. There may need to be 
something more between satellites able to seve the same area to handle the last 
couple of hops, but as a general case 'that-a-way' could be 'good enough' 
routing.

>> So the satellite needs to know what are good next hops to send traffic to 
>> that's headed in different directions, not global knowledge of the entire 
>> constellation.
>
> True, but that requires a knowledge of direction and a (tunnel) comms layer 
> below the Internet-wide IP routing. And the question is how that's best 
> designed.

this the 'IP block tied to geo area' to identify this

>> With only 'nearby' node knowledge needed, I could see low-power 
>> omnidirectional RF beacons being used for nodes to advertise their 
>> existance and routing config to the network. I expect that if they were 
>> doing this, that it would show up in their FCC filings and everyone would 
>> be talking about it, but it's a thought for future versions.
> I'm speculating here, but from an engineering point of view it makes most 
> sense to go with an almanach approach to constellation configuration, i.e., 
> let each bird know where all other birds are at any one time by providing 
> their orbital parameters to every satellite (slow rate of change and easy 
> O(N) computation on the satellite). Dishys have a built-in GPS receiver, so 
> can basically figure out which cell they're in (once movement between cells 
> is allowed). Currently, I suspect that Starlink run a protocol where each 
> bird knows which cell it's responsible for at any one time, and where the 
> bird beacons to Dishys "Cell X, it's me, bird Y" and where the Dishys then 
> get orbital parameters either from the bird or an almanach to align their 
> phased arrays to the bird.

I agree, but I'll add that it's more likely that the dishy knows where it is and 
looks on the air to see what birds it can hear at what signal strength and then 
picks one, rather than it being a purely static assignment (ground clutter 
interfering with the signal in some directions, and things I hear about people 
'tricking' the system to get coverage outside the supported areas)

>> All of this requies some way to tell from the packet where on earth (or in 
>> space) the destination is. If you tunnel over IPv6, you can have enough IPs 
>> to allocate them based on lat/long/ground/orbit so that the routing nodes 
>> (which know where they are) can know what direction to send things. 
> That's indeed an option, although you'd still need a way of reverse mapping: 
> If I have an IPv6 address I know to be in cell X in Ontario, which of my 
> peers has a route to there?

using the mobile-ip approach, when you connect to the network your dishy 
registers it's location and establishes a tunnel endpoint (the cell you are in). 
when sending to another dishy, there would need to be a lookup to find it's cell 
location, and therefor it's tunnel endpoint. once you have these endpoints, the 
satellites can route the traffic betwen them.

>> Given the existance of mobile endpoints (vehicles, aircraft, spacecraft) 
>> this starts sounding like the mobile IP problem (how can you continue a 
>> persistant connection when moving from network to network), but since all 
>> the entry/exit points are under their control, you can tunnel around and 
>> have a protocol to reconfigure the tunnels as endpoints move.
>
> Indeed, and that tunnel configuration protocol is where the challenge lies 
> ;-)
>
> For mobile networks, this task is a little easier as the network between the 
> entry & exit points is more or less static. Plus, mobile nodes move at most 
> at hundreds of miles per hour, not tens of thousands.

still the case here. the tunnels end on the dishy, which don't move that much 
either (except for spacecraft, which can participate more directly). As I said 
above there is more complexity for the last hop handoff as the dishy is 
switchign from satellite to satellite, but they are aleady doing something 
there.

> BTW: Starlink will probably want to avoid ground relays like the plague. The 
> bulk of their customers won't be stock brokers, and their need will be for 
> bandwidth rather than minimal latency. The bottlenecks here are between 
> ground and birds, and between birds. Basically, the nominal capacity of each 
> Starlink bird is purportedly somewhere around the 20 Gb/s mark in the 
> currently deployed generation. Now, as a rule of thumb, the average Internet 
> user in the world last year used a continuous average of 1 Mb/s of data. If 
> we ignore for a moment that connections are bi-directional and that there is 
> such a thing as peak time, then that means that under the current bent pipe 
> model, each Starlink satellite is restricted to at most 20k users. If they 
> want to beat existing providers on the ground that already offer a few Mb/s, 
> they need to slash that by at least a factor of 5, again being really 
> conservative here.

20k users in a geo area. but the 20k users in the LA area won't affect the 20k 
users in the London area as they are on completely different satellites/ground 
stations.

My sister is a prime starlink customer, the best she can get is 2Mb (Rural Mi), 
she loves the starlink beta. I am in the LA suburbs, but the best I can get 
other than cable is bonded DSL to give me 8/2 (I finally was able to get cable, 
but would get starlink to be my redundant link, and for special events when I'm 
not home).

There are large areas where you are competing against dialup, cell service (with 
it's caps) hughs geo service (with it's caps), or wireless ISPs.

> Now those ground providers do have one competitive advantage, and that is 
> that while their lines are long and low bandwidth, they're not shared, at 
> least not to the closest CDN server farms, where 70% or so of today's traffic 
> originates.

'not shared'?? cable is shared, uplinks are underprovisioned. some places it's 
not bad, others it's horrid (when I first got Internet back around 2002, I was 
on shared cable and switching to 128k SDSL was an improvement in real 
performance, if not advertised rates)

> If two Starlink customers want to watch the same cat video on FB, 
> the video has to traverse the satellite medium twice, whereas if two farmers 
> in the Midwest want to do the same on Verizon, they get two copies off their 
> local CDN and you don't get as much as a blip on regional or international 
> fibre cables (which generally run at rates > 20 Gb/s these days).

that last mile to the farmers may be a lot harder than you think

> So if Starlink want to keep customers happy with dozens of Mb/s on demand, 
> each bird will at most be able to handle a few hundred active customers at a 
> time. Any use of multiple ground relays along a path would cut that number 
> substantially.

if you give each customer 10M (which, if low latency isn't _that_ bad, not 
good for streaming 4k video, but will allow multiple people to stream at lower 
res), a satellite by your numbers will handle 1k users.

I am missing something on your talk about ground relays. a given connection 
would be at most one ground station per endpoint, they aren't going to go to a 
ground station, back to the satellites, to another ground station, just endpoint 
-> satellite network -> ground station -> Internet (and if the far end is also 
on starlink, possibly the same thing at the other end)

Yes, when you have starlink <-> starlink communication, they will want to keep 
it in space, but (unfortunantly), that's not the common usage pattern right now.

David Lang