[Starlink] dynamically adjusting cake to starlink

Wed Jun 9 11:32:46 EDT 2021

>  At any given time, a satellite may have ~10k cells under its footprint
that it can serve, and GSO exclusions can exclude between 15% at high
latitudes and 25% near the Equator. It has three arrays for downlinks and
one array for uplinks, with 16 beams per array. Thus, it can train up to 48
beams towards the ground, and receive from up to 16 locations on the ground
at any given time. Thus, to cover each cell, there is a physical TDM
element, plus an additional multiplexing element for each terminal in each
cell. If we assume a satellite has a capacity of ~20 Gbps, limited by the
Ka links from satellite to gateway, each spot beam is capable of delivering
~416 Mbps through each spot beam. If we assume symmetrical performance, the
16 uplink beams from the one panel would “take in” ~416 Mbps per beam.

I think based on the gateway docs from Anatel, we know this number is
probably closer to ~13-16Gbps per Ka antenna, with likely only 1 active.
Given not every user would be receiving data equally, the peak speed per
cell is probably >400Mbps, we've seen speedtests over 500Mbps.
I'm also guessing radio resource planning is pretty dynamic, or will need
to be, I doubt it's a fixed 200/70ms kinda thing.

On Wed, Jun 9, 2021 at 5:17 AM Mike Puchol <mike at starlink.sx> wrote:

> Greetings Dave,
>
> I have just been introduced into the world of bufferbloat - something I’ve
> had to deal with in my WISP, but never had a proper name for. I’m an
> aerospace engineer by origin, so please forgive my quite likely blunders in
> this space!
>
> As for the topic at hand, I understand the concept - in essence, if we can
> find what the Dishy - satellite - gateway triumvirate is doing in terms of
> capacity and buffering, we can apply that information into our own router’s
> buffer/queue strategy. I have taken a quick look at the document, and there
> is one assumption that is wrong, AFAIK, which is that the satellites are
> “bent pipes” - they actually process packets, which means they are an
> active component of buffering and queuing in the path (for optical links to
> work, satellites would necessarily have to process packets).
>
> At any given time, a satellite may have ~10k cells under its footprint
> that it can serve, and GSO exclusions can exclude between 15% at high
> latitudes and 25% near the Equator. It has three arrays for downlinks and
> one array for uplinks, with 16 beams per array. Thus, it can train up to 48
> beams towards the ground, and receive from up to 16 locations on the ground
> at any given time. Thus, to cover each cell, there is a physical TDM
> element, plus an additional multiplexing element for each terminal in each
> cell. If we assume a satellite has a capacity of ~20 Gbps, limited by the
> Ka links from satellite to gateway, each spot beam is capable of delivering
> ~416 Mbps through each spot beam. If we assume symmetrical performance, the
> 16 uplink beams from the one panel would “take in” ~416 Mbps per beam.
>
> Thus, a satellite is effectively limited to a 75/25 duty cycle between
> downlink and uplink, as it can paint each cell three times for download,
> for every time it receives from it.
>
> We then get to the killer - how many cells can a satellite effectively
> serve? If we take 8000 cells as an average, each downlink spot beam would
> need to care for ~167 cells, and in uplink ~500 cells. Focusing on uplink,
> each cell gets ~2ms of “beam time” to transmit, which is ridiculously low.
> From my tracker simulations, certain areas are served by around 8
> satellites at any given time, so catering for satellite spacing, we could
> assume 10ms of “beam time”. Still to low, as this would only allow for ~170
> packets.
>
> My current estimate from back of the envelope calculations is that
> Starlink uses cell TDM factors per beam of 1 to 5, depending on served
> customer density under the footprint. Thus, worst case, a satellite is
> giving a cell ~200ms of downlink time and ~67ms of uplink time. A Dishy
> terminal will need to buffer traffic while it gets its turn on a spot beam.
>
> Does this make sense, also from observations?
>
> Best,
>
> Mike
> On Jun 9, 2021, 11:12 +0200, Dave Taht <davet at teklibre.net>, wrote:
>
> Dear Mike:
>
> The biggest thing we need is something that can pull the right stats off
> the dishy and dynamically adjust cake (on outbound at least) to have the
> right amount of buffering for the available bandwidth, so as to make for
> better statistical multiplexing (FQ) and active queue management (AQM)
>
> It’s pretty simple: in mangled shell script syntax:
>
> while up, down = getstats()
> do
> tc qdisc change dev eth0 root cake bandwidth $up
> tc qdisc change dev ifb0 root cake bandwidth $down
> done
>
> Which any router directly in front of the dishy can do (which is what
> we’ve been doing)
>
> But whatever magic “getstats()” would need to do is unclear from the stats
> we get out of it, and a better alternative would be for the dishy itself
> and their headends to be doing this with “BQL" backpressure.
>
> As for the huge reductions of latency and jitter under working load, and a
> vast improvement in QoE - for what we’ve been able to achieve thus far, see
> appendix A here:
>
>
> https://docs.google.com/document/d/1rVGC-iNq2NZ0jk4f3IAiVUHz2S9O6P-F3vVZU2yBYtw/edit?usp=sharing
>
> We’ve got plenty more data
> on uploads and downloads and other forms of traffic (starlink is
> optimizing for ping, only, over ipv6. Sigh)…
>
> … and a meeting with some starlink execs at 11AM today.
>
> I’m pretty sure at this point we will be able to make a massive
> improvement in starlink’s network design very quickly, after that meeting.
>
> On Jun 8, 2021, at 2:54 PM, Nathan Owens <nathan at nathan.io> wrote:
>
> I invited Mike, the creator of the site (starlink.sx) to join the list -
> he’s put a crazy amount of work in to figure out which sats are active
> (with advice from Jonathan McDowell), programming GSO exclusion bands, etc.
> His dayjob is in the ISP business.
>
> On Sat, Jun 5, 2021 at 8:31 PM Darrell Budic <budic at onholyground.com>
> wrote:
>
>> https://starlink.sx if you have’t seen it yet. You can locate yourself,
>> and it will make some educated guesses about which satellite to which
>> ground station you’re using. Interesting to see the birds change and the
>> links move between ground stations, lots going on to make these things work.
>> _______________________________________________
>> Starlink mailing list
>> Starlink at lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/starlink
>>
> _______________________________________________
> Starlink mailing list
> Starlink at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink
>
>
> _______________________________________________
> Starlink mailing list
> Starlink at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/starlink/attachments/20210609/a68bbebf/attachment.html>