Re: [Starlink] Starlink "beam spread"

Re: [Starlink] Starlink "beam spread"

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 4915 bytes --]


Hi Dick -
glad you joined in the response, given your experience with spatial multiplexing of RF.
 
On Wednesday, August 31, 2022 3:04pm, starlink-request@lists.bufferbloat.net said:



> Date: Wed, 31 Aug 2022 10:52:15 -0700
> From: "Dick Roy" <dickroy@alum.mit.edu>
> To: "'Mike Puchol'" <mike@starlink.sx>
> Cc: <starlink@lists.bufferbloat.net>
> Subject: Re: [Starlink] Starlink "beam spread"
> Message-ID: <BCE2EFF2444C470C87D9D4C6D4E6E0A9@SRA6>
> Content-Type: text/plain; charset="iso-8859-1"
> On this particular one, the gateway beams are extremely narrow, around 1.5º
> to 2.5º. SpaceX is working on “mega-gateways” where 32 antennas
> will
> co-exist. They are also deploying a new gateway design with a larger
> antenna, and thus narrower beamwidth and more gain, allowing for a
> considerable reduction in TX power.
> 
> [RR] there is a much better way to do this! I sure hope starlink is
> considering it. Large antennas with narrow beam widths are a sledgehammer to
> kill a fly.
> :-)
 I'm pretty sure, from my examination of the dishy array (and the satellite arrays are likely the same) that at the front-end electronics level their use of 64 QAM (six bits of signal quantization in amplitude and phase) isn't gonna allow much signalling in multiple directions at once. That's a sampling theory issue, really.
 
There's a lot of handwaving in the discussion here, mostly focused on "beam width" rather than combined channel capacity. To me that's the "sledgehammer" you are referring to.
 
So that's one aspect. But the other aspect that I'm focused on is not in the arraycom space, it's in the question of how the highly variable (bursty) demand in both directions works out. The low-delay (small packets) at the signalling rate are actually smaller than the number of bits in transit over the 2 msec. path. So the packet level multiplexing here (with more dishy terminals than the 4 240 Mb/sec channels afforded) is also an issue. We aren't talking about one CBR uplink per dishy. It's bitrate is highly variable, given Internet load.
 
Now RRUL from *one dishy* doesn't address this problem properly. You need to look at what happens when all the dishys (uplink and downlink) are wanting to saturate all up and down links. Then you add a new packet that needs low latency (whether it is ACK packet for TCP or a click from a FPS, or an audio frame in a teleconference, latency matters for all of these, and they aren't predictable in advance). How long will it be before an uplink slot is opened? If there is no load, it still requires a clear channel, so it requires waiting till one's turn. Assume that each dishy using the uplink gets one uplink slot in a slot time T. Then the delay till slot available can be short, if the slots are short, all dishys get a turn within 2 msec. or less. But if there is a load in both the up and down direction (and one can easily have such a load from file uploads or file downloads on each dishy, or a fairly big load from UHD video streams to each dishy being "buffered" into a playback buffer on a Smart TV), then latency due to queueing delay can soar.
 
If you don't allow all the dishys to use all the capacity of a satellite, the latency problem gets smaller. But then, of course, you are wasting a bit of capacity to preserve latency for low bitrate traffic.
 
However, high bitrate traffic also has latency requirements. It's not all "background".
 
I'm sure this is something that can be worked out by some algorithm that reaches toward the optimal state by dropping packets to signal TCP (or QUIC) that the transmitter needs to slow down to avoid building up queueing delay. The problem here, though, is that all the focus in this discussion is about "throughput" in a single link, and ignores the burstiness that is driving the multiplexing.
 
My point is that the discussion here has focused entirely on achievable bitrate maximum, with no load and no burstiness.
 
This isn't fixed by having more spatial control, to the extent that is possible. It *is* fixable by suitable Automatic Queue Management in the satellite path (which will be the "bottleneck link" between the home and the public Internet), plus suitable end-to-end protocol congestion management that responds to drops or ECN. (I don't know if the satellite looks at the packet hard enough and has enough state to set the ECN bit or that it has enough state to achieve bloom filter based fair queue dropping).
 
Dave Taht indicates he has some mysterious statistics from (single) dishy experiments. It's a problem that we cannot see into the Starlink black box to understand it fully.
 
But I'm CERTAIN that simplistic analysis about achievable channel bitrates in the abstract will not capture the issues, and that there will be LOTS of systemic problems that are not easily solved by just thinking about rates and beam width.

[-- Attachment #2: Type: text/html, Size: 7749 bytes --]

Re: [Starlink] Starlink "beam spread"

[David Lang]

[-- Attachment #1: Type: text/plain, Size: 1141 bytes --]

On Wed, 31 Aug 2022, David P. Reed via Starlink wrote:

> My point is that the discussion here has focused entirely on achievable bitrate maximum, with no load and no burstiness.

well, to be fair, some of that is in response to 'just looking at the bitrate, 
starlink cannot possibly perform well' posts

> This isn't fixed by having more spatial control, to the extent that is 
> possible.

agreed, the spactial control discussion is just a response to 'it can't 
possibly...' posts

> It *is* fixable by suitable Automatic Queue Management in the 
> satellite path (which will be the "bottleneck link" between the home and the 
> public Internet), plus suitable end-to-end protocol congestion management that 
> responds to drops or ECN. (I don't know if the satellite looks at the packet 
> hard enough and has enough state to set the ECN bit or that it has enough 
> state to achieve bloom filter based fair queue dropping).

if the sats are going to have the ability to route through other satellites, 
they have to be looking at the packets, and once you start doing that, looking 
at ECN should not be a significant load.

David Lang

[-- Attachment #2: Type: text/plain, Size: 149 bytes --]

_______________________________________________
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink

Re: [Starlink] Starlink "beam spread"

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 4842 bytes --]


A couple clarifications below on what I meant.
 
On Wednesday, August 31, 2022 4:28pm, "David Lang" <david@lang.hm> said:



> On Wed, 31 Aug 2022, David P. Reed via Starlink wrote:
> 
> > My point is that the discussion here has focused entirely on achievable
> bitrate maximum, with no load and no burstiness.
> 
> well, to be fair, some of that is in response to 'just looking at the bitrate,
> starlink cannot possibly perform well' posts
Who said that? Well a couple of the people who responded to my initial post about beam spread being the wrong way to look at this seemed to decide they wanted to say that.

> 
> > This isn't fixed by having more spatial control, to the extent that is
> > possible.
> 
> agreed, the spactial control discussion is just a response to 'it can't
> possibly...' posts
Actually, the beam spread (the colorful plots and analysis discussing that that Dave Taht highlighted as being important), is all about "spatial control". That was my original point.
 
> 
> > It *is* fixable by suitable Automatic Queue Management in the
> > satellite path (which will be the "bottleneck link" between the home and the
> > public Internet), plus suitable end-to-end protocol congestion management
> that
> > responds to drops or ECN. (I don't know if the satellite looks at the packet
> > hard enough and has enough state to set the ECN bit or that it has enough
> > state to achieve bloom filter based fair queue dropping).
> 
> if the sats are going to have the ability to route through other satellites,
> they have to be looking at the packets, and once you start doing that, looking
> at ECN should not be a significant load.
 
I agree, but to do good queue management (especially avoiding starvation of some flows) you need to decide what packets on which to set ECN. That means you have to consider the current set of packets in some detail, which requires storing state about some number of IP flows. That's a lot more processing than just looking at a bit. That's what I am referring to here, and I assumed anyone who knows about ECN and AQM and how it works would understand that.
 
I'm not going to reason from "intersatellite" routing being operational until they offer it in operation. It's feasible, sort of. Laser beam aiming is quite different from phased array beam steering, and though they may have tested it between two satellites, that makes it a "link technology" not a network. (you can steer a laser beam by moving lightweight mirrors, I know. But tracking isn't so easy when both satellites are moving relative to each other - it seems like way beyond the technology base that Starlink has put in its satellites so far. But who knows.
--------------
As far as "intersatellite" routing being out there soon, well, there's no evidence it's happening soon. The one thing that may be an indication is that Starlink is claiming it will have achieved extensive maritime coverage in 2023. Since that can't be achieved by placing ground stations over all the oceans involved so that ships can get one "bounce" to some ground station attached to fiber, I'm guessing they may be providing such service by some additional satellite based"backbone" in space.
 
Given the low density of ships, and the fact that ships currently happily tolerate long space delays in the ways they currently offer Internet service far at sea, using "geosynchronous" two way systems positioned in fixed locations over the oceans, it's possible that the Starlink constellation might be using some "backbone" of another structure to get a one hop to groundstation repeater. These wouldn't be LEO, and they might not even be Starlink owned. That's all guesswork on my part.
 
Starlink is very eager to provide coverage at sea because it is a huge marketing win, and probably doable without too much effort. They announced a partnership with Royal Carribean a couple days ago.
 
Satellite "mesh routing" doesn't need to be made to work to provide maritime service away from land. If there's not too much aggregate load.
 
What's interesting to me is that their coverage map definitely doesn't cover Africa, South America, Cuba, large parts of Asia, and it isn't planned - if they had "mesh routing" working among satellites, those would be easy. But instead, they seem to be focused on the satellite one-bounce architecture (what the satellite industry calls "bent-pipe" however it is done).
 
A reason we don't do dynamic mesh routing in the public internet is that backbone concentration works very well as an engineering solution. I suspect that the Starlink guys are taking the lower risk path.

> 
> David Lang_______________________________________________
> Starlink mailing list
> Starlink@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink
> 

[-- Attachment #2: Type: text/html, Size: 7757 bytes --]

Re: [Starlink] Starlink "beam spread"

[David Lang]

On Wed, 31 Aug 2022, David P. Reed wrote:

> What's interesting to me is that their coverage map definitely doesn't cover 
> Africa, South America, Cuba, large parts of Asia, and it isn't planned - if 
> they had "mesh routing" working among satellites, those would be easy. But 
> instead, they seem to be focused on the satellite one-bounce architecture 
> (what the satellite industry calls "bent-pipe" however it is done).

The countries covered in the coverage map seems to be as much or more restricted 
by regulations as anything technical.

David Lang

Re: [Starlink] Starlink "beam spread"

[Mike Puchol]

[-- Attachment #1: Type: text/plain, Size: 2916 bytes --]

The primary reason for -not- offering service in any given country is, primarily, regulatory (see the South Africa case). Once they
> I'm not going to reason from "intersatellite" routing being operational until they offer it in operation. It's feasible, sort of. Laser beam aiming is quite different from phased array beam steering, and though they may have tested it between two satellites, that makes it a "link technology" not a network. (you can steer a laser beam by moving lightweight mirrors, I know. But tracking isn't so easy when both satellites are moving relative to each other - it seems like way beyond the technology base that Starlink has put in its satellites so far. But who knows.

We operate several of these in Kenya: https://x.company/projects/taara

They offer 20 Gbps at distances of 20km, and they operate under considerably more vibration, motion, and scintillation than you have in space. They have no issue keeping track of each other once initial acquisition is made. SpaceX launched 10 satellites into polar orbit in Jan 2021, which it used to test and characterize the ISL optical heads - you could see them positioning the satellites in configurations to test side-looking (thus cross-plane), and at different altitudes (cross-shell), and even parallel links to characterize hardware differences (we did this with ours in Kenya too). It was fascinating to watch. I’m quite certain the least problem for Starlink (unless they made major boo-boos in hardware or software) is acquisition and tracking.

A very good book (but not cheap) on the topic is "Free Space Optical Communication” by Hemani Kaushal.
> As far as "intersatellite" routing being out there soon, well, there's no evidence it's happening soon.

There is circumstantial evidence from a user in Nigeria that was getting service and exiting via London, there is no evidence that any of the gateways in Nigeria are operational, so ISL could have played a role: https://www.reddit.com/r/Starlink/comments/wwg0nc/starlink_speed_test_in_nigeria/

Best,

Mike
On Aug 31, 2022, 23:33 +0200, David Lang via Starlink <starlink@lists.bufferbloat.net>, wrote:
> On Wed, 31 Aug 2022, David P. Reed wrote:
>
> > What's interesting to me is that their coverage map definitely doesn't cover
> > Africa, South America, Cuba, large parts of Asia, and it isn't planned - if
> > they had "mesh routing" working among satellites, those would be easy. But
> > instead, they seem to be focused on the satellite one-bounce architecture
> > (what the satellite industry calls "bent-pipe" however it is done).
>
> The countries covered in the coverage map seems to be as much or more restricted
> by regulations as anything technical.
>
> David Lang
> _______________________________________________
> Starlink mailing list
> Starlink@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink

[-- Attachment #2: Type: text/html, Size: 4071 bytes --]

Re: [Starlink] Starlink "beam spread"

[Ulrich Speidel]

[-- Attachment #1: Type: text/plain, Size: 840 bytes --]

On 1/09/2022 7:05 pm, Mike Puchol via Starlink wrote:
>
> There is circumstantial evidence from a user in Nigeria that was 
> getting service and exiting via London, there is no evidence that any 
> of the gateways in Nigeria are operational, so ISL could have played a 
> role: 
> https://www.reddit.com/r/Starlink/comments/wwg0nc/starlink_speed_test_in_nigeria/ 
> <https://www.reddit.com/r/Starlink/comments/wwg0nc/starlink_speed_test_in_nigeria/>
>
Where does it mention that it was exiting via London?

-- 

****************************************************************
Dr. Ulrich Speidel

School of Computer Science

Room 303S.594 (City Campus)

The University of Auckland
u.speidel@auckland.ac.nz  
http://www.cs.auckland.ac.nz/~ulrich/
****************************************************************



[-- Attachment #2: Type: text/html, Size: 1611 bytes --]

Re: [Starlink] Starlink "beam spread"

[David Lang]

On Wed, 31 Aug 2022, David P. Reed wrote:

> I'm not going to reason from "intersatellite" routing being operational until 
> they offer it in operation. It's feasible, sort of. Laser beam aiming is quite 
> different from phased array beam steering, and though they may have tested it 
> between two satellites, that makes it a "link technology" not a network. (you 
> can steer a laser beam by moving lightweight mirrors, I know. But tracking 
> isn't so easy when both satellites are moving relative to each other - it 
> seems like way beyond the technology base that Starlink has put in its 
> satellites so far. But who knows.

They have been launching laser enabled satellites for a while now. I suspect
that the only way we will really know when they are enabled is when we see
coverage expand to the poles and mid-ocean (unless they make announcements about
it)

I doubt that they would be launching laser enabled satellites that could not
track each other.

David Lang

[Starlink] CDNs in space!

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 4081 bytes --]


Having looked into this a lot, CDNs don't account for very much Internet traffic. There's a lot of marketing propaganda out there that suggests this might be true, but when you try to track down the source, it's almost always from an old slide deck presented by a company that sells CDN servers.
 
That doesn't mean that CDN servers don't help a fair bit, but no, today's Internet wouldn't fall apart if we didn't have CDN servers withint 2 msec. of the edge.
 
Also, CDN's need to be BIG to hold all the videos that people might choose to watch at any particular time.
 
So I'm just pointing out that the business case for CDN's in space to merely solve Starlink's potential issues is probably not great. Maybe 10 years from now. Maybe never. The idea that everyone watches TV and the same few seconds of content of a few shows that are extraordinarily popular - well, that dog don't hunt. It doesn't justify multicast either.
 
So let's improve the discussion here. The Internet, for the forseeable future, at the edge, is unicast.
 
CDN's reduce costs for big companies like Netflix, not because they are "close to the watcher" but because it is easier to not have everything centralized in a single point of failure and needing a huge pipe because all customers aggregate demand into their central HQ.
 
There are some advantages at the top level peering in distributing CDN sites into the networks of various access providers (like Comcast) because you can get cheaper pricing.
 
But lets think about CDNs in space. They don't deal with the actual bottleneck in each satellite to ground. The uplink from central ground stations has LOTS of capacity, it won't be the bottleneck if Starlink or any other satellite service for that matter balances its design.
 
And if they get inter-sat links (laser or maybe RF, if the interference problem with other satellites is resolved at the WRC or other international body that regulates RF in space) working, where are the CDNs going to maintain state? The reachable ones for any satellite in the constellation will be moving relative to the user's dishy, unless it the CDN is in geosync orbit).
 
Starlink isn't a media company. It doesn't want to own all the content, or even host all the content.
 
So I'm pretty darn skeptical!
 
At the moment, Starlink is a "last mile" service in a virtual sense. It connects from the public internet at some pretty high performance point of presence, hopefully has a pretty good latency, and doesn't let its users saturate any satellite (lest huge queueing delay build up). Given that, anywhere there is a ground station the public fiber backbone can reach all the CDNs.
 
It's hard to capitalize a last mile fiber service. Each home passed costs about $2000, if we want to connect up (that includes almost all actual rural residences, but not all the uninhabited parts of the planet). For whatever reason, the telcos want no one else to put in fiber, but also don't see much profit in extending their coverage, either, because they aren't allowed to charge for all the content like they used to. Fiber's great advantage is that it has extremely low operational expense.
 
This is what Starlink fills in for. It has lower capex, but HUGE opex. It can't perform or scale very well compared to the asymptotic fiber solution. So it has a pretty good short-term competitive position. And they've gotten very smart in finding early customers who are eager to buy in.
 
However, it's important to realize that what drives this all is how the Internet is used and who needs it where, and how much they are willing to pay.
 
One thing is clear - Starlink isn't the Internet of the future. It's filling a niche (a large one, but a niche).
 
The mistake Motorola made with Iridium was in not realizing that cellular telephony was in all respects a better answer, and would be cheaper, too. People did talk about putting CDNs in space with Iridium, too.
 
But one needs to understand what CDNs are useful for, and really understand what part of the problem that is.
 
 

[-- Attachment #2: Type: text/html, Size: 7835 bytes --]

Re: [Starlink] CDNs in space!

[Mike Puchol]

[-- Attachment #1: Type: text/plain, Size: 5283 bytes --]

I will speak from the side of a tiny WISP in Kenya (15k customers) who was offered a free OCA by Netflix, after they noticed certain levels of demand from our AS.

This OCA is a 1U server with 144TB of storage. It now supplies 10% of our customer traffic.

Not having to pull all that traffic from eg South Africa is good for us, good for Netflix, and good for the customer.

I have had the firm belief, for over a year now,  that SpaceX will field OCA-type content caches in the satellites. They add more satellites than gateway capacity, so having a significant portion of traffic delivered over ISL from CDNs in some satellites (you don’t need them all to have a cache) is a clear benefit.

The critical problem to solve is cache hit ratio, considering content wanted from multiple markets and regions.

My other prediction is that eventually they will have to move to optical gateway, as the terahertz gap will get in the way.

Best,

Mike
On Aug 31, 2022, 22:24 +0200, David P. Reed via Starlink <starlink@lists.bufferbloat.net>, wrote:
> Having looked into this a lot, CDNs don't account for very much Internet traffic. There's a lot of marketing propaganda out there that suggests this might be true, but when you try to track down the source, it's almost always from an old slide deck presented by a company that sells CDN servers.
> That doesn't mean that CDN servers don't help a fair bit, but no, today's Internet wouldn't fall apart if we didn't have CDN servers withint 2 msec. of the edge.
> Also, CDN's need to be BIG to hold all the videos that people might choose to watch at any particular time.
> So I'm just pointing out that the business case for CDN's in space to merely solve Starlink's potential issues is probably not great. Maybe 10 years from now. Maybe never. The idea that everyone watches TV and the same few seconds of content of a few shows that are extraordinarily popular - well, that dog don't hunt. It doesn't justify multicast either.
> So let's improve the discussion here. The Internet, for the forseeable future, at the edge, is unicast.
> CDN's reduce costs for big companies like Netflix, not because they are "close to the watcher" but because it is easier to not have everything centralized in a single point of failure and needing a huge pipe because all customers aggregate demand into their central HQ.
> There are some advantages at the top level peering in distributing CDN sites into the networks of various access providers (like Comcast) because you can get cheaper pricing.
> But lets think about CDNs in space. They don't deal with the actual bottleneck in each satellite to ground. The uplink from central ground stations has LOTS of capacity, it won't be the bottleneck if Starlink or any other satellite service for that matter balances its design.
> And if they get inter-sat links (laser or maybe RF, if the interference problem with other satellites is resolved at the WRC or other international body that regulates RF in space) working, where are the CDNs going to maintain state? The reachable ones for any satellite in the constellation will be moving relative to the user's dishy, unless it the CDN is in geosync orbit).
> Starlink isn't a media company. It doesn't want to own all the content, or even host all the content.
> So I'm pretty darn skeptical!
> At the moment, Starlink is a "last mile" service in a virtual sense. It connects from the public internet at some pretty high performance point of presence, hopefully has a pretty good latency, and doesn't let its users saturate any satellite (lest huge queueing delay build up). Given that, anywhere there is a ground station the public fiber backbone can reach all the CDNs.
> It's hard to capitalize a last mile fiber service. Each home passed costs about $2000, if we want to connect up (that includes almost all actual rural residences, but not all the uninhabited parts of the planet). For whatever reason, the telcos want no one else to put in fiber, but also don't see much profit in extending their coverage, either, because they aren't allowed to charge for all the content like they used to. Fiber's great advantage is that it has extremely low operational expense.
> This is what Starlink fills in for. It has lower capex, but HUGE opex. It can't perform or scale very well compared to the asymptotic fiber solution. So it has a pretty good short-term competitive position. And they've gotten very smart in finding early customers who are eager to buy in.
> However, it's important to realize that what drives this all is how the Internet is used and who needs it where, and how much they are willing to pay.
> One thing is clear - Starlink isn't the Internet of the future. It's filling a niche (a large one, but a niche).
> The mistake Motorola made with Iridium was in not realizing that cellular telephony was in all respects a better answer, and would be cheaper, too. People did talk about putting CDNs in space with Iridium, too.
> But one needs to understand what CDNs are useful for, and really understand what part of the problem that is.
> _______________________________________________
> Starlink mailing list
> Starlink@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink

[-- Attachment #2: Type: text/html, Size: 10782 bytes --]

Re: [Starlink] CDNs in space!

[Radostin Stoyanov]

[-- Attachment #1: Type: text/plain, Size: 412 bytes --]


On 31/08/2022 21:24, David P. Reed via Starlink wrote:
> However, it's important to realize that what drives this all is how 
> the Internet is used and who needs it where, and how much they are 
> willing to pay.

There is also some interesting work into how satellite-based CDNs could 
be used to provide global high-speed Internet service from above the sky:
https://patents.google.com/patent/US10419106B1/en

[-- Attachment #2: Type: text/html, Size: 874 bytes --]

Re: [Starlink] CDNs in space!

[Ulrich Speidel]

[-- Attachment #1: Type: text/plain, Size: 7273 bytes --]

On 1/09/2022 8:24 am, David P. Reed via Starlink wrote:
>
> Having looked into this a lot, CDNs don't account for very much 
> Internet traffic. There's a lot of marketing propaganda out there that 
> suggests this might be true, but when you try to track down the 
> source, it's almost always from an old slide deck presented by a 
> company that sells CDN servers.
>
Well, most web sites may not be using CDNs but almost all frequently 
used sites do. This ranges from the Netflixes and Vimeos of this world 
to Facebook, Twitter, pretty much all significant media organisations 
(even our granny paper, the NZ Herald, does), most larger universities 
(we use Panopto for our lecture recordings and other videos, and most of 
our peer institutions do the same, and they all use CDNs). Most web 
sites that I use on a daily basis are CDN-based. General consensus now 
is that currently, CDN traffic accounts for >70% of Internet traffic by 
volume and growing, and that's what matters here.

If you didn't have CDNs but origin servers only, traffic on 
international links would be much, much higher than it is now. Read: It 
would kill them.

> That doesn't mean that CDN servers don't help a fair bit, but no, 
> today's Internet wouldn't fall apart if we didn't have CDN servers 
> withint 2 msec. of the edge.
>
> Also, CDN's need to be BIG to hold all the videos that people might 
> choose to watch at any particular time.
>
They are big. But they don't need to hold all videos people might choose 
to watch. A CDN server only needs to hold each video that at least one 
user in that neck of the woods has already requested, and that only for 
a given amount of time. But here's an added point: What users want to 
see is highly geographically correlated. Terrestrial CDN servers only 
serve a particular region. A CDN server on a LEO would have to serve 
every region of the world.
>
> So I'm just pointing out that the business case for CDN's in space to 
> merely solve Starlink's potential issues is probably not great. Maybe 
> 10 years from now. Maybe never. The idea that everyone watches TV and 
> the same few seconds of content of a few shows that are 
> extraordinarily popular - well, that dog don't hunt. It doesn't 
> justify multicast either.
>
> So let's improve the discussion here. The Internet, for the forseeable 
> future, at the edge, is unicast.
>
Yes! The problem is that, for any direct-to-site satellite service, the 
edge is on the wrong side of the satellite link.
>
> CDN's reduce costs for big companies like Netflix, not because they 
> are "close to the watcher" but because it is easier to not have 
> everything centralized in a single point of failure and needing a huge 
> pipe because all customers aggregate demand into their central HQ.
>
> There are some advantages at the top level peering in distributing CDN 
> sites into the networks of various access providers (like Comcast) 
> because you can get cheaper pricing.
>
> But lets think about CDNs in space. They don't deal with the actual 
> bottleneck in each satellite to ground.
>
Exactly.
>
> The uplink from central ground stations has LOTS of capacity, it won't 
> be the bottleneck if Starlink or any other satellite service for that 
> matter balances its design.
>
> And if they get inter-sat links (laser or maybe RF, if the 
> interference problem with other satellites is resolved at the WRC or 
> other international body that regulates RF in space) working, where 
> are the CDNs going to maintain state? The reachable ones for any 
> satellite in the constellation will be moving relative to the user's 
> dishy, unless it the CDN is in geosync orbit).
>
> Starlink isn't a media company. It doesn't want to own all the 
> content, or even host all the content.
>
But it's got to transport all the content, each and every time somebody 
wants it. No international subsea fibre cable needs to do this.
>
> So I'm pretty darn skeptical!
>
> At the moment, Starlink is a "last mile" service in a virtual sense. 
> It connects from the public internet at some pretty high performance 
> point of presence, hopefully has a pretty good latency, and doesn't 
> let its users saturate any satellite (lest huge queueing delay build 
> up). Given that, anywhere there is a ground station the public fiber 
> backbone can reach all the CDNs.
>
The problem is though that in reality, Starlink is a severely 
bandwidth-constrained "last thousand mile" service. If the CDN server 
sits on the public fibre backbone, then every time it serves a piece of 
content for the 2nd or subsequent time, it takes up scarce capacity on 
the satellite. That isn't the case in (proper) terrestrial networks, 
where aggregate bandwidth increases the closer you get to the end 
customer. My suburb here has almost all houses on 1 Gb/s FTTH now, so 
collectively, we have well over 1 Tb/s on our doorsteps. Granted, the 
CDNs that serve us probably only have between 1 and 10 Gb/s output, but 
there is just no bottleneck between them and us.
>
> It's hard to capitalize a last mile fiber service. Each home passed 
> costs about $2000, if we want to connect up (that includes almost all 
> actual rural residences, but not all the uninhabited parts of the 
> planet). For whatever reason, the telcos want no one else to put in 
> fiber, but also don't see much profit in extending their coverage, 
> either, because they aren't allowed to charge for all the content like 
> they used to. Fiber's great advantage is that it has extremely low 
> operational expense.
>
$2000 is about a year's worth of Starlink service with Dishy, right? I 
think again this boils down to what is very much a US-specific problem.
>
> This is what Starlink fills in for. It has lower capex, but HUGE opex. 
> It can't perform or scale very well compared to the asymptotic fiber 
> solution. So it has a pretty good short-term competitive position. And 
> they've gotten very smart in finding early customers who are eager to 
> buy in.
>
Indeed.
>
> However, it's important to realize that what drives this all is how 
> the Internet is used and who needs it where, and how much they are 
> willing to pay.
>
> One thing is clear - Starlink isn't the Internet of the future. It's 
> filling a niche (a large one, but a niche).
>
Exactly.
>
> The mistake Motorola made with Iridium was in not realizing that 
> cellular telephony was in all respects a better answer, and would be 
> cheaper, too. People did talk about putting CDNs in space with 
> Iridium, too.
>
> But one needs to understand what CDNs are useful for, and really 
> understand what part of the problem that is.
>
>
> _______________________________________________
> Starlink mailing list
> Starlink@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink
>
-- 
****************************************************************
Dr. Ulrich Speidel

School of Computer Science

Room 303S.594 (City Campus)

The University of Auckland
u.speidel@auckland.ac.nz  
http://www.cs.auckland.ac.nz/~ulrich/
****************************************************************



[-- Attachment #2: Type: text/html, Size: 15149 bytes --]

Re: [Starlink] CDNs in space!

[Brandon Butterworth]

On Wed Aug 31, 2022 at 04:24:21PM -0400, David P. Reed via Starlink wrote:
> Having looked into this a lot, CDNs don't account for very much Internet traffic

It is sufficient for many ISPs to host for free the VoD suppliers caches
(including ours).

> That doesn't mean that CDN servers don't help a fair bit

I think the CDN industry, like anti virus software vendors, has done
us a disservice and made themselved self perpetuating. Their goal to
interpose themselves and take a chunk of cash has taken some pressure
off backbone growth (and cash that would have paid for it)

CDNs may have helped at the time but the 40 / 100G decision was due
to delayed need for 100G (on servers but that rolls into network too)
so it's hard to say if we'd have gone faster sooner or been delayed
waiting for technology instead.

Regardless it's left us with people feeling they need to pay CDNs for
large traffic needs and that leads to slower backbone growth.

> Also, CDN's need to be BIG to hold all the videos that people might
> choose to watch at any particular time.

They don't need to hold everything, just the sufficiently highest requested
set to save enough bandwidth to be worth the expense. SSD are large now too
(100TB but good luck space qualifying that). Being global is going to
make that harder, eg our content is largely UK limited so is a waste of
space flying over other countries.
  
> So I'm just pointing out that the business case for CDN's in space to
> merely solve Starlink's potential issues is probably not great

I agree, we're just mulling over if, and why, that might change.

> The idea that everyone watches TV and the same few seconds of content
> of a few shows that are extraordinarily popular - well, that dog don't
> hunt. It doesn't justify multicast either.

People do but not on the Internet, a lot of people watch linear TV
still despite the claims of Internet VoD suppliers who would have
you think they are the only game now.

For the BBC about 5 to 10% of viewing is VoD. We've not moved the
remaining 90% to the internet yet as there is no multicast (some
FTTH providers are deploying it internally though) and CDNs are
a poor approximation to multicast.

> So let's improve the discussion here. The Internet, for the
> forseeable future, at the edge, is unicast.

Forever most likely

We aim to move all that linear over but slowly to allow the net to grow
with it. We could not move it in one go today (though we're a step closer
in the UK with the national fibre plan) except where there is multicast.
In moving it people habits may change and it all becomes time dilated
to VoD.

> Starlink isn't a media company. It doesn't want to own all the content,
> or even host all the content.

Give them time. Everyone looks to move up the stack where there is more
money

> One thing is clear - Starlink isn't the Internet of the future. It's
> filling a niche (a large one, but a niche).

Yes, that's where we started some of this discussion, some think they
are an alternative and for some they are as needs are diverse.

brandon