<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div class="moz-cite-prefix">On 15/05/2023 3:33 pm, David Lang
wrote:<br>
</div>
<blockquote type="cite" cite="mid:9q29o7n2-69rs-3os1-s93q-0795262qs3o1@ynat.uz">
On Mon, 15 May 2023, Ulrich Speidel wrote:<br>
<br>
> On 14/05/2023 9:00 pm, David Lang wrote:<br>
>> On Sun, 14 May 2023, Ulrich Speidel wrote:<br>
>> <br>
>> >> I just discovered that someone is manufacturing
an adapter so you no <br>
>> >> longer have to cut the cable<br>
>> >><br>
>> >> <br>
>> <a href="https://www.amazon.com/YAOSHENG-Rectangular-Adapter-Connect-Injector/dp/B0BYJTHX4P" moz-do-not-send="true">https://www.amazon.com/YAOSHENG-Rectangular-Adapter-Connect-Injector/dp/B0BYJTHX4P</a><br>
<br>
>> >><br>
>> > I'll see whether I can get hold of one of these.
Cutting a cable on a <br>
>> > university IT asset as an academic is not allowed
here, except if it <br>
>> > doesn't meet electrical safety standards.<br>
</blockquote>
OK, we have one on order, along with PoE injector and power supply.
Don't hold your breath, though, I'll be out of the country when it
arrives and it'll be late July before I get to play with it.<br>
<blockquote type="cite" cite="mid:9q29o7n2-69rs-3os1-s93q-0795262qs3o1@ynat.uz">
>> ><br>
>> > Alternatively, has anyone tried the standard
Starlink Ethernet adapter with <br>
>> > a PoE injector instead of the WiFi box? The adapter
above seems to be like <br>
>> > the Starlink one (which also inserts into the cable
between Dishy and <br>
>> > router).<br>
>> <br>
>> that connects you a 2nd ethernet port on the router, not
on the dishy<br>
>> <br>
>> I just ordered one of those adapters, it will take a few
weeks to arrive.<br>
> How do we know that the Amazon version doesn't do the same?<br>
<br>
because it doesn't involve the router at all. It allows you to
replace the <br>
router with anything you want.<br>
<br>
People have documented how to cut the cable and crimp on a RJ45
connector, use a <br>
standard PoE injector, and connect to any router you want. I was
preparing to do <br>
that (and probably still will for one cable to use a different
locations to <br>
avoid having a 75 ft cable from the dish mounted on the roof of my
van to the <br>
router a couple feet away), This appears to allow me to do the
same functional <br>
thing, but without cutting the cable.<br>
</blockquote>
Let's see whether they actually work any different ;-) They're sure
in the same position in the cable.<br>
<blockquote type="cite" cite="mid:9q29o7n2-69rs-3os1-s93q-0795262qs3o1@ynat.uz">
<br>
>> >> > I suspect they're handing over whole cells,
not individual users, at a <br>
>> >> > time.<br>
>> >><br>
>> >> I would guess the same (remember, in spite of
them having launched >4000 <br>
>> >> satellites, this is still the early days, with
the network changing as <br>
>> >> more launching)<br>
>> >><br>
>> >> We've seen that it seems that there is only one
satellite serving any cell <br>
>> >> one time.<br>
>> <br>
>> > But the reverse is almost certainly not true: Each
satellite must serve <br>
>> > multiple cells.<br>
>> <br>
>> true, but while the satellite over a given area will
change, the usage in <br>
>> that area isn't changing that much<br>
<br>
> Exactly. But your underlying queue sits on the satellite, not
in the area.<br>
<br>
only if the satellite is where you have more input than output.
That may be the <br>
case for users uploading, but for users downloading, I would
expect that the <br>
bandwidth bottleneck would be from the Internet connected ground
station to the <br>
satellite, with the satellite serving many cells but only having
one uplink.<br>
</blockquote>
<p>Leaving lasers aside for the moment.<br>
</p>
<p>I'd expect there to be one queue for each satellite uplink at the
gateway ground station, and that the occupancy of that queue
depends on how much demand the users on that satellite currently
produce. So as a remote terminal switches satellites, even if the
ground station remains the same, it sees different queuing delays
for its inbound traffic at the ground station.</p>
<p>For the uplink from the user terminal, we can't have multiple
users accessing the same uplink channel (however you define
"channel" - frequency, time slot, spreading code, beam,
polarisation, any combination thereof, ...) simultaneously as they
are not able to coordinate and you wouldn't want random access for
your main data link channel because of the hidden node collisions
this would produce (a random access channel paired with an access
grant channel is a different story). So you'd get slot assignments
from the satellite obviously, and the queue for one of these sits
at the user terminal. But what isn't clear to me is whether the
satellites are truly only handled by a single ground station at a
time, or perhaps by multiple ground stations. If it's the latter,
then you might end up with a situation where you have more traffic
arriving at the satellite than it can dispatch to its ground
station(s), and then you'd need a queue in the uplink direction
also. <br>
</p>
<p>Similarly, if the combined uplinks from the ground station are
able to deliver more data than the satellite can downlink to its
users through its current slot assignments, we need a queuing
system on the satellite in that direction, too. <br>
</p>
<p>Add lasers in, and it seems like having some sort of buffer on
the satellites is a must. <br>
</p>
<blockquote type="cite" cite="mid:9q29o7n2-69rs-3os1-s93q-0795262qs3o1@ynat.uz">
<br>
>> >> But remember that the system does know how much
usage there is in the cell <br>
>> >> before they do the handoff. It's unknown if they
do anything with that, or <br>
>> >> if they are just relaying based on geography. We
also don't know what the <br>
>> >> bandwidth to the ground stations is compared to
the dishy.<br>
>> <br>
>> > Well, we do know for NZ, sort of, based on the
licences Starlink has here.<br>
>> <br>
>> what is the ground station bandwith?<br>
><br>
> <a href="https://rrf.rsm.govt.nz/ui/search/licence" moz-do-not-send="true">https://rrf.rsm.govt.nz/ui/search/licence</a>
- seach for "Starlink"<br>
><br>
> ...all NZ licences in all their glory. Looking at Starlink
SES (satellite <br>
> earth station) TX (which is the interesting direction I
guess):<br>
><br>
> - Awarua, Puwera, Hinds, Clevedon, Cromwell, Te Hana:
29750.000000 TX (BW = <br>
> 500 MHz)<br>
> - Awarua, Puwera, Hinds, Clevedon, Cromwell, Te Hana:
28850.000000 TX (BW = <br>
> 500 MHz)<br>
> - Awarua, Puwera, Hinds, Clevedon, Cromwell, Te Hana:
28350.000000 TX (BW = <br>
> 500 MHz)<br>
> - Awarua, Puwera, Hinds, Clevedon, Cromwell, Te Hana:
28250.000000 TX (BW = <br>
> 500 MHz)<br>
> - Awarua, Puwera, Hinds, Clevedon, Cromwell, Te Hana:
27750.000000 TX (BW = <br>
> 500 MHz)<br>
><br>
> So 2.5 GHz up, licensed from 6 ground stations. Now I'm not
convinced that <br>
> they would use all of those from all locations simultaneously
because of the <br>
> risk of off-beam interference. They'll all be transmitting
south, ballpark. <br>
> If there was full re-use at all ground stations, we'd be
looking at 15 GHz. <br>
> If they are able to re-use on all antennas at each ground
station, then we're <br>
> looking at 9 golf balls each in Puwera, Te Hana, Clevedon,
Hinds and <br>
> Cromwell, and an unknown number at Awarua. Assuming 9 there,
we'd be looking <br>
> at 135 GHz all up max.<br>
><br>
> Awarua and Cromwell are 175 km apart, Hinds another 220 km
from Cromwell, <br>
> then it's a hop of about 830 km to Clevedon, and from there
another 100 km to <br>
> Te Hana, which is another 53 km from Puwera, so keeping them
all out of each <br>
> other's hair all the time might be a bit difficult.<br>
><br>
> Lots of other interesting info in the licenses, such as EIRP,
in case you're <br>
> wanting to do link budgets.<br>
<br>
I was asking more in terms of Gb/s rather than MHz of bandwidth.
Dedicated <br>
ground stations with bigger antennas, better filters, more
processing and <br>
overall a much higher budget can get much better data rates out of
a given <br>
amount of bandwidth than the user end stations will.<br>
<br>
it's also possible (especially with bigger antennas) for one
ground station <br>
location to talk to multiple different satellites at once (the
aiming of the <br>
antennas can isolate the signals from each other)<br>
</blockquote>
<p>Well, the Gb/s is what a link budget would give you if you knew
the modulation scheme(s) and any FEC used. The ground station
antennas are normal parabolic dishes in radomes for all we can
tell, and are all the same size, so you can kind of estimate
aperture and hence gain reasonably well. Path loss depends a
little on distance and path quality (weather / rain fade), and we
don't really know in how far their modems use adaptive rates to
cope with this (my Ookla tests during Cyclone Gabrielle most
certainly don't rule this out - rates went down both ways during
the thick of it). I guess we know relatively little about the
on-board phased array on the satellites (apart from very loose
bounds), which restricts our ability to say much about gain there
(and potential for spatial separation / re-use of beam
frequencies). We also don't know how Starlink manages its
frequency allocation across its ground stations. <br>
</p>
<p>It's certainly noticeable here that they seem to have sets of
three grouped together in a relatively compact geographical area
(you could visit all NZ North Island ground stations in a day by
car from Auckland, Auckland traffic notwithstanding, and at a
stretch could do the same down south from Hinds to Awarua if you
manage to ignore the scenery, but getting from the southernmost
North Island ground station to the northernmost South Island one
is basically a two day drive plus ferry trip).<br>
</p>
<blockquote type="cite" cite="mid:9q29o7n2-69rs-3os1-s93q-0795262qs3o1@ynat.uz">
<br>
>> As latency changes, figuring out if it's extra distance
that must be <br>
>> traveled, or buffering is hard. does the latency stay
roughly the same until <br>
>> the next satellite change? or does it taper off?<br>
<br>
> Good question. You would expect step changes in physical
latency between <br>
> satellites, but also gradual change related to satellite
movement. Plus of <br>
> course any rubble thrown into any queue by something suddenly
turning up on <br>
> that path. Don't forget that it's not just cells now, we're
also talking up- <br>
> and downlink for the laser ISLs, at least in some places.<br>
<br>
how far do the satellites move in 15 min and what effect would
that have on <br>
latency (I would assume that most of the time, the satellites are
switched to as <br>
they are getting nearer the two stations, so most of the time, I
would expect a <br>
slight reduction in latency for ~7 min and then a slight increase
for ~7 min, <br>
but I would not expect that this would be a large variation<br>
</blockquote>
<p>Dishy tracks most satellites for significantly less than 15
minutes, and for a relatively small part of their orbit. Let me
explain:</p>
<p><img src="cid:part1.xUdg0mYi.m35A0p0I@auckland.ac.nz" moz-do-not-send="false"><br>
<br>
This is an obstruction map obtained with starlink-grpc-tools
(<a class="moz-txt-link-freetext" href="https://github.com/sparky8512/starlink-grpc-tools">https://github.com/sparky8512/starlink-grpc-tools</a>). The way to
read this is in polar coordinates: The centre of the image is the
dishy boresight (direction of surface normal), distance from the
centre is elevation measured as an angle from the surface normal,
and direction from the centre is essentially the azimuth - top is
north, left is west, bottom is south, and right is east. The white
tracks are the satellites dishy uses, and a graph like this gets
built up over time, one track at a time. Notice how short the
tracks are - they don't follow the satellite for long - typically
under a minute. The red bits are satellites getting obscured by
the edge of our roof.<br>
</p>
<p>I've also attached a time lapse movie of how one of these graphs
builds up - if I correctly remember (the script is on another
machine), one frame in the video corresponds to 5 seconds. <br>
</p>
<p>Conclusion: latency change from tracking one satellite is smaller
than the latency difference as you jump between satellites. You
could be looking at several 100 km of path difference here. In an
instant. Even that, at 300,000 km/s of propagation speed, is only
in the order of maybe 1 ms or so - peanuts compared to the RTTs in
the dozens of ms that we're seeing. But if you get thrown from one
queue onto another as you get handed over - what does that do to
the remote TCP stack that's serving you? <br>
</p>
<blockquote type="cite" cite="mid:9q29o7n2-69rs-3os1-s93q-0795262qs3o1@ynat.uz">
<br>
>> If it stays the same, I would suspect that you are
actually hitting a <br>
>> different ground station and there is a VPN backhaul to
your egress point to <br>
>> the regular Internet (which doesn't support mobile IP
addresses) for that <br>
>> cycle. If it tapers off, then I could buy bufferbloat
that gets resolved as <br>
>> TCP backs off.<br>
><br>
> Yes, quite sorting out which part of your latency is what is
the million <br>
> dollar question here...<br>
><br>
> We saw significant RTT changes here during the recent cyclone
over periods of <br>
> several hours, and these came in steps (see below), with the
initial change <br>
> being a downward one. Averages are over 60 pings (the time
scale isn't 100% <br>
> true as we used "one ping, one second" timing) here.<br>
><br>
><br>
> We're still not sure whether to attribute this to load change
or ground <br>
> station changes. There were a lot of power outages,
especially in Auckland's <br>
> lifestyle block belt, which teems with Starlink users, but
all three North <br>
> Island ground stations were also in areas affected by power
outages (although <br>
> the power companies concerned don't provide the level of
detail to establish <br>
> whether they were affected). It's also not clear what, if
any, backup power <br>
> arrangements they have). At ~25 ms, the step changes in RTT
are too large be <br>
> the result of a switch in ground stations, though, the path
differences just <br>
> aren't that large. You'd also expect a ground station outage
to result in <br>
> longer RTTs, not shorter ones, if you need to re-route via
another ground <br>
> station. One explanation might be users getting cut off if
they relied on one <br>
> particular ground station for bent pipe ops - but that would
not explain this <br>
> order of magnitude effect as I'd expect that number to be
small. So maybe <br>
> power outages at the user end after all. But that would then
tell us that <br>
> these are load-dependent queuing delays. Moreover, since
those load changes <br>
> wouldn't have involved the router at our site, we can
conclude that these are <br>
> queue sojourn times in the Starlink network.<br>
<br>
I have two starlink dishes in the southern california area, I'm
going to put <br>
one on the low-priority mobile plan shortly. These are primarily
used for backup <br>
communication, so I would be happy to add something to them to do
latency <br>
monitoring. In looking at what geo-location reports my location
as, I see it <br>
wander up and down the west coast, from the Los Angeles area all
the way up to <br>
Canada.<br>
</blockquote>
Would be worthwhile to also do traceroutes to various places to see
where you emerge from the satellite side of things.<br>
<blockquote type="cite" cite="mid:9q29o7n2-69rs-3os1-s93q-0795262qs3o1@ynat.uz">
<br>
>> I think that active queue management on the sending side
of the bottleneck <br>
>> will handle it fairly well. It doesn't have to do
calculations based on what <br>
>> the bandwidth is, it just needs to know what it has
pending to go out.<br>
<br>
> Understood - but your customer for AQM is the sending TCP
client, and there <br>
> are two questions here: (a) Does your AQM handle rapid load
changes and (b) <br>
> how do your TCP clients actually respond to your AQM's
handling?<br>
<br>
AQM allocates the available bandwidth between different
connections (usually <br>
different users)<br>
</blockquote>
But it does this under the assumption that the vector for changes in
bandwidth availability is the incoming traffic, which AQM gives
(indirect) feedback to, right?<br>
<blockquote type="cite" cite="mid:9q29o7n2-69rs-3os1-s93q-0795262qs3o1@ynat.uz">
<br>
When it does this indirectly for inbound traffic by delaying acks,
the results <br>
depend on the senders handling of these indirect signals that were
never <br>
intended for this purpose.<br>
<br>
But when it does this directly on the sending side, it doesn't
matter what the <br>
senders want, their data WILL be managed to the priority/bandwidth
that the AQM <br>
sets, and eventually their feedback is dropped packets, which
everyone who is <br>
legitimate responds to. </blockquote>
<p>Understood. You build a control loop, where the latency is the
delay in the control signal.</p>
<p>Classically, you have a physical bottleneck that the AQM manages,
where the physical bandwidth doesn't change. <br>
</p>
<p>The available bandwidth changes, (mostly) as a result of TCP
connections (or similarly behaved UDP applications) joining in
slow start, or disappearing. <br>
</p>
<p>Basically, your queues grow and shrink one packet at a time.</p>
<p>Your control signal allows you (if they're well behaved) throttle
/ accelerate senders. <br>
</p>
<p>What you don't get are quantum jumps in queue occupancy, jump
changes in underlying physical bandwidth, or a whole set of new
senders that are completely oblivious to any of your previous
control signals. But you get all that with satellite handovers
like these.<br>
</p>
<p>So what if the response you elicit in this way is to a queue
scenario that no longer applies?</p>
<blockquote type="cite" cite="mid:9q29o7n2-69rs-3os1-s93q-0795262qs3o1@ynat.uz">But even
if they don't respond (say a ping flood or DoS <br>
attack), the AQM will limit the damage to that connection,
allowing the other <br>
connections trying to use that link to continue to function.<br>
</blockquote>
All understood.<br>
<blockquote type="cite" cite="mid:9q29o7n2-69rs-3os1-s93q-0795262qs3o1@ynat.uz">
<br>
David Lang<br>
</blockquote>
<pre class="moz-signature" cols="72">--
****************************************************************
Dr. Ulrich Speidel
School of Computer Science
Room 303S.594 (City Campus)
The University of Auckland
<a class="moz-txt-link-abbreviated" href="mailto:u.speidel@auckland.ac.nz">u.speidel@auckland.ac.nz</a>
<a class="moz-txt-link-freetext" href="http://www.cs.auckland.ac.nz/~ulrich/">http://www.cs.auckland.ac.nz/~ulrich/</a>
****************************************************************
</pre>
</body>
</html>