[Starlink] starlink upload behavior
Dave Taht
dave.taht at gmail.com
Thu Sep 1 15:50:01 EDT 2022
On Thu, Sep 1, 2022 at 12:24 PM Luis A. Cornejo
<luis.a.cornejo at gmail.com> wrote:
>
> Dave,
>
> Did you leave your ingress at 0?
I could never find an operating point that was suitable.
> How about ack filtering and overhead on egress? Do you mind sharing your tc qdisc commands?
ack-filter works better than ack-filter-aggressive, though I switch
back and forth.
> Have you run the auto rate script?
I used to - but my own usage is principally bound by annoyance at
upload speeds, so I reverted to just using ack-filtering and a 6Mbit
rate on uploads for the sqm-scripts. See attached.
For new subscribers to this list, the genesis of it all was the idea
of doing a cerowrt II project
targetted at making all of openwrt easily capable of interoperating
with starlink's products, and
in at least a few ways, superior. Despite pitching this in multiple
directions, no funding arrived, and Starlink stopped communicating
with us at all...
https://docs.google.com/document/d/1rVGC-iNq2NZ0jk4f3IAiVUHz2S9O6P-F3vVZU2yBYtw/edit#heading=h.qev8j7cst4lc
and instead we've been poking at various subprojects in loose
formation, off of that list. Notably, the cake-autorate work hit 1.0
with some decent solutions for LTE/5G that I hope more are using, some
details on that here:
https://forum.openwrt.org/t/cake-w-adaptive-bandwidth/135379
Given how the starlink network is (d)evolving, and my continued,
fervent hope they are upgrading their dishys and downlinks to manage
congestion better, I went back to paid work and polishing up the
openwrt 22.03 release. (I think we fixed the last major wifi bug a
week ago).
LibreQos is experiencing a small explosion of popularity, and I hope
more small ISPs are leaping on that. When we started developing cake
in 2015, XDP, ebpf didn't exist, and htb was too locky to scale much
past a gbit, we recently got LibreQos past 10gbits and 5000 60/6 mbit
users on really cheap hardware. Next stop, 20Gbit!
https://github.com/rchac/LibreQoS
I picked up a pretty cool FPGA the other day, as the stop after that
is 100Gbits. 6ns per packet is difficult.
A couple days worth of dishy cake stats:
tc -s qdisc show dev wlp3s0
qdisc cake 818c: root refcnt 2 bandwidth 6Mbit diffserv3
triple-isolate nonat nowash ack-filter split-gso rtt 100.0ms raw
overhead 0
Sent 1090433207 bytes 3498702 pkt (dropped 240608, overlimits 2592116
requeues 0)
backlog 0b 0p requeues 0
memory used: 68Kb of 4Mb
capacity estimate: 6Mbit
min/max network layer size: 42 / 1514
min/max overhead-adjusted size: 42 / 1514
average network hdr offset: 14
Bulk Best Effort Voice
thresh 375Kbit 6Mbit 1500Kbit
target 48.4ms 5.0ms 12.1ms
interval 143.4ms 100.0ms 107.1ms
pk_delay 0us 3.2ms 384us
av_delay 0us 415us 15us
sp_delay 0us 3us 2us
backlog 0b 0b 0b
pkts 0 3732804 6506
bytes 0 1105842362 295688
way_inds 0 254569 0
way_miss 0 32630 84
way_cols 0 0 0
drops 0 46 0
marks 0 160 0
ack_drop 0 240562 0
sp_flows 0 2 0
bk_flows 0 1 0
un_flows 0 0 0
max_len 0 13446 329
quantum 300 300 300
For contrast here's two days worth of stats for a 60Mbit customer of
this wisp. Overall we see about a 1% drop rate...
qdisc cake c538: parent 7:1218 bandwidth unlimited diffserv4 triple-isolate nona
t nowash ack-filter split-gso rtt 100ms raw overhead 0
Sent 1068282731 bytes 6570868 pkt (dropped 187845, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
memory used: 292608b of 15140Kb
capacity estimate: 0bit
min/max network layer size: 60 / 1494
min/max overhead-adjusted size: 60 / 1494
average network hdr offset: 14
Bulk Best Effort Video Voice
thresh 0bit 0bit 0bit 0bit
target 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms
pk_delay 1.08ms 196us 130us 44us
av_delay 26us 4us 3us 2us
sp_delay 0us 0us 1us 0us
backlog 0b 0b 0b 0b
pkts 258 6753105 320 5030
bytes 15480 1081835162 24846 860791
way_inds 0 499764 0 0
way_miss 90 63092 229 309
way_cols 0 0 0 0
drops 0 1062 0 2
marks 0 32 0 0
ack_drop 0 186781 0 0
sp_flows 0 1 0 1
bk_flows 0 1 0 0
un_flows 0 0 0 0
max_len 60 1494 1422 590
quantum 1514 1514 1514 1514
>
> -Luis
>
> On Wed, Aug 31, 2022 at 10:22 AM Dave Taht via Starlink <starlink at lists.bufferbloat.net> wrote:
>>
>> I have just been leaving cake at 6Mbit on the upload and that fq and
>> control make it much
>> more tolerable for my mix of videoconferencing and uploads. That said,
>> I hadn't looked
>> at the native performance in a while which is markedly better than it
>> was a few months ago, at least for this quick test run.
>>
>> If anyone can make sense of these... for the first 1/3 of the trace
>> throughput is low and RTTs are *NICE*,
>> the second looks like a sat switch - still nice... then another sat
>> switch where I get full upload throughput
>> and the latency grows significantly.
>>
>> If anyone is into packet captures:
>>
>> http://www.taht.net/~d/starlink-1-auto.cap # starlink through their wifi
>> http://www.taht.net/~d/starlink-1-cake6.cap # starlink through their
>> wifi, with cake bandwidth 6mbit on mine
>>
>> Graphs produced with xplot.
>> _______________________________________________
>> Starlink mailing list
>> Starlink at lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/starlink
--
FQ World Domination pending: https://blog.cerowrt.org/post/state_of_fq_codel/
Dave Täht CEO, TekLibre, LLC
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wlp3s0.iface.conf
Type: application/octet-stream
Size: 999 bytes
Desc: not available
URL: <https://lists.bufferbloat.net/pipermail/starlink/attachments/20220901/70c66f9e/attachment.obj>
More information about the Starlink
mailing list