[Bloat] [Starlink] an rtt_fair test?
Daniel Sterling
sterling.daniel at gmail.com
Sun Apr 17 12:54:44 EDT 2022
test from my network running customized openwrt w/ cake
tar file here:
rtt_fair4be-2022-04-17T124830.462924.dss-cake.f...
<https://drive.google.com/file/d/1MzuHQfEsl7xQFXe0F7n506KYhRAE4jXI/view?usp=drive_web>
root at DESKTOP-DLUP89P:~# flent -x --socket-stats --step-size=.05 -H
de.starlink.taht.net -H london.starlink.taht.net -H s
ingapore.starlink.taht.net -H fremont.starlink.taht.net -t 'dss-cake'
rtt_fair4be
Started Flent 1.3.2 using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to
./rtt_fair4be-2022-04-17T124830.462924.dss-cake.flent.gz.
Summary of rtt_fair4be test run from 2022-04-17 16:48:30.462924
Title: 'dss-cake'
avg median # data pts
Ping (ms) ICMP1 de.starlink.taht.net :
119.54 114.00 ms 1398
Ping (ms) ICMP2 london.starlink.taht.net :
107.97 104.00 ms 1398
Ping (ms) ICMP3 singapore.starlink.taht.net :
272.62 268.00 ms 1398
Ping (ms) ICMP4 fremont.starlink.taht.net :
89.15 84.60 ms 1398
Ping (ms) avg :
147.32 N/A ms 1398
TCP download BE1 de.starlink.taht.net :
25.01 24.81 Mbits/s 1398
TCP download BE2 london.starlink.taht.net :
23.00 22.75 Mbits/s 1398
TCP download BE3 singapore.starlink.taht.net :
20.26 21.01 Mbits/s 1398
TCP download BE4 fremont.starlink.taht.net :
35.64 32.94 Mbits/s 1398
TCP download avg :
25.98 N/A Mbits/s 1398
TCP download fairness :
0.95 N/A Mbits/s 1398
TCP download sum :
103.91 N/A Mbits/s 1398
TCP upload BE1 de.starlink.taht.net :
7.76 8.00 Mbits/s 1398
TCP upload BE1 de.starlink.taht.net::tcp_cwnd :
88.38 91.00 1067
TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate :
7.52 7.57 1064
TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate :
9.82 9.73 1066
TCP upload BE1 de.starlink.taht.net::tcp_rtt :
130.68 125.68 1062
TCP upload BE1 de.starlink.taht.net::tcp_rtt_var :
5.38 4.25 1062
TCP upload BE2 london.starlink.taht.net :
7.53 7.91 Mbits/s 1398
TCP upload BE2 london.starlink.taht.net::tcp_cwnd :
78.47 81.00 1059
TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate :
7.26 7.34 1056
TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate :
9.48 9.45 1057
TCP upload BE2 london.starlink.taht.net::tcp_rtt :
120.05 114.79 1056
TCP upload BE2 london.starlink.taht.net::tcp_rtt_var :
5.25 4.15 1056
TCP upload BE3 singapore.starlink.taht.net :
3.98 4.23 Mbits/s 1398
TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd :
100.96 105.00 1066
TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :
3.95 4.10 1056
TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate :
5.07 5.15 1061
TCP upload BE3 singapore.starlink.taht.net::tcp_rtt :
291.42 286.12 1060
TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var :
7.62 5.64 1060
TCP upload BE4 fremont.starlink.taht.net :
8.41 8.56 Mbits/s 1398
TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd :
75.40 73.00 1060
TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate :
8.04 8.00 1057
TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate :
10.62 10.31 1058
TCP upload BE4 fremont.starlink.taht.net::tcp_rtt :
103.74 98.50 1057
TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var :
4.97 3.96 1057
TCP upload avg :
6.92 N/A Mbits/s 1398
TCP upload fairness :
0.94 N/A Mbits/s 1398
TCP upload sum :
27.68 N/A Mbits/s 1398
root at DESKTOP-DLUP89P:~#
On Sun, Apr 17, 2022 at 11:36 AM jordan--- via Bloat <
bloat at lists.bufferbloat.net> wrote:
> You're welcome! Here's the files Flent generated:
> https://1drv.ms/u/s!Ap4u4Rte63FqjqoSQXX444OWLvl8FQ?e=f22Nt7
>
> And thanks for the link! It honestly doesn't look too bad to implement so
> I might give it a shot today if time allows.
>
> I've also been contemplating replacing it with something like a Mikrotik
> since cake is now integrated into them. I've been testing one at home and
> made a post with a bunch of flent tests to the cake thread in their forum
> if you're interested in some data from a Charter Spectrum cable connection:
> https://forum.mikrotik.com/viewtopic.php?t=179307#p925485
>
> The OneDrive link to my flent files should still be active in that post as
> well.
>
> -Jordan
>
> -----Original Message-----
> From: Dave Taht <dave.taht at gmail.com>
> Sent: Sunday, April 17, 2022 10:57 AM
> To: Jordan Szuch <jordan at inacomptc.com>
> Cc: flent-users <flent-users at flent.org>; bloat <
> bloat at lists.bufferbloat.net>; starlink at lists.bufferbloat.net
> Subject: Re: [Starlink] an rtt_fair test?
>
> Thank you for testing!!! If you could stash the *.flent.gz files somewhere
> I could get at, that would be great. You can also browse the plots using
> the flent-gui to see more detail over time.
>
> Cake, long available as an out of tree build for the edgerouter, has both
> dsl compensation and an ack-filter which would help A LOT, in your case. i
> wish I'd pushed harder to get dsl compensation into the edgerouter smart
> queues code...
>
>
> https://community.ui.com/questions/Cake-compiled-for-the-EdgeRouter-devices/fc1ff27c-f321-4344-8737-fcc755cae8a2
>
> Given your 500kbit upload speed the minimum latency inflation under the
> structure of this test (4 upload flows) should have been around 150ms (4 *
> 25ms + ack overhead), but without looking at the *.gz files I don't know
> what your baseline latency was to any of these sites. (it would be good if
> flent reported on the text output what the minimum actually was)
>
> On Sun, Apr 17, 2022 at 7:35 AM <jordan at inacomptc.com> wrote:
> >
> > Hi Dave,
> >
> > This isn't specifically cake related but I figured it could be
> interesting, especially as these are from a location where I've been
> investigating setting up something with cake. These are statistics from a
> relatives location where they only have a DSL line from CenturyLink
> available to them. There's a little Debian based VM I setup for a couple
> random things so I was able to run flent from that. They have a EdgeRouter
> X serving as the router so they can at least take advantage of fq_codel
> through the routers Smart Queue feature. I ran a quick speed test, then ran
> flent without sqm, and then with sqm. Sorry if this gets a bit long and
> hopefully it stays readable.
> >
> > #############
> > Speedtest by Ookla
> >
> > Server: 123.Net, Inc. - Southfield, MI (id = 15044)
> > ISP: CenturyLink
> > Latency: 45.45 ms (194.34 ms jitter)
> > Download: 8.88 Mbps (data used: 12.6 MB )
> > Upload: 0.64 Mbps (data used: 962.7 kB )
> > Packet Loss: 5.1%
> > Result URL:
> > https://www.speedtest.net/result/c/799ba46b-d3d4-4d54-ac76-118fac847b7
> > 3
> >
> > ###########
> > No SQM or QOS
> >
> > flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H
> > london.starlink.taht.net -H singapore.starlink.taht.net -H
> fremont.starlink.taht.net -t
> nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be Started
> Flent 2.0.0 using Python 3.9.2.
> > Starting rtt_fair4be test. Expected run time: 70 seconds.
> > Data file written to
> > ./rtt_fair4be-2022-04-17T145553.865990.nosqm_erx_rttfair4be_8Mbpsdown_
> > 500Kbpsup_centurylinkdsl.flent.gz
> >
> > Summary of rtt_fair4be test run from 2022-04-17 13:55:53.865990
> > Title: 'nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'
> >
> >
> avg median # data pts
> > Ping (ms) ICMP1 de.starlink.taht.net :
> 1026.06 920.00 ms 1383
> > Ping (ms) ICMP2 london.starlink.taht.net :
> 1009.28 887.00 ms 1383
> > Ping (ms) ICMP3 singapore.starlink.taht.net :
> 1121.09 1012.00 ms 1383
> > Ping (ms) ICMP4 fremont.starlink.taht.net :
> 943.77 853.00 ms 1383
> > Ping (ms) avg :
> 1025.05 N/A ms 1383
> > TCP download BE1 de.starlink.taht.net :
> 1.36 2.00 Mbits/s 1383
> > TCP download BE2 london.starlink.taht.net :
> 0.72 0.65 Mbits/s 1383
> > TCP download BE3 singapore.starlink.taht.net :
> 1.03 1.74 Mbits/s 1383
> > TCP download BE4 fremont.starlink.taht.net :
> 2.11 2.88 Mbits/s 1383
> > TCP download avg :
> 1.30 N/A Mbits/s 1383
> > TCP download fairness :
> 0.86 N/A Mbits/s 1383
> > TCP download sum :
> 5.22 N/A Mbits/s 1383
> > TCP upload BE1 de.starlink.taht.net :
> 0.12 0.20 Mbits/s 1383
> > TCP upload BE1 de.starlink.taht.net::tcp_cwnd :
> 16.85 14.00 1044
> > TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate :
> 0.12 0.12 1036
> > TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate :
> 0.35 0.26 1042
> > TCP upload BE1 de.starlink.taht.net::tcp_rtt :
> 1128.57 954.47 1020
> > TCP upload BE1 de.starlink.taht.net::tcp_rtt_var :
> 68.22 50.08 1020
> > TCP upload BE2 london.starlink.taht.net :
> 0.12 0.15 Mbits/s 1383
> > TCP upload BE2 london.starlink.taht.net::tcp_cwnd :
> 17.33 11.00 1046
> > TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate :
> 0.12 0.11 1041
> > TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate :
> 0.38 0.26 1044
> > TCP upload BE2 london.starlink.taht.net::tcp_rtt :
> 1166.74 991.48 1043
> > TCP upload BE2 london.starlink.taht.net::tcp_rtt_var :
> 79.99 68.27 1043
> > TCP upload BE3 singapore.starlink.taht.net :
> 0.07 0.13 Mbits/s 1383
> > TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd :
> 13.60 9.00 1042
> > TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :
> 0.07 0.08 1013
> > TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate :
> 0.23 0.20 1031
> > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt :
> 1302.23 1129.07 967
> > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var :
> 113.67 98.84 967
> > TCP upload BE4 fremont.starlink.taht.net :
> 0.11 0.20 Mbits/s 1383
> > TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd :
> 17.55 11.00 1038
> > TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate :
> 0.11 0.10 1035
> > TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate :
> 0.32 0.23 1036
> > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt :
> 1094.64 887.54 1033
> > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var :
> 67.20 59.16 1033
> > TCP upload avg :
> 0.10 N/A Mbits/s 1383
> > TCP upload fairness :
> 0.96 N/A Mbits/s 1383
> > TCP upload sum :
> 0.42 N/A Mbits/s 1383
> >
> > ##########
> > with smart queue (fq_codel) on the EdgeRouter
> >
> > flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H
> > london.starlink.taht.net -H singapore.starlink.taht.net -H
> fremont.starlink.taht.net -t
> fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be
> Started Flent 2.0.0 using Python 3.9.2.
> > Starting rtt_fair4be test. Expected run time: 70 seconds.
> > Data file written to
> > ./rtt_fair4be-2022-04-17T150339.549817.fqcodel_erx_rttfair4be_8Mbpsdow
> > n_500Kbpsup_centurylinkdsl.flent.gz
> >
> > Summary of rtt_fair4be test run from 2022-04-17 14:03:39.549817
> > Title: 'fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'
> >
> >
> avg median # data pts
> > Ping (ms) ICMP1 de.starlink.taht.net :
> 392.93 208.00 ms 1384
> > Ping (ms) ICMP2 london.starlink.taht.net :
> 373.94 196.00 ms 1384
> > Ping (ms) ICMP3 singapore.starlink.taht.net :
> 508.15 319.00 ms 1384
> > Ping (ms) ICMP4 fremont.starlink.taht.net :
> 346.23 152.00 ms 1384
> > Ping (ms) avg :
> 405.31 N/A ms 1384
> > TCP download BE1 de.starlink.taht.net :
> 1.36 1.62 Mbits/s 1384
> > TCP download BE2 london.starlink.taht.net :
> 1.47 1.63 Mbits/s 1384
> > TCP download BE3 singapore.starlink.taht.net :
> 1.55 1.60 Mbits/s 1384
> > TCP download BE4 fremont.starlink.taht.net :
> 1.50 1.80 Mbits/s 1384
> > TCP download avg :
> 1.47 N/A Mbits/s 1384
> > TCP download fairness :
> 1.00 N/A Mbits/s 1384
> > TCP download sum :
> 5.88 N/A Mbits/s 1384
> > TCP upload BE1 de.starlink.taht.net :
> 0.09 0.85 Mbits/s 1384
> > TCP upload BE1 de.starlink.taht.net::tcp_cwnd :
> 4.50 4.00 1035
> > TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate :
> 0.08 0.08 1030
> > TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate :
> 0.17 0.15 1032
> > TCP upload BE1 de.starlink.taht.net::tcp_rtt :
> 458.11 318.82 1027
> > TCP upload BE1 de.starlink.taht.net::tcp_rtt_var :
> 95.76 61.88 1027
> > TCP upload BE2 london.starlink.taht.net :
> 0.09 0.93 Mbits/s 1384
> > TCP upload BE2 london.starlink.taht.net::tcp_cwnd :
> 4.17 4.00 1038
> > TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate :
> 0.08 0.08 1033
> > TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate :
> 0.17 0.14 1036
> > TCP upload BE2 london.starlink.taht.net::tcp_rtt :
> 452.67 322.49 1033
> > TCP upload BE2 london.starlink.taht.net::tcp_rtt_var :
> 96.87 64.28 1033
> > TCP upload BE3 singapore.starlink.taht.net :
> 0.06 0.44 Mbits/s 1384
> > TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd :
> 4.16 4.00 1040
> > TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :
> 0.06 0.06 1031
> > TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate :
> 0.12 0.11 1036
> > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt :
> 577.75 438.61 1027
> > TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var :
> 107.05 72.97 1027
> > TCP upload BE4 fremont.starlink.taht.net :
> 0.10 1.48 Mbits/s 1384
> > TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd :
> 4.45 4.00 1020
> > TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate :
> 0.09 0.09 1017
> > TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate :
> 0.21 0.16 1019
> > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt :
> 439.05 285.86 1018
> > TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var :
> 81.90 56.33 1018
> > TCP upload avg :
> 0.08 N/A Mbits/s 1384
> > TCP upload fairness :
> 0.97 N/A Mbits/s 1384
> > TCP upload sum :
> 0.34 N/A Mbits/s 1384
> >
> >
> > EdgeRouter Settings
> >
> > smart-queue DSL {
> > download {
> > ecn enable
> > flows 1024
> > fq-quantum 300
> > limit 10240
> > rate 8mbit
> > }
> > upload {
> > ecn enable
> > flows 1024
> > fq-quantum 300
> > limit 10240
> > rate 600kbit
> > target 15ms
> > }
> > wan-interface pppoe0
> >
> > Also, here's the VMs qdisc settings. Which I now realize probably
> > aren't the best
> >
> > sysctl -a | grep qdisc
> > net.core.default_qdisc = pfifo_fast
> >
> > tc -s qdisc
> > qdisc noqueue 0: dev lo root refcnt 2
> > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b
> > 0p requeues 0 qdisc mq 0: dev eth0 root Sent 628201 bytes 8643 pkt
> > (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 qdisc
> > pfifo_fast 0: dev eth0 parent :1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1
> > 1 1 1 1 1 Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0
> > requeues 0) backlog 0b 0p requeues 0
> >
> >
> >
> > -Jordan
> >
> > -----Original Message-----
> > From: Starlink <starlink-bounces at lists.bufferbloat.net> On Behalf Of
> > Dave Taht
> > Sent: Saturday, April 16, 2022 9:40 PM
> > To: flent-users <flent-users at flent.org>; bloat
> > <bloat at lists.bufferbloat.net>; starlink at lists.bufferbloat.net
> > Subject: [Starlink] an rtt_fair test?
> >
> > if anyone has a bit of spare time this holiday, I'd like to collect a
> few results from various networks around the web this week, not just on the
> turris. Example command line and outputs here:
> >
> > https://forum.turris.cz/t/sqm-on-turris-flent-benchmarks/17048/
> >
> > --
> > I tried to build a better future, a few times:
> > https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> >
> > Dave Täht CEO, TekLibre, LLC
> > _______________________________________________
> > Starlink mailing list
> > Starlink at lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/starlink
> >
>
>
> --
> I tried to build a better future, a few times:
> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
>
> Dave Täht CEO, TekLibre, LLC
>
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/bloat/attachments/20220417/4944b33a/attachment-0001.html>
More information about the Bloat
mailing list