[Bloat] [Starlink] an rtt_fair test?

jordan at inacomptc.com jordan at inacomptc.com
Sun Apr 17 10:35:38 EDT 2022


Hi Dave,

This isn't specifically cake related but I figured it could be interesting, especially as these are from a location where I've been investigating setting up something with cake. These are statistics from a relatives location where they only have a DSL line from CenturyLink available to them. There's a little Debian based VM I setup for a couple random things so I was able to run flent from that. They have a EdgeRouter X serving as the router so they can at least take advantage of fq_codel through the routers Smart Queue feature. I ran a quick speed test, then ran flent without sqm, and then with sqm. Sorry if this gets a bit long and hopefully it stays readable.

#############
Speedtest by Ookla

     Server: 123.Net, Inc. - Southfield, MI (id = 15044)
        ISP: CenturyLink
    Latency:    45.45 ms   (194.34 ms jitter)
   Download:     8.88 Mbps (data used: 12.6 MB )
     Upload:     0.64 Mbps (data used: 962.7 kB )
Packet Loss:     5.1%
 Result URL: https://www.speedtest.net/result/c/799ba46b-d3d4-4d54-ac76-118fac847b73

###########
No SQM or QOS

flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net -t nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be
Started Flent 2.0.0 using Python 3.9.2.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to ./rtt_fair4be-2022-04-17T145553.865990.nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl.flent.gz

Summary of rtt_fair4be test run from 2022-04-17 13:55:53.865990
  Title: 'nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'

                                                                          avg       median          # data pts
 Ping (ms) ICMP1 de.starlink.taht.net                          :      1026.06       920.00 ms             1383
 Ping (ms) ICMP2 london.starlink.taht.net                      :      1009.28       887.00 ms             1383
 Ping (ms) ICMP3 singapore.starlink.taht.net                   :      1121.09      1012.00 ms             1383
 Ping (ms) ICMP4 fremont.starlink.taht.net                     :       943.77       853.00 ms             1383
 Ping (ms) avg                                                 :      1025.05          N/A ms             1383
 TCP download BE1 de.starlink.taht.net                         :         1.36         2.00 Mbits/s        1383
 TCP download BE2 london.starlink.taht.net                     :         0.72         0.65 Mbits/s        1383
 TCP download BE3 singapore.starlink.taht.net                  :         1.03         1.74 Mbits/s        1383
 TCP download BE4 fremont.starlink.taht.net                    :         2.11         2.88 Mbits/s        1383
 TCP download avg                                              :         1.30          N/A Mbits/s        1383
 TCP download fairness                                         :         0.86          N/A Mbits/s        1383
 TCP download sum                                              :         5.22          N/A Mbits/s        1383
 TCP upload BE1 de.starlink.taht.net                           :         0.12         0.20 Mbits/s        1383
 TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :        16.85        14.00                1044
 TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :         0.12         0.12                1036
 TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :         0.35         0.26                1042
 TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :      1128.57       954.47                1020
 TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :        68.22        50.08                1020
 TCP upload BE2 london.starlink.taht.net                       :         0.12         0.15 Mbits/s        1383
 TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :        17.33        11.00                1046
 TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :         0.12         0.11                1041
 TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :         0.38         0.26                1044
 TCP upload BE2 london.starlink.taht.net::tcp_rtt              :      1166.74       991.48                1043
 TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :        79.99        68.27                1043
 TCP upload BE3 singapore.starlink.taht.net                    :         0.07         0.13 Mbits/s        1383
 TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :        13.60         9.00                1042
 TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :         0.07         0.08                1013
 TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :         0.23         0.20                1031
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :      1302.23      1129.07                 967
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :       113.67        98.84                 967
 TCP upload BE4 fremont.starlink.taht.net                      :         0.11         0.20 Mbits/s        1383
 TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :        17.55        11.00                1038
 TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :         0.11         0.10                1035
 TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :         0.32         0.23                1036
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :      1094.64       887.54                1033
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :        67.20        59.16                1033
 TCP upload avg                                                :         0.10          N/A Mbits/s        1383
 TCP upload fairness                                           :         0.96          N/A Mbits/s        1383
 TCP upload sum                                                :         0.42          N/A Mbits/s        1383

##########
with smart queue (fq_codel) on the EdgeRouter

flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net -t fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be
Started Flent 2.0.0 using Python 3.9.2.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to ./rtt_fair4be-2022-04-17T150339.549817.fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl.flent.gz

Summary of rtt_fair4be test run from 2022-04-17 14:03:39.549817
  Title: 'fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'

                                                                          avg       median          # data pts
 Ping (ms) ICMP1 de.starlink.taht.net                          :       392.93       208.00 ms             1384
 Ping (ms) ICMP2 london.starlink.taht.net                      :       373.94       196.00 ms             1384
 Ping (ms) ICMP3 singapore.starlink.taht.net                   :       508.15       319.00 ms             1384
 Ping (ms) ICMP4 fremont.starlink.taht.net                     :       346.23       152.00 ms             1384
 Ping (ms) avg                                                 :       405.31          N/A ms             1384
 TCP download BE1 de.starlink.taht.net                         :         1.36         1.62 Mbits/s        1384
 TCP download BE2 london.starlink.taht.net                     :         1.47         1.63 Mbits/s        1384
 TCP download BE3 singapore.starlink.taht.net                  :         1.55         1.60 Mbits/s        1384
 TCP download BE4 fremont.starlink.taht.net                    :         1.50         1.80 Mbits/s        1384
 TCP download avg                                              :         1.47          N/A Mbits/s        1384
 TCP download fairness                                         :         1.00          N/A Mbits/s        1384
 TCP download sum                                              :         5.88          N/A Mbits/s        1384
 TCP upload BE1 de.starlink.taht.net                           :         0.09         0.85 Mbits/s        1384
 TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :         4.50         4.00                1035
 TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :         0.08         0.08                1030
 TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :         0.17         0.15                1032
 TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :       458.11       318.82                1027
 TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :        95.76        61.88                1027
 TCP upload BE2 london.starlink.taht.net                       :         0.09         0.93 Mbits/s        1384
 TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :         4.17         4.00                1038
 TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :         0.08         0.08                1033
 TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :         0.17         0.14                1036
 TCP upload BE2 london.starlink.taht.net::tcp_rtt              :       452.67       322.49                1033
 TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :        96.87        64.28                1033
 TCP upload BE3 singapore.starlink.taht.net                    :         0.06         0.44 Mbits/s        1384
 TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :         4.16         4.00                1040
 TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :         0.06         0.06                1031
 TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :         0.12         0.11                1036
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :       577.75       438.61                1027
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :       107.05        72.97                1027
 TCP upload BE4 fremont.starlink.taht.net                      :         0.10         1.48 Mbits/s        1384
 TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :         4.45         4.00                1020
 TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :         0.09         0.09                1017
 TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :         0.21         0.16                1019
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :       439.05       285.86                1018
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :        81.90        56.33                1018
 TCP upload avg                                                :         0.08          N/A Mbits/s        1384
 TCP upload fairness                                           :         0.97          N/A Mbits/s        1384
 TCP upload sum                                                :         0.34          N/A Mbits/s        1384


EdgeRouter Settings

smart-queue DSL {
        download {
            ecn enable
            flows 1024
            fq-quantum 300
            limit 10240
            rate 8mbit
        }
        upload {
            ecn enable
            flows 1024
            fq-quantum 300
            limit 10240
            rate 600kbit
            target 15ms
        }
        wan-interface pppoe0

Also, here's the VMs qdisc settings. Which I now realize probably aren't the best

sysctl -a | grep qdisc
net.core.default_qdisc = pfifo_fast

tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
 Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth0 parent :1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0



-Jordan

-----Original Message-----
From: Starlink <starlink-bounces at lists.bufferbloat.net> On Behalf Of Dave Taht
Sent: Saturday, April 16, 2022 9:40 PM
To: flent-users <flent-users at flent.org>; bloat <bloat at lists.bufferbloat.net>; starlink at lists.bufferbloat.net
Subject: [Starlink] an rtt_fair test?

if anyone has a bit of spare time this holiday, I'd like to collect a few results from various networks around the web this week, not just on the turris. Example command line and outputs here:

https://forum.turris.cz/t/sqm-on-turris-flent-benchmarks/17048/

--
I tried to build a better future, a few times:
https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org

Dave Täht CEO, TekLibre, LLC
_______________________________________________
Starlink mailing list
Starlink at lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink



More information about the Bloat mailing list