[Bloat] [Starlink] an rtt_fair test?

Daniel Sterling sterling.daniel at gmail.com
Sun Apr 17 12:54:44 EDT 2022


test from my network running customized openwrt w/ cake

tar file here:

 rtt_fair4be-2022-04-17T124830.462924.dss-cake.f...
<https://drive.google.com/file/d/1MzuHQfEsl7xQFXe0F7n506KYhRAE4jXI/view?usp=drive_web>


root at DESKTOP-DLUP89P:~# flent -x --socket-stats --step-size=.05 -H
de.starlink.taht.net -H london.starlink.taht.net -H s
ingapore.starlink.taht.net -H fremont.starlink.taht.net -t 'dss-cake'
rtt_fair4be
Started Flent 1.3.2 using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to
./rtt_fair4be-2022-04-17T124830.462924.dss-cake.flent.gz.

Summary of rtt_fair4be test run from 2022-04-17 16:48:30.462924
  Title: 'dss-cake'


avg       median          # data pts
 Ping (ms) ICMP1 de.starlink.taht.net                          :
119.54       114.00 ms             1398
 Ping (ms) ICMP2 london.starlink.taht.net                      :
107.97       104.00 ms             1398
 Ping (ms) ICMP3 singapore.starlink.taht.net                   :
272.62       268.00 ms             1398
 Ping (ms) ICMP4 fremont.starlink.taht.net                     :
 89.15        84.60 ms             1398
 Ping (ms) avg                                                 :
147.32          N/A ms             1398
 TCP download BE1 de.starlink.taht.net                         :
 25.01        24.81 Mbits/s        1398
 TCP download BE2 london.starlink.taht.net                     :
 23.00        22.75 Mbits/s        1398
 TCP download BE3 singapore.starlink.taht.net                  :
 20.26        21.01 Mbits/s        1398
 TCP download BE4 fremont.starlink.taht.net                    :
 35.64        32.94 Mbits/s        1398
 TCP download avg                                              :
 25.98          N/A Mbits/s        1398
 TCP download fairness                                         :
0.95          N/A Mbits/s        1398
 TCP download sum                                              :
103.91          N/A Mbits/s        1398
 TCP upload BE1 de.starlink.taht.net                           :
7.76         8.00 Mbits/s        1398
 TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :
 88.38        91.00                1067
 TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :
7.52         7.57                1064
 TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :
9.82         9.73                1066
 TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :
130.68       125.68                1062
 TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :
5.38         4.25                1062
 TCP upload BE2 london.starlink.taht.net                       :
7.53         7.91 Mbits/s        1398
 TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :
 78.47        81.00                1059
 TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :
7.26         7.34                1056
 TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :
9.48         9.45                1057
 TCP upload BE2 london.starlink.taht.net::tcp_rtt              :
120.05       114.79                1056
 TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :
5.25         4.15                1056
 TCP upload BE3 singapore.starlink.taht.net                    :
3.98         4.23 Mbits/s        1398
 TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :
100.96       105.00                1066
 TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :
3.95         4.10                1056
 TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :
5.07         5.15                1061
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :
291.42       286.12                1060
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :
7.62         5.64                1060
 TCP upload BE4 fremont.starlink.taht.net                      :
8.41         8.56 Mbits/s        1398
 TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :
 75.40        73.00                1060
 TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :
8.04         8.00                1057
 TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :
 10.62        10.31                1058
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :
103.74        98.50                1057
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :
4.97         3.96                1057
 TCP upload avg                                                :
6.92          N/A Mbits/s        1398
 TCP upload fairness                                           :
0.94          N/A Mbits/s        1398
 TCP upload sum                                                :
 27.68          N/A Mbits/s        1398
root at DESKTOP-DLUP89P:~#


On Sun, Apr 17, 2022 at 11:36 AM jordan--- via Bloat <
bloat at lists.bufferbloat.net> wrote:

> You're welcome! Here's the files Flent generated:
> https://1drv.ms/u/s!Ap4u4Rte63FqjqoSQXX444OWLvl8FQ?e=f22Nt7
>
> And thanks for the link! It honestly doesn't look too bad to implement so
> I might give it a shot today if time allows.
>
> I've also been contemplating replacing it with something like a Mikrotik
> since cake is now integrated into them. I've been testing one at home and
> made a post with a bunch of flent tests to the cake thread in their forum
> if you're interested in some data from a Charter Spectrum cable connection:
> https://forum.mikrotik.com/viewtopic.php?t=179307#p925485
>
> The OneDrive link to my flent files should still be active in that post as
> well.
>
> -Jordan
>
> -----Original Message-----
> From: Dave Taht <dave.taht at gmail.com>
> Sent: Sunday, April 17, 2022 10:57 AM
> To: Jordan Szuch <jordan at inacomptc.com>
> Cc: flent-users <flent-users at flent.org>; bloat <
> bloat at lists.bufferbloat.net>; starlink at lists.bufferbloat.net
> Subject: Re: [Starlink] an rtt_fair test?
>
> Thank you for testing!!! If you could stash the *.flent.gz files somewhere
> I could get at, that would be great. You can also browse the plots using
> the flent-gui to see more detail over time.
>
> Cake, long available as an out of tree build for the edgerouter, has both
> dsl compensation and an ack-filter which would help A LOT, in your case. i
> wish I'd pushed harder to get dsl compensation into the edgerouter smart
> queues code...
>
>
> https://community.ui.com/questions/Cake-compiled-for-the-EdgeRouter-devices/fc1ff27c-f321-4344-8737-fcc755cae8a2
>
> Given your 500kbit upload speed the minimum latency inflation under the
> structure of this test (4 upload flows) should have been around 150ms (4 *
> 25ms + ack overhead), but without looking at the *.gz files I don't know
> what your baseline latency was to any of these sites. (it would be good if
> flent reported on the text output what the minimum actually was)
>
> On Sun, Apr 17, 2022 at 7:35 AM <jordan at inacomptc.com> wrote:
> >
> > Hi Dave,
> >
> > This isn't specifically cake related but I figured it could be
> interesting, especially as these are from a location where I've been
> investigating setting up something with cake. These are statistics from a
> relatives location where they only have a DSL line from CenturyLink
> available to them. There's a little Debian based VM I setup for a couple
> random things so I was able to run flent from that. They have a EdgeRouter
> X serving as the router so they can at least take advantage of fq_codel
> through the routers Smart Queue feature. I ran a quick speed test, then ran
> flent without sqm, and then with sqm. Sorry if this gets a bit long and
> hopefully it stays readable.
> >
> > #############
> > Speedtest by Ookla
> >
> >      Server: 123.Net, Inc. - Southfield, MI (id = 15044)
> >         ISP: CenturyLink
> >     Latency:    45.45 ms   (194.34 ms jitter)
> >    Download:     8.88 Mbps (data used: 12.6 MB )
> >      Upload:     0.64 Mbps (data used: 962.7 kB )
> > Packet Loss:     5.1%
> >  Result URL:
> > https://www.speedtest.net/result/c/799ba46b-d3d4-4d54-ac76-118fac847b7
> > 3
> >
> > ###########
> > No SQM or QOS
> >
> > flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H
> > london.starlink.taht.net -H singapore.starlink.taht.net -H
> fremont.starlink.taht.net -t
> nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be Started
> Flent 2.0.0 using Python 3.9.2.
> > Starting rtt_fair4be test. Expected run time: 70 seconds.
> > Data file written to
> > ./rtt_fair4be-2022-04-17T145553.865990.nosqm_erx_rttfair4be_8Mbpsdown_
> > 500Kbpsup_centurylinkdsl.flent.gz
> >
> > Summary of rtt_fair4be test run from 2022-04-17 13:55:53.865990
> >   Title: 'nosqm_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'
> >
> >
>  avg       median          # data pts
> >  Ping (ms) ICMP1 de.starlink.taht.net                          :
> 1026.06       920.00 ms             1383
> >  Ping (ms) ICMP2 london.starlink.taht.net                      :
> 1009.28       887.00 ms             1383
> >  Ping (ms) ICMP3 singapore.starlink.taht.net                   :
> 1121.09      1012.00 ms             1383
> >  Ping (ms) ICMP4 fremont.starlink.taht.net                     :
>  943.77       853.00 ms             1383
> >  Ping (ms) avg                                                 :
> 1025.05          N/A ms             1383
> >  TCP download BE1 de.starlink.taht.net                         :
>  1.36         2.00 Mbits/s        1383
> >  TCP download BE2 london.starlink.taht.net                     :
>  0.72         0.65 Mbits/s        1383
> >  TCP download BE3 singapore.starlink.taht.net                  :
>  1.03         1.74 Mbits/s        1383
> >  TCP download BE4 fremont.starlink.taht.net                    :
>  2.11         2.88 Mbits/s        1383
> >  TCP download avg                                              :
>  1.30          N/A Mbits/s        1383
> >  TCP download fairness                                         :
>  0.86          N/A Mbits/s        1383
> >  TCP download sum                                              :
>  5.22          N/A Mbits/s        1383
> >  TCP upload BE1 de.starlink.taht.net                           :
>  0.12         0.20 Mbits/s        1383
> >  TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :
> 16.85        14.00                1044
> >  TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :
>  0.12         0.12                1036
> >  TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :
>  0.35         0.26                1042
> >  TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :
> 1128.57       954.47                1020
> >  TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :
> 68.22        50.08                1020
> >  TCP upload BE2 london.starlink.taht.net                       :
>  0.12         0.15 Mbits/s        1383
> >  TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :
> 17.33        11.00                1046
> >  TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :
>  0.12         0.11                1041
> >  TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :
>  0.38         0.26                1044
> >  TCP upload BE2 london.starlink.taht.net::tcp_rtt              :
> 1166.74       991.48                1043
> >  TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :
> 79.99        68.27                1043
> >  TCP upload BE3 singapore.starlink.taht.net                    :
>  0.07         0.13 Mbits/s        1383
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :
> 13.60         9.00                1042
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :
>  0.07         0.08                1013
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :
>  0.23         0.20                1031
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :
> 1302.23      1129.07                 967
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :
>  113.67        98.84                 967
> >  TCP upload BE4 fremont.starlink.taht.net                      :
>  0.11         0.20 Mbits/s        1383
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :
> 17.55        11.00                1038
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :
>  0.11         0.10                1035
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :
>  0.32         0.23                1036
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :
> 1094.64       887.54                1033
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :
> 67.20        59.16                1033
> >  TCP upload avg                                                :
>  0.10          N/A Mbits/s        1383
> >  TCP upload fairness                                           :
>  0.96          N/A Mbits/s        1383
> >  TCP upload sum                                                :
>  0.42          N/A Mbits/s        1383
> >
> > ##########
> > with smart queue (fq_codel) on the EdgeRouter
> >
> > flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H
> > london.starlink.taht.net -H singapore.starlink.taht.net -H
> fremont.starlink.taht.net -t
> fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl rtt_fair4be
> Started Flent 2.0.0 using Python 3.9.2.
> > Starting rtt_fair4be test. Expected run time: 70 seconds.
> > Data file written to
> > ./rtt_fair4be-2022-04-17T150339.549817.fqcodel_erx_rttfair4be_8Mbpsdow
> > n_500Kbpsup_centurylinkdsl.flent.gz
> >
> > Summary of rtt_fair4be test run from 2022-04-17 14:03:39.549817
> >   Title: 'fqcodel_erx_rttfair4be_8Mbpsdown_500Kbpsup_centurylinkdsl'
> >
> >
>  avg       median          # data pts
> >  Ping (ms) ICMP1 de.starlink.taht.net                          :
>  392.93       208.00 ms             1384
> >  Ping (ms) ICMP2 london.starlink.taht.net                      :
>  373.94       196.00 ms             1384
> >  Ping (ms) ICMP3 singapore.starlink.taht.net                   :
>  508.15       319.00 ms             1384
> >  Ping (ms) ICMP4 fremont.starlink.taht.net                     :
>  346.23       152.00 ms             1384
> >  Ping (ms) avg                                                 :
>  405.31          N/A ms             1384
> >  TCP download BE1 de.starlink.taht.net                         :
>  1.36         1.62 Mbits/s        1384
> >  TCP download BE2 london.starlink.taht.net                     :
>  1.47         1.63 Mbits/s        1384
> >  TCP download BE3 singapore.starlink.taht.net                  :
>  1.55         1.60 Mbits/s        1384
> >  TCP download BE4 fremont.starlink.taht.net                    :
>  1.50         1.80 Mbits/s        1384
> >  TCP download avg                                              :
>  1.47          N/A Mbits/s        1384
> >  TCP download fairness                                         :
>  1.00          N/A Mbits/s        1384
> >  TCP download sum                                              :
>  5.88          N/A Mbits/s        1384
> >  TCP upload BE1 de.starlink.taht.net                           :
>  0.09         0.85 Mbits/s        1384
> >  TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :
>  4.50         4.00                1035
> >  TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :
>  0.08         0.08                1030
> >  TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :
>  0.17         0.15                1032
> >  TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :
>  458.11       318.82                1027
> >  TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :
> 95.76        61.88                1027
> >  TCP upload BE2 london.starlink.taht.net                       :
>  0.09         0.93 Mbits/s        1384
> >  TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :
>  4.17         4.00                1038
> >  TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :
>  0.08         0.08                1033
> >  TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :
>  0.17         0.14                1036
> >  TCP upload BE2 london.starlink.taht.net::tcp_rtt              :
>  452.67       322.49                1033
> >  TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :
> 96.87        64.28                1033
> >  TCP upload BE3 singapore.starlink.taht.net                    :
>  0.06         0.44 Mbits/s        1384
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :
>  4.16         4.00                1040
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :
>  0.06         0.06                1031
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :
>  0.12         0.11                1036
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :
>  577.75       438.61                1027
> >  TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :
>  107.05        72.97                1027
> >  TCP upload BE4 fremont.starlink.taht.net                      :
>  0.10         1.48 Mbits/s        1384
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :
>  4.45         4.00                1020
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :
>  0.09         0.09                1017
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :
>  0.21         0.16                1019
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :
>  439.05       285.86                1018
> >  TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :
> 81.90        56.33                1018
> >  TCP upload avg                                                :
>  0.08          N/A Mbits/s        1384
> >  TCP upload fairness                                           :
>  0.97          N/A Mbits/s        1384
> >  TCP upload sum                                                :
>  0.34          N/A Mbits/s        1384
> >
> >
> > EdgeRouter Settings
> >
> > smart-queue DSL {
> >         download {
> >             ecn enable
> >             flows 1024
> >             fq-quantum 300
> >             limit 10240
> >             rate 8mbit
> >         }
> >         upload {
> >             ecn enable
> >             flows 1024
> >             fq-quantum 300
> >             limit 10240
> >             rate 600kbit
> >             target 15ms
> >         }
> >         wan-interface pppoe0
> >
> > Also, here's the VMs qdisc settings. Which I now realize probably
> > aren't the best
> >
> > sysctl -a | grep qdisc
> > net.core.default_qdisc = pfifo_fast
> >
> > tc -s qdisc
> > qdisc noqueue 0: dev lo root refcnt 2
> >  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)  backlog 0b
> > 0p requeues 0 qdisc mq 0: dev eth0 root  Sent 628201 bytes 8643 pkt
> > (dropped 0, overlimits 0 requeues 0)  backlog 0b 0p requeues 0 qdisc
> > pfifo_fast 0: dev eth0 parent :1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1
> > 1 1 1 1 1  Sent 628201 bytes 8643 pkt (dropped 0, overlimits 0
> > requeues 0)  backlog 0b 0p requeues 0
> >
> >
> >
> > -Jordan
> >
> > -----Original Message-----
> > From: Starlink <starlink-bounces at lists.bufferbloat.net> On Behalf Of
> > Dave Taht
> > Sent: Saturday, April 16, 2022 9:40 PM
> > To: flent-users <flent-users at flent.org>; bloat
> > <bloat at lists.bufferbloat.net>; starlink at lists.bufferbloat.net
> > Subject: [Starlink] an rtt_fair test?
> >
> > if anyone has a bit of spare time this holiday, I'd like to collect a
> few results from various networks around the web this week, not just on the
> turris. Example command line and outputs here:
> >
> > https://forum.turris.cz/t/sqm-on-turris-flent-benchmarks/17048/
> >
> > --
> > I tried to build a better future, a few times:
> > https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> >
> > Dave Täht CEO, TekLibre, LLC
> > _______________________________________________
> > Starlink mailing list
> > Starlink at lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/starlink
> >
>
>
> --
> I tried to build a better future, a few times:
> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
>
> Dave Täht CEO, TekLibre, LLC
>
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/bloat/attachments/20220417/4944b33a/attachment-0001.html>


More information about the Bloat mailing list