[Rpm] rpm meeting notes from last week
Will Hawkins
hawkinsw at obs.cr
Sun Mar 27 18:26:21 EDT 2022
On Sun, Mar 27, 2022 at 2:34 PM Sebastian Moeller via Rpm
<rpm at lists.bufferbloat.net> wrote:
>
> Dear RPMers,
>
> I just managed to install and test goresponsiveness on an Ubuntu20.4 LTS linux host (where the default go is a bit on the old side), and I am 100% not confident that the test runs sufficiently long to either saturate the link or to measure the saturating load sufficiently precise and IMHO most likely neither...
>
> Here is the text of an issue I posted as an issue to the nice goresponsiveness project:
>
> The total measurement duration of the test against apples servers appears quite short. It would be nice if the duration could be controlled from the client (or if that is not according to the spec if the servers could be configured to measure for longer durations):
>
> user at ubuntu20.4LTS:~/CODE/goresponsiveness$ time ./networkQuality --config mensura.cdn-apple.com --port 443 --path /api/v1/gm/config --timeout 60
> 03-27-2022 17:51:51 UTC Go Responsiveness to mensura.cdn-apple.com:443...
> Download: 87.499 Mbps ( 10.937 MBps), using 12 parallel connections.
> Upload: 26.531 Mbps ( 3.316 MBps), using 12 parallel connections.
> Total RTTs measured: 5
> RPM: 829
>
> real 0m5.504s
> user 0m1.277s
> sys 0m1.225s
>
>
> ~6 seconds, that is even shorter than run-of-the-mill speedtests (which IMHO tend to clock in at ~10 seconds per direction)... This might or might not result in a saturating load, but it most certainly will result in an imprecise throughput measurement...
>
> Experience with bufferbloat measurements seems to indicate that >= 20-30 seconds are required to really saturate a link.
>
> Trying to confirm this with flent/netperf:
>
> date ; ping -c 10 netperf-eu.bufferbloat.net ; ./run-flent --ipv4 -l 30 -H netperf-eu.bufferbloat.net rrul_var --remote-metadata=root at 192.168.42.1 --test-parameter=cpu_stats_hosts=root at 192.168.42.1 --step-size=.05 --socket-stats --test-parameter bidir_streams=8 --test-parameter markings=0,0,0,0,0,0,0,0 --test-parameter ping_hosts=1.1.1.1 -D . -t IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.6-pppoe-wan-eth2.7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_netperf-eu.bufferbloat.net --log-file
>
> [...]
>
> Ping (ms) ICMP 1.1.1.1 (extra) : 12.20 12.10 21.91 ms 914
> Ping (ms) avg : 26.11 N/A N/A ms 914
> Ping (ms)::ICMP : 25.30 25.10 35.11 ms 914
> Ping (ms)::UDP 0 (0) : 25.42 25.43 33.43 ms 914
> Ping (ms)::UDP 1 (0) : 25.36 25.32 33.77 ms 914
> Ping (ms)::UDP 2 (0) : 26.99 26.88 36.13 ms 914
> Ping (ms)::UDP 3 (0) : 25.27 25.20 34.42 ms 914
> Ping (ms)::UDP 4 (0) : 25.26 25.22 33.32 ms 914
> Ping (ms)::UDP 5 (0) : 27.99 27.85 37.20 ms 914
> Ping (ms)::UDP 6 (0) : 27.95 27.89 36.29 ms 914
> Ping (ms)::UDP 7 (0) : 25.42 25.32 34.56 ms 914
> TCP download avg : 11.93 N/A N/A Mbits/s 914
> TCP download sum : 95.44 N/A N/A Mbits/s 914
> TCP download::0 (0) : 11.85 12.26 13.97 Mbits/s 914
> TCP download::1 (0) : 11.81 12.28 13.81 Mbits/s 914
> TCP download::2 (0) : 12.13 12.31 16.28 Mbits/s 914
> TCP download::3 (0) : 12.04 12.31 13.76 Mbits/s 914
> TCP download::4 (0) : 11.78 12.28 13.83 Mbits/s 914
> TCP download::5 (0) : 12.07 12.30 14.44 Mbits/s 914
> TCP download::6 (0) : 11.78 12.29 13.67 Mbits/s 914
> TCP download::7 (0) : 11.98 12.30 13.92 Mbits/s 914
> TCP totals : 126.34 N/A N/A Mbits/s 914
> TCP upload avg : 3.86 N/A N/A Mbits/s 914
> TCP upload sum : 30.90 N/A N/A Mbits/s 914
> [...]
>
> This is a bidirectional test that only reports actual TCP goodput, neither the latency probes nor the reverse ACK traffic is accounted and yet:
> TCP download sum: 95.44
> TCP upload sum: 30.90
>
> which compared to the actual shape settings of 105/36 (with 34 bytes overhead) seems sane:
> theoretical maximal throughput:
> IPv4
> 105 * ((1500-8-20-20-12)/(1500+26)) = 99.08 Mbps
> 36 * ((1500-8-20-20-12)/(1500+26)) = 33.97 Mbps
> IPv6
> 105 * ((1500-8-40-20-12)/(1500+26)) = 97.71 Mbps
> 36 * ((1500-8-40-20-12)/(1500+26)) = 33.50 Mbps
>
> Compared to these numbers the reported 87.499/26.531 do not seems either saturating or precise, probably neither.... I do wonder how reliable the RPM number is gong to be if the "saturate the link" part apparently failed?
>
> Question: is full saturation actually intended here, or is "sufficiently high load" the goal (if the later what constitutes sufficiently high)?
According to the spec, you may not get completely full saturation. The
definition of saturation per the spec is fairly specific: See 4.1.4 of
https://github.com/network-quality/draft-ietf-ippm-responsiveness/blob/master/draft-ietf-ippm-responsiveness.txt.
We are still working through some of the timing issues in the
goresponsiveness client and doing our best. Thank you for your
patience and helpful, constructive and kind feedback.
Will
>
> Regards
> Sebastian
>
> P.S.: When disabling my traffic shaper I get:
>
> user at ubuntu20.4LTS:~/CODE/flent$ time ../goresponsiveness/networkQuality --config mensura.cdn-apple.com --port 443 --path /api/v1/gm/config --timeout 60
> 03-27-2022 18:20:06 UTC Go Responsiveness to mensura.cdn-apple.com:443...
> Download: 99.390 Mbps ( 12.424 MBps), using 12 parallel connections.
> Upload: 27.156 Mbps ( 3.395 MBps), using 20 parallel connections.
> Total RTTs measured: 5
> RPM: 512
>
> real 0m9.728s
> user 0m1.763s
> sys 0m1.660s
> user at ubuntu20.4LTS:~/CODE/flent$ time ../goresponsiveness/networkQuality --config mensura.cdn-apple.com --port 443 --path /api/v1/gm/config --timeout 60
> 03-27-2022 18:20:27 UTC Go Responsiveness to mensura.cdn-apple.com:443...
> Download: 97.608 Mbps ( 12.201 MBps), using 12 parallel connections.
> Upload: 27.375 Mbps ( 3.422 MBps), using 24 parallel connections.
> Total RTTs measured: 5
> RPM: 366
>
> real 0m12.926s
> user 0m2.330s
> sys 0m2.442s
>
> Which for the download approaches/exceeds the theoretical limit* yet for the upload still falls a bit short of the 8 flow flent result
>
>
> *) This is nothing particular unique to goresponsiveness most multi-stream tests fail to actually report robust and reliable values for thoughput, even though this "ain't rocket science". Instead of trying to discount the TCP start-up phase somehow, simply run the test long enough so the start up does not dominate the result and just divide total transported volume by total duration... yes that is going to slightly underestimate the total link capacity, but that seems IMHO considerably more benign/useful than reporting 99.39 of 99.08 possible or rather 97.71 possible , since the actual test uses IPv6 on my link...
>
>
>
> > On Mar 27, 2022, at 16:25, Dave Taht via Rpm <rpm at lists.bufferbloat.net> wrote:
> >
> > Sasha has re-organized the meeting notes we've been taking which will
> > be permanently up here:
> >
> > https://docs.google.com/document/d/19K14vrBLNvX_KOdRMK_s0SCYTnj70IJ-dQubL37kQgw/edit
> >
> > The public meetings continue at 10AM PDT tuesdays.
> >
> > One thing came up in that meeting that bears further reflection: The
> > structure of the the responsiveness test is e2e, and differentiating
> > between the wifi hop and/or sourcing tests from the router - is hard.
> > In the latter case, we are asking a router not only to saturate its
> > network connection, but to do it using crypto, which is a mighty task
> > to ask of a router that's job is primarily routing (with offloads).
> >
> > A thought might be to attempt p2p tests between say, an ios phone and
> > an osx laptop, which would exercise the router as a wifi router. A
> > docker container for the server would be straightforward. (It's still
> > hard for me to
> > trust containers or vm's to run fast or accurately enough, too).
> >
> > Former case is still e2e but with the icmp unreachable idea I described in wtbb.
> >
> > --
> > I tried to build a better future, a few times:
> > https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> >
> > Dave Täht CEO, TekLibre, LLC
> > _______________________________________________
> > Rpm mailing list
> > Rpm at lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/rpm
>
> _______________________________________________
> Rpm mailing list
> Rpm at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/rpm
More information about the Rpm
mailing list