From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id AD7D33CB35 for ; Sun, 27 Mar 2022 14:34:11 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1648406050; bh=y6n6ToJB3Ffa9X+lnLMNNDK363CpVoFii1p9PdLKY48=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=CWUwgkNu/7KXmnEpRDjxpzJ/5bJHKZ0dWfNy7PwFlHaISX5Ceei1LN+q3+fPgr+sC tf1iZ75hKKRPFstXl8xqMQJCLt2m4s10Is8zydMgFsBa/mWx3q8c8AKslOUpv//A/a agg92FSaBj9hJRe8nJ9UdYYfl1ye9cIs1jimaDR4= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from smtpclient.apple ([77.1.234.251]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MBm1U-1nkavm0CSq-00CAev; Sun, 27 Mar 2022 20:34:10 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) From: Sebastian Moeller In-Reply-To: Date: Sun, 27 Mar 2022 20:34:09 +0200 Cc: Rpm Content-Transfer-Encoding: quoted-printable Message-Id: <0D7D64F2-984B-428D-AC42-C862AB141E01@gmx.de> References: To: =?utf-8?Q?Dave_T=C3=A4ht?= X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Provags-ID: V03:K1:SogsRe4wKODn4qvDxuDZJ+puLjFOo8E5daOKj3JtctjPzMNVmkV /HYUf+GkSTA0nmT6Ps3rp/axbasia93HF47ApvRZdNktTu/lnEaQndpR4JoS1IexKGPt9/R GMCmwWUwsvYuUgmBVQx8ajNKBpv7g2BadLSSVtKdEnNJ1QqdsXxWPxa6AkGizlH7gO8y/8a hRU1NQ8aiEIypPKHgQ1cQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:ewxUZkIr4ig=:Gsb0pTq1/UmrhMc0Z1VEUx MeIe69yrwyBthNM9sQZr9TdyaBHO1gCPTcm+8S/G02eV18oh/mnCmXOH8cfk+kz6zpJHAVx6N KlXWbHsGQ3gvD6f6KTwI4U0HL9p7vUREPVfVlrhUo68oVqa+aeXKJ7LZiPnR15dUMMvMnJS2a ClqVlvpB1CS0YnS9h7iHd4yvQN+kpMzCzdl6Yp4jDTZzxGxdLiEq/uo/mKfKfyDNushJpUAIZ VDEWemjX7PVjeF6GSc5vkxAuXBMLUy+dF87yIYPMCkiZB1tqqtooSaIFsaztA67Or5AY+JFAU lP5jlpq8Gy/y9X84gGegr9eFwdjDMQlfn2eozsQTtosVYmZ2UbpWVtpPLcxy/W9av36JthhiL ijDsFou9lNNTJcGSZ2hlWlRkbAk3DP4537bQL2YY+NgI2fynGxa8Qxqn7yBVByHPs2Itnknol 6oy/EEQQAbF1kDQCWAmmTa1uPFM7ho6805/wEESUxXkP2y9gQMMOdy5tA6DkpmihnCgT7duRY j5S4n6EJZPv7kZegXmGZuisQGssg1cjSL+MVAp6S/X/ddcxnB3pCY7DmlKtSU7ipLtdWP7bhK msaPuesURUe9AehjJI09WmDwFgH/QmR1aC7gj1t43wKJIlGo7PCf9yaviKSJhozzHaOZ98a28 owPmgGzc4eAJzTT3IPf28gtjamZZ0AnDu4j1vHfF0Dm5tMoezsRJiTOtd69nLEVtYDAQ8WdA1 8d1J//eSpwtjTnZi9E2KQhpbbtCS0Y7z2Zd5fk3P3cF3glglyvIBFeuO0PDL1nk6fa69jGOAD QvAU0f+GW4exv/gaKyPpcj28jIa8JpiSSMd6uiG0KnB8jUE27CrSh10DwbEIzQWlyUO2JOIQZ S/w1Ql5iPMynzE5sAyIoTFOz2zo2HyjdPJ70ULnk57D20L7LzL/08TJSgBMin+GripUk9G0kr OMzAzDqQPreSBNi8OvU1h/Ta2GxDBEtQzkTLA2ScEi9BUVW7vpCVe3DTUWTLn/g5VyiZ8AF7T jxA9RMFc7BPEoxPN3TPapBBaUn85AbrMpSb9BQ6xvqZwsCSazfCdHJtp91Vn2IP5W+Zzdkog7 Tb0zmoI5XVm4A0= Subject: Re: [Rpm] rpm meeting notes from last week X-BeenThere: rpm@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: revolutions per minute - a new metric for measuring responsiveness List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Mar 2022 18:34:12 -0000 Dear RPMers, I just managed to install and test goresponsiveness on an Ubuntu20.4 LTS = linux host (where the default go is a bit on the old side), and I am = 100% not confident that the test runs sufficiently long to either = saturate the link or to measure the saturating load sufficiently precise = and IMHO most likely neither... Here is the text of an issue I posted as an issue to the nice = goresponsiveness project: The total measurement duration of the test against apples servers = appears quite short. It would be nice if the duration could be = controlled from the client (or if that is not according to the spec if = the servers could be configured to measure for longer durations): user@ubuntu20.4LTS:~/CODE/goresponsiveness$ time ./networkQuality = --config mensura.cdn-apple.com --port 443 --path /api/v1/gm/config = --timeout 60 03-27-2022 17:51:51 UTC Go Responsiveness to = mensura.cdn-apple.com:443... Download: 87.499 Mbps ( 10.937 MBps), using 12 parallel connections. Upload: 26.531 Mbps ( 3.316 MBps), using 12 parallel connections. Total RTTs measured: 5 RPM: 829 real 0m5.504s user 0m1.277s sys 0m1.225s ~6 seconds, that is even shorter than run-of-the-mill speedtests (which = IMHO tend to clock in at ~10 seconds per direction)... This might or = might not result in a saturating load, but it most certainly will result = in an imprecise throughput measurement... Experience with bufferbloat measurements seems to indicate that >=3D = 20-30 seconds are required to really saturate a link. Trying to confirm this with flent/netperf: date ; ping -c 10 netperf-eu.bufferbloat.net ; ./run-flent --ipv4 -l 30 = -H netperf-eu.bufferbloat.net rrul_var = --remote-metadata=3Droot@192.168.42.1 = --test-parameter=3Dcpu_stats_hosts=3Droot@192.168.42.1 --step-size=3D.05 = --socket-stats --test-parameter bidir_streams=3D8 --test-parameter = markings=3D0,0,0,0,0,0,0,0 --test-parameter ping_hosts=3D1.1.1.1 -D . -t = IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of= 116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.6-pppoe-wan-eth2.7_2_br= idged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_netperf-eu.bufferbloa= t.net --log-file [...] Ping (ms) ICMP 1.1.1.1 (extra) : 12.20 12.10 = 21.91 ms 914 Ping (ms) avg : 26.11 N/A = N/A ms 914 Ping (ms)::ICMP : 25.30 25.10 = 35.11 ms 914 Ping (ms)::UDP 0 (0) : 25.42 25.43 = 33.43 ms 914 Ping (ms)::UDP 1 (0) : 25.36 25.32 = 33.77 ms 914 Ping (ms)::UDP 2 (0) : 26.99 26.88 = 36.13 ms 914 Ping (ms)::UDP 3 (0) : 25.27 25.20 = 34.42 ms 914 Ping (ms)::UDP 4 (0) : 25.26 25.22 = 33.32 ms 914 Ping (ms)::UDP 5 (0) : 27.99 27.85 = 37.20 ms 914 Ping (ms)::UDP 6 (0) : 27.95 27.89 = 36.29 ms 914 Ping (ms)::UDP 7 (0) : 25.42 25.32 = 34.56 ms 914 TCP download avg : 11.93 N/A = N/A Mbits/s 914 TCP download sum : 95.44 N/A = N/A Mbits/s 914 TCP download::0 (0) : 11.85 12.26 = 13.97 Mbits/s 914 TCP download::1 (0) : 11.81 12.28 = 13.81 Mbits/s 914 TCP download::2 (0) : 12.13 12.31 = 16.28 Mbits/s 914 TCP download::3 (0) : 12.04 12.31 = 13.76 Mbits/s 914 TCP download::4 (0) : 11.78 12.28 = 13.83 Mbits/s 914 TCP download::5 (0) : 12.07 12.30 = 14.44 Mbits/s 914 TCP download::6 (0) : 11.78 12.29 = 13.67 Mbits/s 914 TCP download::7 (0) : 11.98 12.30 = 13.92 Mbits/s 914 TCP totals : 126.34 N/A = N/A Mbits/s 914 TCP upload avg : 3.86 N/A = N/A Mbits/s 914 TCP upload sum : 30.90 N/A = N/A Mbits/s 914 [...] This is a bidirectional test that only reports actual TCP goodput, = neither the latency probes nor the reverse ACK traffic is accounted and = yet: TCP download sum: 95.44 TCP upload sum: 30.90=20 which compared to the actual shape settings of 105/36 (with 34 bytes = overhead) seems sane: theoretical maximal throughput: IPv4 105 * ((1500-8-20-20-12)/(1500+26)) =3D 99.08 Mbps 36 * ((1500-8-20-20-12)/(1500+26)) =3D 33.97 Mbps IPv6 105 * ((1500-8-40-20-12)/(1500+26)) =3D 97.71 Mbps 36 * ((1500-8-40-20-12)/(1500+26)) =3D 33.50 Mbps Compared to these numbers the reported 87.499/26.531 do not seems either = saturating or precise, probably neither.... I do wonder how reliable the = RPM number is gong to be if the "saturate the link" part apparently = failed? Question: is full saturation actually intended here, or is "sufficiently = high load" the goal (if the later what constitutes sufficiently high)? Regards Sebastian P.S.: When disabling my traffic shaper I get: user@ubuntu20.4LTS:~/CODE/flent$ time ../goresponsiveness/networkQuality = --config mensura.cdn-apple.com --port 443 --path /api/v1/gm/config = --timeout 60 03-27-2022 18:20:06 UTC Go Responsiveness to = mensura.cdn-apple.com:443... Download: 99.390 Mbps ( 12.424 MBps), using 12 parallel connections. Upload: 27.156 Mbps ( 3.395 MBps), using 20 parallel connections. Total RTTs measured: 5 RPM: 512 real 0m9.728s user 0m1.763s sys 0m1.660s user@ubuntu20.4LTS:~/CODE/flent$ time ../goresponsiveness/networkQuality = --config mensura.cdn-apple.com --port 443 --path /api/v1/gm/config = --timeout 60 03-27-2022 18:20:27 UTC Go Responsiveness to = mensura.cdn-apple.com:443... Download: 97.608 Mbps ( 12.201 MBps), using 12 parallel connections. Upload: 27.375 Mbps ( 3.422 MBps), using 24 parallel connections. Total RTTs measured: 5 RPM: 366 real 0m12.926s user 0m2.330s sys 0m2.442s Which for the download approaches/exceeds the theoretical limit* yet for = the upload still falls a bit short of the 8 flow flent result *) This is nothing particular unique to goresponsiveness most = multi-stream tests fail to actually report robust and reliable values = for thoughput, even though this "ain't rocket science". Instead of = trying to discount the TCP start-up phase somehow, simply run the test = long enough so the start up does not dominate the result and just divide = total transported volume by total duration... yes that is going to = slightly underestimate the total link capacity, but that seems IMHO = considerably more benign/useful than reporting 99.39 of 99.08 possible = or rather 97.71 possible , since the actual test uses IPv6 on my link... > On Mar 27, 2022, at 16:25, Dave Taht via Rpm = wrote: >=20 > Sasha has re-organized the meeting notes we've been taking which will > be permanently up here: >=20 > = https://docs.google.com/document/d/19K14vrBLNvX_KOdRMK_s0SCYTnj70IJ-dQubL3= 7kQgw/edit >=20 > The public meetings continue at 10AM PDT tuesdays. >=20 > One thing came up in that meeting that bears further reflection: The > structure of the the responsiveness test is e2e, and differentiating > between the wifi hop and/or sourcing tests from the router - is hard. > In the latter case, we are asking a router not only to saturate its > network connection, but to do it using crypto, which is a mighty task > to ask of a router that's job is primarily routing (with offloads). >=20 > A thought might be to attempt p2p tests between say, an ios phone and > an osx laptop, which would exercise the router as a wifi router. A > docker container for the server would be straightforward. (It's still > hard for me to > trust containers or vm's to run fast or accurately enough, too). >=20 > Former case is still e2e but with the icmp unreachable idea I = described in wtbb. >=20 > --=20 > I tried to build a better future, a few times: > https://wayforward.archive.org/?site=3Dhttps%3A%2F%2Fwww.icei.org >=20 > Dave T=C3=A4ht CEO, TekLibre, LLC > _______________________________________________ > Rpm mailing list > Rpm@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/rpm