From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 1F4CB3CB35 for ; Sun, 27 Mar 2022 18:26:34 -0400 (EDT) Received: by mail-ej1-x629.google.com with SMTP id r13so25095031ejd.5 for ; Sun, 27 Mar 2022 15:26:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=obs-cr.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=n0vXdTZAEHb1J6TDn+n7AmPJe80bZVlWthpoNsNDYmc=; b=ikAGqlojzP7na5OThL7xh0rFJ3xakHq4VLIS1AW23GVw6uTM28e8hj1LSXNPkhlS8+ WfV5HnOvxZkKaQvOdwx0pdf47NtBaBh6Szqh0DP33RecVbT2BRlpy6rolr4ZfA7TdEQ6 2aVwokNdOG2gc30DOFyMIBK7Dq0CtVfYZUcCx00sNtAEvJmGXVkXNjZV4jJqdS1ef9Gi gQlkL+PMlvxjjinHHk/jeOmdPQubnHAIeAPWO2U3xAKUVHG+F004K0Z+2YP1jAfa5zag hrqfcQRtg1MTv/FYw9UYi7SSKZjlj0mFA8P0nljfnDbuxkQwn2fWxSJfaWTZx2X8wPPQ rMwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=n0vXdTZAEHb1J6TDn+n7AmPJe80bZVlWthpoNsNDYmc=; b=8Ipd7FGovZZbC+H1DAQfxO9iVbIsPheCDNylEo88xW72Wxe/YX8o3bX0/2PudWfRPi AzYXx/yDIKRIPD7ur4REDlBqo4oTEfNE+Tf7EVaQZAhSZlVteT1g8NiJ5+KF8/mohvzc h2UoJE+HPMRwahEhm4hh1dWiPLf3Y27dpffxif1nNZ5j3JqDbfEDReIxDA0LsgRzwKUz 7eM6ylF22AhX7e2r0ZpSHJwMF6qrxhsJ43BgLPT6yroMtA2JtQt0/PmIb+XMUhERhbXk DrxXHfW5+8PtujVz5wSHsGlvrRjb0oDomx8a8hBCrrKCjRS2U05uE+AtcIqQiAcKQrSh 1jag== X-Gm-Message-State: AOAM530eW+iO3zsw2VSLNB2uwe9lQhpQQoMRPpQkLBF+0giuy4uCKMfS XMTVPxVNS+7eExxSKIXS4Jl4qOuYQsy1j1aHLGQP6A== X-Google-Smtp-Source: ABdhPJwFTtApz/1qUkT2kX+KfPdqbTOSrz3vsRCBJJ9vcmEguePY6/OyFCM7eIhYpNnEl9j1zmh4e7osYlxZklG/ijI= X-Received: by 2002:a17:906:53c7:b0:6ce:6f32:ce53 with SMTP id p7-20020a17090653c700b006ce6f32ce53mr24767442ejo.352.1648419992479; Sun, 27 Mar 2022 15:26:32 -0700 (PDT) MIME-Version: 1.0 References: <0D7D64F2-984B-428D-AC42-C862AB141E01@gmx.de> In-Reply-To: <0D7D64F2-984B-428D-AC42-C862AB141E01@gmx.de> From: Will Hawkins Date: Sun, 27 Mar 2022 18:26:21 -0400 Message-ID: To: Sebastian Moeller Cc: =?UTF-8?Q?Dave_T=C3=A4ht?= , Rpm Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Rpm] rpm meeting notes from last week X-BeenThere: rpm@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: revolutions per minute - a new metric for measuring responsiveness List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Mar 2022 22:26:34 -0000 On Sun, Mar 27, 2022 at 2:34 PM Sebastian Moeller via Rpm wrote: > > Dear RPMers, > > I just managed to install and test goresponsiveness on an Ubuntu20.4 LTS = linux host (where the default go is a bit on the old side), and I am 100% n= ot confident that the test runs sufficiently long to either saturate the li= nk or to measure the saturating load sufficiently precise and IMHO most lik= ely neither... > > Here is the text of an issue I posted as an issue to the nice goresponsiv= eness project: > > The total measurement duration of the test against apples servers appears= quite short. It would be nice if the duration could be controlled from the= client (or if that is not according to the spec if the servers could be co= nfigured to measure for longer durations): > > user@ubuntu20.4LTS:~/CODE/goresponsiveness$ time ./networkQuality --confi= g mensura.cdn-apple.com --port 443 --path /api/v1/gm/config --timeout 60 > 03-27-2022 17:51:51 UTC Go Responsiveness to mensura.cdn-apple.com:443... > Download: 87.499 Mbps ( 10.937 MBps), using 12 parallel connections. > Upload: 26.531 Mbps ( 3.316 MBps), using 12 parallel connections. > Total RTTs measured: 5 > RPM: 829 > > real 0m5.504s > user 0m1.277s > sys 0m1.225s > > > ~6 seconds, that is even shorter than run-of-the-mill speedtests (which I= MHO tend to clock in at ~10 seconds per direction)... This might or might n= ot result in a saturating load, but it most certainly will result in an imp= recise throughput measurement... > > Experience with bufferbloat measurements seems to indicate that >=3D 20-3= 0 seconds are required to really saturate a link. > > Trying to confirm this with flent/netperf: > > date ; ping -c 10 netperf-eu.bufferbloat.net ; ./run-flent --ipv4 -l 30 -= H netperf-eu.bufferbloat.net rrul_var --remote-metadata=3Droot@192.168.42.1= --test-parameter=3Dcpu_stats_hosts=3Droot@192.168.42.1 --step-size=3D.05 -= -socket-stats --test-parameter bidir_streams=3D8 --test-parameter markings= =3D0,0,0,0,0,0,0,0 --test-parameter ping_hosts=3D1.1.1.1 -D . -t IPv4_SQM_c= ake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_wor= k-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.6-pppoe-wan-eth2.7_2_bridged-BTHH5A= -OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_netperf-eu.bufferbloat.net --log-f= ile > > [...] > > Ping (ms) ICMP 1.1.1.1 (extra) : 12.20 12.10 2= 1.91 ms 914 > Ping (ms) avg : 26.11 N/A = N/A ms 914 > Ping (ms)::ICMP : 25.30 25.10 = 35.11 ms 914 > Ping (ms)::UDP 0 (0) : 25.42 25.43 = 33.43 ms 914 > Ping (ms)::UDP 1 (0) : 25.36 25.32 = 33.77 ms 914 > Ping (ms)::UDP 2 (0) : 26.99 26.88 = 36.13 ms 914 > Ping (ms)::UDP 3 (0) : 25.27 25.20 = 34.42 ms 914 > Ping (ms)::UDP 4 (0) : 25.26 25.22 = 33.32 ms 914 > Ping (ms)::UDP 5 (0) : 27.99 27.85 = 37.20 ms 914 > Ping (ms)::UDP 6 (0) : 27.95 27.89 = 36.29 ms 914 > Ping (ms)::UDP 7 (0) : 25.42 25.32 = 34.56 ms 914 > TCP download avg : 11.93 N/A = N/A Mbits/s 914 > TCP download sum : 95.44 N/A = N/A Mbits/s 914 > TCP download::0 (0) : 11.85 12.26 = 13.97 Mbits/s 914 > TCP download::1 (0) : 11.81 12.28 = 13.81 Mbits/s 914 > TCP download::2 (0) : 12.13 12.31 = 16.28 Mbits/s 914 > TCP download::3 (0) : 12.04 12.31 = 13.76 Mbits/s 914 > TCP download::4 (0) : 11.78 12.28 = 13.83 Mbits/s 914 > TCP download::5 (0) : 12.07 12.30 = 14.44 Mbits/s 914 > TCP download::6 (0) : 11.78 12.29 = 13.67 Mbits/s 914 > TCP download::7 (0) : 11.98 12.30 = 13.92 Mbits/s 914 > TCP totals : 126.34 N/A = N/A Mbits/s 914 > TCP upload avg : 3.86 N/A = N/A Mbits/s 914 > TCP upload sum : 30.90 N/A = N/A Mbits/s 914 > [...] > > This is a bidirectional test that only reports actual TCP goodput, neithe= r the latency probes nor the reverse ACK traffic is accounted and yet: > TCP download sum: 95.44 > TCP upload sum: 30.90 > > which compared to the actual shape settings of 105/36 (with 34 bytes over= head) seems sane: > theoretical maximal throughput: > IPv4 > 105 * ((1500-8-20-20-12)/(1500+26)) =3D 99.08 Mbps > 36 * ((1500-8-20-20-12)/(1500+26)) =3D 33.97 Mbps > IPv6 > 105 * ((1500-8-40-20-12)/(1500+26)) =3D 97.71 Mbps > 36 * ((1500-8-40-20-12)/(1500+26)) =3D 33.50 Mbps > > Compared to these numbers the reported 87.499/26.531 do not seems either = saturating or precise, probably neither.... I do wonder how reliable the RP= M number is gong to be if the "saturate the link" part apparently failed? > > Question: is full saturation actually intended here, or is "sufficiently = high load" the goal (if the later what constitutes sufficiently high)? According to the spec, you may not get completely full saturation. The definition of saturation per the spec is fairly specific: See 4.1.4 of https://github.com/network-quality/draft-ietf-ippm-responsiveness/blob/mast= er/draft-ietf-ippm-responsiveness.txt. We are still working through some of the timing issues in the goresponsiveness client and doing our best. Thank you for your patience and helpful, constructive and kind feedback. Will > > Regards > Sebastian > > P.S.: When disabling my traffic shaper I get: > > user@ubuntu20.4LTS:~/CODE/flent$ time ../goresponsiveness/networkQuality = --config mensura.cdn-apple.com --port 443 --path /api/v1/gm/config --timeou= t 60 > 03-27-2022 18:20:06 UTC Go Responsiveness to mensura.cdn-apple.com:443... > Download: 99.390 Mbps ( 12.424 MBps), using 12 parallel connections. > Upload: 27.156 Mbps ( 3.395 MBps), using 20 parallel connections. > Total RTTs measured: 5 > RPM: 512 > > real 0m9.728s > user 0m1.763s > sys 0m1.660s > user@ubuntu20.4LTS:~/CODE/flent$ time ../goresponsiveness/networkQuality = --config mensura.cdn-apple.com --port 443 --path /api/v1/gm/config --timeou= t 60 > 03-27-2022 18:20:27 UTC Go Responsiveness to mensura.cdn-apple.com:443... > Download: 97.608 Mbps ( 12.201 MBps), using 12 parallel connections. > Upload: 27.375 Mbps ( 3.422 MBps), using 24 parallel connections. > Total RTTs measured: 5 > RPM: 366 > > real 0m12.926s > user 0m2.330s > sys 0m2.442s > > Which for the download approaches/exceeds the theoretical limit* yet for = the upload still falls a bit short of the 8 flow flent result > > > *) This is nothing particular unique to goresponsiveness most multi-strea= m tests fail to actually report robust and reliable values for thoughput, e= ven though this "ain't rocket science". Instead of trying to discount the T= CP start-up phase somehow, simply run the test long enough so the start up = does not dominate the result and just divide total transported volume by to= tal duration... yes that is going to slightly underestimate the total link = capacity, but that seems IMHO considerably more benign/useful than reportin= g 99.39 of 99.08 possible or rather 97.71 possible , since the actual test = uses IPv6 on my link... > > > > > On Mar 27, 2022, at 16:25, Dave Taht via Rpm wrote: > > > > Sasha has re-organized the meeting notes we've been taking which will > > be permanently up here: > > > > https://docs.google.com/document/d/19K14vrBLNvX_KOdRMK_s0SCYTnj70IJ-dQu= bL37kQgw/edit > > > > The public meetings continue at 10AM PDT tuesdays. > > > > One thing came up in that meeting that bears further reflection: The > > structure of the the responsiveness test is e2e, and differentiating > > between the wifi hop and/or sourcing tests from the router - is hard. > > In the latter case, we are asking a router not only to saturate its > > network connection, but to do it using crypto, which is a mighty task > > to ask of a router that's job is primarily routing (with offloads). > > > > A thought might be to attempt p2p tests between say, an ios phone and > > an osx laptop, which would exercise the router as a wifi router. A > > docker container for the server would be straightforward. (It's still > > hard for me to > > trust containers or vm's to run fast or accurately enough, too). > > > > Former case is still e2e but with the icmp unreachable idea I described= in wtbb. > > > > -- > > I tried to build a better future, a few times: > > https://wayforward.archive.org/?site=3Dhttps%3A%2F%2Fwww.icei.org > > > > Dave T=C3=A4ht CEO, TekLibre, LLC > > _______________________________________________ > > Rpm mailing list > > Rpm@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/rpm > > _______________________________________________ > Rpm mailing list > Rpm@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/rpm