From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 6E9C43B29E for ; Fri, 4 Nov 2022 15:10:15 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1667589005; bh=ojU6mn+fUL0HUFoFIIWdCnHhJetrLwtKouPLvIS1A8o=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=o6obM+dPa6YUKQ9YSM77u2mdAyyvIXZw9Dx8BMmZaJFlimIFsG50BfhXhmLYggWbW S/0BdRGD/pG0L5zDOM1ot2UqedZtLnr3iDXINt+hkhAhatyyLe7nfmVFq0SGrBt3Qm MXPXODZCO3Gn6aIrs8jm+7wj9xwf3dxboiKRJjXJHlYqCQ+m3MiZ7673+VLvbhMxgC gelCyskHLa145ip6LHip2qsZ61buUnwDItCiPn5R3AYH3amd/aJBlR5620qXwW2ZEe Fc3IEUre/9EAkli1Roo6/chp380c/krMgXzLaJbuNo89Bh/M5oi6r33JaclzoJ/hPW P9SDrsSpYQc8Q== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from smtpclient.apple ([77.6.118.159]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MFKGP-1ooY320xsv-00FkZe; Fri, 04 Nov 2022 20:10:05 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) From: Sebastian Moeller In-Reply-To: Date: Fri, 4 Nov 2022 20:10:03 +0100 Cc: =?utf-8?Q?Dave_T=C3=A4ht?= , rjmcmahon , Rpm , "ippm@ietf.org" , Will Hawkins Content-Transfer-Encoding: quoted-printable Message-Id: References: To: "MORTON JR., AL" X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Provags-ID: V03:K1:Lzb1nTMgNWwV7qhVRou+wvRHEQvQx2DcsJfJ17O/lvVp0sOmexD ZlLSVcUGJuU0nxfjIezkPFr7YB6BeWgyNJtRcwiC/4Eerrc5IWLkWa1C0e7jhdop4V5FqyT lZWoMyFWR8dJfFztFoewBRYpVp4JtVpJuv6dZ2aR1tIArloF5qKkHIgcNWtDzejEPpnvDCw BZzLdmlVKEHKtkcLue6Iw== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:5toHsOaKXYE=;5nuVZHjPSweOhxGDyBnmv3krpHM f6uS+voysxIv5FEVLpuXmVgPBMNIwAzIgX+4lPGEOE8+VAf0GPVZodsfAx11d8KYxmkdnzed8 1DmFcpKMiiyZiUi7xj8Af2QgmWX67b/ORxTgYtdp9h4lTSKoJQaK+F3JyVZfXXGzhCFlSRuV5 lYUuFIPT4iuga+wdV0iASlOFQqF26g0eKuWmM/9My0UF8zbr2hN55gjr/+rif2QQZ+GmgtbNw 5P1OD8+keQpA01tS1CBknM8tnN18x6mlcJMhVKsK8ZvGgmv6SaOwTUtb44TKjzFW6xiG6G4Zu 25ROFDGcU00a5H2oNE9jHkfU4OOFOWllkmw2HZ+8rRjelx7ws7HA0H7ABp88yML6HJuJNPmTq a9AGxmozZCnyn6mKoKbt0pks+/zYtmKlvg2ADgXK2hKQtZd6hWcxGeMPS4rE21zD81KiKMKRq pDPVuoqwztYKVz8/hNxxQqvZrp8wndo4P7r10rnvpj4zax4l1MWRpYTOclqz5cgWa1fQs9yBn qjbE6+DEOUk2WszP84a+VPFOtaMWhzLE2dpwuHGW3074j+few4HOzJNbMAb3cwuB3txfLxs94 ZU7vD3kipjm40ntnw8Z7A16AFGSUIz49zvNaTSFcx2r6Ls3/DqDLngnRD6eu1YuHoKPEO23th EhobUVUKMBDmd8pLwY2rV6SsB5YmMsw/Ze4WWD4kaOnsDepuQQYvNQfbkWL6g7iCp9V4YjUtA 2n5r7YGuVLubupnwjf5ZGzeCGDZ5rL9Ff4wZFeYaGaSYxtZRH2iCbHu2pboQtwTPbZZ5SrrkB Gle5NbKUZm9ae1PU/Nsz9lECiYy5k7DwPvwNA2iA1CaKu76626Uni/jpXBby8rnFuQA82t/q+ vzgW9xh6GR/d35jJJwyJIJGGGx7AOyVhjBia9U0Z6roZSCwOyL8/wLl8bsn5aNLKiX1+hZ9xL xY/9wiBZkCNDtDoOYmQMHx3mD2M= Subject: Re: [Rpm] [ippm] Preliminary measurement comparison of "Working Latency" metrics X-BeenThere: rpm@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: revolutions per minute - a new metric for measuring responsiveness List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Nov 2022 19:10:15 -0000 Hi Al, > On Nov 4, 2022, at 18:14, MORTON JR., AL via Rpm = wrote: >=20 > Hi all, >=20 > I have been working through the threads on misery metrics and = lightweight sensing of bw and buffering, and with those insights and = gold-nuggets in mind, I'm iterating through testing the combinations of = tools that Dave That and Bob McMahon suggested. >=20 > earlier this week, Dave wrote: >> How does networkQuality compare vs a vs your tool vs a vs = goresponsiveness? >=20 > goresponsiveness installed flawlessly - very nice instructions and = getting-started info. >=20 > Comparison of NetQual (networkQuality -vs), UDPST -A c, = gores/goresponsiveness >=20 > Working Latency & Capacity Summary on DOCSIS 3.1 access with 1 Gbps = down-stream service > (capacity in Mbps, delays in ms, h and m are RPM categories, High and = Medium) >=20 > NetQual UDPST (RFC 9097) gores > DnCap RPM DelLD DelMin DnCap RTTmin RTTrange DnCap = RPM DelLD > 882 788 m 76ms 8ms 967 28 0-16 127 = (1382) 43ms=20 > 892 1036 h 58ms 8 966 27 0-18 128 = (2124) 28ms > 887 1304 h 46ms 6 969 27 0-18 130 = (1478) 41ms > 885 1008 h 60ms 8 967 28 0-22 127 = (1490) 40ms > 894 1383 h 43ms 11 967 28 0-15 133 = (2731) 22ms >=20 > NetQual UDPST (RFC 9097) gores > UpCap RPM DelLD DelMin UpCap RTTmin RTTrange UpCap = RPM DelLD > 21 327 m 183ms 8ms 22 (51) 28 0-253 12 = =20 > 21 413 m 145ms 8 22 (43) 28 0-255 15 = =20 > 22 273 m 220ms 6 23 (53) 28 0-259 10 = =20 > 21 377 m 159ms 8 23 (51) 28 0-250 10 = =20 > 22 281 m 214ms 11 23 (52) 28 0-250 6 = =20 >=20 > These tests were conducted in a round-robin fashion to minimize the = possibility of network variations between measurements: > NetQual - rest - UDPST-Dn - rest- UDPST-Up - rest - gores - rest - = repeat=20 >=20 > NetQual indicates the same reduced capacity in Downstream when = compared to UDPST (940Mbps is the max for TCP payloads, while 967-970 is = max for IP-layer capacity, dep. on VLAN tag). [SM] DOCSIS ISPs traditionally provision higher peak rates than = they advertise, with a DOCSIS modem/router with > 1 Gbps LAN capacity = (even via bonded LAG) people in Germany routinely measure TCP/IPv4/HTTP = goodput in the 1050 Mbps range. But typically gigabit ethernet limits = the practically achievable throughput somewhat: Ethernet payload rate: 1000 * ((1500)/(1500 + 38)) =3D = 975.29 Mbps Ethernet payload rate +VLAN: 1000 * ((1500)/(1500 + 38 + 4)) = =3D 972.76 Mbps IPv4 payload (ethernet+VLAN): 1000 * ((1500 - 20)/(1500 + 38 + = 4)) =3D 959.79 Mbps IPv6 payload (ethernet+VLAN): 1000 * ((1500 - 40)/(1500 + 38 + = 4)) =3D 946.82 Mbps IPv4/TCP payload (ethernet+VLAN): 1000 * ((1500 - 20 - 20)/(1500 + = 38 + 4)) =3D 946.82 Mbps IPv6/TCP payload (ethernet+VLAN): 1000 * ((1500 - 20 - 40)/(1500 + = 38 + 4)) =3D 933.85 Mbps IPv4/TCP/RFC1323 timestamps payload: 1000 * ((1500 - 12 - 20 - = 20)/(1500 + 38 + 4)) =3D 939.04 Mbps IPv6/TCP/RFC1323 timestamps payload: 1000 * ((1500 - 12 - 20 - = 40)/(1500 + 38 + 4)) =3D 926.07 Mbps Speedtests tend to report IPv4/TCP timestamps might be on depending on = the OS, but on-line speedtest almost never return the simple average = rate over the measurement duration, but play "clever" tricks to exclude = the start-up phase and to aggregate over multiple flows that invariably = ending with results that tend to exceed the hard limits shown above... > Upstream capacities are not very different (a factor that made TCP = methods more viable many years ago when most consumer access speeds were = limited to 10's of Megabits). [SM] My take on this is that this is partly due to the goal of = ramping up very quickly and get away with a short measurement duration. = That causes imprecision. As Dave said flent's RRUL test defaults to 60 = seconds and I often ran/run it for 5 to 10 minutes to get somewhat more = reliable numbers (and for timecourses to look at and reason about). > gores reports significantly lower capacity in both downstream and = upstream measurements, a factor of 7 less than NetQual for downstream. = Interestingly, the reduced capacity (taken as the working load) results = in higher responsiveness: RPM meas are higher and loaded delays are = lower for downstream. [SM] Yepp, if you only fill-up the queue partially you will = harvest less queueing delay and hence retain more responsiveness, albeit = this really just seems to be better interpreted as go-responsiveness = failing to achieve "working-conditions".=20 I tend to run gores like this: time ./networkQuality --config mensura.cdn-apple.com --port 443 --path = /api/v1/gm/config --sattimeout 60 --extended-stats --logger-filename = go_networkQuality_$(date +%Y%m%d_%H%M%S) --sattimeout 60 extends the time out for the saturation measurement = somewhat (before I saw it simply failing on a 1 Gbps access link, it did = give some diagnostic message though). >=20 > The comparison of networkQuality and goresponsiveness is somewhat = confounded by the need to use the Apple server infrastructure for both = these methods (the documentation provides this option - thanks!). I = don't have admin access to our server at the moment. But the measured = differences are large despite the confounding factor.=20 [SM] Puzzled, I thought when comparing the two ntworkQuality = variants having the same back-end sort of helps be reducing the = differences to the clients? But comparison to UDPST suffers somewhat. >=20 > goresponsiveness has its own, very different output format than = networkQuality. There isn't a comparable "-v" option other than -debug = (which is extremely detailed). gores only reports RPM for the = downstream. [SM] I agree it would be nice if gores would grow a sequential = mode as well. > I hope that these results will prompt more of the principle = evangelists and coders to weigh-in. [SM] No luck ;) but at least "amateur hour" (aka me) is ready to = discuss. >=20 > It's also worth noting that the RTTrange reported by UDPST is the = range above the minimum RTT, and represents the entire 10 second test. = The consistency of the maximum of the range (~255ms) seems to indicate = that UDPST has characterized the length of the upstream buffer during = measurements. [SM] thanks for explaining that. Regards Sebastian >=20 > I'm sure there is more to observe that is prompted by these = measurements; comments welcome! >=20 > Al >=20 >=20 >> -----Original Message----- >> From: ippm On Behalf Of MORTON JR., AL >> Sent: Tuesday, November 1, 2022 10:51 AM >> To: Dave Taht >> Cc: ippm@ietf.org; Rpm >> Subject: Re: [ippm] Preliminary measurement comparison of "Working = Latency" >> metrics >>=20 >> *** Security Advisory: This Message Originated Outside of AT&T ***. >> Reference http://cso.att.com/EmailSecurity/IDSP.html for more = information. >>=20 >> Hi Dave, >> Thanks for trying UDPST (RFC 9097)! >>=20 >> Something you might try with starlink: >> use the -X option and UDPST will generate random payloads. >>=20 >> The measurements with -X will reflect the uncompressed that are = possible. >> I tried this on a ship-board Internet access: uncompressed rate was = 100kbps. >>=20 >> A few more quick replies below, >> Al >>=20 >>> -----Original Message----- >>> From: Dave Taht >>> Sent: Tuesday, November 1, 2022 12:22 AM >>> To: MORTON JR., AL >>> Cc: ippm@ietf.org; Rpm >>> Subject: Re: [ippm] Preliminary measurement comparison of "Working = Latency" >>> metrics >>>=20 >>> Dear Al: >>>=20 >>> OK, I took your udpst tool for a spin. >>>=20 >>> NICE! 120k binary (I STILL work on machines with only 4MB of flash), >>> good feature set, VERY fast, >> [acm] >> Len Ciavattone (my partner in crime on several measurement projects) = is the >> lead coder: he has implemented many measurement tools extremely = efficiently, >> this one in C-lang. >>=20 >>> and in very brief testing, seemed >>> to be accurate in the starlink case, though it's hard to tell with >>> them as the rate changes every 15s. >> [acm] >> Great! Our guiding principle developing UDPST has been to test the = accuracy of >> measurements against a ground-truth. It pays-off. >>=20 >>>=20 >>> I filed a couple bug reports on trivial stuff: >>>=20 >> = https://urldefense.com/v3/__https://github.com/BroadbandForum/obudpst/issu= es/8 >>>=20 >> = __;!!BhdT!iufMVqCyoH_CQ2AXdNJV1QSjl_7srzb_IznWE87U6E583vleKCIKpL3PBLwci5nO= IEUc >>> SswH_opYmEw$ >> [acm] >> much appreciated... We have an OB-UDPST project meeting this Friday, = can >> discuss then. >>=20 >>>=20 >>> (Adding diffserv and ecn washing or marking detection would be a = nice >>> feature to have) >>>=20 >>> Aside from the sheer joy coming from the "it compiles! and runs!" >>> phase I haven't looked much further. >>>=20 >>> I left a copy running on one of my starlink testbeds - >>> fremont.starlink.taht.net - if anyone wants to try it. It's >>> instrumented with netperf, flent, irtt, iperf2 (not quite the latest >>> version from bob, but close), and now udpst, and good to about a = gbit. >>>=20 >>> nice tool! >> [acm] >> Thanks again! >>=20 >>>=20 >>> Has anyone here played with crusader? ( >>>=20 >> = https://urldefense.com/v3/__https://github.com/Zoxc/crusader__;!!BhdT!iufM= VqCy >>> = oH_CQ2AXdNJV1QSjl_7srzb_IznWE87U6E583vleKCIKpL3PBLwci5nOIEUcSswHm4Wtzjc$ = ) >>>=20 >>> On Mon, Oct 31, 2022 at 4:30 PM Dave Taht = wrote: >>>>=20 >>>> On Mon, Oct 31, 2022 at 1:41 PM MORTON JR., AL = wrote: >>>>=20 >>>>>> have you tried irtt? >>>=20 >> = (https://urldefense.com/v3/__https://github.com/heistp/irtt__;!!BhdT!iufMV= qCyo >>> = H_CQ2AXdNJV1QSjl_7srzb_IznWE87U6E583vleKCIKpL3PBLwci5nOIEUcSswHBSirIcE$ = ) >>>>> I have not. Seems like a reasonable tool for UDP testing. The = feature I >>> didn't like in my scan of the documentation is the use of = Inter-packet delay >>> variation (IPDV) instead of packet delay variation (PDV): variation = from the >>> minimum (or reference) delay. The morbidly curious can find my = analysis in >> RFC >>> 5481: >>>=20 >> = https://urldefense.com/v3/__https://datatracker.ietf.org/doc/html/rfc5481_= _;!! >>>=20 >> = BhdT!iufMVqCyoH_CQ2AXdNJV1QSjl_7srzb_IznWE87U6E583vleKCIKpL3PBLwci5nOIEUcS= swHt >>> T7QSlc$ >>>>=20 >>>> irtt was meant to simulate high speed voip and one day >>>> videoconferencing. Please inspect the json output >>>> for other metrics. Due to OS limits it is typically only accurate = to a >>>> 3ms interval. One thing it does admirably is begin to expose the >>>> sordid sump of L2 behaviors in 4g, 5g, wifi, and other wireless >>>> technologies, as well as request/grant systems like cable and gpon, >>>> especially when otherwise idle. >>>>=20 >>>> Here is a highres plot of starlink's behaviors from last year: >>>> = https://urldefense.com/v3/__https://forum.openwrt.org/t/cake-w-adaptive- >>> bandwidth- >>>=20 >> = historic/108848/3238__;!!BhdT!iufMVqCyoH_CQ2AXdNJV1QSjl_7srzb_IznWE87U6E58= 3vle >>> KCIKpL3PBLwci5nOIEUcSswH_5Sms3w$ >>>>=20 >>>> clearly showing them "optimizing for bandwidth" and changing next = sat >>>> hop, and about a 40ms interval of buffering between these switches. >>>> I'd published elsewhere, if anyone cares, a preliminary study of = what >>>> starlink's default behaviors did to cubic and BBR... >>>>=20 >>>>>=20 >>>>> irtt's use of IPDV means that the results won=E2=80=99t compare = with UDPST, and >>> possibly networkQuality. But I may give it a try anyway... >>>>=20 >>>> The more the merrier! Someday the "right" metrics will arrive. >>>>=20 >>>> As a side note, this paper focuses on RAN uplink latency >>>>=20 >>>=20 >> = https://urldefense.com/v3/__https://dl.ifip.org/db/conf/itc/itc2021/157074= 0615 >>>=20 >> = .pdf__;!!BhdT!iufMVqCyoH_CQ2AXdNJV1QSjl_7srzb_IznWE87U6E583vleKCIKpL3PBLwc= i5nO >>> IEUcSswHgvqerjg$ which I think >>>> is a major barrier to most forms of 5G actually achieving good >>>> performance in a FPS game, if it is true for more RANs. I'd like = more >>>> to be testing uplink latencies idle and with load, on all >>>> technologies. >>>>=20 >>>>>=20 >>>>> thanks again, Dave. >>>>> Al >>>>>=20 >>>>>> -----Original Message----- >>>>>> From: Dave Taht >>>>>> Sent: Monday, October 31, 2022 12:52 PM >>>>>> To: MORTON JR., AL >>>>>> Cc: ippm@ietf.org; Rpm >>>>>> Subject: Re: [ippm] Preliminary measurement comparison of = "Working >>> Latency" >>>>>> metrics >>>>>>=20 >>>>>> Thank you very much for the steer to RFC9097. I'd completely = missed >>> that. >>>>>>=20 >>>>>> On Mon, Oct 31, 2022 at 9:04 AM MORTON JR., AL >> wrote: >>>>>>>=20 >>>>>>> (astute readers may have guessed that I pressed "send" too soon = on >>> previous >>>>>> message...) >>>>>>>=20 >>>>>>> I also conducted upstream tests this time, here are the results: >>>>>>> (capacity in Mbps, delays in ms, h and m are RPM categories, = High >> and >>>>>> Medium) >>>>>>>=20 >>>>>>> Net Qual UDPST (RFC9097) >> Ookla >>>>>>> UpCap RPM DelLD DelMin UpCap RTTmin RTTrange >> UpCap >>>>>> Ping(no load) >>>>>>> 34 1821 h 33ms 11ms 23 (42) 28 0-252 = 22 >>> 8 >>>>>>> 22 281 m 214ms 8ms 27 (52) 25 5-248 = 22 >>> 8 >>>>>>> 22 290 m 207ms 8ms 27 (55) 28 0-253 = 22 >>> 9 >>>>>>> 21 330 m 182ms 11ms 23 (44) 28 0-255 = 22 >>> 7 >>>>>>> 22 334 m 180ms 9ms 33 (56) 25 0-255 = 22 >>> 9 >>>>>>>=20 >>>>>>> The Upstream capacity measurements reflect an interesting = feature >> that >>> we >>>>>> can reliably and repeatably measure with UDPST. The first ~3 = seconds >> of >>>>>> upstream data experience a "turbo mode" of ~50Mbps. UDPST = displays >> this >>>>>> behavior in its 1 second sub-interval measurements and has a = bimodal >>> reporting >>>>>> option that divides the complete measurement interval in two time >>> intervals to >>>>>> report an initial (turbo) max capacity and a steady-state max = capacity >>> for the >>>>>> later intervals. The UDPST capacity results present both = measurements: >>> steady- >>>>>> state first. >>>>>>=20 >>>>>> Certainly we can expect bi-model distributions from many ISPs, = as, for >>>>>> one thing, the "speedboost" concept remains popular, except that = it's >>>>>> misnamed, as it should be called speed-subtract or speed-lose. = Worse, >>>>>> it is often configured "sneakily", in that it doesn't kick in for = the >>>>>> typical observed duration of the test, for some, they cut the >>>>>> available bandwidth about 20s in, others, 1 or 5 minutes. >>>>>>=20 >>>>>> One of my biggest issues with the rpm spec so far is that it = should, >>>>>> at least, sometimes, run randomly longer than the overly short >>>>>> interval it runs for and the tools also allow for manual override = of >>> length. >>>>>>=20 >>>>>> we caught a lot of tomfoolery with flent's rrul test running by >> default >>> for >>>>>> 1m. >>>>>>=20 >>>>>> Also, AQMs on the path can take a while to find the optimal drop = or >> mark >>> rate. >>>>>>=20 >>>>>>>=20 >>>>>>> The capacity processing in networkQuality and Ookla appear to = report >>> the >>>>>> steady-state result. >>>>>>=20 >>>>>> Ookla used to basically report the last result. Also it's not a = good >>>>>> indicator of web traffic behavior at all, watching the curve >>>>>> go up much more slowly in their test on say, fiber 2ms, vs = starlink, >>>>>> (40ms).... >>>>>>=20 >>>>>> So adding another mode - how quickly is peak bandwidth actually >>>>>> reached, would be nice. >>>>>>=20 >>>>>> I haven't poked into the current iteration of the = goresponsiveness >>>>>> test at all: = https://urldefense.com/v3/__https://github.com/network- >>>>>>=20 >>>=20 >> = quality/goresponsiveness__;!!BhdT!giGhURYxqguQCyB3NT8rE0vADdzxcQ2eCzfS4NRM= sdvb >>>>>> K2bOqw0uMPbFeJ7PxzxTc48iQFubYTxxmyA$ it >>>>>> would be good to try collecting more statistics and histograms = and >>>>>> methods of analyzing the data in that libre-source version. >>>>>>=20 >>>>>> How does networkQuality compare vs a vs your tool vs a vs >>> goresponsiveness? >>>>>>=20 >>>>>>> I watched the upstream capacity measurements on the Ookla app, = and >>> could >>>>>> easily see the initial rise to 40-50Mbps, then the drop to = ~22Mbps for >>> most of >>>>>> the test which determined the final result. >>>>>>=20 >>>>>> I tend to get upset when I see ookla's new test flash a peak = result in >>>>>> the seconds and then settle on some lower number somehow. >>>>>> So far as I know they are only sampling the latency every 250ms. >>>>>>=20 >>>>>>>=20 >>>>>>> The working latency is about 200ms in networkQuality and about = 280ms >>> as >>>>>> measured by UDPST (RFC9097). Note that the networkQuality minimum >> delay >>> is >>>>>> ~20ms lower than the UDPST RTTmin, so this accounts for some of = the >>> difference >>>>>> in working latency. Also, we used the very dynamic Type C load >>>>>> adjustment/search algorithm in UDPST during all of this testing, = which >>> could >>>>>> explain the higher working latency to some degree. >>>>>>>=20 >>>>>>> So, it's worth noting that the measurements needed for assessing >>> working >>>>>> latency/responsiveness are available in the UDPST utility, and = that >> the >>> UDPST >>>>>> measurements are conducted on UDP transport (used by a growing >> fraction >>> of >>>>>> Internet traffic). >>>>>>=20 >>>>>> Thx, didn't know of this work til now! >>>>>>=20 >>>>>> have you tried irtt? >>>>>>=20 >>>>>>>=20 >>>>>>> comments welcome of course, >>>>>>> Al >>>>>>>=20 >>>>>>>> -----Original Message----- >>>>>>>> From: ippm On Behalf Of MORTON JR., AL >>>>>>>> Sent: Sunday, October 30, 2022 8:09 PM >>>>>>>> To: ippm@ietf.org >>>>>>>> Subject: Re: [ippm] Preliminary measurement comparison of = "Working >>>>>> Latency" >>>>>>>> metrics >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> Hi again RPM friends and IPPM'ers, >>>>>>>>=20 >>>>>>>> As promised, I repeated the tests shared last week, this time >> using >>> both >>>>>> the >>>>>>>> verbose (-v) and sequential (-s) dwn/up test options of >>> networkQuality. I >>>>>>>> followed Sebastian's calculations as well. >>>>>>>>=20 >>>>>>>> Working Latency & Capacity Summary >>>>>>>>=20 >>>>>>>> Net Qual UDPST >>> Ookla >>>>>>>> DnCap RPM DelLD DelMin DnCap RTTmin RTTrange >>> DnCap >>>>>>>> Ping(no load) >>>>>>>> 885 916 m 66ms 8ms 970 28 0-20 >> 940 >>> 8 >>>>>>>> 888 1355 h 44ms 8ms 966 28 0-23 >> 940 >>> 8 >>>>>>>> 891 1109 h 54ms 8ms 968 27 0-19 >> 940 >>> 9 >>>>>>>> 887 1141 h 53ms 11ms 966 27 0-18 >> 937 >>> 7 >>>>>>>> 884 1151 h 52ms 9ms 968 28 0-20 >> 937 >>> 9 >>>>>>>>=20 >>>>>>>> With the sequential test option, I noticed that networkQuality >>> achieved >>>>>> nearly >>>>>>>> the maximum capacity reported almost immediately at the start = of a >>> test. >>>>>>>> However, the reported capacities are low by about 60Mbps, >> especially >>> when >>>>>>>> compared to the Ookla TCP measurements. >>>>>>>>=20 >>>>>>>> The loaded delay (DelLD) is similar to the UDPST RTTmin + (the >> high >>> end of >>>>>> the >>>>>>>> RTTrange), for example 54ms compared to (27+19=3D46). Most of = the >>>>>> networkQuality >>>>>>>> RPM measurements were categorized as "High". There doesn't seem = to >>> be much >>>>>>>> buffering in the downstream direction. >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>> -----Original Message----- >>>>>>>>> From: ippm On Behalf Of MORTON JR., AL >>>>>>>>> Sent: Monday, October 24, 2022 6:36 PM >>>>>>>>> To: ippm@ietf.org >>>>>>>>> Subject: [ippm] Preliminary measurement comparison of "Working >>> Latency" >>>>>>>>> metrics >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> Hi RPM friends and IPPM'ers, >>>>>>>>>=20 >>>>>>>>> I was wondering what a comparison of some of the "working >> latency" >>>>>> metrics >>>>>>>>> would look like, so I ran some tests using a service on DOCSIS >>> 3.1, with >>>>>> the >>>>>>>>> downlink provisioned for 1Gbps. >>>>>>>>>=20 >>>>>>>>> I intended to run apple's networkQuality, UDPST (RFC9097), and >>> Ookla >>>>>>>> Speedtest >>>>>>>>> with as similar connectivity as possible (but we know that the >>> traffic >>>>>> will >>>>>>>>> diverge to different servers and we can't change that aspect). >>>>>>>>>=20 >>>>>>>>> Here's a quick summary of yesterday's results: >>>>>>>>>=20 >>>>>>>>> Working Latency & Capacity Summary >>>>>>>>>=20 >>>>>>>>> Net Qual UDPST Ookla >>>>>>>>> DnCap RPM DnCap RTTmin RTTVarRnge DnCap >>> Ping(no >>>>>> load) >>>>>>>>> 878 62 970 28 0-19 941 = 6 >>>>>>>>> 891 92 970 27 0-20 940 = 7 >>>>>>>>> 891 120 966 28 0-22 937 = 9 >>>>>>>>> 890 112 970 28 0-21 940 = 8 >>>>>>>>> 903 70 970 28 0-16 935 = 9 >>>>>>>>>=20 >>>>>>>>> Note: all RPM values were categorized as Low. >>>>>>>>>=20 >>>>>>>>> networkQuality downstream capacities are always on the low = side >>> compared >>>>>> to >>>>>>>>> others. We would expect about 940Mbps for TCP, and that's = mostly >>> what >>>>>> Ookla >>>>>>>>> achieved. I think that a longer test duration might be needed = to >>> achieve >>>>>> the >>>>>>>>> actual 1Gbps capacity with networkQuality; intermediate values >>> observed >>>>>> were >>>>>>>>> certainly headed in the right direction. (I recently upgraded = to >>>>>> Monterey >>>>>>>> 12.6 >>>>>>>>> on my MacBook, so should have the latest version.) >>>>>>>>>=20 >>>>>>>>> Also, as Sebastian Moeller's message to the list reminded me, = I >>> should >>>>>> have >>>>>>>>> run the tests with the -v option to help with comparisons. = I'll >>> repeat >>>>>> this >>>>>>>>> test when I can make time. >>>>>>>>>=20 >>>>>>>>> The UDPST measurements of RTTmin (minimum RTT observed during >> the >>> test) >>>>>> and >>>>>>>>> the range of variation above the minimum (RTTVarRnge) add-up = to >>> very >>>>>>>>> reasonable responsiveness IMO, so I'm not clear why RPM graded >>> this >>>>>> access >>>>>>>> and >>>>>>>>> path as "Low". The UDPST server I'm using is in NJ, and I'm in >>> Chicago >>>>>>>>> conducting tests, so the minimum 28ms is typical. UDPST >>> measurements >>>>>> were >>>>>>>> run >>>>>>>>> on an Ubuntu VM in my MacBook. >>>>>>>>>=20 >>>>>>>>> The big disappointment was that the Ookla desktop app I = updated >>> over the >>>>>>>>> weekend did not include the new responsiveness metric! I >> included >>> the >>>>>> ping >>>>>>>>> results anyway, and it was clearly using a server in the = nearby >>> area. >>>>>>>>>=20 >>>>>>>>> So, I have some more work to do, but I hope this is = interesting- >>> enough >>>>>> to >>>>>>>>> start some comparison discussions, and bring-out some >> suggestions. >>>>>>>>>=20 >>>>>>>>> happy testing all, >>>>>>>>> Al >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> _______________________________________________ >>>>>>>>> ippm mailing list >>>>>>>>> ippm@ietf.org >>>>>>>>>=20 >>>>>>>>=20 >>>>>>=20 >>>=20 >> = https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ippm__;!= !Bhd >>>>>>>>>=20 >>>>>>>>=20 >>>>>>=20 >>>=20 >> = T!hd5MvMQw5eiICQbsfoNaZBUS38yP4YIodBvz1kV5VsX_cGIugVnz5iIkNqi6fRfIQzWef_xK= qg4$ >>>>>>>>=20 >>>>>>>> _______________________________________________ >>>>>>>> ippm mailing list >>>>>>>> ippm@ietf.org >>>>>>>>=20 >>>>>>=20 >>>=20 >> = https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ippm__;!= !Bhd >>>>>>>> T!g- >>>>>>=20 >>> = FsktB_l9MMSGNUge6FXDkL1npaKtKcyDtWLcTZGpCunxNNCcTImH8YjC9eUT262Wd8q1EBpiw$= >>>>>>>=20 >>>>>>> _______________________________________________ >>>>>>> ippm mailing list >>>>>>> ippm@ietf.org >>>>>>>=20 >>>>>>=20 >>>=20 >> = https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ippm__;!= !Bhd >>>>>>=20 >>>=20 >> = T!giGhURYxqguQCyB3NT8rE0vADdzxcQ2eCzfS4NRMsdvbK2bOqw0uMPbFeJ7PxzxTc48iQFub= _gMs >>>>>> KXU$ >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> -- >>>>>> This song goes out to all the folk that thought Stadia would = work: >>>>>> = https://urldefense.com/v3/__https://www.linkedin.com/posts/dtaht_the- >>> mushroom- >>>>>> song-activity-6981366665607352320- >>>>>>=20 >>>=20 >> = FXtz__;!!BhdT!giGhURYxqguQCyB3NT8rE0vADdzxcQ2eCzfS4NRMsdvbK2bOqw0uMPbFeJ7P= xzxT >>>>>> c48iQFub34zz4iE$ >>>>>> Dave T=C3=A4ht CEO, TekLibre, LLC >>>>=20 >>>>=20 >>>>=20 >>>> -- >>>> This song goes out to all the folk that thought Stadia would work: >>>> = https://urldefense.com/v3/__https://www.linkedin.com/posts/dtaht_the- >>> mushroom-song-activity-6981366665607352320- >>>=20 >> = FXtz__;!!BhdT!iufMVqCyoH_CQ2AXdNJV1QSjl_7srzb_IznWE87U6E583vleKCIKpL3PBLwc= i5nO >>> IEUcSswHLHDpSWs$ >>>> Dave T=C3=A4ht CEO, TekLibre, LLC >>>=20 >>>=20 >>>=20 >>> -- >>> This song goes out to all the folk that thought Stadia would work: >>> = https://urldefense.com/v3/__https://www.linkedin.com/posts/dtaht_the- >> mushroom- >>> song-activity-6981366665607352320- >>>=20 >> = FXtz__;!!BhdT!iufMVqCyoH_CQ2AXdNJV1QSjl_7srzb_IznWE87U6E583vleKCIKpL3PBLwc= i5nO >>> IEUcSswHLHDpSWs$ >>> Dave T=C3=A4ht CEO, TekLibre, LLC >> _______________________________________________ >> ippm mailing list >> ippm@ietf.org >> = https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ippm__;!= !Bhd >> = T!jOVBx7DlKXbDMiZqaYSUhBtkSdUvfGpYUyGvLerdLsLBJZPMzEGcbhC9ZSzsZOd1dYC-rDt9= HLI$ > _______________________________________________ > Rpm mailing list > Rpm@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/rpm