From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from sender4-op-o12.zoho.com (sender4-op-o12.zoho.com [136.143.188.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id AF8EE3B2A4 for ; Thu, 3 Nov 2022 18:10:11 -0400 (EDT) ARC-Seal: i=1; a=rsa-sha256; t=1667513402; cv=none; d=zohomail.com; s=zohoarc; b=NKIWgg8RQssmds1Sppw7V/P2O4KKrBjhGdcYRewV83jTJe2oYBrQYDk93lKJgEu+7BkQzbcGmU/cpcESX7LGVJ4IEvl8aRqrXQ5+oXXj5NeJBHoBL/1n4RP7dQVHbulVMjqlYvyF/aZ5m2WCy1T3QHKiM1BArAHfDhpdt5UNanc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1667513402; h=Content-Type:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=QitTC7DOz1duY12L6vlfRJlMXlW4cXEmSZS/YJ50ncI=; b=OmS4AAVJVyirEV834GRUZKoeUWff5lLB96UdC6DDSaH7VXLJ75/EL4pCKPJHobX9wCc7wkFDReL+RhVPS2U1Sr/kFaSEZY4xosDi20ZErtnpcCML4XnV2BZsgC4fYM4BS1g/vuCmfp6a/TVgVvONuqns8QSEiCjvfIzUGiSAOWA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=jonathanfoulkes.com; spf=pass smtp.mailfrom=jf@jonathanfoulkes.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1667513402; s=zoho; d=jonathanfoulkes.com; i=jf@jonathanfoulkes.com; h=From:From:Message-Id:Message-Id:Content-Type:Mime-Version:Subject:Subject:Date:Date:In-Reply-To:Cc:Cc:To:To:References:Reply-To; bh=QitTC7DOz1duY12L6vlfRJlMXlW4cXEmSZS/YJ50ncI=; b=CClv8z7i0dzrcIrIMczfyZn8qIs271e+MLQETGW/Qvo4DUO5TEu5U5lwL3N4YceS Q4v5xYu1TmfZZRw3FJ8vGHfqhr8wQ6AE4vNxQngMn5lAc3ssGER3PFnv1HewutOoTee 6zj2T3wi+pTrpjQ2mLs1a6vX0oIiqiF9wpVbBO4c= Received: from smtpclient.apple (h38.249.20.98.dynamic.ip.windstream.net [98.20.249.38]) by mx.zohomail.com with SMTPS id 1667513395013902.9548689374309; Thu, 3 Nov 2022 15:09:55 -0700 (PDT) From: "jf@jonathanfoulkes.com" Message-Id: <49CC4E18-2165-4027-BA1D-B3C998BBF54D@jonathanfoulkes.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_899FB761-506A-4738-B244-6CF9FD9DB09E" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.200.110.1.12\)) Date: Thu, 3 Nov 2022 18:09:43 -0400 In-Reply-To: Cc: Rpm To: Christoph Paasch References: X-Mailer: Apple Mail (2.3731.200.110.1.12) X-ZohoMailClient: External Subject: Re: [Rpm] Changes to RPM calculation for MacOS Ventura? X-BeenThere: rpm@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: revolutions per minute - a new metric for measuring responsiveness List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Nov 2022 22:10:12 -0000 --Apple-Mail=_899FB761-506A-4738-B244-6CF9FD9DB09E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi Christoph, Thanks for the reply, it clarifies why the metric would be different but = it then leads to questions regarding how / where bufferbloat is = occurring on the links creating the load. I noted the tc stats show a = peak delay of 81ms on download and 139ms on upload, so there is indeed = some queue build-up in the router. So even testing cutting QoS to 50% of the upstream capacity and still = getting no better than medium RPM rates. Here is the test run on that unit set for roughly half ( 80 / 10 ) of = the upstream line ( 180 / 24 ), this one from a 12.6 OS =3D=3D=3D=3D SUMMARY =3D=3D=3D=3D = =20 Upload capacity: 8.679 Mbps Download capacity: 75.213 Mbps Upload flows: 20 Download flows: 12 Upload Responsiveness: High (2659 RPM) Download Responsiveness: High (2587 RPM) Base RTT: 14 Start: 11/3/22, 4:05:01 PM End: 11/3/22, 4:05:26 PM OS Version: Version 12.6 (Build 21G115) And this one from Ventura (13) =3D=3D=3D=3D SUMMARY =3D=3D=3D=3D Uplink capacity: 9.328 Mbps (Accuracy: High) Downlink capacity: 76.555 Mbps (Accuracy: High) Uplink Responsiveness: Low (143 RPM) (Accuracy: High) Downlink Responsiveness: Medium (380 RPM) (Accuracy: High) Idle Latency: 29.000 milli-seconds (Accuracy: High) Interface: en6 Uplink bytes transferred: 16.734 MB Downlink bytes transferred: 85.637 MB Uplink Flow count: 20 Downlink Flow count: 12 Start: 11/3/22, 4:03:33 PM End: 11/3/22, 4:03:58 PM OS Version: Version 13.0 (Build 22A380) Does all-out use of ECN cause a penalty? On download, we recorded 9504 marks, but only 38 drops. So the flows = should have been well managed with all the early feedback to senders.=20 Do the metrics look for drops and thus this low drop rate seems like = there is bloat given the amount of traffic in flight? The device under test is an MT7621 running stock OpenWRT 22.03.2 with = SQM installed and using layer-cake. But we see similar metrics on an = i5-4200 x86 box with Intel NICs. So it=E2=80=99s not horsepower related. I just retested on the x86, with the same ballpark results. I=E2=80=99ll re-test tomorrow with all the ECN features on the Mac and = the router disabled to see what that does to the metrics. Thanks, Jonathan > On Nov 1, 2022, at 5:52 PM, Christoph Paasch = wrote: >=20 > Hello Jonathan, >=20 >> On Oct 28, 2022, at 2:45 PM, jf--- via Rpm = wrote: >>=20 >> Hopefully, Christoph can provide some details on the changes from the = prior networkQuality test, as we=E2=80=99re seeing some pretty large = changes in results for the latest RPM tests. >>=20 >> Where before we=E2=80=99d see results in the >1,500 RPM (and multiple = >2,000 RPM results) for a DOCSIS 3.1 line with QoS enabled (180 down/35 = up), it now returns peak download RPM of ~600 and ~800 for upload. >>=20 >> latest results: >>=20 >> =3D=3D=3D=3D SUMMARY =3D=3D=3D=3D >> Uplink capacity: 25.480 Mbps (Accuracy: High) >> Downlink capacity: 137.768 Mbps (Accuracy: High) >> Uplink Responsiveness: Medium (385 RPM) (Accuracy: High) >> Downlink Responsiveness: Medium (376 RPM) (Accuracy: High) >> Idle Latency: 43.875 milli-seconds (Accuracy: High) >> Interface: en8 >> Uplink bytes transferred: 35.015 MB >> Downlink bytes transferred: 154.649 MB >> Uplink Flow count: 16 >> Downlink Flow count: 12 >> Start: 10/28/22, 5:12:30 PM >> End: 10/28/22, 5:12:54 PM >> OS Version: Version 13.0 (Build 22A380) >>=20 >> Latencies (as monitored via PingPlotter) stay absolutely steady = during these tests, >>=20 >> So unless my ISP coincidentally started having major service issues, = I=E2=80=99m scratching my head as to why. >>=20 >> For contrast, the Ookla result is as follows: = https://www.speedtest.net/result/13865976456 with 15ms down, 18ms up = loaded latencies. >=20 > In Ventura, we started adding the latency on the load-generating = connections to the final RPM-calulcation as well. The formula being used = is now exactly what is in the v01 IETF draft. >=20 > Very likely the bottleneck in your network does FQ, and so latency on = separate connections is very low, while your load-generating connections = are still bufferbloated. >=20 >=20 > Ookla measures latency only on separate connections, thus will also be = heavily impacted by FQ. >=20 >=20 > Does that clarify it? >=20 >=20 > Cheers, > Christoph=20 >=20 >=20 >>=20 >> Further machine details: MacBook Pro 16=E2=80=9D (2019) using a USB-C = to Ethernet adapter. >> I run with full ECN enabled: >> sudo sysctl -w net.inet.tcp.disable_tcp_heuristics=3D1=20 >>=20 >> sudo sysctl -w net.inet.tcp.ecn_initiate_out=3D1 >>=20 >> sudo sysctl -w net.inet.tcp.ecn_negotiate_in=3D1 >>=20 >> and also with instant ack replies: >>=20 >> sysctl net.inet.tcp.delayed_ack >> net.inet.tcp.delayed_ack: 0 >>=20 >> I did try with delayed_ack=3D1, and the results were about the same. >>=20 >> Thanks in advance, >>=20 >> Jonathan Foulkes >>=20 >> _______________________________________________ >> Rpm mailing list >> Rpm@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/rpm >=20 --Apple-Mail=_899FB761-506A-4738-B244-6CF9FD9DB09E Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi = Christoph,

Thanks for the reply, it clarifies why the = metric would be different but it then leads to questions regarding how / = where bufferbloat is occurring on the links creating the load. I noted = the tc stats show a peak delay of 81ms on download and 139ms on upload, = so there is indeed some queue build-up in the = router.

So even testing cutting QoS to 50% of = the upstream capacity and still getting no better than medium RPM = rates.

Here is the test run on that unit set = for roughly half ( 80 / 10 ) of the upstream line ( 180 / 24 ), this one = from a 12.6 OS

=3D=3D=3D=3D SUMMARY =3D=3D=3D= =3D                   =                     =                     =                     =           
Upload capacity: = 8.679 Mbps
Download capacity: 75.213 Mbps
Upload = flows: 20
Download flows: 12
Upload Responsiveness: = High (2659 RPM)
Download Responsiveness: High (2587 = RPM)
Base RTT: 14
Start: 11/3/22, 4:05:01 = PM
End: 11/3/22, 4:05:26 PM
OS Version: Version 12.6 = (Build 21G115)

And this one from Ventura = (13)

=3D=3D=3D=3D SUMMARY = =3D=3D=3D=3D
Uplink capacity: 9.328 Mbps (Accuracy: = High)
Downlink capacity: 76.555 Mbps (Accuracy: = High)
Uplink Responsiveness: Low (143 RPM) (Accuracy: = High)
Downlink Responsiveness: Medium (380 RPM) (Accuracy: = High)
Idle Latency: 29.000 milli-seconds (Accuracy: = High)
Interface: en6
Uplink bytes transferred: = 16.734 MB
Downlink bytes transferred: 85.637 = MB
Uplink Flow count: 20
Downlink Flow count: = 12
Start: 11/3/22, 4:03:33 PM
End: 11/3/22, 4:03:58 = PM
OS Version: Version 13.0 (Build = 22A380)

Does all-out use of ECN cause a = penalty?

On download, we recorded 9504 marks, = but only 38 drops. So the flows should have been well managed with all = the early feedback to senders. 

Do the = metrics look for drops and thus this low drop rate seems like there is = bloat given the amount of traffic in = flight?

The device under test is an MT7621 = running stock OpenWRT 22.03.2 with SQM installed and using layer-cake. = But we see similar metrics on an i5-4200 x86 box with Intel NICs. So = it=E2=80=99s not horsepower related.
I just retested on the = x86, with the same ballpark results.

I=E2=80=99ll= re-test tomorrow with all the ECN features on the Mac and the router = disabled to see what that does to the = metrics.

Thanks,

Jonatha= n



On Nov 1, 2022, at 5:52 PM, Christoph Paasch = <cpaasch@apple.com> wrote:

Hello Jonathan,

On Oct 28, 2022, at 2:45 PM, jf--- via Rpm = <rpm@lists.bufferbloat.net> wrote:

Hopefully, Christoph can provide some details on the = changes from the prior networkQuality test, as we=E2=80=99re seeing some = pretty large changes in results for the latest RPM = tests.

Where before we=E2=80=99d see results in the = >1,500 RPM (and multiple >2,000 RPM results) for a DOCSIS 3.1 line = with QoS enabled (180 down/35 up), it now returns peak download RPM of = ~600 and ~800 for upload.

latest = results:

=3D=3D=3D=3D SUMMARY = =3D=3D=3D=3D
Uplink capacity: 25.480 Mbps (Accuracy: = High)
Downlink capacity: 137.768 Mbps (Accuracy: = High)
Uplink Responsiveness: Medium (385 RPM) (Accuracy: = High)
Downlink Responsiveness: Medium (376 RPM) (Accuracy: = High)
Idle Latency: 43.875 milli-seconds (Accuracy: = High)
Interface: en8
Uplink bytes transferred: = 35.015 MB
Downlink bytes transferred: 154.649 = MB
Uplink Flow count: 16
Downlink Flow count: = 12
Start: 10/28/22, 5:12:30 PM
End: 10/28/22, = 5:12:54 PM
OS Version: Version 13.0 (Build = 22A380)

Latencies (as monitored via = PingPlotter) stay absolutely steady during these = tests,

So unless my ISP coincidentally started = having major service issues, I=E2=80=99m scratching my head as to = why.

For contrast, the Ookla result is as = follows: https://www.speedtes= t.net/result/13865976456  with 15ms down, 18ms up loaded = latencies.

In Ventura, = we started adding the latency on the load-generating connections to the = final RPM-calulcation as well. The formula being used is now exactly = what is in the v01 IETF draft.

Very likely the = bottleneck in your network does FQ, and so latency on separate = connections is very low, while your load-generating connections are = still bufferbloated.


Ookla = measures latency only on separate connections, thus will also be heavily = impacted by FQ.


Does that = clarify = it?


Cheers,
Christoph&nb= sp;



Further machine = details: MacBook Pro 16=E2=80=9D (2019) using a USB-C to Ethernet = adapter.
I run with full ECN enabled:

sudo sysctl -w = net.inet.tcp.disable_tcp_heuristics=3D1 

sudo sysctl -w net.inet.tcp.ecn_initiate_out=3D1

sudo sysctl -w = net.inet.tcp.ecn_negotiate_in=3D1

a= nd also with instant ack replies:

sysctl = net.inet.tcp.delayed_ack
net.inet.tcp.delayed_ack: = 0

I did try with delayed_ack=3D1, and the = results were about the same.

Thanks in = advance,

Jonathan = Foulkes

________________________________________= _______
Rpm mailing = list
Rpm@lists.bufferbloat.net
https://lists.bufferbloat.net/listinf= o/rpm


<= /div>= --Apple-Mail=_899FB761-506A-4738-B244-6CF9FD9DB09E--