[Rpm] Changes to RPM calculation for MacOS Ventura?

Thu Nov 10 15:41:32 EST 2022

> On Nov 4, 2022, at 1:50 PM, jf at jonathanfoulkes.com wrote:
> 
> As I mentioned, here are the results of running with ECN enabled vs disabled.
> 
> First, we retest with ECN enabled at the router, the SQM instance and fully enabled on the MacOS (see a prior post with the settings).
> 
> This is a DOCSIS 3.0 line at 300Mbps down and 25Mbps Up, router is an x86 i5-4200 running OpenWRT 22.03 with layer_cake QoS.
> 
> ==== SUMMARY ====
> Uplink capacity: 20.971 Mbps (Accuracy: High)
> Downlink capacity: 267.643 Mbps (Accuracy: High)
> Uplink Responsiveness: Medium (550 RPM) (Accuracy: High)
> Downlink Responsiveness: Medium (900 RPM) (Accuracy: High)
> Idle Latency: 21.417 milli-seconds (Accuracy: High)
> Interface: en6
> Uplink bytes transferred: 24.359 MB
> Downlink bytes transferred: 298.730 MB
> Uplink Flow count: 12
> Downlink Flow count: 12
> Start: 11/4/22, 3:41:30 PM
> End: 11/4/22, 3:41:49 PM
> OS Version: Version 13.0 (Build 22A380)
> 
> Then I disabled ECN support on the router, the sqm instance, and the MacOS, then re-ran a couple of times, and this is a representative result:
> 
> ==== SUMMARY ====
> Uplink capacity: 20.961 Mbps (Accuracy: High)
> Downlink capacity: 255.146 Mbps (Accuracy: High)
> Uplink Responsiveness: Medium (475 RPM) (Accuracy: High)
> Downlink Responsiveness: Medium (591 RPM) (Accuracy: High)
> Idle Latency: 23.583 milli-seconds (Accuracy: High)
> Interface: en6
> Uplink bytes transferred: 24.593 MB
> Downlink bytes transferred: 278.096 MB
> Uplink Flow count: 12
> Downlink Flow count: 12
> Start: 11/4/22, 4:11:59 PM
> End: 11/4/22, 4:12:18 PM
> OS Version: Version 13.0 (Build 22A380)
> 
> The absence of ECN does have an impact, most significantly on download, where it drops from 900 to 591 RPM

That’s interesting and good to know that the tool can measure the positive impact of ECN!

> My takeaway is that ECN does help with responsiveness overall.
> 
> What I’m still puzzled by here is the bloat and why changes to QoS settings do not make a difference.

 That is interesting indeed. What configurations are you testing? At this level, pcaps may be helpful.

Christoph

> 
> I have questions about how the lag within a bulk flow is a factor in the calculation, but I’ll first go read the latest version of the document.
> In case I missed it, is there a discussion thread someone can point me to where this change is mentioned?

The draft itself has had latency on bulk-flows since always. We just hadn’t backed this into the implementation, due to a bad server-side responsiveness which we have addressed now.

> 
> Thanks,
> 
> Jonathan
> 
> 
>> On Nov 3, 2022, at 6:09 PM, jf--- via Rpm <rpm at lists.bufferbloat.net> wrote:
>> 
>> Hi Christoph,
>> 
>> Thanks for the reply, it clarifies why the metric would be different but it then leads to questions regarding how / where bufferbloat is occurring on the links creating the load. I noted the tc stats show a peak delay of 81ms on download and 139ms on upload, so there is indeed some queue build-up in the router.
>> 
>> So even testing cutting QoS to 50% of the upstream capacity and still getting no better than medium RPM rates.
>> 
>> Here is the test run on that unit set for roughly half ( 80 / 10 ) of the upstream line ( 180 / 24 ), this one from a 12.6 OS
>> 
>> ==== SUMMARY ====                                                                                         
>> Upload capacity: 8.679 Mbps
>> Download capacity: 75.213 Mbps
>> Upload flows: 20
>> Download flows: 12
>> Upload Responsiveness: High (2659 RPM)
>> Download Responsiveness: High (2587 RPM)
>> Base RTT: 14
>> Start: 11/3/22, 4:05:01 PM
>> End: 11/3/22, 4:05:26 PM
>> OS Version: Version 12.6 (Build 21G115)
>> 
>> And this one from Ventura (13)
>> 
>> ==== SUMMARY ====
>> Uplink capacity: 9.328 Mbps (Accuracy: High)
>> Downlink capacity: 76.555 Mbps (Accuracy: High)
>> Uplink Responsiveness: Low (143 RPM) (Accuracy: High)
>> Downlink Responsiveness: Medium (380 RPM) (Accuracy: High)
>> Idle Latency: 29.000 milli-seconds (Accuracy: High)
>> Interface: en6
>> Uplink bytes transferred: 16.734 MB
>> Downlink bytes transferred: 85.637 MB
>> Uplink Flow count: 20
>> Downlink Flow count: 12
>> Start: 11/3/22, 4:03:33 PM
>> End: 11/3/22, 4:03:58 PM
>> OS Version: Version 13.0 (Build 22A380)
>> 
>> Does all-out use of ECN cause a penalty?
>> 
>> On download, we recorded 9504 marks, but only 38 drops. So the flows should have been well managed with all the early feedback to senders. 
>> 
>> Do the metrics look for drops and thus this low drop rate seems like there is bloat given the amount of traffic in flight?
>> 
>> The device under test is an MT7621 running stock OpenWRT 22.03.2 with SQM installed and using layer-cake. But we see similar metrics on an i5-4200 x86 box with Intel NICs. So it’s not horsepower related.
>> I just retested on the x86, with the same ballpark results.
>> 
>> I’ll re-test tomorrow with all the ECN features on the Mac and the router disabled to see what that does to the metrics.
>> 
>> Thanks,
>> 
>> Jonathan
>> 
>> 
>> 
>>> On Nov 1, 2022, at 5:52 PM, Christoph Paasch <cpaasch at apple.com> wrote:
>>> 
>>> Hello Jonathan,
>>> 
>>>> On Oct 28, 2022, at 2:45 PM, jf--- via Rpm <rpm at lists.bufferbloat.net> wrote:
>>>> 
>>>> Hopefully, Christoph can provide some details on the changes from the prior networkQuality test, as we’re seeing some pretty large changes in results for the latest RPM tests.
>>>> 
>>>> Where before we’d see results in the >1,500 RPM (and multiple >2,000 RPM results) for a DOCSIS 3.1 line with QoS enabled (180 down/35 up), it now returns peak download RPM of ~600 and ~800 for upload.
>>>> 
>>>> latest results:
>>>> 
>>>> ==== SUMMARY ====
>>>> Uplink capacity: 25.480 Mbps (Accuracy: High)
>>>> Downlink capacity: 137.768 Mbps (Accuracy: High)
>>>> Uplink Responsiveness: Medium (385 RPM) (Accuracy: High)
>>>> Downlink Responsiveness: Medium (376 RPM) (Accuracy: High)
>>>> Idle Latency: 43.875 milli-seconds (Accuracy: High)
>>>> Interface: en8
>>>> Uplink bytes transferred: 35.015 MB
>>>> Downlink bytes transferred: 154.649 MB
>>>> Uplink Flow count: 16
>>>> Downlink Flow count: 12
>>>> Start: 10/28/22, 5:12:30 PM
>>>> End: 10/28/22, 5:12:54 PM
>>>> OS Version: Version 13.0 (Build 22A380)
>>>> 
>>>> Latencies (as monitored via PingPlotter) stay absolutely steady during these tests,
>>>> 
>>>> So unless my ISP coincidentally started having major service issues, I’m scratching my head as to why.
>>>> 
>>>> For contrast, the Ookla result is as follows: https://www.speedtest.net/result/13865976456  with 15ms down, 18ms up loaded latencies.
>>> 
>>> In Ventura, we started adding the latency on the load-generating connections to the final RPM-calulcation as well. The formula being used is now exactly what is in the v01 IETF draft.
>>> 
>>> Very likely the bottleneck in your network does FQ, and so latency on separate connections is very low, while your load-generating connections are still bufferbloated.
>>> 
>>> 
>>> Ookla measures latency only on separate connections, thus will also be heavily impacted by FQ.
>>> 
>>> 
>>> Does that clarify it?
>>> 
>>> 
>>> Cheers,
>>> Christoph 
>>> 
>>> 
>>>> 
>>>> Further machine details: MacBook Pro 16” (2019) using a USB-C to Ethernet adapter.
>>>> I run with full ECN enabled:
>>>> sudo sysctl -w net.inet.tcp.disable_tcp_heuristics=1 
>>>> 
>>>> sudo sysctl -w net.inet.tcp.ecn_initiate_out=1
>>>> 
>>>> sudo sysctl -w net.inet.tcp.ecn_negotiate_in=1
>>>> 
>>>> and also with instant ack replies:
>>>> 
>>>> sysctl net.inet.tcp.delayed_ack
>>>> net.inet.tcp.delayed_ack: 0
>>>> 
>>>> I did try with delayed_ack=1, and the results were about the same.
>>>> 
>>>> Thanks in advance,
>>>> 
>>>> Jonathan Foulkes
>>>> 
>>>> _______________________________________________
>>>> Rpm mailing list
>>>> Rpm at lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/rpm
>>> 
>> 
>> _______________________________________________
>> Rpm mailing list
>> Rpm at lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/rpm
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/rpm/attachments/20221110/a5c6588e/attachment.html>