[Bloat] An ACM Queue article about bloat and LibraQoS
David Collier-Brown
davec-b at rogers.com
Tue Jul 16 10:25:54 EDT 2024
Many thanks!
--dave
On 2024-07-16 09:48, Sebastian Moeller via Bloat wrote:
> Hi Dave,
>
>
>
>> On 15. Jul 2024, at 16:26, David Collier-Brown <davec-b at rogers.com> wrote:
>>
>> Let's see... Found them!
>> score A+ D %
>> up 9.1 31.8 29%
>> down 163 456.2 36%
>> A+ figures were only 20-30% of the bloated D figure (:-()
>> I showed the article with both graphs to an outside reader and he spotted it instantly, and asked why I was only getting a tiny fraction of the "bloated" throughput.
>> --dave
> [SM] OK, that clearly is a lot of throughput sacrifice... are you sure your link is as optimised as possible? If not I wozuld be happy to help.
> Regarding other tools for measurements that include latency-under-load:
>
> A) Ookla's speedtest.net <http://speedtest.net/> will also report the interquartil means for latency under load
> Example: https://www.speedtest.net/result/16503399049
>
> B) Cloudflare's speed.cloudflare.com <http://speed.cloudflare.com/> will also report some scores
> Here is an article delving into how these scores are derived from the measured data
> https://blog.cloudflare.com/aim-database-for-internet-quality
>
>
> c) Apple's networkQuality tool (on macos/ios) or the open source go version thereof: https://github.com/network-quality/goresponsiveness
> user at laptop goresponsiveness % networkQuality -v -s
> ==== Verbose Results ====
> ---
> Capacity:
> ---
> Uplink capacity: 39.688 Mbps
> Accuracy: High
> Uplink bytes transferred: 106.702 MB
> Uplink Flow count: 16
> Downlink capacity: 74.621 Mbps
> Accuracy: High
> Downlink bytes transferred: 50.644 MB
> Downlink Flow count: 6
> ---
> Latency:
> ---
> Idle Latency:
> 2618 RPM (22.917 milliseconds)
> Transport: 3178 RPM (18.875 milliseconds)
> Security: 1832 RPM (32.750 milliseconds)
> HTTP: 3503 RPM (17.125 milliseconds)
> Accuracy: High
> Uplink Responsiveness: Medium
> 296 RPM (202.171 milliseconds)
> Transport: 3390 RPM (17.699 milliseconds)
> Security: 2260 RPM (26.538 milliseconds)
> HTTP: 3276 RPM (18.312 milliseconds)
> HTTP loaded: 155 RPM (386.579 milliseconds)
> Accuracy: High
> Downlink Responsiveness: Medium
> 919 RPM (65.240 milliseconds)
> Transport: 3564 RPM (16.833 milliseconds)
> Security: 2488 RPM (24.115 milliseconds)
> HTTP: 3440 RPM (17.438 milliseconds)
> HTTP loaded: 430 RPM (139.362 milliseconds)
> Accuracy: High
> ---
> Protocols Used:
> ---
> HTTP/2: 100%
> ---
> Transport-layer info:
> ---
> ECN Disabled: 100%
> ---
> Other Info:
> ---
> Test Endpoint: defra1-edge-bx-006.aaplimg.com
> Interface: en7
> Start: 2024-07-16 15:10:23.240
> End: 2024-07-16 15:10:51.921
> OS Version: Version 14.5 (Build 23F79)
>
> Here the RPM for Uplink and Downlink HTTP are most relevant (these measure inter-flow latency increases that is how much the latency of flows stayiung below their capacity share is effected by parallel capacity seeking flows, HTTP loaded measures the self congestion of the load bearing flows)
>
> Here an invocation of the opensource version (using apple's servers):
>
> ./networkQuality --detailed --relative-rpm --config mensura.cdn-apple.com --port 443 --path /api/v1/gm/config --rpm.timeout 120 --extended-stats --rpm.mnp 32 --logger-filename go_networkQuality_20240716_151413
> 07-16-2024 13:14:13 UTC Go Responsiveness to mensura.cdn-apple.com:443...
> Unbounded Baseline RPM: 2857 (P90)
> Unbounded Baseline RPM: 3322 (Single-Sided 5% Trimmed Mean)
> Baseline RPM: 2827 (P90)
> Baseline RPM: 3327 (Single-Sided 5% Trimmed Mean)
> Results:
> ========
> Download:
> Throughput: 68.989 Mbps (8.624 MBps), using 26 parallel connections.
> Extended Statistics:
> Maximum Segment Size: 1208
> Total Bytes Retransmitted: 0
> Retransmission Ratio: 0.00%
> Total Bytes Reordered: 140449506
> Average RTT: 26.85185185185185
> Unbounded RPM Calculation Statistics:
> Total Self Probes: 1922
> Total Foreign Probes: 5766
> Trimmed Self Probes Count: 96 of 1922
> Trimmed Foreign Probes Count: 288 of 5766
> P90 Self RTT: 0.635866s
> P90 Foreign RTT: 0.020506s
> P90 RTT: 0.328186s
> Trimmed Mean Self RTT: 0.166220s
> Trimmed Mean Foreign RTT: 0.018372s
> Trimmed Mean RTT: 0.092296s
> RPM Calculation Statistics:
> Total Self Probes: 399
> Total Foreign Probes: 1197
> Trimmed Self Probes Count: 19 of 399
> Trimmed Foreign Probes Count: 59 of 1197
> P90 Self RTT: 0.806346s
> P90 Foreign RTT: 0.020808s
> P90 RTT: 0.413577s
> Trimmed Mean Self RTT: 0.329165s
> Trimmed Mean Foreign RTT: 0.019207s
> Trimmed Mean RTT: 0.174186s
> RPM: 145 (P90)
> RPM: 344 (Single-Sided 5% Trimmed Mean)
> ========
> Upload:
> Note: Test did not run to stability (responsiveness), these results are estimates.
> Throughput: 38.249 Mbps (4.781 MBps), using 32 parallel connections.
> Extended Statistics:
> Maximum Segment Size: 1208
> Total Bytes Retransmitted: 102005114
> Retransmission Ratio: 15.48%
> Total Bytes Reordered: 0
> Average RTT: 43.75
> Unbounded RPM Calculation Statistics:
> Total Self Probes: 11091
> Total Foreign Probes: 33273
> Trimmed Self Probes Count: 554 of 11091
> Trimmed Foreign Probes Count: 1663 of 33273
> P90 Self RTT: 1.564135s
> P90 Foreign RTT: 0.222845s
> P90 RTT: 0.893490s
> Trimmed Mean Self RTT: 0.111610s
> Trimmed Mean Foreign RTT: 0.018731s
> Trimmed Mean RTT: 0.065171s
> RPM Calculation Statistics:
> Total Self Probes: 381
> Total Foreign Probes: 1143
> Trimmed Self Probes Count: 19 of 381
> Trimmed Foreign Probes Count: 57 of 1143
> P90 Self RTT: 1.131356s
> P90 Foreign RTT: 0.270268s
> P90 RTT: 0.700812s
> Trimmed Mean Self RTT: 0.139588s
> Trimmed Mean Foreign RTT: 0.020490s
> Trimmed Mean RTT: 0.080039s
> RPM: 86 (P90)
> RPM: 750 (Single-Sided 5% Trimmed Mean)
> ========
> Unbounded Final RPM Calculation stats:
> Total Self Probes: 13013
> Total Foreign Probes: 39039
> Trimmed Self Probes Count: 650 of 13013
> Trimmed Foreign Probes Count: 1951 of 39039
> P90 Self RTT: 1.372974s
> P90 Foreign RTT: 0.154186s
> P90 RTT: 0.763580s
> Trimmed Mean Self RTT: 0.117897s
> Trimmed Mean Foreign RTT: 0.018549s
> Trimmed Mean RTT: 0.068223s
>
> Unbounded Final RPM: 79 (P90)
> Unbounded Final RPM: 879 (Single-Sided 5% Trimmed Mean)
>
> Final RPM Calculation stats:
> Total Self Probes: 11490
> Total Foreign Probes: 34470
> Trimmed Self Probes Count: 574 of 11490
> Trimmed Foreign Probes Count: 1723 of 34470
> P90 Self RTT: 1.517121s
> P90 Foreign RTT: 0.208764s
> P90 RTT: 0.862942s
> Trimmed Mean Self RTT: 0.113263s
> Trimmed Mean Foreign RTT: 0.018745s
> Trimmed Mean RTT: 0.066004s
>
> Final RPM: 70 (P90) RTT: 0.862942s
> Final RPM: 909 (Single-Sided 5% Trimmed Mean) RTT: 0.066004s
> Final RPM (Self Only): 40 (P90) RTT: 1.517121s
> Final RPM (Self Only): 530 (Single-Sided 5% Trimmed Mean) RTT: 0.113263s
> Final RPM (Foreign Only): 287 (P90) RTT: 0.208764s
> Final RPM (Foreign Only): 3201 (Single-Sided 5% Trimmed Mean) RTT: 0.018745s
> Working-Conditions Effect: Final RPM is 2% of baseline RPM (P90)
> Working-Conditions Effect: Final RPM is 27% of baseline RPM (Single-Sided 5% Trimmed Mean)
>
> real 2m56.763s
> user 0m16.186s
> sys 0m26.759s
>
> Here Foreign means inter-flow measurements, Self meand intra-flow latency.
>
>
>
> And of course if you have your own server in the cloud there is always flent's RRUL test...
>
> Regards
> Sebastian
>
> P.S.: For an article about libreqos, maybe the beofre and after of a link of one of the libreqos using ISPs might be the best test?
>
>
>>
>> On 2024-07-15 09:12, Sebastian Moeller wrote:
>>> Hi Dave,
>>>
>>>
>>>
>>>> On 15. Jul 2024, at 12:59, David Collier-Brown via Bloat <bloat at lists.bufferbloat.net> wrote:
>>>>
>>>> On 2024-07-15 05:18, Sebastian Moeller via Bloat wrote:
>>>>
>>>>> Hi Dave,
>>>>>
>>>>> nice! I only would have wished for the A-grade waveform result as well, it is a bit of a downer that the last figure show the unpleasant 'before' status only.
>>>>>
>>>>> Regards
>>>>> Sebastian
>>>>>
>>>> That was a deliberate action on my part: the "after" status shows much smaller throughput numbers, leading observant readers to ask if improving latency didn't destroy one's bandwidth.
>>>>
>>> [SM] Ah, I see. But just for our discussion here, how much (potential) capacity did you need to treade in for decent responsiveness?
>>>
>>>
>>>> I suspect that the throughputs "with bloat" are greatly exaggerated... from counting the extra data that fills the bloated buffers (;-))
>>>>
>>> [SM] Good point... especially for upload tests, I occasionally see numbers which I know for a fact to be impossible (abovre the gross capacity of my access link) and I wonder whether all tests actually do proper reporting of the grand average capacity numbers. This should not be rocket science: take the start time of the first maesasurement flow and the end time of the last measurement flow and divide the total number of payload bytes trasferred by the total time this took.
>>>
>>>
>>>> --dave
>>>>
>>>>
>>>>
>>>>>> On 14. Jul 2024, at 23:30, David Collier-Brown via Bloat <bloat at lists.bufferbloat.net> wrote:
>>>>>>
>>>>>> I have a new article on ACM Queue, at https://dl.acm.org/doi/pdf/10.1145/3674953
>>>>>> "If you're an ISP and all your customers hate you, take heart. This is now a solvable problem."
>>>>>> It's part of a tongue-in-cheek series called "You Don't Know Jack", and it's about Bufferbloat and the LibreQoS project.
>>>>>> --dave
>>>>>> --
>>>>>> David Collier-Brown, | Always do right. This will gratify
>>>>>> System Programmer and Author | some people and astonish the rest
>>>>>> davecb at spamcop.net | -- Mark Twain
>>>>>> _______________________________________________
>>>>>> Bloat mailing list
>>>>>> Bloat at lists.bufferbloat.net
>>>>>> https://lists.bufferbloat.net/listinfo/bloat
>>>>>>
>>>>> _______________________________________________
>>>>> Bloat mailing list
>>>>> Bloat at lists.bufferbloat.net
>>>>> https://lists.bufferbloat.net/listinfo/bloat
>>>>>
>>>> --
>>>> David Collier-Brown, | Always do right. This will gratify
>>>> System Programmer and Author | some people and astonish the rest
>>>> davecb at spamcop.net | -- Mark Twain
>>>>
>>>> _______________________________________________
>>>> Bloat mailing list
>>>> Bloat at lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/bloat
>>>>
>> --
>> David Collier-Brown, | Always do right. This will gratify
>> System Programmer and Author | some people and astonish the rest
>> davecb at spamcop.net | -- Mark Twain
>
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
--
David Collier-Brown, | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
davecb at spamcop.net | -- Mark Twain
More information about the Bloat
mailing list