[Bloat] [Rpm] so great to see ISPs that care
rjmcmahon
rjmcmahon at rjmcmahon.com
Sun Mar 12 17:20:00 EDT 2023
for completeness, here is a concurrent "working load" example:
[root at ryzen3950 iperf2-code]# iperf -c 192.168.1.58%enp4s0 -i 1 -e
--bounceback --working-load=up,4 -t 3
------------------------------------------------------------
Client connecting to 192.168.1.58, TCP port 5001 with pid 3125575 via
enp4s0 (1 flows)
Write buffer size: 100 Byte
Bursting: 100 Byte writes 10 times every 1.00 second(s)
Bounce-back test (size= 100 Byte) (server hold req=0 usecs &
tcp_quickack)
TOS set to 0x0 and nodelay (Nagle off)
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 2] local 192.168.1.69%enp4s0 port 49268 connected with 192.168.1.58
port 5001 (bb w/quickack len/hold=100/0) (sock=7)
(icwnd/mss/irtt=14/1448/243) (ct=0.29 ms) on 2023-03-12 14:18:25.658
(PDT)
[ 5] local 192.168.1.69%enp4s0 port 49244 connected with 192.168.1.58
port 5001 (prefetch=16384) (sock=3) (qack) (icwnd/mss/irtt=14/1448/260)
(ct=0.31 ms) on 2023-03-12 14:18:25.658 (PDT)
[ 4] local 192.168.1.69%enp4s0 port 49254 connected with 192.168.1.58
port 5001 (prefetch=16384) (sock=4) (qack) (icwnd/mss/irtt=14/1448/295)
(ct=0.35 ms) on 2023-03-12 14:18:25.658 (PDT)
[ 1] local 192.168.1.69%enp4s0 port 49256 connected with 192.168.1.58
port 5001 (prefetch=16384) (sock=6) (qack) (icwnd/mss/irtt=14/1448/270)
(ct=0.31 ms) on 2023-03-12 14:18:25.658 (PDT)
[ 3] local 192.168.1.69%enp4s0 port 49252 connected with 192.168.1.58
port 5001 (prefetch=16384) (sock=5) (qack) (icwnd/mss/irtt=14/1448/263)
(ct=0.31 ms) on 2023-03-12 14:18:25.658 (PDT)
[ ID] Interval Transfer Bandwidth Write/Err Rtry
Cwnd/RTT(var) NetPwr
[ 5] 0.00-1.00 sec 41.8 MBytes 351 Mbits/sec 438252/0 3
73K/53(3) us 826892
[ 1] 0.00-1.00 sec 39.3 MBytes 330 Mbits/sec 412404/0 24
39K/45(3) us 916455
[ ID] Interval Transfer Bandwidth BB
cnt=avg/min/max/stdev Rtry Cwnd/RTT RPS
[ 2] 0.00-1.00 sec 1.95 KBytes 16.0 Kbits/sec
10=0.323/0.093/2.147/0.641 ms 0 14K/119 us 3098 rps
[ 4] 0.00-1.00 sec 34.2 MBytes 287 Mbits/sec 358210/0 15
55K/53(3) us 675869
[ 3] 0.00-1.00 sec 33.4 MBytes 280 Mbits/sec 349927/0 11
127K/53(4) us 660241
[SUM] 0.00-1.00 sec 109 MBytes 917 Mbits/sec 1146389/0 29
[ 5] 1.00-2.00 sec 42.1 MBytes 353 Mbits/sec 441376/0 1
73K/55(9) us 802502
[ 1] 1.00-2.00 sec 39.6 MBytes 333 Mbits/sec 415644/0 0
39K/51(6) us 814988
[ 2] 1.00-2.00 sec 1.95 KBytes 16.0 Kbits/sec
10=0.079/0.056/0.127/0.019 ms 0 14K/67 us 12658 rps
[ 4] 1.00-2.00 sec 33.8 MBytes 283 Mbits/sec 354150/0 0
55K/58(7) us 610603
[ 3] 1.00-2.00 sec 33.7 MBytes 283 Mbits/sec 353392/0 2
127K/53(6) us 666777
[SUM] 1.00-2.00 sec 110 MBytes 919 Mbits/sec 1148918/0 3
[ 5] 2.00-3.00 sec 42.2 MBytes 354 Mbits/sec 442685/0 0
73K/50(8) us 885370
[ 1] 2.00-3.00 sec 36.9 MBytes 310 Mbits/sec 387381/0 0
39K/48(4) us 807044
[ 2] 2.00-3.00 sec 1.95 KBytes 16.0 Kbits/sec
10=0.073/0.058/0.093/0.012 ms 0 14K/60 us 13774 rps
[ 4] 2.00-3.00 sec 33.9 MBytes 284 Mbits/sec 355533/0 0
55K/52(4) us 683717
[ 3] 2.00-3.00 sec 29.4 MBytes 247 Mbits/sec 308725/0 1
127K/54(4) us 571713
[SUM] 2.00-3.00 sec 106 MBytes 886 Mbits/sec 1106943/0 1
[ 5] 0.00-3.00 sec 126 MBytes 353 Mbits/sec 1322314/0 4
73K/57(18) us 773072
[ 2] 0.00-3.00 sec 7.81 KBytes 21.3 Kbits/sec
40=0.134/0.053/2.147/0.328 ms 0 14K/58 us 7489 rps
[ 2] 0.00-3.00 sec BB8(f)-PDF: bin(w=100us):cnt(40)=1:31,2:8,22:1
(5.00/95.00/99.7%=1/2/22,Outliers=1,obl/obu=0/0)
[ 3] 0.00-3.00 sec 96.5 MBytes 270 Mbits/sec 1012045/0 14
127K/57(6) us 591693
[ 1] 0.00-3.00 sec 116 MBytes 324 Mbits/sec 1215431/0 24
39K/51(5) us 794234
[ 4] 0.00-3.00 sec 102 MBytes 285 Mbits/sec 1067895/0 15
55K/55(9) us 647061
[SUM] 0.00-3.00 sec 324 MBytes 907 Mbits/sec 3402254/0 33
[ CT] final connect times (min/avg/max/stdev) = 0.292/0.316/0.352/22.075
ms (tot/err) = 5/0
> iperf 2 uses responses per second and also provides the bounce back
> times as well as one way delays.
>
> The hypothesis is that network engineers have to fix KPI issues,
> including latency, ahead of shipping products.
>
> Asking companies to act on consumer complaints is way too late. It's
> also extremely costly. Those running Amazon customer service can
> explain how these consumer calls about their devices cause things like
> device returns (as that's all the call support can provide.) This
> wastes energy to physically ship things back, causes a stack of
> working items that now go to ewaste, etc.
>
> It's really on network operators, suppliers and device mfgs to get
> ahead of this years before consumers get their stuff.
>
> As a side note, many devices select their WiFi chanspec (AP channel+)
> based on the strongest RSSI. The network paths should be based on KPIs
> like low latency. Strong signal just means an AP is yelling to loudly
> and interfering with the neighbors. Try the optimal AP chanspec that
> has 10dB separation per spatial dimension and the whole apartment
> complex would be better for it.
>
> We're so focused on buffer bloat we're ignoring everything else where
> incremental engineering has led to poor products & offerings.
>
> [rjmcmahon at ryzen3950 iperf2-code]$ iperf -c 192.168.1.72 -i 1 -e
> --bounceback --trip-times
> ------------------------------------------------------------
> Client connecting to 192.168.1.72, TCP port 5001 with pid 3123814 (1
> flows)
> Write buffer size: 100 Byte
> Bursting: 100 Byte writes 10 times every 1.00 second(s)
> Bounce-back test (size= 100 Byte) (server hold req=0 usecs &
> tcp_quickack)
> TOS set to 0x0 and nodelay (Nagle off)
> TCP window size: 16.0 KByte (default)
> Event based writes (pending queue watermark at 16384 bytes)
> ------------------------------------------------------------
> [ 1] local 192.168.1.69%enp4s0 port 41336 connected with 192.168.1.72
> port 5001 (prefetch=16384) (bb w/quickack len/hold=100/0) (trip-times)
> (sock=3) (icwnd/mss/irtt=14/1448/284) (ct=0.33 ms) on 2023-03-12
> 14:01:24.820 (PDT)
> [ ID] Interval Transfer Bandwidth BB
> cnt=avg/min/max/stdev Rtry Cwnd/RTT RPS
> [ 1] 0.00-1.00 sec 1.95 KBytes 16.0 Kbits/sec
> 10=0.311/0.209/0.755/0.159 ms 0 14K/202 us 3220 rps
> [ 1] 1.00-2.00 sec 1.95 KBytes 16.0 Kbits/sec
> 10=0.254/0.180/0.335/0.051 ms 0 14K/210 us 3934 rps
> [ 1] 2.00-3.00 sec 1.95 KBytes 16.0 Kbits/sec
> 10=0.266/0.168/0.468/0.088 ms 0 14K/210 us 3754 rps
> [ 1] 3.00-4.00 sec 1.95 KBytes 16.0 Kbits/sec
> 10=0.294/0.184/0.442/0.078 ms 0 14K/233 us 3396 rps
> [ 1] 4.00-5.00 sec 1.95 KBytes 16.0 Kbits/sec
> 10=0.263/0.150/0.427/0.077 ms 0 14K/215 us 3802 rps
> [ 1] 5.00-6.00 sec 1.95 KBytes 16.0 Kbits/sec
> 10=0.325/0.237/0.409/0.056 ms 0 14K/258 us 3077 rps
> [ 1] 6.00-7.00 sec 1.95 KBytes 16.0 Kbits/sec
> 10=0.259/0.165/0.410/0.077 ms 0 14K/219 us 3857 rps
> [ 1] 7.00-8.00 sec 1.95 KBytes 16.0 Kbits/sec
> 10=0.277/0.193/0.415/0.068 ms 0 14K/224 us 3608 rps
> [ 1] 8.00-9.00 sec 1.95 KBytes 16.0 Kbits/sec
> 10=0.292/0.206/0.465/0.072 ms 0 14K/231 us 3420 rps
> [ 1] 9.00-10.00 sec 1.95 KBytes 16.0 Kbits/sec
> 10=0.256/0.157/0.439/0.082 ms 0 14K/211 us 3908 rps
> [ 1] 0.00-10.01 sec 19.5 KBytes 16.0 Kbits/sec
> 100=0.280/0.150/0.755/0.085 ms 0 14K/1033 us 3573 rps
> [ 1] 0.00-10.01 sec OWD Delays (ms) Cnt=100
> To=0.169/0.074/0.318/0.056 From=0.105/0.055/0.162/0.024
> Asymmetry=0.065/0.000/0.172/0.049 3573 rps
> [ 1] 0.00-10.01 sec BB8(f)-PDF:
> bin(w=100us):cnt(100)=2:14,3:57,4:20,5:8,8:1
> (5.00/95.00/99.7%=2/5/8,Outliers=0,obl/obu=0/0)
>
>
> Bob
>> Dave,
>>
>> your presentation was awesome, I fully agree with you ;). I very much
>> liked your practical funnel demonstration which was boiled down to the
>> bare minimum (I only partly asked myself, will the liquid spill in in
>> your laptops keyboard, and if so is it water-proof, but you clearly
>> had rehearsed/tried that before).
>> BTW, I always have to think of this
>> h++ps://www.youtube.com/watch?v=R7yfISlGLNU somehow when you present
>> live from the marina ;)
>>
>>
>> I am still not through watching all of the presentations and panels,
>> but can already say, team L4S continues to over-promise and
>> under-deliver, but Koen's presentation itself was done well and might
>> (sadly) convince people to buy-in into L4(S) = 2L2L = too little, too
>> late.
>>
>> Stuart's RPM presentation was great, making a convincing point.
>> (Except for pitching L4S and LLD as "solutions", I will accept them as
>> a step in the right direction, but why not go in all the way and
>> embrace proper scheduling?)
>>
>> In detail though, I am not fully convinced about the decision of
>> taking the inverse of delay increase as singular measure here as I
>> consider that as a bit of a squandered opportunity at public
>> outreach/education and as comparing idle and working RPM is
>> non-intuitive, while idle and working RTT can immediately subtracted
>> to see the extent of the queueing damage in actionable terms.
>>
>> Try the same with RPM values:
>>
>> 123-1234567:~ user$ networkQuality -v
>> ==== SUMMARY ====
>>
>> Upload capacity: 22.208 Mbps
>> Download capacity: 88.054 Mbps
>> Upload flows: 12
>> Download flows: 12
>> Responsiveness: High (2622 RPM)
>> Base RTT: 18
>> Start: 3/12/23, 21:00:58
>> End: 3/12/23, 21:01:08
>> OS Version: Version 12.6.3 (Build 21G419)
>>
>> here we can divide 60 [sec/minute] * 1000 [ms/sec] by the RPM [1/min]
>> to get: 60000/2622 = 22.88 ms loaded delay and subtract the base RTT
>> of 18 for 60000/2622 - 18 = 4.88 ~5ms of loaded delay which is a
>> useful quantity when managing a delay budget (this test was performed
>> over wired ethernet with competent AQM and traffic shaping on the
>> link, so no surprise about the outcome there). Let's look at the
>> reverse and convert the base RTT into a base RPM score instead:
>> 6000/18 = 333 rpm, what exactly does the delta RPM of 2622-333 =
>> 2289rpm now tell us about the difference between idle and working
>> conditions? [Well, since conversion is not witchcraft, I will be fine
>> as will other interested in actual evoked delay, but we could have
>> gotten a better measure*]
>>
>> And all for the somewhat unhelpful car analogy... (it is not that for
>> internal combustion engines bigger is necessarily better for RPM,
>> either for torque or fuel efficiency).
>>
>> I guess that ship has sailed though and RPM it is
>>
>> *) Stuart notes that milliseconds and Hertz sound to sciency, but they
>> could simply have given the delay increase in milliseconds a fancier
>> name to solve that specific problem...
>>
>>
>>> On Mar 12, 2023, at 20:31, Dave Taht via Rpm
>>> <rpm at lists.bufferbloat.net> wrote:
>>>
>>> https://www.reddit.com/r/HomeNetworking/comments/11pmc9a/comment/jbypj0z/?context=3
>>>
>>> --
>>> Come Heckle Mar 6-9 at: https://www.understandinglatency.com/
>>> Dave Täht CEO, TekLibre, LLC
>>> _______________________________________________
>>> Rpm mailing list
>>> Rpm at lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/rpm
>>
>> _______________________________________________
>> Rpm mailing list
>> Rpm at lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/rpm
> _______________________________________________
> Rpm mailing list
> Rpm at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/rpm
More information about the Bloat
mailing list