Re: [Cake] dual-src/dsthost unfairness, only with bi-directional traffic

[Pete Heist <pete@heistp.net>]

[-- Attachment #1: Type: text/plain, Size: 1526 bytes --]


> On Jan 3, 2019, at 12:03 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
>> Jon, is there anything I can check by instrumenting the code somewhere
>> specific?
> 
> Is there any way you could test with a bulk UDP flow? I'm wondering
> whether this is a second-order effect where TCP ACKs are limited in a
> way that cause the imbalance? Are you using ACK compression?


Not using ack-filter, if that’s what’s meant by ACK compression. I thought about the TCP ACK traffic, but would be very surprised if that amount of ACK traffic could cause that large of an imbalance, although it’s worth trying to find out.

I tried iperf3 in UDP mode, but cake is treating these flows aggressively. I get the impression that cake penalizes flows heavily that do not respond to congestion control signals. If I pit one 8 TCP flows against a single UDP flow at 40mbit, the UDP flow goes into a death spiral with increasing drops over time (iperf3 output attached).

I’m not sure there’d be any way I can test fairness with iperf3 in UDP mode. We’d need something that has some congestion control feedback, right? Otherwise, I don’t think there are any rates I can choose to both reach saturation and not be severely punished. And if it has congestion control feedback, it has the ACK-like traffic we’re trying to avoid for the test. :)

As another test, I took out the one-armed router and just tried from a client to a server, no VLANs. Same result. So, still stumped. Thank you for the help...


[-- Attachment #2: iperf3_spiral.txt --]
[-- Type: text/plain, Size: 5287 bytes --]

-----------------------------------------------------------
Server listening on 5202
-----------------------------------------------------------
Accepted connection from 10.0.0.239, port 48289
[  5] local 10.0.0.231 port 5202 connected to 10.72.0.239 port 38334
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec  4.20 MBytes  35.3 Mbits/sec  0.467 ms  21/559 (3.8%)  
[  5]   1.00-2.00   sec  4.27 MBytes  35.8 Mbits/sec  0.555 ms  43/589 (7.3%)  
[  5]   2.00-3.00   sec  4.48 MBytes  37.6 Mbits/sec  0.482 ms  69/642 (11%)  
[  5]   3.00-4.00   sec  3.90 MBytes  32.7 Mbits/sec  0.461 ms  87/586 (15%)  
[  5]   4.00-5.00   sec  3.84 MBytes  32.2 Mbits/sec  0.490 ms  111/603 (18%)  
[  5]   5.00-6.00   sec  3.94 MBytes  33.0 Mbits/sec  0.341 ms  130/634 (21%)  
[  5]   6.00-7.00   sec  3.63 MBytes  30.5 Mbits/sec  0.539 ms  144/609 (24%)  
[  5]   7.00-8.00   sec  3.59 MBytes  30.1 Mbits/sec  0.451 ms  159/618 (26%)  
[  5]   8.00-9.00   sec  3.21 MBytes  26.9 Mbits/sec  0.987 ms  181/592 (31%)  
[  5]   9.00-10.00  sec  3.23 MBytes  27.1 Mbits/sec  0.224 ms  225/639 (35%)  
[  5]  10.00-11.00  sec  3.11 MBytes  26.1 Mbits/sec  0.204 ms  214/612 (35%)  
[  5]  11.00-12.00  sec  2.80 MBytes  23.5 Mbits/sec  0.371 ms  229/587 (39%)  
[  5]  12.00-13.00  sec  2.66 MBytes  22.3 Mbits/sec  0.543 ms  254/594 (43%)  
[  5]  13.00-14.00  sec  2.73 MBytes  22.9 Mbits/sec  0.386 ms  292/642 (45%)  
[  5]  14.00-15.00  sec  2.49 MBytes  20.9 Mbits/sec  0.399 ms  298/617 (48%)  
[  5]  15.00-16.00  sec  2.40 MBytes  20.1 Mbits/sec  0.216 ms  288/595 (48%)  
[  5]  16.00-17.00  sec  2.20 MBytes  18.5 Mbits/sec  0.486 ms  327/609 (54%)  
[  5]  17.00-18.00  sec  2.19 MBytes  18.3 Mbits/sec  0.538 ms  344/624 (55%)  
[  5]  18.00-19.00  sec  2.00 MBytes  16.8 Mbits/sec  0.519 ms  321/577 (56%)  
[  5]  19.00-20.00  sec  1.95 MBytes  16.4 Mbits/sec  0.930 ms  369/619 (60%)  
[  5]  20.00-21.00  sec  1.93 MBytes  16.2 Mbits/sec  0.526 ms  377/624 (60%)  
[  5]  21.00-22.00  sec  1.66 MBytes  13.9 Mbits/sec  0.543 ms  374/586 (64%)  
[  5]  22.00-23.00  sec  1.70 MBytes  14.2 Mbits/sec  0.833 ms  412/629 (66%)  
[  5]  23.00-24.00  sec  1.66 MBytes  13.9 Mbits/sec  0.340 ms  402/614 (65%)  
[  5]  24.00-25.00  sec  1.52 MBytes  12.7 Mbits/sec  0.693 ms  431/625 (69%)  
[  5]  25.00-26.00  sec  1.40 MBytes  11.7 Mbits/sec  0.491 ms  404/583 (69%)  
[  5]  26.00-27.00  sec  1.32 MBytes  11.1 Mbits/sec  1.028 ms  456/625 (73%)  
[  5]  27.00-28.00  sec  1.25 MBytes  10.5 Mbits/sec  0.870 ms  427/587 (73%)  
[  5]  28.00-29.00  sec  1.20 MBytes  10.1 Mbits/sec  0.660 ms  479/633 (76%)  
[  5]  29.00-30.00  sec  1.19 MBytes  9.96 Mbits/sec  0.773 ms  466/618 (75%)  
[  5]  30.00-31.00  sec  1.05 MBytes  8.85 Mbits/sec  1.103 ms  455/590 (77%)  
[  5]  31.00-32.00  sec  1.03 MBytes  8.65 Mbits/sec  0.559 ms  488/620 (79%)  
[  5]  32.00-33.00  sec   888 KBytes  7.27 Mbits/sec  0.415 ms  494/605 (82%)  
[  5]  33.00-34.00  sec   896 KBytes  7.34 Mbits/sec  1.023 ms  489/601 (81%)  
[  5]  34.00-35.00  sec   880 KBytes  7.21 Mbits/sec  0.986 ms  519/629 (83%)  
[  5]  35.00-36.00  sec   776 KBytes  6.36 Mbits/sec  0.414 ms  493/590 (84%)  
[  5]  36.00-37.00  sec   800 KBytes  6.55 Mbits/sec  0.845 ms  506/606 (83%)  
[  5]  37.00-38.00  sec   832 KBytes  6.82 Mbits/sec  1.124 ms  536/640 (84%)  
[  5]  38.00-39.00  sec   768 KBytes  6.29 Mbits/sec  0.577 ms  515/611 (84%)  
[  5]  39.00-40.00  sec   728 KBytes  5.96 Mbits/sec  1.269 ms  496/587 (84%)  
[  5]  40.00-41.00  sec   752 KBytes  6.16 Mbits/sec  0.834 ms  544/638 (85%)  
[  5]  41.00-42.00  sec   528 KBytes  4.32 Mbits/sec  1.533 ms  346/412 (84%)  
[  5]  42.00-43.00  sec   552 KBytes  4.52 Mbits/sec  2.008 ms  722/791 (91%)  
[  5]  43.00-44.00  sec   416 KBytes  3.41 Mbits/sec  2.202 ms  528/580 (91%)  
[  5]  44.00-45.00  sec   408 KBytes  3.34 Mbits/sec  2.075 ms  566/617 (92%)  
[  5]  45.00-46.00  sec   512 KBytes  4.19 Mbits/sec  1.629 ms  517/581 (89%)  
[  5]  46.00-47.00  sec   400 KBytes  3.28 Mbits/sec  1.750 ms  584/634 (92%)  
[  5]  47.00-48.00  sec   408 KBytes  3.34 Mbits/sec  1.587 ms  541/592 (91%)  
[  5]  48.00-49.00  sec   504 KBytes  4.13 Mbits/sec  1.344 ms  587/650 (90%)  
[  5]  49.00-50.00  sec   600 KBytes  4.91 Mbits/sec  1.338 ms  522/597 (87%)  
[  5]  50.00-51.00  sec   384 KBytes  3.15 Mbits/sec  2.033 ms  592/640 (92%)  
[  5]  51.00-52.00  sec   504 KBytes  4.13 Mbits/sec  1.566 ms  529/592 (89%)  
[  5]  52.00-53.00  sec   400 KBytes  3.28 Mbits/sec  1.883 ms  508/558 (91%)  
[  5]  53.00-54.00  sec   424 KBytes  3.47 Mbits/sec  1.833 ms  639/692 (92%)  
[  5]  54.00-55.00  sec   280 KBytes  2.29 Mbits/sec  2.004 ms  539/574 (94%)  
[  5]  55.00-56.00  sec   288 KBytes  2.36 Mbits/sec  2.161 ms  522/558 (94%)  
[  5]  56.00-57.00  sec   288 KBytes  2.36 Mbits/sec  2.693 ms  635/671 (95%)  
[  5]  57.00-58.00  sec   264 KBytes  2.16 Mbits/sec  2.225 ms  554/587 (94%)  
[  5]  58.00-59.00  sec   256 KBytes  2.10 Mbits/sec  2.065 ms  571/603 (95%)  
[  5]  59.00-60.00  sec   696 KBytes  5.70 Mbits/sec  0.154 ms  588/675 (87%)  
[  5]  60.00-60.04  sec  24.0 KBytes  4.98 Mbits/sec  0.136 ms  0/3 (0%)