Re: [Cake] lan keyword affects host fairness

[Pete Heist <peteheist@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2735 bytes --]


> On Nov 24, 2017, at 12:21 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Dave Taht <dave@taht.net> writes:
> 
>> Pete Heist <peteheist@gmail.com> writes:
>> 
>>>    On Nov 23, 2017, at 10:44 AM, Jonathan Morton <chromatix99@gmail.com> wrote:
>>> 
>>>    This is most likely an interaction of the AQM with Linux' scheduling
>>>    latency.
>>> 
>>>    At the 'lan' setting, the time comstants are similar in magnitude to the
>>>    delays induced by Linux itself, so congestion might be signalled
>>>    prematurely. The flows will then become sparse and total throughput reduced,
>>>    leaving little or no back-pressure for the fairness logic to work against.
>> 
>> Agreed. 
>> 
>> man page add:
>> 
>> At the 'lan' setting(1ms), the time constants are similar in magnitude
>> to the jitter in the Linux kernel itself, so congestion might be
>> signalled prematurely. The flows will then become sparse and total
>> throughput reduced, leaving little or no back-pressure for the fairness
>> logic to work against. Use the "metro" setting for local lans unless you
>> have a custom kernel.
> 
> Erm, doesn't this make the 'lan' keyword pretty much useless? So why not
> just remove it? Or redefine it to something that actually works? 3ms?

Removing the bandwidth keywords altogether and going back to fq_codel’s specification of target and interval would be my personal preference (unless we can figure out how to make the keywords work well with one another in all cases).

If we want to keep them, I’ll look for more holes where things might not behave as people expect so we can further clear up the docs.

Also, note that even at ‘metro’, fairness is better, but still not fully fair on my hardware:

https://docs.google.com/spreadsheets/d/1SMXWw2fLfmBRU622urfdvA_Ujsuf_KQ4P3uyOH1skOM/edit#gid=2072687073 <https://docs.google.com/spreadsheets/d/1SMXWw2fLfmBRU622urfdvA_Ujsuf_KQ4P3uyOH1skOM/edit#gid=2072687073>

With 950mbit rate limiting, it’s pretty good at 1.16:1. But with bql it’s still ~2:1. At what rtt will fairness actually work? That may depend on hardware, OS version, number of flows, whether bql is being used or not, or other factors. At rtt 100ms fairness worked well for this test with both bql and rate limiting.

Also note that I’m glossing over the fact that on my hardware (Ethernet device with four queues), sometimes there was a bandwidth asymmetry and corresponding fluctuation in fairness when using bql that changed with each flent run, so I had to retry the test once or twice to get a “good” run. I would try to quantify that separately before I get into it further. But that’s something that may also throw folks for a loop.


[-- Attachment #2: Type: text/html, Size: 4058 bytes --]