* [Cake] Fwd: net-next is OPEN...
[not found] <20180416.110000.1863692416063182988.davem@davemloft.net>
@ 2018-04-16 15:01 ` Dave Taht
2018-04-16 15:12 ` Toke Høiland-Jørgensen
0 siblings, 1 reply; 11+ messages in thread
From: Dave Taht @ 2018-04-16 15:01 UTC (permalink / raw)
To: Cake List
do we consider cake ready this time?
---------- Forwarded message ----------
From: David Miller <davem@davemloft.net>
Date: Mon, Apr 16, 2018 at 8:00 AM
Subject: net-next is OPEN...
To: netdev@vger.kernel.org
You all know the drill:
http://vger.kernel.org/~davem/net-next.html
Now, let's see if you guys can avoid drowning me all at once
this time :-)
--
Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Cake] Fwd: net-next is OPEN...
2018-04-16 15:01 ` [Cake] Fwd: net-next is OPEN Dave Taht
@ 2018-04-16 15:12 ` Toke Høiland-Jørgensen
2018-04-16 20:55 ` [Cake] " Pete Heist
0 siblings, 1 reply; 11+ messages in thread
From: Toke Høiland-Jørgensen @ 2018-04-16 15:12 UTC (permalink / raw)
To: Dave Taht, Cake List
Dave Taht <dave.taht@gmail.com> writes:
> do we consider cake ready this time?
I'm not aware of anything outstanding, at least...
-Toke
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Cake] net-next is OPEN...
2018-04-16 15:12 ` Toke Høiland-Jørgensen
@ 2018-04-16 20:55 ` Pete Heist
2018-04-16 21:23 ` Toke Høiland-Jørgensen
2018-04-17 13:52 ` [Cake] net-next is OPEN Jonathan Morton
0 siblings, 2 replies; 11+ messages in thread
From: Pete Heist @ 2018-04-16 20:55 UTC (permalink / raw)
To: Toke Høiland-Jørgensen; +Cc: Dave Taht, Cake List
[-- Attachment #1: Type: text/plain, Size: 975 bytes --]
> On Apr 16, 2018, at 5:12 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>
> Dave Taht <dave.taht@gmail.com> writes:
>
>> do we consider cake ready this time?
>
> I'm not aware of anything outstanding, at least...
Do I have the release cycle right that it would have to be upstreamed by “about 10 weeks” from now and would end up in 4.18 (https://www.kernel.org/doc/Documentation/networking/netdev-FAQ.txt)?
I remember that fairness behavior at low RTTs (< 20ms) needed to be either improved or documented, and don’t see anything about that in the man page in the tc-adv repo thus far. Summarizing the host isolation results at http://www.drhleny.cz/bufferbloat/cake/round2/#hostiso_cake_{rtt}_{qos-id} <http://www.drhleny.cz/bufferbloat/cake/round2/#hostiso_cake_%7Brtt%7D_%7Bqos-id%7D>:
RTT: fairness (1.0 == perfect fairness)
---
100us: 2.22
1ms: 1.7
2ms: 1.6
3ms: 1.42
5ms: 1.31
8ms: 1.16
10ms: 1.12
20ms: 1.02
40ms: 1.017
[-- Attachment #2: Type: text/html, Size: 2057 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Cake] net-next is OPEN...
2018-04-16 20:55 ` [Cake] " Pete Heist
@ 2018-04-16 21:23 ` Toke Høiland-Jørgensen
2018-04-17 2:45 ` [Cake] fairness vs RTT Pete Heist
2018-04-17 13:52 ` [Cake] net-next is OPEN Jonathan Morton
1 sibling, 1 reply; 11+ messages in thread
From: Toke Høiland-Jørgensen @ 2018-04-16 21:23 UTC (permalink / raw)
To: Pete Heist; +Cc: Dave Taht, Cake List
Pete Heist <pete@eventide.io> writes:
>> On Apr 16, 2018, at 5:12 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>
>> Dave Taht <dave.taht@gmail.com> writes:
>>
>>> do we consider cake ready this time?
>>
>> I'm not aware of anything outstanding, at least...
>
> Do I have the release cycle right that it would have to be upstreamed
> by “about 10 weeks” from now and would end up in 4.18
> (https://www.kernel.org/doc/Documentation/networking/netdev-FAQ.txt)?
Assuming it is accepted, yes... :)
> I remember that fairness behavior at low RTTs (< 20ms) needed to be
> either improved or documented, and don’t see anything about that in
> the man page in the tc-adv repo thus far. Summarizing the host
> isolation results at
> http://www.drhleny.cz/bufferbloat/cake/round2/#hostiso_cake_{rtt}_{qos-id}
> <http://www.drhleny.cz/bufferbloat/cake/round2/#hostiso_cake_%7Brtt%7D_%7Bqos-id%7D>:
>
> RTT: fairness (1.0 == perfect fairness)
> ---
> 100us: 2.22
> 1ms: 1.7
> 2ms: 1.6
> 3ms: 1.42
> 5ms: 1.31
> 8ms: 1.16
> 10ms: 1.12
> 20ms: 1.02
> 40ms: 1.017
Erm, what's the metric and which data source are you looking at here?
-Toke
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Cake] fairness vs RTT
2018-04-16 21:23 ` Toke Høiland-Jørgensen
@ 2018-04-17 2:45 ` Pete Heist
2018-04-17 8:24 ` Toke Høiland-Jørgensen
0 siblings, 1 reply; 11+ messages in thread
From: Pete Heist @ 2018-04-17 2:45 UTC (permalink / raw)
To: Toke Høiland-Jørgensen; +Cc: Dave Taht, Cake List
[-- Attachment #1: Type: text/plain, Size: 2542 bytes --]
> On Apr 16, 2018, at 11:23 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>
>> I remember that fairness behavior at low RTTs (< 20ms) needed to be
>> either improved or documented, and don’t see anything about that in
>> the man page in the tc-adv repo thus far. Summarizing the host
>> isolation results at
>> http://www.drhleny.cz/bufferbloat/cake/round2/#hostiso_cake_{rtt}_{qos-id}
>> <http://www.drhleny.cz/bufferbloat/cake/round2/#hostiso_cake_%7Brtt%7D_%7Bqos-id%7D>:
>>
>> RTT: fairness (1.0 == perfect fairness)
>> ---
>> 100us: 2.22
>> 1ms: 1.7
>> 2ms: 1.6
>> 3ms: 1.42
>> 5ms: 1.31
>> 8ms: 1.16
>> 10ms: 1.12
>> 20ms: 1.02
>> 40ms: 1.017
>
> Erm, what's the metric and which data source are you looking at here?
Subject changed...
The clients were as follows:
Client 0- 1 stream up
Client 1- 12 streams up
Client 2- 1 stream down
Client 3- 12 streams down
It looks like what I used before was Client 3’s “TCP Download Sum avg” divided by Client 2’s “TCP Download avg” from the srchost/dsthost tests. The data’s in a table here (see column “Client 3 Mean / Client 2 Mean”):
https://docs.google.com/spreadsheets/d/1e06ZfHSSmecJx9sPU2s2g2GYCS18cMIhBN8PXf1jwaM/edit?usp=sharing <https://docs.google.com/spreadsheets/d/1e06ZfHSSmecJx9sPU2s2g2GYCS18cMIhBN8PXf1jwaM/edit?usp=sharing>
I was calculating by hand before, so the numbers are slightly different, but that’s the idea.
Now, these tests were done at 500Mbit between two cabled APU2s, so we could just be running out of CPU on this hardware at lower RTTs. There are CPU stats included, and a “Flent Data Files” section. Is it possible to tell from this if CPU is the problem? The highest median client load I see looks to be 0.77 for the 40ms tests, for example, with a mean of 0.63.
http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_100us_eg_cakeeth_src_cakeeth_dst_500mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_100us_eg_cakeeth_src_cakeeth_dst_500mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_40ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_40ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html>
If we’re not sure, the tests would have to be redone at a lower bitrate. It would probably be best if someone reproduced these results externally anyway, and perhaps it would also be better to not be mixing upload and download tests at the same time… :)
[-- Attachment #2: Type: text/html, Size: 3874 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Cake] fairness vs RTT
2018-04-17 2:45 ` [Cake] fairness vs RTT Pete Heist
@ 2018-04-17 8:24 ` Toke Høiland-Jørgensen
0 siblings, 0 replies; 11+ messages in thread
From: Toke Høiland-Jørgensen @ 2018-04-17 8:24 UTC (permalink / raw)
To: Pete Heist; +Cc: Dave Taht, Cake List
Pete Heist <pete@eventide.io> writes:
>> On Apr 16, 2018, at 11:23 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>
>>> I remember that fairness behavior at low RTTs (< 20ms) needed to be
>>> either improved or documented, and don’t see anything about that in
>>> the man page in the tc-adv repo thus far. Summarizing the host
>>> isolation results at
>>> http://www.drhleny.cz/bufferbloat/cake/round2/#hostiso_cake_{rtt}_{qos-id}
>>> <http://www.drhleny.cz/bufferbloat/cake/round2/#hostiso_cake_%7Brtt%7D_%7Bqos-id%7D>:
>>>
>>> RTT: fairness (1.0 == perfect fairness)
>>> ---
>>> 100us: 2.22
>>> 1ms: 1.7
>>> 2ms: 1.6
>>> 3ms: 1.42
>>> 5ms: 1.31
>>> 8ms: 1.16
>>> 10ms: 1.12
>>> 20ms: 1.02
>>> 40ms: 1.017
>>
>> Erm, what's the metric and which data source are you looking at here?
>
>
> Subject changed...
>
> The clients were as follows:
> Client 0- 1 stream up
> Client 1- 12 streams up
> Client 2- 1 stream down
> Client 3- 12 streams down
>
> It looks like what I used before was Client 3’s “TCP Download Sum avg” divided by Client 2’s “TCP Download avg” from the srchost/dsthost tests. The data’s in a table here (see column “Client 3 Mean / Client 2 Mean”):
>
> https://docs.google.com/spreadsheets/d/1e06ZfHSSmecJx9sPU2s2g2GYCS18cMIhBN8PXf1jwaM/edit?usp=sharing <https://docs.google.com/spreadsheets/d/1e06ZfHSSmecJx9sPU2s2g2GYCS18cMIhBN8PXf1jwaM/edit?usp=sharing>
>
> I was calculating by hand before, so the numbers are slightly different, but that’s the idea.
>
> Now, these tests were done at 500Mbit between two cabled APU2s, so we
> could just be running out of CPU on this hardware at lower RTTs. There
> are CPU stats included, and a “Flent Data Files” section. Is it
> possible to tell from this if CPU is the problem? The highest median
> client load I see looks to be 0.77 for the 40ms tests, for example,
> with a mean of 0.63.
Well, the CPU usage meter is just summing that first line of /proc/stat;
so yeah, individual CPUs can definitely be 100% loaded. And the fact
that the test only achieves a total of ~200 Mbps rather than the 500
would indicate that this is the case...
-Toke
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Cake] net-next is OPEN...
2018-04-16 20:55 ` [Cake] " Pete Heist
2018-04-16 21:23 ` Toke Høiland-Jørgensen
@ 2018-04-17 13:52 ` Jonathan Morton
2018-04-18 5:43 ` Pete Heist
1 sibling, 1 reply; 11+ messages in thread
From: Jonathan Morton @ 2018-04-17 13:52 UTC (permalink / raw)
To: Pete Heist; +Cc: Toke Høiland-Jørgensen, Cake List
> On 16 Apr, 2018, at 11:55 pm, Pete Heist <pete@eventide.io> wrote:
>
> I remember that fairness behavior at low RTTs (< 20ms) needed to be either improved or documented
The reason for the behaviour, IIRC, was that throughput dropped below 100% when the latency target was reduced too much. Since then there has been a small change which might improve it a little, so a retest would be reasonable.
- Jonathan Morton
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Cake] net-next is OPEN...
2018-04-17 13:52 ` [Cake] net-next is OPEN Jonathan Morton
@ 2018-04-18 5:43 ` Pete Heist
2018-04-19 21:17 ` Pete Heist
0 siblings, 1 reply; 11+ messages in thread
From: Pete Heist @ 2018-04-18 5:43 UTC (permalink / raw)
To: Jonathan Morton; +Cc: Toke Høiland-Jørgensen, Cake List
I also think I saw this happen at lower bandwidths as well, when the CPU wasn’t loaded. What I’ll do is re-test on the current version I have at, say, 50Mbit (or to where load drops substantially), then update to the head and test again, and let you know...
Pete
> On Apr 17, 2018, at 3:52 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
>
>> On 16 Apr, 2018, at 11:55 pm, Pete Heist <pete@eventide.io> wrote:
>>
>> I remember that fairness behavior at low RTTs (< 20ms) needed to be either improved or documented
>
> The reason for the behaviour, IIRC, was that throughput dropped below 100% when the latency target was reduced too much. Since then there has been a small change which might improve it a little, so a retest would be reasonable.
>
> - Jonathan Morton
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Cake] net-next is OPEN...
2018-04-18 5:43 ` Pete Heist
@ 2018-04-19 21:17 ` Pete Heist
2018-04-20 9:32 ` Toke Høiland-Jørgensen
0 siblings, 1 reply; 11+ messages in thread
From: Pete Heist @ 2018-04-19 21:17 UTC (permalink / raw)
To: Jonathan Morton; +Cc: Toke Høiland-Jørgensen, Cake List
> On Apr 18, 2018, at 7:43 AM, Pete Heist <pete@eventide.io> wrote:
>
> I also think I saw this happen at lower bandwidths as well, when the CPU wasn’t loaded. What I’ll do is re-test on the current version I have at, say, 50Mbit (or to where load drops substantially), then update to the head and test again, and let you know...
>
>> On Apr 17, 2018, at 3:52 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
>>
>>> On 16 Apr, 2018, at 11:55 pm, Pete Heist <pete@eventide.io> wrote:
>>>
>>> I remember that fairness behavior at low RTTs (< 20ms) needed to be either improved or documented
>>
>> The reason for the behaviour, IIRC, was that throughput dropped below 100% when the latency target was reduced too much. Since then there has been a small change which might improve it a little, so a retest would be reasonable.
At 50mbit I don’t see nearly as much fairness degradation at low RTTs, although there’s some. Even at 100us, “fairness” is around 1.1 (1.0 being perfectly fair) instead of the 2.x I saw at 500mbit. I do not see much of a difference between the latest code (16d7fed, 2018-04-17) and the previous code I tested (7061401, 2017-12-01), if that info is of use.
RTT: tcp_1up upload Mbps / tcp_12up upload Mbps
7061401 (2017-12-01):
100us: 23.80 / 25.85
1ms: 23.89 / 29.46
10ms: 23.93 / 24.66
40ms: 23.96 / 24.10
100ms: 23.97 / 24.12
16d7fed (2018-04-17):
100us: 23.97 / 26.49
1ms: 23.89 / 26.27
10ms: 23.98 / 26.37
40ms: 23.94 / 24.08
100ms: 23.97 / 24.12
I can post reports / flent files on request.
So it appears this is CPU related, and not worth exploring further(?) and not worth documenting(?) other than that once things have stabilized, documenting how Cake degrades under various extreme conditions would be informative.
Well, here’s to science and a good walk in the weeds…
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Cake] net-next is OPEN...
2018-04-19 21:17 ` Pete Heist
@ 2018-04-20 9:32 ` Toke Høiland-Jørgensen
0 siblings, 0 replies; 11+ messages in thread
From: Toke Høiland-Jørgensen @ 2018-04-20 9:32 UTC (permalink / raw)
To: Pete Heist, Jonathan Morton; +Cc: Cake List
Pete Heist <pete@eventide.io> writes:
>> On Apr 18, 2018, at 7:43 AM, Pete Heist <pete@eventide.io> wrote:
>>
>> I also think I saw this happen at lower bandwidths as well, when the CPU wasn’t loaded. What I’ll do is re-test on the current version I have at, say, 50Mbit (or to where load drops substantially), then update to the head and test again, and let you know...
>>
>>> On Apr 17, 2018, at 3:52 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
>>>
>>>> On 16 Apr, 2018, at 11:55 pm, Pete Heist <pete@eventide.io> wrote:
>>>>
>>>> I remember that fairness behavior at low RTTs (< 20ms) needed to be either improved or documented
>>>
>>> The reason for the behaviour, IIRC, was that throughput dropped below 100% when the latency target was reduced too much. Since then there has been a small change which might improve it a little, so a retest would be reasonable.
>
> At 50mbit I don’t see nearly as much fairness degradation at low RTTs, although there’s some. Even at 100us, “fairness” is around 1.1 (1.0 being perfectly fair) instead of the 2.x I saw at 500mbit. I do not see much of a difference between the latest code (16d7fed, 2018-04-17) and the previous code I tested (7061401, 2017-12-01), if that info is of use.
>
> RTT: tcp_1up upload Mbps / tcp_12up upload Mbps
>
> 7061401 (2017-12-01):
>
> 100us: 23.80 / 25.85
> 1ms: 23.89 / 29.46
> 10ms: 23.93 / 24.66
> 40ms: 23.96 / 24.10
> 100ms: 23.97 / 24.12
>
> 16d7fed (2018-04-17):
>
> 100us: 23.97 / 26.49
> 1ms: 23.89 / 26.27
> 10ms: 23.98 / 26.37
> 40ms: 23.94 / 24.08
> 100ms: 23.97 / 24.12
>
> I can post reports / flent files on request.
>
> So it appears this is CPU related, and not worth exploring further(?)
> and not worth documenting(?) other than that once things have
> stabilized, documenting how Cake degrades under various extreme
> conditions would be informative.
Awesome, thanks for re-testing! :)
-Toke
^ permalink raw reply [flat|nested] 11+ messages in thread
* [Cake] net-next is open
@ 2017-12-01 14:52 Dave Taht
0 siblings, 0 replies; 11+ messages in thread
From: Dave Taht @ 2017-12-01 14:52 UTC (permalink / raw)
To: Cake List
http://vger.kernel.org/~davem/net-next.html
Can I garner a few Signed-off-by's?
I will add tested-by for georgio and pete.
--
Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2018-04-20 9:32 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20180416.110000.1863692416063182988.davem@davemloft.net>
2018-04-16 15:01 ` [Cake] Fwd: net-next is OPEN Dave Taht
2018-04-16 15:12 ` Toke Høiland-Jørgensen
2018-04-16 20:55 ` [Cake] " Pete Heist
2018-04-16 21:23 ` Toke Høiland-Jørgensen
2018-04-17 2:45 ` [Cake] fairness vs RTT Pete Heist
2018-04-17 8:24 ` Toke Høiland-Jørgensen
2018-04-17 13:52 ` [Cake] net-next is OPEN Jonathan Morton
2018-04-18 5:43 ` Pete Heist
2018-04-19 21:17 ` Pete Heist
2018-04-20 9:32 ` Toke Høiland-Jørgensen
2017-12-01 14:52 [Cake] net-next is open Dave Taht
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox