Re: [Cake] Upstream submission of dual-mode fairness patch

[Pete Heist <pete@heistp.net>]

[-- Attachment #1: Type: text/plain, Size: 2990 bytes --]

As another clue, the un-fairness goes away when ECN is enabled on the client (both results with ingress keyword in use):

sysctl -w net.ipv4.tcp_ecn=2

IP1, 1 up: 46.7 Mbit
IP2, 16 up: 46.7 Mbit
IP1, 16 down: 41.0 Mbit
IP2, 1 down: 49.3 Mbit

sysctl -w net.ipv4.tcp_ecn=1

IP1, 1 up: 47.0 Mbit
IP2, 16 up: 47.0 Mbit
IP1, 16 down: 46.1 Mbit
IP2, 1 down: 46.1 Mbit

Also, this is probably just academic, but here are pcaps of "IP1, 16 down", with and without “ingress” and ECN enabled, along with some throughput and RTT graphs of one of the 16 flows for each: https://www.heistp.net/downloads/ingress_fairness/ <https://www.heistp.net/downloads/ingress_fairness/>

I find the different TCP RTT “slots” interesting (see the rtt graphs, all times in ms):

no ingress: 4, 8.5, 13, 17
no ingress+ecn: 8, 12, 16 (a few outliers at 20)
ingress: 4, 8, 12, 16, 20
ingress+ecn: 8, 12, 16 (outliers at 4 and 20 later in test)

> On Mar 3, 2019, at 8:19 AM, Pete Heist <pete@heistp.net> wrote:
> 
> Oops, sorry I didn’t notice this before but it’s the ingress keyword that makes the difference:
> 
> qdisc cake 802c: dev ifbLink2 parent 1:1 bandwidth unlimited besteffort dual-dsthost nonat nowash ingress no-ack-filter no-split-gso rtt 100.0ms raw overhead 0 
> 
> IP1, 1 up: 46.8 Mbit
> IP2, 16 up: 46.8 Mbit
> IP1, 16 down: 41.9 Mbit
> IP2, 1 down: 51.1 Mbit
> 
> qdisc cake 803a: dev ifbLink2 parent 1:1 bandwidth unlimited besteffort dual-dsthost nonat nowash no-ack-filter no-split-gso rtt 100.0ms raw overhead 0 
> 
> IP1, 1 up: 46.6 Mbit
> IP2, 16 up: 46.5 Mbit
> IP1, 16 down: 46.4 Mbit
> IP2, 1 down: 46.5 Mbit
> 
> Also, my setup is different in that I use cake as a leaf to either hfsc or htb. But, it’s adding the ingress keyword that causes the imbalance for me.
> 
>> On Mar 2, 2019, at 11:20 AM, Pete Heist <pete@heistp.net <mailto:pete@heistp.net>> wrote:
>> 
>> Great, thanks for trying it. That strongly suggests a problem with the kernel (or driver) I’m using. Feels better to know it works as it should in recent kernels though...
>> 
>>> On Mar 2, 2019, at 5:47 AM, George Amanakis <gamanakis@gmail.com <mailto:gamanakis@gmail.com>> wrote:
>>> 
>>> On Fri, 2019-03-01 at 22:02 -0500, George Amanakis wrote:
>>>> 
>>>> I will setup a vlan and try again.
>>>> 
>>> 
>>> I replicated Pete's VLAN setup, and I am getting fairness:
>>> 
>>> IP1,2 <---> router enp1s0 / router enp1s0.100 <---> server
>>> 
>>> IP1, 1 up:      46.73 mbit/s
>>> IP2, 32 up:     46.91
>>> IP1, 32 down:   46.70
>>> IP2, 1 down:    46.89
>>> 
>>> qdisc cake 8006: dev enp1s0.100 root refcnt 2 bandwidth 100Mbit
>>> diffserv3 dual-srchost nonat nowash no-ack-filter split-gso nofwmark
>>> rtt 100.0ms noatm overhead 4
>>> 
>>> qdisc cake 8002: dev ifb0 root refcnt 2 bandwidth 100Mbit diffserv3
>>> dual-dsthost nonat nowash no-ack-filter split-gso nofwmark rtt 100.0ms
>>> noatm overhead 4
>>> 
>>> 
>>> 
>> 
> 


[-- Attachment #2: Type: text/html, Size: 5354 bytes --]