From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 523733BA8E for ; Tue, 15 Jan 2019 22:34:38 -0500 (EST) Received: by mail-qk1-x72e.google.com with SMTP id o125so2948755qkf.3 for ; Tue, 15 Jan 2019 19:34:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qVD+TO7PehvoY+xwdTR8xhs+83C7QKmGbDaXRrdTh60=; b=pLp7NGAFpaq2Yp6v5XBYJj3Lb3ZDW6jqFeIeHu2wxo3GYPuUR7Jd0YgSTDWUuQhXjL 2PU5jNEaUtAE2/y3Mx6U7acJYtrENU1PkvCzi2758JPxLpPeA9B0ILPl0kuEgMVISXLR Gu4Vl7chh+5+DoYr6V9Y+VfU6mr9UBhLvy0dscaQ8OcKhYnJ0BC2x+lhcltQ/nbcWMqb wTvaaMwD92lamEmwcfFn5JlTvp5LRQIIWE6jk5yEz/9bJMuBeda3QCnV3E0nsFsIcL2q h6+SA5misc8467CpbQmP5ICkxL5aI0+eNT/J8mz1JLZHvCkgN8lshGyQbmVmh20gNo+a jF6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qVD+TO7PehvoY+xwdTR8xhs+83C7QKmGbDaXRrdTh60=; b=iH7ytHaOtt8eQ6izU3F4ghLcUms8NYPQkajUeDk7K4bu0yHDHbLAZN+V7IoHVXRHRJ jt9fMWm11V4Y6+kWD5T6oU10D+zkgBEo8x/5ETbjHh/iXpJQU4l7CyRv9WmjCfgEdWG0 cKJCloSzm/QX9+0RiUF6JE/y8Z1FVVtt88xH3aoAoTbxQVH9YHQ4sFpIoIPliAW2eR8/ ixUuzd91fOhUa4NVvJMQJA6GKEzBXL3PLsaKzNjbtmw4agP9mi6hXdzH4o5oDmZll6j/ w2QrTDwvbXWYBmtHFf3qfPM4UL9hGHJoZH6Dbj6NICvFJucxHOl6TENsXrFfiNfyHz3p g32Q== X-Gm-Message-State: AJcUukfmdJRVKz/YZqvlVG9W8oR2GQoRrxfYVJX9YCOahuxMNEdd26Yi g4felHXvW3ICvlPKQtblEafU2ZE7x8E= X-Google-Smtp-Source: ALg8bN5Lgt6eGr+WyVFtzz4raY431lkNBt08Rr39swPPSQT7nqA94Byamhd7FkCYfWG8sGPIp/bWZQ== X-Received: by 2002:a37:9f92:: with SMTP id i140mr5079995qke.131.1547609677368; Tue, 15 Jan 2019 19:34:37 -0800 (PST) Received: from vmroute.lan (c-76-114-187-97.hsd1.md.comcast.net. [76.114.187.97]) by smtp.gmail.com with ESMTPSA id v57sm58595348qtv.80.2019.01.15.19.34.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Jan 2019 19:34:36 -0800 (PST) From: George Amanakis To: cake@lists.bufferbloat.net Date: Tue, 15 Jan 2019 22:34:30 -0500 Message-Id: <20190116033430.8446-1-gamanakis@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Cake] dual-src/dsthost unfairness, only with bi-directional traffic X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jan 2019 03:34:38 -0000 A better version of the patch for testing. Setup: IP{1,2}(flent) <----> Router <----> Server(netserver) Router: tc qdisc add dev enp1s0 root cake bandwidth 100mbit dual-srchost besteffort tc qdisc add dev enp4s0 root cake bandwidth 100mbit dual-dsthost besteffort IP1: Data file written to ./tcp_8down-2019-01-15T222742.358874.flent.gz. Summary of tcp_8down test run at 2019-01-16 03:27:42.358874: avg median # data pts Ping (ms) ICMP : 0.86 0.78 ms 342 TCP download avg : 6.16 5.86 Mbits/s 301 TCP download sum : 49.28 46.90 Mbits/s 301 TCP download::1 : 6.23 5.86 Mbits/s 297 TCP download::2 : 6.16 5.87 Mbits/s 297 TCP download::3 : 6.15 5.87 Mbits/s 297 TCP download::4 : 6.14 5.87 Mbits/s 297 TCP download::5 : 6.15 5.87 Mbits/s 297 TCP download::6 : 6.15 5.87 Mbits/s 297 TCP download::7 : 6.15 5.87 Mbits/s 297 TCP download::8 : 6.15 5.87 Mbits/s 297 Data file written to ./tcp_1up-2019-01-15T222743.387906.flent.gz. Summary of tcp_1up test run at 2019-01-16 03:27:43.387906: avg median # data pts Ping (ms) ICMP : 0.87 0.80 ms 343 TCP upload : 47.02 46.20 Mbits/s 265 IP2: Data file written to ./tcp_1up-2019-01-15T222744.371050.flent.gz. Summary of tcp_1up test run at 2019-01-16 03:27:44.371050: avg median # data pts Ping (ms) ICMP : 0.89 0.77 ms 342 TCP upload : 46.89 46.36 Mbits/s 293 Data file written to ./tcp_8down-2019-01-15T222745.382941.flent.gz. Summary of tcp_8down test run at 2019-01-16 03:27:45.382941: avg median # data pts Ping (ms) ICMP : 0.90 0.81 ms 343 TCP download avg : 6.15 5.86 Mbits/s 301 TCP download sum : 49.23 46.91 Mbits/s 301 TCP download::1 : 6.15 5.87 Mbits/s 297 TCP download::2 : 6.15 5.87 Mbits/s 297 TCP download::3 : 6.15 5.87 Mbits/s 296 TCP download::4 : 6.15 5.87 Mbits/s 297 TCP download::5 : 6.15 5.87 Mbits/s 297 TCP download::6 : 6.16 5.87 Mbits/s 297 TCP download::7 : 6.16 5.87 Mbits/s 297 TCP download::8 : 6.16 5.87 Mbits/s 297 --- sch_cake.c | 67 ++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 50 insertions(+), 17 deletions(-) diff --git a/sch_cake.c b/sch_cake.c index d434ae0..962a090 100644 --- a/sch_cake.c +++ b/sch_cake.c @@ -148,6 +148,7 @@ struct cake_host { u32 dsthost_tag; u16 srchost_refcnt; u16 dsthost_refcnt; + u16 bulk_flow_count; }; struct cake_heap_entry { @@ -1921,12 +1922,22 @@ static s32 cake_enqueue(struct sk_buff *skb, struct Qdisc *sch, flow->deficit = (b->flow_quantum * quantum_div[host_load]) >> 16; } else if (flow->set == CAKE_SET_SPARSE_WAIT) { + struct cake_host *srchost = &b->hosts[flow->srchost]; + struct cake_host *dsthost = &b->hosts[flow->dsthost]; + /* this flow was empty, accounted as a sparse flow, but actually * in the bulk rotation. */ flow->set = CAKE_SET_BULK; b->sparse_flow_count--; b->bulk_flow_count++; + + if (cake_dsrc(q->flow_mode)) + srchost->bulk_flow_count++; + + if (cake_ddst(q->flow_mode)) + dsthost->bulk_flow_count++; + } if (q->buffer_used > q->buffer_max_used) @@ -2097,23 +2108,8 @@ retry: dsthost = &b->hosts[flow->dsthost]; host_load = 1; - if (cake_dsrc(q->flow_mode)) - host_load = max(host_load, srchost->srchost_refcnt); - - if (cake_ddst(q->flow_mode)) - host_load = max(host_load, dsthost->dsthost_refcnt); - - WARN_ON(host_load > CAKE_QUEUES); - /* flow isolation (DRR++) */ if (flow->deficit <= 0) { - /* The shifted prandom_u32() is a way to apply dithering to - * avoid accumulating roundoff errors - */ - flow->deficit += (b->flow_quantum * quantum_div[host_load] + - (prandom_u32() >> 16)) >> 16; - list_move_tail(&flow->flowchain, &b->old_flows); - /* Keep all flows with deficits out of the sparse and decaying * rotations. No non-empty flow can go into the decaying * rotation, so they can't get deficits @@ -2122,6 +2118,13 @@ retry: if (flow->head) { b->sparse_flow_count--; b->bulk_flow_count++; + + if (cake_dsrc(q->flow_mode)) + srchost->bulk_flow_count++; + + if (cake_ddst(q->flow_mode)) + dsthost->bulk_flow_count++; + flow->set = CAKE_SET_BULK; } else { /* we've moved it to the bulk rotation for @@ -2131,6 +2134,22 @@ retry: flow->set = CAKE_SET_SPARSE_WAIT; } } + + if (cake_dsrc(q->flow_mode)) + host_load = max(host_load, srchost->bulk_flow_count); + + if (cake_ddst(q->flow_mode)) + host_load = max(host_load, dsthost->bulk_flow_count); + + WARN_ON(host_load > CAKE_QUEUES); + + /* The shifted prandom_u32() is a way to apply dithering to + * avoid accumulating roundoff errors + */ + flow->deficit += (b->flow_quantum * quantum_div[host_load] + + (prandom_u32() >> 16)) >> 16; + list_move_tail(&flow->flowchain, &b->old_flows); + goto retry; } @@ -2151,6 +2170,13 @@ retry: &b->decaying_flows); if (flow->set == CAKE_SET_BULK) { b->bulk_flow_count--; + + if (cake_dsrc(q->flow_mode)) + srchost->bulk_flow_count--; + + if (cake_ddst(q->flow_mode)) + dsthost->bulk_flow_count--; + b->decaying_flow_count++; } else if (flow->set == CAKE_SET_SPARSE || flow->set == CAKE_SET_SPARSE_WAIT) { @@ -2164,9 +2190,16 @@ retry: if (flow->set == CAKE_SET_SPARSE || flow->set == CAKE_SET_SPARSE_WAIT) b->sparse_flow_count--; - else if (flow->set == CAKE_SET_BULK) + else if (flow->set == CAKE_SET_BULK) { b->bulk_flow_count--; - else + + if (cake_dsrc(q->flow_mode)) + srchost->bulk_flow_count--; + + if (cake_ddst(q->flow_mode)) + dsthost->bulk_flow_count--; + + } else b->decaying_flow_count--; flow->set = CAKE_SET_NONE; -- 2.20.1