From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 544A93CB35 for ; Thu, 14 Feb 2019 14:00:29 -0500 (EST) Received: by mail-wr1-x429.google.com with SMTP id q18so7665902wrx.9 for ; Thu, 14 Feb 2019 11:00:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=lUtv5bwdP2wgtyBmX76g5y42ugba0ACdqvFAoTL0ynM=; b=eV2MXgTe2PmZTn82u7SdUL9e0Bdv+7gqVj1XtpH4VjYVVzlYSrHViBHYj9Wrn0AtIt YLmd49Ut0gdqB1+Tp1Lm5AS/OVusTE14JHwVoevOkdgQyyFDsVj5aiqL1McMNYb/502H pabV169O/dxZgoLDyze4ECiOIwOijVFwjeyMjtl96rnBEniRYVghryYtifIKjech6SFZ /aLkHiZbJhH2pRjwFTZI/QdKOKSWjKSgW7Eq0EwCweirhWGKIfoJ3PI74UdyyfQ8B3a6 Kl+XJ84wDYG4TegSoZM0MfE4popxaYsXpdIX0YP4JjFybgRq10stLq9tN+oOqUM8h963 zO7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=lUtv5bwdP2wgtyBmX76g5y42ugba0ACdqvFAoTL0ynM=; b=lT/j3gge7Xlt1z8+C8L9bw94IMaUqbHtzhj/4kE1I/sg4ffRIMjj4OauREylcwPrHM Iff/HWP+uk1SgINf1piQcD+7oYg6hbXwKCh7InfkWcvAuAP7S2onhnAQSOz8epqn3EWt w1bQsGQxGW6JSyADJ3wPRF1Dph/n7IAIznhotO+lY+Z031uR7MAgpA6XbEVK+5NT/xzn FzZ/KuxTGxrPICzjMol6CImetd7TEdrjAmhp3bOaJUmhvC1gZU7CDrV0c41ET7+ndKB6 IkzztUzbCb/50YQi3wLAXp+6U34uzwjIHKz7K4phhOsXYeiWahfct4p9OhLIEVYQ1I+P zpYw== X-Gm-Message-State: AHQUAuaMOvV0MKV7BPrVGD51OhfpzJ+2pJ76RSoQ1ymJMKscmClf3r3x WdFKfZOsNYt5YHJsRjCb5ARAOw== X-Google-Smtp-Source: AHgI3IZZr+oe2Bx/PvPXEcMkDtgr4jiDG5Cb51BBlskGB8RFekDKpRIQnpT4NDBuZGoOHmSsrKW7Zw== X-Received: by 2002:adf:9d1d:: with SMTP id k29mr3806684wre.211.1550170828313; Thu, 14 Feb 2019 11:00:28 -0800 (PST) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id y12sm1797213wmi.7.2019.02.14.11.00.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Feb 2019 11:00:26 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Pete Heist In-Reply-To: <20190214180217.13090-1-gamanakis@gmail.com> Date: Thu, 14 Feb 2019 20:00:25 +0100 Cc: toke@redhat.com, cake@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: References: <87a7iymxie.fsf@toke.dk> <20190214180217.13090-1-gamanakis@gmail.com> To: George Amanakis X-Mailer: Apple Mail (2.3445.9.1) Subject: Re: [Cake] progress? dual-src/dsthost unfairness X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2019 19:00:29 -0000 Thanks for the work on this George! That=E2=80=99s a nice step forward = for Cake=E2=80=99s fairness with bi-directional traffic. :) I=E2=80=99ll try some tests when I get a chance... > On Feb 14, 2019, at 7:02 PM, George Amanakis = wrote: >=20 > I tried Toke's and Jonathan's suggestion, dropping the > sparse_flow_count. Tthe results are the same (fairness). > In a hash collision in this patch the host_bulk_flow_count is not = updated, > does this make sense? >=20 > -------------------8<------------------- > Client A: > Data file written to ./tcp_8down-2019-02-14T115248.702453.flent.gz. > Summary of tcp_8down test run at 2019-02-14 16:52:48.702453: >=20 > avg median # data pts > Ping (ms) ICMP : 0.78 0.69 ms 341 > TCP download avg : 6.00 5.81 Mbits/s 301 > TCP download sum : 48.00 46.51 Mbits/s 301 > TCP download::1 : 6.00 5.82 Mbits/s 297 > TCP download::2 : 5.99 5.82 Mbits/s 297 > TCP download::3 : 6.00 5.82 Mbits/s 297 > TCP download::4 : 6.00 5.82 Mbits/s 297 > TCP download::5 : 6.00 5.82 Mbits/s 297 > TCP download::6 : 6.00 5.82 Mbits/s 298 > TCP download::7 : 6.01 5.82 Mbits/s 297 > TCP download::8 : 6.00 5.82 Mbits/s 298 > Data file written to ./tcp_1up-2019-02-14T115249.700445.flent.gz. > Summary of tcp_1up test run at 2019-02-14 16:52:49.700445: >=20 > avg median # data pts > Ping (ms) ICMP : 0.79 0.69 ms 341 > TCP upload : 47.69 46.33 Mbits/s 266 >=20 >=20 >=20 > Client B: > Data file written to ./tcp_1down-2019-02-14T115250.817948.flent.gz. > Summary of tcp_1down test run at 2019-02-14 16:52:50.817948: >=20 > avg median # data pts > Ping (ms) ICMP : 0.83 0.69 ms 341 > TCP download : 47.82 46.57 Mbits/s 300 > Data file written to ./tcp_8up-2019-02-14T115251.710755.flent.gz. > Summary of tcp_8up test run at 2019-02-14 16:52:51.710755: >=20 > avg median # data pts > Ping (ms) ICMP : 0.78 0.69 ms 341 > TCP upload avg : 5.99 5.79 Mbits/s 301 > TCP upload sum : 47.88 46.30 Mbits/s 301 > TCP upload::1 : 5.98 5.81 Mbits/s 224 > TCP upload::2 : 5.99 5.82 Mbits/s 224 > TCP upload::3 : 5.98 5.77 Mbits/s 219 > TCP upload::4 : 5.98 5.82 Mbits/s 224 > TCP upload::5 : 5.99 5.77 Mbits/s 218 > TCP upload::6 : 5.99 5.77 Mbits/s 221 > TCP upload::7 : 5.98 5.77 Mbits/s 219 > TCP upload::8 : 5.99 5.77 Mbits/s 221 > -------------------8<------------------- >=20 >=20 > --- > sch_cake.c | 84 +++++++++++++++++++++++++++++++++++------------------- > 1 file changed, 55 insertions(+), 29 deletions(-) >=20 > diff --git a/sch_cake.c b/sch_cake.c > index d434ae0..ed3fbd9 100644 > --- a/sch_cake.c > +++ b/sch_cake.c > @@ -146,8 +146,8 @@ struct cake_flow { > struct cake_host { > u32 srchost_tag; > u32 dsthost_tag; > - u16 srchost_refcnt; > - u16 dsthost_refcnt; > + u16 srchost_bulk_flow_count; > + u16 dsthost_bulk_flow_count; > }; >=20 > struct cake_heap_entry { > @@ -844,8 +844,6 @@ skip_hash: > * queue, accept the collision, update the host tags. > */ > q->way_collisions++; > - = q->hosts[q->flows[reduced_hash].srchost].srchost_refcnt--; > - = q->hosts[q->flows[reduced_hash].dsthost].dsthost_refcnt--; > allocate_src =3D cake_dsrc(flow_mode); > allocate_dst =3D cake_ddst(flow_mode); > found: > @@ -865,13 +863,12 @@ found: > } > for (i =3D 0; i < CAKE_SET_WAYS; > i++, k =3D (k + 1) % CAKE_SET_WAYS) { > - if (!q->hosts[outer_hash + = k].srchost_refcnt) > + if (!q->hosts[outer_hash + = k].srchost_bulk_flow_count) > break; > } > q->hosts[outer_hash + k].srchost_tag =3D = srchost_hash; > found_src: > srchost_idx =3D outer_hash + k; > - q->hosts[srchost_idx].srchost_refcnt++; > q->flows[reduced_hash].srchost =3D srchost_idx; > } >=20 > @@ -887,13 +884,12 @@ found_src: > } > for (i =3D 0; i < CAKE_SET_WAYS; > i++, k =3D (k + 1) % CAKE_SET_WAYS) { > - if (!q->hosts[outer_hash + = k].dsthost_refcnt) > + if (!q->hosts[outer_hash + = k].dsthost_bulk_flow_count) > break; > } > q->hosts[outer_hash + k].dsthost_tag =3D = dsthost_hash; > found_dst: > dsthost_idx =3D outer_hash + k; > - q->hosts[dsthost_idx].dsthost_refcnt++; > q->flows[reduced_hash].dsthost =3D dsthost_idx; > } > } > @@ -1913,20 +1909,30 @@ static s32 cake_enqueue(struct sk_buff *skb, = struct Qdisc *sch, > b->sparse_flow_count++; >=20 > if (cake_dsrc(q->flow_mode)) > - host_load =3D max(host_load, = srchost->srchost_refcnt); > + host_load =3D max(host_load, = srchost->srchost_bulk_flow_count); >=20 > if (cake_ddst(q->flow_mode)) > - host_load =3D max(host_load, = dsthost->dsthost_refcnt); > + host_load =3D max(host_load, = dsthost->dsthost_bulk_flow_count); >=20 > flow->deficit =3D (b->flow_quantum * > quantum_div[host_load]) >> 16; > } else if (flow->set =3D=3D CAKE_SET_SPARSE_WAIT) { > + struct cake_host *srchost =3D &b->hosts[flow->srchost]; > + struct cake_host *dsthost =3D &b->hosts[flow->dsthost]; > + > /* this flow was empty, accounted as a sparse flow, but = actually > * in the bulk rotation. > */ > flow->set =3D CAKE_SET_BULK; > b->sparse_flow_count--; > b->bulk_flow_count++; > + > + if (cake_dsrc(q->flow_mode)) > + srchost->srchost_bulk_flow_count++; > + > + if (cake_ddst(q->flow_mode)) > + dsthost->dsthost_bulk_flow_count++; > + > } >=20 > if (q->buffer_used > q->buffer_max_used) > @@ -2097,23 +2103,8 @@ retry: > dsthost =3D &b->hosts[flow->dsthost]; > host_load =3D 1; >=20 > - if (cake_dsrc(q->flow_mode)) > - host_load =3D max(host_load, srchost->srchost_refcnt); > - > - if (cake_ddst(q->flow_mode)) > - host_load =3D max(host_load, dsthost->dsthost_refcnt); > - > - WARN_ON(host_load > CAKE_QUEUES); > - > /* flow isolation (DRR++) */ > if (flow->deficit <=3D 0) { > - /* The shifted prandom_u32() is a way to apply dithering = to > - * avoid accumulating roundoff errors > - */ > - flow->deficit +=3D (b->flow_quantum * = quantum_div[host_load] + > - (prandom_u32() >> 16)) >> 16; > - list_move_tail(&flow->flowchain, &b->old_flows); > - > /* Keep all flows with deficits out of the sparse and = decaying > * rotations. No non-empty flow can go into the = decaying > * rotation, so they can't get deficits > @@ -2122,6 +2113,13 @@ retry: > if (flow->head) { > b->sparse_flow_count--; > b->bulk_flow_count++; > + > + if (cake_dsrc(q->flow_mode)) > + = srchost->srchost_bulk_flow_count++; > + > + if (cake_ddst(q->flow_mode)) > + = dsthost->dsthost_bulk_flow_count++; > + > flow->set =3D CAKE_SET_BULK; > } else { > /* we've moved it to the bulk rotation = for > @@ -2131,6 +2129,22 @@ retry: > flow->set =3D CAKE_SET_SPARSE_WAIT; > } > } > + > + if (cake_dsrc(q->flow_mode)) > + host_load =3D max(host_load, = srchost->srchost_bulk_flow_count); > + > + if (cake_ddst(q->flow_mode)) > + host_load =3D max(host_load, = dsthost->dsthost_bulk_flow_count); > + > + WARN_ON(host_load > CAKE_QUEUES); > + > + /* The shifted prandom_u32() is a way to apply dithering = to > + * avoid accumulating roundoff errors > + */ > + flow->deficit +=3D (b->flow_quantum * = quantum_div[host_load] + > + (prandom_u32() >> 16)) >> 16; > + list_move_tail(&flow->flowchain, &b->old_flows); > + > goto retry; > } >=20 > @@ -2151,6 +2165,13 @@ retry: > &b->decaying_flows); > if (flow->set =3D=3D CAKE_SET_BULK) { > b->bulk_flow_count--; > + > + if (cake_dsrc(q->flow_mode)) > + = srchost->srchost_bulk_flow_count--; > + > + if (cake_ddst(q->flow_mode)) > + = dsthost->dsthost_bulk_flow_count--; > + > b->decaying_flow_count++; > } else if (flow->set =3D=3D = CAKE_SET_SPARSE || > flow->set =3D=3D = CAKE_SET_SPARSE_WAIT) { > @@ -2164,14 +2185,19 @@ retry: > if (flow->set =3D=3D CAKE_SET_SPARSE || > flow->set =3D=3D = CAKE_SET_SPARSE_WAIT) > b->sparse_flow_count--; > - else if (flow->set =3D=3D CAKE_SET_BULK) > + else if (flow->set =3D=3D CAKE_SET_BULK) = { > b->bulk_flow_count--; > - else > + > + if (cake_dsrc(q->flow_mode)) > + = srchost->srchost_bulk_flow_count--; > + > + if (cake_ddst(q->flow_mode)) > + = dsthost->dsthost_bulk_flow_count--; > + > + } else > b->decaying_flow_count--; >=20 > flow->set =3D CAKE_SET_NONE; > - srchost->srchost_refcnt--; > - dsthost->dsthost_refcnt--; > } > goto begin; > } > --=20 > 2.20.1 >=20 > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake