Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; "cake overhead 40" didn't)

[Dave Taht <dave.taht@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2399 bytes --]

On Mon, Jun 29, 2015 at 6:42 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
> HI Mikael,, hi Jonathan,
>
>> [...]
>>
>> These are the results from 50M and 500M, also including 50up and 50down that I added to my test suite script.
>>
>> http://swm.pp.se/aqm/rrul_150629-cake-4.tar
>>
>
> Now both ingress and egress are up to roughly 455Mbps from roughly 360 with cake just playing leaf qdisc for HTB. This looks even better than before…

350 *usec* induced delay on the 50mbit rrul_be test. w00t!

Most of the tests compare well with the reference rangeley data now. I
would like a 900mbit soft shaped result.

1.2ms at 500mbit. Less of a w00t. Possible it is coming from elsewhere
on that path (fq or fq_codel on the server and client?)

cake currently peels at 1ms / flows (more or less)... NAPI is an
issue... hw mq an issue...

There are a half dozen things in the mvneta driver I would try to
reduce it's latency more. The easy ones:

reduce this to 16:

        netif_napi_add(dev, &pp->napi, mvneta_poll, NAPI_POLL_WEIGHT);

Reduce this to 24: (this will also reduce the max outstanding stuff in
the driver by  a LOT, but is still not BQL!)

/* Max number of allowed TCP segments for software TSO */
#define MVNETA_MAX_TSO_SEGS 100

Both of the will improve read side latency at the cost of more sirqs.

I do not know what reducing these will do, and would test both of the
above separately.

/* Coalescing */
#define MVNETA_TXDONE_COAL_PKTS         1
#define MVNETA_RX_COAL_PKTS             32
#define MVNETA_RX_COAL_USEC             100

As for cake itself, eric dumazet told us we dont need atomic ops in it,
and peeling at at even lower threshold has some appeal (to me, anyway)

attached is a patch for that, put it in your feeds/cero/kmod_sched_cake/patches
directory, rebuild (make package/kmod-sched-cake/{clean,compile,install})

(bump up the makefile rel number also, if you want)





> Best Regards
>         Sebastian
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel



-- 
Dave Täht
worldwide bufferbloat report:
http://www.dslreports.com/speedtest/results/bufferbloat
And:
What will it take to vastly improve wifi for everyone?
https://plus.google.com/u/0/explore/makewififast

[-- Attachment #2: 0001-Rid-unneeded-atomic-ops-and-reduce-peeling-threshold.patch --]
[-- Type: text/x-patch, Size: 2510 bytes --]

From 46be609e95474e9db856b5e12756d4a7568adf42 Mon Sep 17 00:00:00 2001
From: Dave Taht <dave.taht@bufferbloat.net>
Date: Mon, 29 Jun 2015 09:38:00 -0700
Subject: [PATCH] Rid unneeded atomic ops and reduce peeling threshold

---
 sch_cake.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/sch_cake.c b/sch_cake.c
index 80e1cb2..9a358b9 100644
--- a/sch_cake.c
+++ b/sch_cake.c
@@ -121,7 +121,7 @@ struct cake_fqcd_sched_data {
 
     struct codel_params cparams;
     u32      drop_overlimit;
-    atomic_t flow_count;
+    u32 flow_count;
 
     struct list_head new_flows; /* list of new flows */
     struct list_head old_flows; /* list of old flows */
@@ -427,7 +427,7 @@ static int cake_enqueue(struct sk_buff *skb, struct Qdisc *sch)
 	 * Split GSO aggregates if they're likely to impair flow isolation
 	 * or if we need to know individual packet sizes for framing overhead.
 	 */
-	if(unlikely((len * max(atomic_read(&fqcd->flow_count), 1)) > q->peel_threshold && skb_is_gso(skb)))
+	if(unlikely((len * max(&fqcd->flow_count, 1)) > q->peel_threshold && skb_is_gso(skb)))
 	{
 		struct sk_buff *segs, *nskb;
 		netdev_features_t features = netif_skb_features(skb);
@@ -477,7 +477,7 @@ static int cake_enqueue(struct sk_buff *skb, struct Qdisc *sch)
 	/* flowchain */
 	if(list_empty(&flow->flowchain)) {
 		list_add_tail(&flow->flowchain, &fqcd->new_flows);
-		atomic_inc(&fqcd->flow_count);
+		fqcd->flow_count+=1;
 		flow->deficit = fqcd->quantum;
 		flow->dropped = 0;
 	}
@@ -615,7 +615,7 @@ retry:
 			list_move_tail(&flow->flowchain, &fqcd->old_flows);
 		} else {
 			list_del_init(&flow->flowchain);
-			atomic_dec(&fqcd->flow_count);
+			fqcd->flow_count-=1;
 		}
 		goto begin;
 	}
@@ -966,7 +966,7 @@ static void cake_reconfigure(struct Qdisc *sch)
 		if(q->buffer_limit < 65536)
 			q->buffer_limit = 65536;
 
-		q->peel_threshold = (q->rate_flags & CAKE_FLAG_ATM) ? 0 : min(65535U, q->rate_bps >> 10);
+		q->peel_threshold = (q->rate_flags & CAKE_FLAG_ATM) ? 0 : min(65535U, q->rate_bps >> 12);
 	} else {
 		q->buffer_limit = 1 << 20;
 		q->peel_threshold = 0;
@@ -1083,7 +1083,7 @@ static int cake_init(struct Qdisc *sch, struct nlattr *opt)
 		fqcd->perturbation = prandom_u32();
 		INIT_LIST_HEAD(&fqcd->new_flows);
 		INIT_LIST_HEAD(&fqcd->old_flows);
-		atomic_set(&fqcd->flow_count, 0);
+		fqcd->flow_count = 0;
 		/* codel_params_init(&fqcd->cparams); */
 
 		fqcd->flows    = cake_zalloc(fqcd->flows_cnt * sizeof(struct cake_fqcd_flow));
-- 
1.9.1