[Bloat] Fwd: performance testing on the WRT1200AC

[Dave Taht]

a wider audience for the issues in new consumer hardware seems desirable.

forwarding with permission.


---------- Forwarded message ----------
From: Dave Taht <dave.taht at gmail.com>
Date: Sun, Jun 14, 2015 at 8:41 AM
Subject: Re: performance testing on the WRT1200AC
To: Mikael Abrahamsson <swmike at swm.pp.se>, Aaron Wood <woody77 at gmail.com>


Dear Mikael:

netperf-wrapper has been renamed to flent. :) Quite a bit of new stuff
is dropping into it, one of my favorite tests is the new qdisc_stats
test (which I run at the same time as another test). It hasn't been
tested on a multi-queue interface (and doesn't work with openwrt's sh
implementation dang it). But do a pull anyway. :)

On Sun, Jun 14, 2015 at 8:18 AM, Mikael Abrahamsson <swmike at swm.pp.se> wrote:
>
> Hi,
>
> I want to do some more demanding testing of the WRT1200AC. Currently it's
> running a few days old openwrt CC. It comes with the below qdisc setting. I
> will be testing it using the following setup:
>
> linux-switch-wrt1200ac-linux
>
> All links above are gigabit ethernet links.
>
> My plan is to for instance run netperf-wrapper with a few different tests.
>
> Would it strain the WRT1200AC if I configured it to shape to 900 megabit/s
> bidirectionallty? I guess in order to actually achieve a little bit of

My original tests with the 1900AC showed htb peaking out with sqm +
offloads at about 550/650mbit on the rrul test. (I can't remember if
nat was on or off, but I think off)

but that was months ago. I have a huge hope that cake will do better
on this platform and recently (yesterday) I think got that to the
point where we could push it to openwrt to be built regularly.

Aaron, cc'd, has done quite a bit of work with the 1900, and I think
he started running into trouble at 200mbit.

> buffering, I'm going to have to run below wirespeed? Because I can't get
> more than 1 gigabit/s of traffic to the wrt1200ac because of above layout,
> so doing bidirectional shaping to 900 on eth0 (WAN PORT) would at least give
> it a bit more to do and also give a chance to induce some buffering?

Ain't it a bitch? A thought would be to also exercise the wifi a bit
to drive it past gigE overall. So have two clients running flent tests
simultaneously, one on wifi, one on ethernet, and there you go,
driving it into overload.

> Do you have some other ideas for testing? I am mostly interested in making
> sure the CPU is fast enough to do AQM at gig speeds...

Well, there are other issues.

A) The mvneta ethernet driver in the 1900 did not support BQL when
last I looked, supplying insufficient backpressure to the upper
layers.

B) The multiqueued hardware applies a bit of fq for you automagically,
BUT, even if BQL was in place, BQL's buffering is additive per
hardware queue, so it tends to

what I saw was nearly no drops in the qdisc. I don't think I even saw
maxpacket grow (a sure sign you are backlogging in the qdisc) I ended
up disabling the hardware mq multiqueue[1] stuff entirely by "tc qdisc
add dev eth0 root fq_codel", and even then, see A) - but I did finally
see maxpacket grow...

C) to realize to my horror that they had very aggressively implemented
GRO for everything, giving us 64k "packets" to deal with coming in
from the gigE ethernet... which interacted rather badly with the
10Mbit outgoing interface I had at the time.

and that explained why nearly all the QoS systems as deployed in this
generation of router were doing so badly...

which led to a change in codel's control law (upstream in linux, not
sure in openwrt), and ultimately frantic activity in cake to do
peeling apart of superpackets like that.

I applaud further testing, and I would love it if you could verify
that the GRO problem remains and that it's hard to get sufficient
backpressure (and latencies should grow a lot) when driven with wifi+
ethernet

On simple single threaded up or down tests I was able to get full gigE
throughput out of the 1900's wan interface, but disabling offloads was
quite damaging, as was mixed traffic like rrul_50 up, which makes GRO
far less effective.

I wish I had time to go and add BQL. I requested it of the author, no response.

>
> root at OpenWrt:/tmp# tc qdisc

tc -s qdisc show # -s is more revealing

> qdisc mq 0: dev eth0 root
> qdisc fq_codel 0: dev eth0 parent :1 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth0 parent :2 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth0 parent :3 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth0 parent :4 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth0 parent :5 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth0 parent :6 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth0 parent :7 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth0 parent :8 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc mq 0: dev eth1 root
> qdisc fq_codel 0: dev eth1 parent :1 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth1 parent :2 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth1 parent :3 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth1 parent :4 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth1 parent :5 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth1 parent :6 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth1 parent :7 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth1 parent :8 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc mq 0: dev wlan0 root
> qdisc fq_codel 0: dev wlan0 parent :1 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan0 parent :2 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan0 parent :3 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan0 parent :4 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
>
>
> --
> Mikael Abrahamsson    email: swmike at swm.pp.se



--
Dave Täht
What will it take to vastly improve wifi for everyone?
https://plus.google.com/u/0/explore/makewififast


-- 
Dave Täht
What will it take to vastly improve wifi for everyone?
https://plus.google.com/u/0/explore/makewififast