From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-x22e.google.com (mail-oi0-x22e.google.com [IPv6:2607:f8b0:4003:c06::22e]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id D510421F36B for ; Sun, 14 Jun 2015 10:39:07 -0700 (PDT) Received: by oihd6 with SMTP id d6so46384879oih.2 for ; Sun, 14 Jun 2015 10:39:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=OKYAVHE9iWPwKiqiIqA7g1V7Re9XamdEY6eDGbTG6cQ=; b=UnAyqIx5kC2r/o4oq+RWykJMPsWGlnAsiNEQYzAtLdFJRBeD8Fq7GF7d7xc7DuZiiB fLWYEsvAUwRvfcvOJYs5zAPxU8Lm6eONKEPtOlFfORyjzLryrB22WOcOgmhMyuzRqspx iEAok/TH44tiUQpuv7MbUO9kKwBze+C8EcLkpU+yPCo27o3+ZYbsx58zxmVfW5/im2oA OH/hcFiW1i4fPBX8Z1JhRGXNcXtqmk1VE6kImVfF8BYRgnR0XxcnwPpjK0pihgB1c0vv 58J40oIkfjrMKnQZtHtTRQlm2if4VkxTvr/pwbzXuBqMObJ/ysAcbip22NlHknzAii/M RXvg== MIME-Version: 1.0 X-Received: by 10.182.214.37 with SMTP id nx5mr20475425obc.56.1434303546826; Sun, 14 Jun 2015 10:39:06 -0700 (PDT) Received: by 10.202.105.129 with HTTP; Sun, 14 Jun 2015 10:39:06 -0700 (PDT) In-Reply-To: References: Date: Sun, 14 Jun 2015 10:39:06 -0700 Message-ID: From: Dave Taht To: bloat Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: [Bloat] Fwd: performance testing on the WRT1200AC X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jun 2015 17:39:36 -0000 a wider audience for the issues in new consumer hardware seems desirable. forwarding with permission. ---------- Forwarded message ---------- From: Dave Taht Date: Sun, Jun 14, 2015 at 8:41 AM Subject: Re: performance testing on the WRT1200AC To: Mikael Abrahamsson , Aaron Wood Dear Mikael: netperf-wrapper has been renamed to flent. :) Quite a bit of new stuff is dropping into it, one of my favorite tests is the new qdisc_stats test (which I run at the same time as another test). It hasn't been tested on a multi-queue interface (and doesn't work with openwrt's sh implementation dang it). But do a pull anyway. :) On Sun, Jun 14, 2015 at 8:18 AM, Mikael Abrahamsson wrot= e: > > Hi, > > I want to do some more demanding testing of the WRT1200AC. Currently it's > running a few days old openwrt CC. It comes with the below qdisc setting.= I > will be testing it using the following setup: > > linux-switch-wrt1200ac-linux > > All links above are gigabit ethernet links. > > My plan is to for instance run netperf-wrapper with a few different tests= . > > Would it strain the WRT1200AC if I configured it to shape to 900 megabit/= s > bidirectionallty? I guess in order to actually achieve a little bit of My original tests with the 1900AC showed htb peaking out with sqm + offloads at about 550/650mbit on the rrul test. (I can't remember if nat was on or off, but I think off) but that was months ago. I have a huge hope that cake will do better on this platform and recently (yesterday) I think got that to the point where we could push it to openwrt to be built regularly. Aaron, cc'd, has done quite a bit of work with the 1900, and I think he started running into trouble at 200mbit. > buffering, I'm going to have to run below wirespeed? Because I can't get > more than 1 gigabit/s of traffic to the wrt1200ac because of above layout= , > so doing bidirectional shaping to 900 on eth0 (WAN PORT) would at least g= ive > it a bit more to do and also give a chance to induce some buffering? Ain't it a bitch? A thought would be to also exercise the wifi a bit to drive it past gigE overall. So have two clients running flent tests simultaneously, one on wifi, one on ethernet, and there you go, driving it into overload. > Do you have some other ideas for testing? I am mostly interested in makin= g > sure the CPU is fast enough to do AQM at gig speeds... Well, there are other issues. A) The mvneta ethernet driver in the 1900 did not support BQL when last I looked, supplying insufficient backpressure to the upper layers. B) The multiqueued hardware applies a bit of fq for you automagically, BUT, even if BQL was in place, BQL's buffering is additive per hardware queue, so it tends to what I saw was nearly no drops in the qdisc. I don't think I even saw maxpacket grow (a sure sign you are backlogging in the qdisc) I ended up disabling the hardware mq multiqueue[1] stuff entirely by "tc qdisc add dev eth0 root fq_codel", and even then, see A) - but I did finally see maxpacket grow... C) to realize to my horror that they had very aggressively implemented GRO for everything, giving us 64k "packets" to deal with coming in from the gigE ethernet... which interacted rather badly with the 10Mbit outgoing interface I had at the time. and that explained why nearly all the QoS systems as deployed in this generation of router were doing so badly... which led to a change in codel's control law (upstream in linux, not sure in openwrt), and ultimately frantic activity in cake to do peeling apart of superpackets like that. I applaud further testing, and I would love it if you could verify that the GRO problem remains and that it's hard to get sufficient backpressure (and latencies should grow a lot) when driven with wifi+ ethernet On simple single threaded up or down tests I was able to get full gigE throughput out of the 1900's wan interface, but disabling offloads was quite damaging, as was mixed traffic like rrul_50 up, which makes GRO far less effective. I wish I had time to go and add BQL. I requested it of the author, no respo= nse. > > root@OpenWrt:/tmp# tc qdisc tc -s qdisc show # -s is more revealing > qdisc mq 0: dev eth0 root > qdisc fq_codel 0: dev eth0 parent :1 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth0 parent :2 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth0 parent :3 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth0 parent :4 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth0 parent :5 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth0 parent :6 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth0 parent :7 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth0 parent :8 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc mq 0: dev eth1 root > qdisc fq_codel 0: dev eth1 parent :1 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth1 parent :2 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth1 parent :3 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth1 parent :4 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth1 parent :5 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth1 parent :6 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth1 parent :7 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth1 parent :8 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc mq 0: dev wlan0 root > qdisc fq_codel 0: dev wlan0 parent :1 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev wlan0 parent :2 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev wlan0 parent :3 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev wlan0 parent :4 limit 1024p flows 1024 quantum 300 > target 5.0ms interval 100.0ms ecn > > > -- > Mikael Abrahamsson email: swmike@swm.pp.se -- Dave T=C3=A4ht What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast --=20 Dave T=C3=A4ht What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast