From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x241.google.com (mail-wm0-x241.google.com [IPv6:2a00:1450:400c:c09::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 7F87A3B2A2; Sun, 11 Dec 2016 06:54:45 -0500 (EST) Received: by mail-wm0-x241.google.com with SMTP id m203so4797999wma.3; Sun, 11 Dec 2016 03:54:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date:cc:message-id:references :to; bh=TZMu1B3QMlHilnDz23ssD1qrFnidBWviF3bFdncqgVU=; b=uZ573Gx6CJPOLh7LegHvMqXhzEmPvZdm+EjNAAYqDhl2iMckrSAXOkIFfG0EZSJj7p mhH16wTKfZd9KqzZZoBna3V+tyZlNmRC+MrO+66jNfD8wAcrReIwxw1L2k/KCyh/ohrV 1WhX2IbjR6OdvngMa/CQCCpaPrpNTrKpvmnwyL2+3XXiDP0w/tRrLfkK+3iYwnMbRfjp 0HlZH/VgvLZlf+DYEV4+gQkrCnjZRe0gkfQKk4ZitkkeH4xxsp9BzWso2NreOY1hTBM4 7z9RAWwJsC2vL/yfOz0+rhn/IrOev/oCaFptVcj7Cz1sHp/Nqdr7UlLn9WE/qE4tNd7V iu/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=TZMu1B3QMlHilnDz23ssD1qrFnidBWviF3bFdncqgVU=; b=mfUDKHeZrpE7PjUsBQI5KtvIO/KWbKuqQeYC5VajVVujjVXDFbGh3zAaDLSN64rnyU hM+2ZmQfkAhDRZUSLmRBq4ynwFHhpTrhwiWn+/Y/FU7BWkbrcsl6GU0q8MR01suOJ+KF gWH+VPzArCkdo2s6szCrvtO/u1dUiBDD6novxLdJkvoLRXHo5Rzs5zExItlzqKQ4wr+1 axUkpc//27GZJZ3assXkyEVfEat4HpOKeImX7CHbLZjyC0CJkjctwHXnuuc2H0b8vQIm fyVgRdZpPluoyt6DaBPTdB/jJd15LHAc5X+2TgyPJ0H6zso0WWZK5kbyzZgfOGHKMXR/ 4TFA== X-Gm-Message-State: AKaTC03oHqLXROU5o/hBhYRnTklJuuOkXDx4JVUhNDRevCuc4I0IPQ4snviS6f+Gx48s2A== X-Received: by 10.28.209.67 with SMTP id i64mr14383912wmg.48.1481457284454; Sun, 11 Dec 2016 03:54:44 -0800 (PST) Received: from [10.72.0.20] (h-1169.lbcfree.net. [185.99.119.68]) by smtp.gmail.com with ESMTPSA id x7sm7855658wjp.18.2016.12.11.03.54.43 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 11 Dec 2016 03:54:44 -0800 (PST) Content-Type: multipart/alternative; boundary="Apple-Mail=_1F85D268-D8B1-499F-9A35-082242137CC4" Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Phineas Gage In-Reply-To: <5EA3C6E6-76C7-4347-9B14-3C310271A8FD@gmail.com> Date: Sun, 11 Dec 2016 12:54:43 +0100 Cc: codel@lists.bufferbloat.net, make-wifi-fast@lists.bufferbloat.net Message-Id: <6C732BCD-1069-40FF-AEAC-A5A0683945D6@gmail.com> References: <9BA0F7D8-5A25-4033-A101-38BF001B74EB@gmail.com> <5EA3C6E6-76C7-4347-9B14-3C310271A8FD@gmail.com> To: Jonathan Morton X-Mailer: Apple Mail (2.3124) Subject: Re: [Make-wifi-fast] Software rate limiting with fq_codel for point-to-point WiFi backhaul links X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 11:54:45 -0000 --Apple-Mail=_1F85D268-D8B1-499F-9A35-082242137CC4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Thanks for the hint Jonathon. In general, this appears to be effective, = at least for the flent rrul test. Below are flent rrul_noclassification run graphs (flent.gz files = available) and the script I used to set up fq_codel. I see close to link = rate with iperf3 tests, which test throughput in only one direction, yet = can still control the queue reasonably well when there=E2=80=99s = bidirectional traffic. For example, I=E2=80=99ve got the total rate = limited to 90 Mbit, and each average flow in a flent rrul test runs = around 10 Mbit. There are four in each direction, eight total, so I see = around 80 Mbit of TCP throughput along with the low rate traffic at = around 6 ms latency, which is not bad. If you set up fq_codel like this on both ends of the link, it doesn=E2=80=99= t perform quite as well- latency goes up and throughput down a bit. = There are probably some "unwanted interactions" when you do this. So = it=E2=80=99s best to set it up on only end end of the link, probably the = end whose egress heads towards the Internet if applicable, but I=E2=80=99m= still thinking that through. Again, for those reading this later, it would be best to just run = fq_codel (or Cake but I=E2=80=99ll get to that next) on the egress right = on the hardware with the WiFi interface attached. The interface will = rate limit the queue automatically when there=E2=80=99s bidirectional = traffic, and account for variable speed, so that should be a superior = configuration. But this is a solution for when that isn=E2=80=99t = possible, and the AQM has to be run on separate hardware, connected via = Ethernet. I still want to do more testing for my WISP, both with Cake and the new = LEDE builds as well, in various configurations, so I hope to do that = next and will start a new thread when there are results. For now, this = =E2=80=9Chalf duplex rate limiting=E2=80=9D by running AQM on a common = IFB interface may help people in some situations! = https://dl.dropboxusercontent.com/u/71551878/bufferbloat/fq_codel_90mbit.p= ng = = https://dl.dropboxusercontent.com/u/71551878/bufferbloat/fq_codel_90mbit_b= oth.png = https://dl.dropboxusercontent.com/u/71551878/bufferbloat/qos.sh = > On Dec 9, 2016, at 1:37 PM, Phineas Gage wrote: >=20 > Ok, I had not realized that, thanks. :) >=20 > I=E2=80=99ve not seen this done anywhere, has anyone tried it? = Otherwise I=E2=80=99ll give it a try and write back what I find. >=20 > In this case, the throughput for the backhaul links =E2=80=9Cshould=E2=80= =9D be mostly stable, and we=E2=80=99ll just accept any variation as = =E2=80=9Cno worse than before=E2=80=9D. >=20 > It's true, I also want to try Cake (anywhere I wrote fq_codel that = could be substituted with Cake), and I see from here = (https://www.bufferbloat.net/projects/codel/wiki/Cake/#installing-cake-out= -of-tree-on-linux = ) that it should work on the 3.16.7 kernel I need to = target. Voyage Linux doesn=E2=80=99t install with kernel sources, but I = should be able to get that compiled with their SDK. >=20 >> On Dec 9, 2016, at 12:39 PM, Jonathan Morton > wrote: >>=20 >>> On 9 Dec, 2016, at 12:12, Phineas Gage > wrote: >>>=20 >>> Given the half-duplex nature of 802.11 WiFi, is it possible to use = fq_codel with software rate limiting on separate hardware from the WiFi = radio, while still allowing at or near the full WiFi link rate? >>=20 >> Given that you can=E2=80=99t reliably predict the actual wifi = throughput from userspace, and that it will vary over time due to = external interference and path attenuation, that would be difficult. >>=20 >> However, you *can* loop both the ingress and egress traffic through a = common IFB interface, and shape that - using Cake, even. That sounds = like what you=E2=80=99re trying to experiment with. >>=20 >> - Jonathan Morton >>=20 >=20 --Apple-Mail=_1F85D268-D8B1-499F-9A35-082242137CC4 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
Thanks for the hint Jonathon. In general, = this appears to be effective, at least for the flent rrul = test.

Below = are flent rrul_noclassification run graphs (flent.gz files available) = and the script I used to set up fq_codel. I see close to link rate with = iperf3 tests, which test throughput in only one direction, yet can still = control the queue reasonably well when there=E2=80=99s bidirectional = traffic. For example, I=E2=80=99ve got the total rate limited to 90 = Mbit, and each average flow in a flent rrul test runs around 10 Mbit. = There are four in each direction, eight total, so I see around 80 Mbit = of TCP throughput along with the low rate traffic at around 6 ms = latency, which is not bad.

If you set up fq_codel like this on both ends of the link, it = doesn=E2=80=99t perform quite as well- latency goes up and throughput = down a bit. There are probably some "unwanted interactions" when you do = this. So it=E2=80=99s best to set it up on only end end of the link, = probably the end whose egress heads towards the Internet if applicable, = but I=E2=80=99m still thinking that through.

Again, for those reading this later, it = would be best to just run fq_codel (or Cake but I=E2=80=99ll get to that = next) on the egress right on the hardware with the WiFi interface = attached. The interface will rate limit the queue automatically when = there=E2=80=99s bidirectional traffic, and account for variable speed, = so that should be a superior configuration. But this is a solution for = when that isn=E2=80=99t possible, and the AQM has to be run on separate = hardware, connected via Ethernet.

I still want to do more testing for my = WISP, both with Cake and the new LEDE builds as well, in various = configurations, so I hope to do that next and will start a new thread = when there are results. For now, this =E2=80=9Chalf duplex rate = limiting=E2=80=9D by running AQM on a common IFB interface may help = people in some situations!

https://dl.dropboxusercontent.com/u/71551878/bufferbloat/fq_cod= el_90mbit.png

https://dl.dropboxusercontent.com/u/71551878/bufferbloat/fq_cod= el_90mbit_both.png

https://dl.dropboxusercontent.com/u/71551878/bufferbloat/qos.sh=

On Dec 9, 2016, at 1:37 PM, = Phineas Gage <phineas919@gmail.com> wrote:

Ok, I had not realized that, thanks. :)

I=E2=80=99ve not seen = this done anywhere, has anyone tried it? Otherwise I=E2=80=99ll give it = a try and write back what I find.

In this case, the throughput for the = backhaul links =E2=80=9Cshould=E2=80=9D be mostly stable, and we=E2=80=99l= l just accept any variation as =E2=80=9Cno worse than = before=E2=80=9D.

It's true, I also want to try Cake (anywhere I wrote fq_codel = that could be substituted with Cake), and I see from here (https://www.bufferbloat.net/projects/codel/wiki/Cake/#installin= g-cake-out-of-tree-on-linux) that it should work on the 3.16.7 = kernel I need to target. Voyage Linux doesn=E2=80=99t install with = kernel sources, but I should be able to get that compiled with their = SDK.

On Dec 9, 2016, at 12:39 PM, Jonathan Morton = <chromatix99@gmail.com> wrote:

On 9 Dec, = 2016, at 12:12, Phineas Gage <phineas919@gmail.com> wrote:

Given the half-duplex nature of 802.11 WiFi, is it possible = to use fq_codel with software rate limiting on separate hardware from = the WiFi radio, while still allowing at or near the full WiFi link = rate?

Given that you can=E2=80=99= t reliably predict the actual wifi throughput from userspace, and that = it will vary over time due to external interference and path = attenuation, that would be difficult.

However, you *can* loop both the ingress and egress traffic = through a common IFB interface, and shape that - using Cake, even. =  That sounds like what you=E2=80=99re trying to experiment with.

- Jonathan Morton



= --Apple-Mail=_1F85D268-D8B1-499F-9A35-082242137CC4--