[Make-wifi-fast] Software rate limiting with fq_codel for point-to-point WiFi backhaul links

[Phineas Gage]

Given the half-duplex nature of 802.11 WiFi, is it possible to use fq_codel with software rate limiting on separate hardware from the WiFi radio, while still allowing at or near the full WiFi link rate?

Note #1: This is probably more of a job for the new "Make WiFi Fast" code in LEDE, but so far I can’t run that in the environment I’m working in (more below).

Note #2: It also makes me think, I wonder how this problem is handled in the new Make WiFi Fast code, as that could be another big advantage for it, if the "half duplex rate limiting” I’m thinking of can’t be done in Linux (more below).

Note #3: This question may not be specific to fq_codel, or even WiFi, as it could apply to any half duplex physical media, with any queueing discipline, but this list seems more active than most. :)

How I got here: I arrived at this question because I started doing Flent RRUL runs over a local WiFi link and fq_codel was struggling to get latency down. I was using HTB to rate limit both input and output to 85 Mbps, which is a little less than the WiFi’s one-way throughput as reported by iperf3 (95 Mbps). Then it dawned on me that 802.11 WiFi is half duplex, so I need to limit the _combined_ rate. And indeed, simultaneous iperf3 throughput tests in both directions add up to a total combined rate of around 95 Mbps. I set the HTB input and output rate to 40 Mbps each, and average latency under load during an RRUL test dropped right away from ~30ms to ~5ms, and looked smooth, so I knew I was finally in control of the queue. 

New Problem: But now that my latency is reduced, I have a new problem. I’m rate limiting to around half my possible link rate in each direction, so if I’m only doing a download or upload on the link, I’m losing around half my potential throughput! That led to my question. Is it even possible to gain this throughput back without losing control of the queue? I would need the queueing in Linux to be aware that the physical media is half duplex, and rate limit the _total_ input and output traffic, not limit the input and output individually. Let’s call it “half duplex rate limiting.” AFAIK there’s no way to do this with IFB or Linux traffic control in general. Is there?

Background: I’ll back up a bit and add some background on what I’m doing, for those interested. I’m offering to help my community WISP try to reduce latency in their network by deploying fq_codel where possible. For those who saw my earlier WiFi related post, that was unrelated and on a different site, where fq_codel on the edge router did improve latency there. For this WISP however, since we can’t practically install and manage edge routers with fq_codel at every end user site, I’m doing some research to find out if fq_codel can at least help on the internal point-to-point backhaul links that connect their nodes. Currently, sfq is used for those. Unlike a typical edge router, which limits to “Internet speed”, these backhaul links usually operate at or near the maximum link rate of the WiFi hardware. So it’s not like a typical CPE device whose link rate might be well above the observed Internet rate, making the “half duplex problem” irrelevant, if that makes sense.

The WISP serves around 800 members, and is built with a combination of fiber and point-to-point WiFi links of various speeds and frequencies. Here’s a map:

http://mapa.czfree.net/#lat=50.76816800116274&lng=15.066890716552734&zoom=13&autofilter=1&type=k&geolocate=98%7C114%7C111%7C117%7C109%7C111%7C118%7C115%7C107%7C97&node=6101&aponly=1&bbonly=1&actlink=1&actnode=1& <http://mapa.czfree.net/#lat=50.76816800116274&lng=15.066890716552734&zoom=13&autofilter=1&type=k&geolocate=98|114|111|117|109|111|118|115|107|97&node=6101&aponly=1&bbonly=1&actlink=1&actnode=1&>

Most of the nodes use Ubiquiti hardware for the radios and custom hardware for the routers, connected to the radios via Ethernet, and most of which run on 1 GHz AMD G-T40E processors and run a customized distribution of Voyage Linux 0.10 (Debian 8 based distribution for embedded hardware). The distro ships with kernel 3.16.7, which does have a working sch_fq_codel module. To get oriented, I started with some Flent RRUL runs on my test setup below, and that’s where I ran into this question. Next, I’ll test on some actual hardware targeted for deployment to understand its fq_codel throughput limits, but not before I understand this “half duplex problem” better.

Test setup:

PC #1  <->  WiFi (802.11n, 130 Mbps Tx Rate, RSSI -60)  <->  AirPort Express  <->  100Mbit Ethernet  <->  PC #2

PC #1: Runs a Voyage Linux VM (more on that later), with the Flent client and HTB + fq_codel, and is connected via the PC's WiFi adapter to the Airport Express.

PC #2: Runs netserver for flent to connect to, and is connected via Ethernet to the Airport Express.

Thanks for any input.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/make-wifi-fast/attachments/20161209/730a5508/attachment.html>