[Codel] [Make-wifi-fast] Software rate limiting with fq_codel for point-to-point WiFi backhaul links

[David Lang]

If you are able to make the settings on the outgoing side of both the ends of 
the link, the need to estimate bandwidth should pretty much go away.

The whole mess of estimating bandwidth and throttling to keep below that 
estimate is only needed when the device(s) that you have control over are not 
the ones directly adjacent to the bottleneck. If you are directly adjacent to 
the bottleneck, then you don't need to guess how much data is too much, you see 
it directly by watching the queues build up (at which point, fq_codel kicks in 
and forces things to back off)

you really want to be running fq_codel on the 'radios', that's the bottleneck 
point as you shift from ethernet to wifi.

I would guess that the airtime fairness work probably won't make a huge 
difference in your case as the backhaul networks are all operating at or near 
the same speeds, but it may be that it will help by better grouping traffic into 
transmission bursts.

David Lang

On Fri, 9 Dec 2016, Phineas Gage wrote:

> Given the half-duplex nature of 802.11 WiFi, is it possible to use fq_codel with software rate limiting on separate hardware from the WiFi radio, while still allowing at or near the full WiFi link rate?
>
> Note #1: This is probably more of a job for the new "Make WiFi Fast" code in LEDE, but so far I can’t run that in the environment I’m working in (more below).
>
> Note #2: It also makes me think, I wonder how this problem is handled in the new Make WiFi Fast code, as that could be another big advantage for it, if the "half duplex rate limiting” I’m thinking of can’t be done in Linux (more below).
>
> Note #3: This question may not be specific to fq_codel, or even WiFi, as it could apply to any half duplex physical media, with any queueing discipline, but this list seems more active than most. :)
>
> How I got here: I arrived at this question because I started doing Flent RRUL runs over a local WiFi link and fq_codel was struggling to get latency down. I was using HTB to rate limit both input and output to 85 Mbps, which is a little less than the WiFi’s one-way throughput as reported by iperf3 (95 Mbps). Then it dawned on me that 802.11 WiFi is half duplex, so I need to limit the _combined_ rate. And indeed, simultaneous iperf3 throughput tests in both directions add up to a total combined rate of around 95 Mbps. I set the HTB input and output rate to 40 Mbps each, and average latency under load during an RRUL test dropped right away from ~30ms to ~5ms, and looked smooth, so I knew I was finally in control of the queue.
>
> New Problem: But now that my latency is reduced, I have a new problem. I’m rate limiting to around half my possible link rate in each direction, so if I’m only doing a download or upload on the link, I’m losing around half my potential throughput! That led to my question. Is it even possible to gain this throughput back without losing control of the queue? I would need the queueing in Linux to be aware that the physical media is half duplex, and rate limit the _total_ input and output traffic, not limit the input and output individually. Let’s call it “half duplex rate limiting.” AFAIK there’s no way to do this with IFB or Linux traffic control in general. Is there?
>
> Background: I’ll back up a bit and add some background on what I’m doing, for 
> those interested. I’m offering to help my community WISP try to reduce latency 
> in their network by deploying fq_codel where possible. For those who saw my 
> earlier WiFi related post, that was unrelated and on a different site, where 
> fq_codel on the edge router did improve latency there. For this WISP however, 
> since we can’t practically install and manage edge routers with fq_codel at 
> every end user site, I’m doing some research to find out if fq_codel can at 
> least help on the internal point-to-point backhaul links that connect their 
> nodes. Currently, sfq is used for those. Unlike a typical edge router, which 
> limits to “Internet speed”, these backhaul links usually operate at or near 
> the maximum link rate of the WiFi hardware. So it’s not like a typical CPE 
> device whose link rate might be well above the observed Internet rate, making 
> the “half duplex problem” irrelevant, if that makes sense.
>
> The WISP serves around 800 members, and is built with a combination of fiber and point-to-point WiFi links of various speeds and frequencies. Here’s a map:
>
> http://mapa.czfree.net/#lat=50.76816800116274&lng=15.066890716552734&zoom=13&autofilter=1&type=k&geolocate=98%7C114%7C111%7C117%7C109%7C111%7C118%7C115%7C107%7C97&node=6101&aponly=1&bbonly=1&actlink=1&actnode=1& <http://mapa.czfree.net/#lat=50.76816800116274&lng=15.066890716552734&zoom=13&autofilter=1&type=k&geolocate=98|114|111|117|109|111|118|115|107|97&node=6101&aponly=1&bbonly=1&actlink=1&actnode=1&>
>
> Most of the nodes use Ubiquiti hardware for the radios and custom hardware for 
> the routers, connected to the radios via Ethernet, and most of which run on 1 
> GHz AMD G-T40E processors and run a customized distribution of Voyage Linux 
> 0.10 (Debian 8 based distribution for embedded hardware). The distro ships 
> with kernel 3.16.7, which does have a working sch_fq_codel module. To get 
> oriented, I started with some Flent RRUL runs on my test setup below, and 
> that’s where I ran into this question. Next, I’ll test on some actual hardware 
> targeted for deployment to understand its fq_codel throughput limits, but not 
> before I understand this “half duplex problem” better.
>
> Test setup:
>
> PC #1  <->  WiFi (802.11n, 130 Mbps Tx Rate, RSSI -60)  <->  AirPort Express  <->  100Mbit Ethernet  <->  PC #2
>
> PC #1: Runs a Voyage Linux VM (more on that later), with the Flent client and HTB + fq_codel, and is connected via the PC's WiFi adapter to the Airport Express.
>
> PC #2: Runs netserver for flent to connect to, and is connected via Ethernet to the Airport Express.
>
> Thanks for any input.