[Codel] Using fq_codel with a WiFi uplink to the Internet

[Phineas Gage]

> On Sep 23, 2016, at 1:54 PM, Phineas Gage <phineas919 at gmail.com> wrote:
> 
>>> Option A: We can choose a maximum of 40/40 Mbit with 1:5 aggregation, meaning we could get 40 Mbit, but we could also get a lot less at times (8 Mbit I assume), depending on other network load.
>> 
>> The 1:5 aggregation is worth exploring a bit more deeply.  Usually this refers to the ratio between the total bandwidth provisioned across all customers (in some region and some service class) and the minimum backhaul capacity within and at the far edge of the ISP’s network.  The theory is that customers tend not to use all of their bandwidth all of the time, though they do tend to use all of it some of the time.  This theory works fairly well in practice as long as the traffic patterns are not too correlated and are distributed over a sufficient number of customers.
>> 
>> In order to be dragged down to 8Mbps, you would have to see all the other customers sharing your backhaul trying to use 8Mbps or more (on average) at the same time.  This is unlikely to occur often; consider major sportsball championship final matches, or national disasters such as one we recently had an anniversary of, as the most likely triggers.  Under such circumstances, you’ll need to step in and manually experiment to find out what works, but shaping at 8Mbps would be a reasonable default reaction.
>> 
>> Of more concern to you is how much external load is needed to pull your share of the backhaul significantly below 40Mbps - or, alternatively, below the 20Mbps you can get with the more expensive uncontended service.  This will vary depending on how many customers the 5:1 average is spread over - you can’t actually calculate it from the information given.
>> 
>> So the question is whether they have a 40/40 backhaul shared among 5 customers, or a 1G/1G backhaul which they plan to share among up to 125 customers, each with 40/40 service.  I think the latter is a perfectly reasonable possibility, and would be much more robust than the former option which would give you a reduction in throughput as soon as *any* of the other customers on the same backhaul segment did *anything*.
>> 
>> It’s a question worth asking your WISP.  Be ready to point out that you don’t plan to actually *use* 40Mbps full-time, but that your AQM solution depends on knowing how much bandwidth is available for when *your* customers randomly demand it.
> 
> Thanks a lot for this great information, I’ll ask them more about what’s behind their aggregation ratio. As I mentioned in one reply earlier, I may ask for flexibility in our contract, try the aggregated service, then switch to guaranteed service depending on the actual performance of their network. If I can rate limit to 30/30 and fq_codel, and it’s 99% effective, I’d rather take that than settle for a guaranteed 20/20 on-season and 6/6 off-season to get to the same annual contract price.

Following up on the WISP results, I tested the 40 Mbps non-guaranteed connection for a week or so, and the truth is that the speed varies much more than expected. Although I once saw a 40 Mbps download, that’s more of a black swan. Download throughput during Internet rush hour regularly dropped to 5-10 Mbps, and I once saw 1 Mbps. Upload throughput stays higher. But I can’t see how “aggregation 1:5” has any meaning for me, as I saw speeds below 8 Mbps. There is probably no way I can effectively control either the upload or download queues effectively with this type of service.

Also, they seem to prioritize traffic to speedtest.net, because while the graphs to those servers looked pretty good and consistent (I scripted a test every 30 minutes using speedtest-cli), throughput reported by using wget to various high-powered Ubuntu servers around us in Europe told a different story. During the day, 25-30 Mbps was routine, and in the evening, 5-10 Mbps was routine, with outliers on either end. This was of course with a cabled client and no other traffic from us on the connection. So I’ll be finding out more details about the “guaranteed” 10 Mbps service.

Also a couple other comments to anyone interested:

- I saw a post about recent work getting Cake to run on EdgeOS, so I expressed interest in that for the EdgeRouter X, which I installed yesterday: http://community.ubnt.com/t5/EdgeMAX/Cake-compiled-for-the-ERL/td-p/1679844

- I might make it to the OpenWrt summit (http://openwrtsummit.org/) in Berlin next Thursday, 10/13, in case I catch anyone there.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/codel/attachments/20161003/2038c811/attachment.html>