[Bloat] Tuning fq_codel: are there more best practices for slow connections? (<1mbit)

[Sebastian Moeller]

Hi cloneman,


> On Nov 2, 2017, at 07:01, cloneman <bufferbloat at flamingpc.com> wrote:
> 
> I'm trying to gather advice for people stuck on older connections. It appears that having dedictated /micromanged tc classes greatly outperforms the "no knobs" fq_codel approach for connections with  slow upload speed.
> 
> When running a single file upload @350kbps , I've observed the competing ICMP traffic quickly begin to drop (fq_codel) or be delayed considerably ( under sfq). From reading the tuning best practices page is not optimized for this scenario. (<2.5mbps)
> (https://www.bufferbloat.net/projects/codel/wiki/Best_practices_for_benchmarking_Codel_and_FQ_Codel/) fq_codel 

	This page was last updated 2014, it seems we learned a few tricks since then. May I recommend you look into sqm-scripts (see https://lede-project.org/docs/user-guide/sqm for a user guide) if only to look at how we recommend to configure fq_codel currently.
	One of the biggest issues with fq_codel at slow speeds (exactly speeds below (1526*8)/0.005 = 2441600 bps) is that even a single packet might use up most of or even exceed  the 5ms default "target" duration that fq_codel/codel default to. This is a problem because exceeding that time will cause fq_codel to switch into drop mode and your experience will be choppy. We now recommend to simply increase target to be at least around 1.5 MTU worth of transfer time, for fast links this will be smaller than the default 5ms,  so we do nothing but for slower links it will not so we adjust things. fq_codel has no way of knowing the available bandwidth and hence can not auto compensate for that.


> 
> Of particular concern is that a no-knobs SFQ works better for me than an untuned codel ( more delay but much less loss for small flows). People just flipping the fq_codel button on their router at these low speeds could be doing themselves a disservice.

	I would in all modesty recommend that people rather look into using sqm-scripts (at least if using lede/openwrt or other linux based distributions) that should give a better starting point for their own experiments and modifications. So I am not saying do not experiment yousrseld, but rather start from a known decent starting point (also sqm-scripts easily handles user supplied scripts and makes it quite comfortable to get started with playing with qdisc's and traffic shapers).

> 
> I've toyed with increasing the target and this does solve the excessive drops. I haven't played with limit and quantum all that much. 

	Target is exactly the value of interest here (except it seems reasonable to also adjust interval as target is supposed to be in the range of 5-10% of interval, so if target changes, interval might need to change as well)

> 
> My go-to solution for this would be different classes, a.k.a. traditional QoS. But ,  wouldn't it be possible to tune fq_codel punish the large flows 'properly' for this very low bandwidth scenario? Surely <1kb ICMP packets can squeeze through properly without being dropped if there is 350kbps available, if the competing flow is managed correctly.

	At 2.5Mbps one full MTU packet in transfer will make all other flows with wueued packets exceed the default 5ms target and hence change them into drop mode, extending target is the right thing to do here... Other than that fq_codel does try to boost sparse flows, but you will only see this once you extended target appropriately.

> 
> I could create a class filter by packet length, thereby moving ICMP/VoIP to its own tc class, but  this goes against "no knobs" it seems like I'm re-inventing the wheel of fair queuing - shouldn't the smallest flows never be delayed/dropped automatically?

	Not dropping based on a feature like length will simply invite abuse, so I would not go there.

> 
> Lowering Quantum below 1500 is confusing, serving a fractional packet in a time interval?

	No not serving fractional packets, but taking multiple rounds through the "scheduler" before a flow with a large queued packet will be allows to send, this also should allow for smaller  packets to squeeze by faster (without affecting bandwidth fairness).

> 
> Is there real value in tuning fq_codel for these connections or should people migrate to something else like nfq_codel?

	Again, maybe just use sqm-scripts might slove most of these issues in a user friendly fashion. sqm-scripts also allows to easily use the experimental cake qdisc which has quite a number of cool tricks up its sleeve, like pirecing though NAT to get the "real" source and destination IP addresses which can be used to easily configure a made in which cake tries to achieve fairness by the number of concurrently active internal host IPs (and that for many end users seems to be sufficient to not having to bother any further with twiddling with qos/q\aqm settings).

Best Regards 


> 
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat