[Starlink] FQ_Codel

[Sebastian Moeller]

Hi David,


> On Jun 8, 2022, at 22:49, David P. Reed <dpreed at deepplum.com> wrote:
> 
> No, I don't think so. However, a hidden (not often discussed) way that using FQ-codel in a home router works is that you have to artificially restrict the total bitrate in both directions used by your home router to talk to the access provider link.

	I am not sure I fully agree with the "hidden (not often discussed)" qualification, when I explain sqm principles I start by explaining queues* and bottleneck queues and the need to create an artificial bottleneck to get the queueing under our control where we can employ decent scheduling and AQM to minimize the downsides of excessive queueing. E.g in https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details we write:

"Basic Settings - the details…

SQM is designed to manage the queues of packets waiting to be sent across the slowest (bottleneck) link, which is usually your connection to the Internet. The algorithm cannot automatically adapt to network conditions on DSL, cable modems or GPON without any settings. Since the majority of ISP provided configurations for buffering are broken today, you need take control of the bottleneck link away from the ISP and move it into the router so it can be fixed. You do this by entering link speeds that are a few percent below the actual speeds."



*) Ad little as I understand them, I do not claim to be an expert on queueing theory.


> Typically, it is recommended to use 95% of the upload/download speeds of that link as the limit. This forces packets to be dropped when the constraint is exceeded. Now this forces congestion control signals (dropped packets) to be observed by both ends. (In a cable DOCSIS system, this allows the edge to manage the throughput of the CMTS for the local endpoint, because the CMTS won't drop packets when it should - because configuring DOCSIS 3.1 CMTS's is often done in a way that causes bufferbloat in CMTS. DOCSIS 2 always had bufferbloat in the CMTS).
>  
> Starlink doesn't sell you a stable "max rate" - instead that rate varies depending on traffic, and can't be easily measured.
> So to configure the dishy or an edge router connected to it correctly, you need to enforce such a limit such that it actually causes FQ-codel to see dropped packets.
> On Wednesday, June 8, 2022 3:12pm, "warren ponder" <wponder11 at gmail.com> said:
> 
> So this is really helpful. Is it fair to say then that end users with SQM and fq_codel on a Starlink connection should essentially not turn on SQM.and.just leave it off?
> 
> On Wed, Jun 8, 2022, 11:47 AM David P. Reed <dpreed at deepplum.com> wrote:
> I'm just going to remind folks that fixing bufferbloat in Starlink won't be possible with FQ-Codel in the CPE equipment. If that were possible, it could be fixed entirely in a box sitting between the dishy and the user's "home network".
>  
> Evidence exists that the bulk of the "bloat" can exist, not just in the dishy, but also in the central "access point" where satellites in a coverage region direct all the traffic from and to the public Internet. This connection from the region becomes bloated if the inbound link and outbound link become "underprovisioned" for peak rates of all the served dishy terminals.
> That public-Internet to StarLink access point (is there a more distinct, precise name) can develop a very long delay queue.  For the same reason that bufferbloat always gets designed in - memory is cheap and plentiful, so instead of dropping packets to minimize latency, the link just stores packets until multiple seconds worth of traffic build up on one or both ends of that link.
>  
> This problem can be solved only by dropping packets (with packet drop rate mitigated by ECN-marking) to match the desired round-trip latency across the entire Internet. Typically, this queue size should max out and start dropping packets at about 2 * cross-Internet desired latency * bit-rate of this link.
> Cross-Internet desired latency can be selected these days by using light-speed in fiber between one side of the North American continent and the other - around 15 msec. is appropriate. (which should be the worst case end-to-end latency observed using Starlink, and is around the 20 msec. number bandied about by Musk - though he really never understood what end-to-end latency means).
>  
>  
> Now it may be that the dishy itself also has such bloat built in, which would make FQ-Codel in the dishy also important.
>  
> The problem called bufferbloat occurs whenever ANY router on ANY end-to-end shared path allows such queueing delay to accumulate before shortening the queue.
>  
> It really frustrates me that memory keeps being added to router outbound buffers anywhere. And it may be that the reason is that almost nobody who designs packet forwarding systems understands Queueing Theory at all! It certainly doesn't help that "packet drops" (even one or two per second) are considered a failure of the equipment.
>  
> FQ-codel is great, but why it works is that it makes the choice of what packet to drop far better (by being fair and a little bit elastic). However, the lack of FQ-Codel doesn't fix system-level bufferbloat.
>  
>  
> _______________________________________________
> Starlink mailing list
> Starlink at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink
> _______________________________________________
> Starlink mailing list
> Starlink at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink