From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 6A0E03B2A4 for ; Thu, 9 Jun 2022 04:58:20 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1654765097; bh=vB8ZQAVALAWgYLM5GWtX0J2XyE+oZ6ukyQJb6eb+Bxg=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=EJ9wOdE8RDhsmeUM6dWfcl9VHWOuPKTGAVFNHofzvtmP4K5ntqi0Z1Ykr9I+427pu gBA21GE24B9X98S7Rg4HdydmszolzqeKA7bQL4FuMdS6iq/alfvzsmEXEGC78X77Ni WSrGtD6EJuYK1KC9+j+ebOc3ZKxmraCp5ETj8zUQ= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from smtpclient.apple ([134.76.241.253]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1N33Ed-1nbkoe06mr-013Mko; Thu, 09 Jun 2022 10:58:17 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.100.31\)) From: Sebastian Moeller X-Priority: 3 (Normal) In-Reply-To: <1654721363.567314628@apps.rackspace.com> Date: Thu, 9 Jun 2022 10:58:16 +0200 Cc: warren ponder , starlink@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: <73873391-9F9F-4850-A270-7FFDB01F929F@gmx.de> References: <1654714026.72728578@apps.rackspace.com> <1654721363.567314628@apps.rackspace.com> To: "David P. Reed" X-Mailer: Apple Mail (2.3696.100.31) X-Provags-ID: V03:K1:uLQrGXRR2vycerSEHb9iyhknTNX2I6nsyFbZHLPHI79eKPuIUZ8 X01qNnTb2mQIXyMR525FX2zGvkvwd9tkgnRSjEJr9GVWNSTDIk65tkgRM79oHhpBIh4Q5ZP BfCuJ5lPIxyGrqBGDGsP4cEFX/y6DecedM9lBsSmHmrdr0klHCXfX5ASzlg31jN22/R6v8b WF3gjncvU4WXpVXaFQ5Ww== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:W+cGmgM8+PU=:VUZ3q/lBvrpS4jto9Tdnqa VEYiASoqa1SA3h8GnhYIBHOPNtflvr7Ys9xn/r+rwdGoNP7fFTfoES+ru7DSHXaT5pn8RYMFs 8r/HyrzVezgly0xLboLBSqn3YmL9Baz4coUsJaL0TklM2rh98ySfovDFvQ6Y3+HsEgoC/2MyL cuHxmJUCG6tCcp2/HzgGacmcT2gyvQU/wJ0xLP1vbhP+LbrNLB0N7b0aMCKfRJNY+BdP/xJNd A5teSYRntXBcYffoA9sU/xBrNmeMe1FfZlKPDmgvBIkmHr2SUq8bP6kN7j/J2OYOdL3Z1Hm4X oO6TUe8qUFD4UtwKPy6JljIX1Uq21xMynh5d1JylbRfQjQXBL26i05Q/1PqK82z3+ht7Si94J yGPIovE3mV5I5u++DtWDCddH/z6agtKn83UrsejnqdjtJe7Vlt8nSJVTSYsSsHlh1KYHWvOX5 izOQQg5KyDBcJLrULNI1nXKYbMiUkeKskuwpsifEtt8s5sYtIPWeXklo7LZhOCftwppnVml7Z vkumgTdErnNi0sEakXSm2EWxX8hNbN5NEq+zjWF/65SyDFIKVPcyN9m0qqQvn+4J+ICdj3G3t DEWeiMjj/qMaiViXHChLWW4ueeDkPO7JSlFRwbS0Iuh7Ge1lMFZpEmsWIJDM9B+2QZw13ZaDz PWX5Ci4P2Mb1wjfqOuhG6NDAQEmzLA7i2VmCvRx3RSRjHFyXOosWARJCLR7TSr/4q1Q0n48+W 1PxcYAtSbqbzxk6nw1m5xTzvyHmlG9a8UKoO2WAj5El9EVc6QQ4DWhbnEKslRPONFtYB3CAx5 dZf6vnQCyiFJ5KCnoI5g3ns+k8TEJdA9VUOeCjaX7BiKzNmr/Xg0TpnVhy/0+14aOvqgsuEH/ 83XX3YUP6d3/3GYZ+OvE+QhzZ0Jb2IDy4UCmCGJIflSQsO/1eW/8FjTXkmhrPe6liUkoPngd7 V1ClcNn5fLKT+ie56SN1MJ+5O0qUsXGGKPFDkQQ/iiX7i5o3a2c0QNeZX5GFzcMy1ZlRlDt+R Kcb/S7+UAh8wUvMjHN6xABugI3Wrhj5+Mj0iu0Zb0CBhH+m2aaBPXL4hw/lUodtxCu1eAyDKz 56iOMysN72cN58ZskQlZQNJiIRVn1Qj9mGr3oGZFzFdHVTXTKUelcMZRg== Subject: Re: [Starlink] FQ_Codel X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jun 2022 08:58:20 -0000 Hi David, > On Jun 8, 2022, at 22:49, David P. Reed wrote: >=20 > No, I don't think so. However, a hidden (not often discussed) way that = using FQ-codel in a home router works is that you have to artificially = restrict the total bitrate in both directions used by your home router = to talk to the access provider link. I am not sure I fully agree with the "hidden (not often = discussed)" qualification, when I explain sqm principles I start by = explaining queues* and bottleneck queues and the need to create an = artificial bottleneck to get the queueing under our control where we can = employ decent scheduling and AQM to minimize the downsides of excessive = queueing. E.g in = https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details = we write: "Basic Settings - the details=E2=80=A6 SQM is designed to manage the queues of packets waiting to be sent = across the slowest (bottleneck) link, which is usually your connection = to the Internet. The algorithm cannot automatically adapt to network = conditions on DSL, cable modems or GPON without any settings. Since the = majority of ISP provided configurations for buffering are broken today, = you need take control of the bottleneck link away from the ISP and move = it into the router so it can be fixed. You do this by entering link = speeds that are a few percent below the actual speeds." *) Ad little as I understand them, I do not claim to be an expert on = queueing theory. > Typically, it is recommended to use 95% of the upload/download speeds = of that link as the limit. This forces packets to be dropped when the = constraint is exceeded. Now this forces congestion control signals = (dropped packets) to be observed by both ends. (In a cable DOCSIS = system, this allows the edge to manage the throughput of the CMTS for = the local endpoint, because the CMTS won't drop packets when it should - = because configuring DOCSIS 3.1 CMTS's is often done in a way that causes = bufferbloat in CMTS. DOCSIS 2 always had bufferbloat in the CMTS). > =20 > Starlink doesn't sell you a stable "max rate" - instead that rate = varies depending on traffic, and can't be easily measured. > So to configure the dishy or an edge router connected to it correctly, = you need to enforce such a limit such that it actually causes FQ-codel = to see dropped packets. > On Wednesday, June 8, 2022 3:12pm, "warren ponder" = said: >=20 > So this is really helpful. Is it fair to say then that end users with = SQM and fq_codel on a Starlink connection should essentially not turn on = SQM.and.just leave it off? >=20 > On Wed, Jun 8, 2022, 11:47 AM David P. Reed = wrote: > I'm just going to remind folks that fixing bufferbloat in Starlink = won't be possible with FQ-Codel in the CPE equipment. If that were = possible, it could be fixed entirely in a box sitting between the dishy = and the user's "home network". > =20 > Evidence exists that the bulk of the "bloat" can exist, not just in = the dishy, but also in the central "access point" where satellites in a = coverage region direct all the traffic from and to the public Internet. = This connection from the region becomes bloated if the inbound link and = outbound link become "underprovisioned" for peak rates of all the served = dishy terminals. > That public-Internet to StarLink access point (is there a more = distinct, precise name) can develop a very long delay queue. For the = same reason that bufferbloat always gets designed in - memory is cheap = and plentiful, so instead of dropping packets to minimize latency, the = link just stores packets until multiple seconds worth of traffic build = up on one or both ends of that link. > =20 > This problem can be solved only by dropping packets (with packet drop = rate mitigated by ECN-marking) to match the desired round-trip latency = across the entire Internet. Typically, this queue size should max out = and start dropping packets at about 2 * cross-Internet desired latency * = bit-rate of this link. > Cross-Internet desired latency can be selected these days by using = light-speed in fiber between one side of the North American continent = and the other - around 15 msec. is appropriate. (which should be the = worst case end-to-end latency observed using Starlink, and is around the = 20 msec. number bandied about by Musk - though he really never = understood what end-to-end latency means). > =20 > =20 > Now it may be that the dishy itself also has such bloat built in, = which would make FQ-Codel in the dishy also important. > =20 > The problem called bufferbloat occurs whenever ANY router on ANY = end-to-end shared path allows such queueing delay to accumulate before = shortening the queue. > =20 > It really frustrates me that memory keeps being added to router = outbound buffers anywhere. And it may be that the reason is that almost = nobody who designs packet forwarding systems understands Queueing Theory = at all! It certainly doesn't help that "packet drops" (even one or two = per second) are considered a failure of the equipment. > =20 > FQ-codel is great, but why it works is that it makes the choice of = what packet to drop far better (by being fair and a little bit elastic). = However, the lack of FQ-Codel doesn't fix system-level bufferbloat. > =20 > =20 > _______________________________________________ > Starlink mailing list > Starlink@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/starlink > _______________________________________________ > Starlink mailing list > Starlink@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/starlink