From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 193FC3B2A4 for ; Thu, 9 Jun 2022 04:51:02 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1654764660; bh=RhKyBxbrnQ0CoWIFqwXg6ddEV70M2WLudjOIFrKZHbk=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=Dbq8QI/EdG5hNiWSUMfYrs+ATfDabyS8v7zyuHDFAYyNOD0vyy7ioIp6wT/kSkvLU vt8dXI1uZLqFTCzXX9JNz7TMt2vrg2uXFqHqReazXxMHt69Frawyj7dACHffB0iml+ RzhwUFFuTKdcdX1LOZgp6t5FTEuMb1CGgxL4MQ/o= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from smtpclient.apple ([134.76.241.253]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1Md6Mj-1nQANT19pD-00aGZs; Thu, 09 Jun 2022 10:51:00 +0200 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.100.31\)) From: Sebastian Moeller X-Priority: 3 (Normal) In-Reply-To: <1654714026.72728578@apps.rackspace.com> Date: Thu, 9 Jun 2022 10:50:59 +0200 Cc: starlink@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: <35A226DB-8907-4213-BDEF-A8CFED81F1F1@gmx.de> References: <1654714026.72728578@apps.rackspace.com> To: "David P. Reed" X-Mailer: Apple Mail (2.3696.100.31) X-Provags-ID: V03:K1:IaZWheB/a3A3d1eUSZJWjbN63OdT0iQS0EV6PNUs6RzVDDOmjFE XorCSC1xLxOdemcOVjnhqSid742Q3OaHBDX0HJWS0GK5m9YYULzeH5IwoTX33CMPjk7P9qg 4pLd/xCuad1ReEm4z2jAq2z9XvCP2rU9iUnbAy5NsB7l/CaShpH8Nm6FkSiBIt9sUORJPwo TvClnJMvrcx1xEBfft36g== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:sqitXvRewuY=:+qzO8vE1v1sTkx/hjVaHWt Y/pNgAFkXIAaaOGrPHD6N1YFkkxqBln0hqh1ViTJHiPTXm4jyjbcf0YG4glPR1eSX4Qsi2RFe dggO2ZQD2ZeMj6njRda6E4M2nz7umus3sT25B8/6bpmcLuEyJsZWr6YZ0c8VfTpTrQuf9ZT1S Km0pTAdZGvf7RtrvBcr0hsu3jSPE36yqjJAT41u3IJm+5p9dCRhFPnVF/ZqwTkrjLv9dOZNrj fY0cxpJ17VHv+/OugkJ+ZWGbxiDIeT8lO4+4Ja2djSIVN5SGvovVnSl3SoF3pLfh+sIgIdNos nKZ5dUfek3Qb42BgJSfeFDRmuZsFYrpQ1X2B86fqatPmjMbfCL1cjjs+ysBsAurjjqUu02mC5 AWDX6/Ekkuzm/6S2++p75Ds6N0toxhGtK0bjLkpPT0cY0xmC6T/43cbmLVoSgJD6Tx/BLmi72 bppY+eM+pbXN8ZEigw0BUmCkP6RietC9RdxMwqkxnKpB5zy/x/R15RBtgxmT2LffnZXswTaJV oinj3wHwRjcURTbxIb02EAEN3DsOuejvNQUi7rzyrNHr4qVpTZH24/VghXfBJcldtyIZEorc5 Vt6hVPHcUdmPZck4cY6vEiTP/33LO0g5dNWbxl1j3ufJ6BZy3DsSPvYUKjAB3t3HTbr76OzkY R4b1izz8p6szDEMuXzawBcHwQcLooeLcl159e5iPfkPE/I7eCKEGV5RyN/N3dA1FFtSkoQO2s bdhcVeF3IULK5gCvozBvcaN03UdeiQxPyREbOHvChczqi6d24wtghucCDMAl9Fkv3st0Tojz7 uRdrTRDHYs+fqWFWkJNc0D+AOi0xqf1yGXTbBa6v+Cg+bXtfOi4r5u4WlDjLq3R5cBKY+3HBK tTHNpN4sihIMO8JoyxAQ+S6SQU1TusdbiEy4PbuwclOO3Rlv++EGNwsTOAMCTfHk7qAqkkn10 0L+o3Aeojxssi2InYlASSEMTseYKUFVmEQqMjncZk+dq0znWv7qfsUy0131vMM623YN9M8aGP yyUleSb8Z8yQlofl124CyntLMRVFgbPm+g+NTEE5wgjgrkVo9XdNupvXyGjGrSc8N7E+ON5xp CcjHY70tLrHRgSf4p6mGtyw2N/5nsFeCqGd7WBxAHD9WsC7zPO6n1CTzw== Subject: Re: [Starlink] FQ_Codel X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jun 2022 08:51:03 -0000 Hi David, > On Jun 8, 2022, at 20:47, David P. Reed wrote: >=20 > I'm just going to remind folks that fixing bufferbloat in Starlink = won't be possible with FQ-Codel in the CPE equipment. If that were = possible, it could be fixed entirely in a box sitting between the dishy = and the user's "home network". While we can not fix it, we can remedy it to some degree. > =20 > Evidence exists that the bulk of the "bloat" can exist, not just in = the dishy, but also in the central "access point" where satellites in a = coverage region direct all the traffic from and to the public Internet. = This connection from the region becomes bloated if the inbound link and = outbound link become "underprovisioned" for peak rates of all the served = dishy terminals. > That public-Internet to StarLink access point (is there a more = distinct, precise name) can develop a very long delay queue. For the = same reason that bufferbloat always gets designed in - memory is cheap = and plentiful, ... but CPU cycles are still precious, it is this combination = that results in the "over-sized, but under-managed" combination that is = so atrocious for latency under load/working latency. (Granted, decent = management typically means that the queues never really grow to fill = large memory buffers, but that also means that reserving large amounts = of memory for buffering does not hurt anymore). > so instead of dropping packets to minimize latency, the link just = stores packets until multiple seconds worth of traffic build up on one = or both ends of that link. As long as they service the remote stations in a somewhat = predictable/fair rpund-robin fashion under load we should be able to = remedy that though. > This problem can be solved only by dropping packets (with packet drop = rate mitigated by ECN-marking) to match the desired round-trip latency = across the entire Internet. Typically, this queue size should max out = and start dropping packets at about 2 * cross-Internet desired latency * = bit-rate of this link. > Cross-Internet desired latency can be selected these days by using = light-speed in fiber between one side of the North American continent = and the other - around 15 msec. is appropriate. (which should be the = worst case end-to-end latency observed using Starlink, and is around the = 20 msec. number bandied about by Musk - though he really never = understood what end-to-end latency means). > =20 > =20 > Now it may be that the dishy itself also has such bloat built in, = which would make FQ-Codel in the dishy also important. > =20 > The problem called bufferbloat occurs whenever ANY router on ANY = end-to-end shared path allows such queueing delay to accumulate before = shortening the queue. > =20 > It really frustrates me that memory keeps being added to router = outbound buffers anywhere. And it may be that the reason is that almost = nobody who designs packet forwarding systems understands Queueing Theory = at all! It certainly doesn't help that "packet drops" (even one or two = per second) are considered a failure of the equipment. > =20 > FQ-codel is great, but why it works is that it makes the choice of = what packet to drop far better (by being fair and a little bit elastic). = However, the lack of FQ-Codel doesn't fix system-level bufferbloat. I would have guessed tat the FQ scheduler alone already helps a = lot, as it restricts the pain from over-committing to the hash-bucket = housing the offending flow. Sure selecting the most effective packet(s) = to drop also helps, but FQ alone will already help non-capacity-seeking = flows (that stay below their capacity share) a lot if competing with = capacity seeking traffic on the same link. Regards Sebastian > =20 > =20 > _______________________________________________ > Starlink mailing list > Starlink@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/starlink