From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp102.iad3a.emailsrvr.com (smtp102.iad3a.emailsrvr.com [173.203.187.102]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 207ED3B2A4 for ; Wed, 8 Jun 2022 16:49:24 -0400 (EDT) Received: from app17.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by smtp5.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 9FD70252EE; Wed, 8 Jun 2022 16:49:23 -0400 (EDT) Received: from deepplum.com (localhost.localdomain [127.0.0.1]) by app17.wa-webapps.iad3a (Postfix) with ESMTP id 8B2B5E195D; Wed, 8 Jun 2022 16:49:23 -0400 (EDT) Received: by apps.rackspace.com (Authenticated sender: dpreed@deepplum.com, from: dpreed@deepplum.com) with HTTP; Wed, 8 Jun 2022 16:49:23 -0400 (EDT) X-Auth-ID: dpreed@deepplum.com Date: Wed, 8 Jun 2022 16:49:23 -0400 (EDT) From: "David P. Reed" To: "warren ponder" Cc: starlink@lists.bufferbloat.net MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20220608164923000000_55052" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: References: <1654714026.72728578@apps.rackspace.com> X-Client-IP: 209.6.168.128 Message-ID: <1654721363.567314628@apps.rackspace.com> X-Mailer: webmail/19.0.15-RC X-Classification-ID: 3fd80ff3-57c8-4601-a25d-447133e6bd5e-1-1 Subject: Re: [Starlink] =?utf-8?q?FQ=5FCodel?= X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jun 2022 20:49:24 -0000 ------=_20220608164923000000_55052 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0ANo, I don't think so. However, a hidden (not often discussed) way that u= sing FQ-codel in a home router works is that you have to artificially restr= ict the total bitrate in both directions used by your home router to talk t= o the access provider link.=0A =0ATypically, it is recommended to use 95% o= f the upload/download speeds of that link as the limit. This forces packets= to be dropped when the constraint is exceeded. Now this forces congestion = control signals (dropped packets) to be observed by both ends. (In a cable = DOCSIS system, this allows the edge to manage the throughput of the CMTS fo= r the local endpoint, because the CMTS won't drop packets when it should - = because configuring DOCSIS 3.1 CMTS's is often done in a way that causes bu= fferbloat in CMTS. DOCSIS 2 always had bufferbloat in the CMTS).=0A =0AStar= link doesn't sell you a stable "max rate" - instead that rate varies depend= ing on traffic, and can't be easily measured.=0ASo to configure the dishy o= r an edge router connected to it correctly, you need to enforce such a limi= t such that it actually causes FQ-codel to see dropped packets.=0AOn Wednes= day, June 8, 2022 3:12pm, "warren ponder" said:=0A=0A= =0A=0ASo this is really helpful. Is it fair to say then that end users with= SQM and fq_codel on a Starlink connection should essentially not turn on S= QM.and.just leave it off?=0A=0A=0AOn Wed, Jun 8, 2022, 11:47 AM David P. Re= ed <[ dpreed@deepplum.com ]( mailto:dpreed@deepplum.com )> wrote:=0AI'm jus= t going to remind folks that fixing bufferbloat in Starlink won't be possib= le with FQ-Codel in the CPE equipment. If that were possible, it could be f= ixed entirely in a box sitting between the dishy and the user's "home netwo= rk".=0A =0AEvidence exists that the bulk of the "bloat" can exist, not just= in the dishy, but also in the central "access point" where satellites in a= coverage region direct all the traffic from and to the public Internet. Th= is connection from the region becomes bloated if the inbound link and outbo= und link become "underprovisioned" for peak rates of all the served dishy t= erminals.=0AThat public-Internet to StarLink access point (is there a more = distinct, precise name) can develop a very long delay queue. For the same = reason that bufferbloat always gets designed in - memory is cheap and plent= iful, so instead of dropping packets to minimize latency, the link just sto= res packets until multiple seconds worth of traffic build up on one or both= ends of that link.=0A =0AThis problem can be solved only by dropping packe= ts (with packet drop rate mitigated by ECN-marking) to match the desired ro= und-trip latency across the entire Internet. Typically, this queue size sho= uld max out and start dropping packets at about 2 * cross-Internet desired = latency * bit-rate of this link.=0ACross-Internet desired latency can be se= lected these days by using light-speed in fiber between one side of the Nor= th American continent and the other - around 15 msec. is appropriate. (whic= h should be the worst case end-to-end latency observed using Starlink, and = is around the 20 msec. number bandied about by Musk - though he really neve= r understood what end-to-end latency means).=0A =0A =0ANow it may be that t= he dishy itself also has such bloat built in, which would make FQ-Codel in = the dishy also important.=0A =0AThe problem called bufferbloat occurs whene= ver ANY router on ANY end-to-end shared path allows such queueing delay to = accumulate before shortening the queue.=0A =0AIt really frustrates me that = memory keeps being added to router outbound buffers anywhere. And it may be= that the reason is that almost nobody who designs packet forwarding system= s understands Queueing Theory at all! It certainly doesn't help that "packe= t drops" (even one or two per second) are considered a failure of the equip= ment.=0A =0AFQ-codel is great, but why it works is that it makes the choice= of what packet to drop far better (by being fair and a little bit elastic)= . However, the lack of FQ-Codel doesn't fix system-level bufferbloat.=0A = =0A=0A _______________________________________________=0A Starlink mailing = list=0A[ Starlink@lists.bufferbloat.net ]( mailto:Starlink@lists.bufferbloa= t.net )=0A[ https://lists.bufferbloat.net/listinfo/starlink ]( https://list= s.bufferbloat.net/listinfo/starlink ) ------=_20220608164923000000_55052 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

No, I don't think so. = However, a hidden (not often discussed) way that using FQ-codel in a home r= outer works is that you have to artificially restrict the total bitrate in = both directions used by your home router to talk to the access provider lin= k.

=0A

 

=0A

Typically, = it is recommended to use 95% of the upload/download speeds of that link as = the limit. This forces packets to be dropped when the constraint is exceede= d. Now this forces congestion control signals (dropped packets) to be obser= ved by both ends. (In a cable DOCSIS system, this allows the edge to manage= the throughput of the CMTS for the local endpoint, because the CMTS won't = drop packets when it should - because configuring DOCSIS 3.1 CMTS's is ofte= n done in a way that causes bufferbloat in CMTS. DOCSIS 2 always had buffer= bloat in the CMTS).

=0A

 

=0A

Starlink doesn't sell you a stable "max rate" - instead that rate var= ies depending on traffic, and can't be easily measured.

=0A

So to configure the dishy or an edge router connected to it correct= ly, you need to enforce such a limit such that it actually causes FQ-codel = to see dropped packets.

=0A

On Wednesday, June 8, 20= 22 3:12pm, "warren ponder" <wponder11@gmail.com> said:

=0A

=0A
So this is really helpful. Is = it fair to say then that end users with SQM and fq_codel on a Starlink conn= ection should essentially not turn on SQM.and.just leave it off?
=0A=0A
=0A
= On Wed, Jun 8, 2022, 11:47 AM David P. Reed <dpreed@deepplum.com> wrote:
=0A
=0A

I'm just going to remind folks that fi= xing bufferbloat in Starlink won't be possible with FQ-Codel in the CPE equ= ipment. If that were possible, it could be fixed entirely in a box sitting = between the dishy and the user's "home network".

=0A

&nb= sp;

=0A

Evidence exists that the bulk of the "bloat" can= exist, not just in the dishy, but also in the central "access point" where= satellites in a coverage region direct all the traffic from and to the pub= lic Internet. This connection from the region becomes bloated if the inboun= d link and outbound link become "underprovisioned" for peak rates of all th= e served dishy terminals.

=0A

That public-Internet to St= arLink access point (is there a more distinct, precise name) can develop a = very long delay queue.  For the same reason that bufferbloat always ge= ts designed in - memory is cheap and plentiful, so instead of dropping pack= ets to minimize latency, the link just stores packets until multiple second= s worth of traffic build up on one or both ends of that link.

=0A

 

=0A

This problem can be solved only by = dropping packets (with packet drop rate mitigated by ECN-marking) to match = the desired round-trip latency across the entire Internet. Typically, this = queue size should max out and start dropping packets at about 2 * cross-Int= ernet desired latency * bit-rate of this link.

=0A

Cross= -Internet desired latency can be selected these days by using light-speed i= n fiber between one side of the North American continent and the other - ar= ound 15 msec. is appropriate. (which should be the worst case end-to-end la= tency observed using Starlink, and is around the 20 msec. number bandied ab= out by Musk - though he really never understood what end-to-end latency mea= ns).

=0A

 

=0A

 

=0A

Now it may be that the dishy itself also has such bloat built = in, which would make FQ-Codel in the dishy also important.

=0A

 

=0A

The problem called bufferbloat occur= s whenever ANY router on ANY end-to-end shared path allows such queueing de= lay to accumulate before shortening the queue.

=0A

 = ;

=0A

It really frustrates me that memory keeps being ad= ded to router outbound buffers anywhere. And it may be that the reason is t= hat almost nobody who designs packet forwarding systems understands Queuein= g Theory at all! It certainly doesn't help that "packet drops" (even one or= two per second) are considered a failure of the equipment.

=0A

 

=0A

FQ-codel is great, but why it works = is that it makes the choice of what packet to drop far better (by being fai= r and a little bit elastic). However, the lack of FQ-Codel doesn't fix syst= em-level bufferbloat.

=0A

 

=0A
=0A

 

=0A=0A_______________________________________________
Starlink mailing = list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink=0A
=0A
 ------=_20220608164923000000_55052--