From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp64.iad3a.emailsrvr.com (smtp64.iad3a.emailsrvr.com [173.203.187.64]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 124F63B29E for ; Thu, 1 Sep 2022 13:08:48 -0400 (EDT) Received: from app14.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by smtp17.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 8001822362; Thu, 1 Sep 2022 13:08:48 -0400 (EDT) Received: from deepplum.com (localhost.localdomain [127.0.0.1]) by app14.wa-webapps.iad3a (Postfix) with ESMTP id 69402600BA; Thu, 1 Sep 2022 13:08:48 -0400 (EDT) Received: by apps.rackspace.com (Authenticated sender: dpreed@deepplum.com, from: dpreed@deepplum.com) with HTTP; Thu, 1 Sep 2022 13:08:48 -0400 (EDT) X-Auth-ID: dpreed@deepplum.com Date: Thu, 1 Sep 2022 13:08:48 -0400 (EDT) From: "David P. Reed" To: starlink@lists.bufferbloat.net MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20220901130848000000_75271" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: References: X-Client-IP: 209.6.168.128 Message-ID: <1662052128.388811686@apps.rackspace.com> X-Mailer: webmail/19.0.18-RC X-Classification-ID: 10c89328-be14-40f2-b370-5ff631d004e9-1-1 Subject: Re: [Starlink] Starlink Digest, Vol 18, Issue 1 X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Sep 2022 17:08:49 -0000 ------=_20220901130848000000_75271 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0AHi Sebastian -=0Aregarding slow start and video, I was just thinking and= I came up with a possible answer regarding the slow start issue compatible= with the Internet architecture. Read the whole piece below, I hope it make= s sense.=0ASince the "bottleneck link" in Starlink will almost certainly be= the downlink shared among multiple dishys, which to me is like the CMTS in= a cable modem system, I think the reasoning is more general than Starlink.= Oddly, :-) the idea has to do with AQM and TCP flow control. Our common i= nterest.=0A =0AIt's a concept, not a full implementation. It doesn't requir= e massive rethinking of the whole Internet. Which means, unlike the QUIC pr= oject, it can be experimented with in just a part of the Internet, but woul= d require a video-server partner and a bottleneck-link operator partner.=0A= =0APlease steal and improve if you like it.=0A =0A- David=0A =0A> Date: Th= u, 1 Sep 2022 09:58:16 +0200=0A=0A> From: Sebastian Moeller =0A> To: Ulrich Speidel =0A> Cc: David Lang , Ulrich Speidel via Starlink=0A> =0A> Subject: Re: [Starlink] Starlink "beam spread"=0A =0A=0A> I a= m prepared to eat crow on this in the future, but I am highly skeptical abo= ut=0A> CDNs in space (in spite of it being a cool project from the technolo= gical side).=0A>=0AYou and me both...=0A> =0A> *) As it looks slow start is= getting a bad rep from multiple sides, but I see not=0A> better alternativ= e out there that solves the challenge slow-start tackles in a=0A> better wa= y. Namely gradual ramping and probing of sending rates/congestion windows= =0A> to avoid collapse, this in turn means that short flows will never reac= h capacity,=0A> the solution to which might well be, use longer flows then.= ..=0A=0A=0AI'm old enough to remember how the TCP slow start problem was fi= rst dealt with in HTTP pretty well (good enough for the time, and focused o= n the end-to-end picture of the WWW architecture (REST).=0AMy friend Jim Ge= ttys was involved, as were a couple of others - I think Jeff Mogul, too.=0A= =0AThe answer was HTTP 1.1. That is, using a single TCP flow for all HTTP = requests to a server. That would increase the flow to get through slow star= t more quickly (aggregating many flows) and holding the flow open, assuming= (almost always correctly) that between the user's clicks and the multiple = parts of content, this would keep the flow out of slow start. (HTTP 1.1's m= ultiplexing of requests onto a single TCP flow had other advantages, too - = OS Internet stacks couldn't handle many simultaneous HTTP connections, addi= ng more delay when a web page was sourced from many servers).=0A =0AAs far = as "pretty good" solution Akamai also solved the bulk of the Web's problem= - NOT by caching at all, but by letting ISPs *purchase* server capacity cl= oser to the bulk of their consumers, and letting that *purchased* capacity = pay for Akamai's costs. This moved the function of what to cache "out of th= e network" to the edge, and didn't require ANY changes to HTTP (in fact, it= tended to concentrate more traffic into a single HTTP 1.1 flow, to a share= d Akamai server.=0A =0AYou may not see where I'm going with this yet - but = actually it's an End-to-End Argument against putting "function into the net= work" when the function is about the end points. With HTTP 1.1, a small cha= nge *at the endpoints of the Web* was easier and simpler than some hypothet= ical "intelligence" in the network that would obviate "slow start". With Ak= amai, a small change that allowed heavily trafficed web services to "buy se= rver capacity" in Akamai's server fabric, which was "at the endpoints of th= e Internet" itself just distributed around the edges of the Internet was go= od enough.=0A =0ASo, let's look at video traffic. Well, first of all, any r= esponse time needs it has is mostly fixed by buffering at the receiver. Not= needing any fixes "in the network" - you want to receive video, allocate a= big buffer and fill it at the start. (This is where slow start gets in the= way, perhaps).=0AI'm pretty sure *most* video watching doesn't go over RTP= anymore - it goes over TCP now. [Dropped frames are interpolated, but inst= ead retransmitted, because the buffer is sufficient in all video watching g= ear at the viewing end. RTP is useful for two-way videoconferencing, but th= at's because conferencing has the 100 msec. need for RTT, unless you are co= nferencing with Mars or the Moon.]=0A =0ASo "slow start" of some sort is th= e current solution that deals with the congestion burst that would disrupt = others early in a video watching session.=0A =0ABut I think you are right -= there are better ways to deal with the shared bottleneck link that a new v= ideo session might encounter. - something like slow start is needed ONLY if= the bottleneck is *shared* among multiple users.=0A =0AThe problem is the = potential of "starvation" of other flows at the bottleneck link along the v= ideo's path.=0A =0AAnd this is a problem we now have one partial solution f= or, but it's not deployed where it needs to be. That is, in a word - "fair"= queuing plus aggressive packet dropping (and maybe aggressive ECN marking)= . Notice I say it needs to be at the *bottleneck link*, not the home router= , which isn't the bottleneck in practice.=0ANow "fair" queueing's most impo= rtant property is that it eliminates starvation or (near starvation) caused= by one flow against another.=0A =0AAnd the key about putting it in the bot= tleneck link (with packet drops being the default means of signalling incip= ient congestion) is that the bottleneck link is the only place in the netwo= rk where any computer knows what the competing traffic is. You need to some= how get both the new flow and the competing flows to negotiate a new "fair"= balance of usage, without building up unacceptable queueing delay.=0A =0AS= o where is the "bottleneck link" today? Well, in cable Internet, it's the = CMTS almost all the time. That is, not at the premises of the watcher. This= is where the congestion needs to be signalled to ALL users from. Some need= to slow down (including the new video watching stream, which has to slow d= own relative to what it is asking for which is to fill up the buffer as fas= t as possible when you start watching).=0A =0AThe problem with video watchi= ng is that when you tune to a new video, you want the buffer to fill "as fa= st as possible", but the others sharing the Internet resources don't want t= o be "starved". They want "fair" queuing (not Harrison Bergeron fairness - = look that up), but a "best efforts" (to use the Cerf and Kahn phrase) fairn= ess, which is non-starvation.=0A =0ABut fair queueing only works if the sou= rces of the traffic cut their rate as a result of congestion signalling.=0A= The real "fairness" requires pushing the problem to the endpoints. It's an = end-to-end argument. That's why we need to replace "slow start" gradually w= ith something better.=0A =0AI don't know what the better thing is, but it n= eeds to trigger the AQM mechanism more quickly than does slow start (includ= ing causing early signals to the sharers of the bottleneck link).=0AIf we a= ssume that packet drops are the only effective signals (because ECN just ha= s failed to be sufficiently widely deployed - a problem that really, really= bothers me), that means that during the initial phase of a TCP session you= need some packets to get dropped between source and sink, and also some pa= ckets on the competing flows to get dropped. But you don't want too much mu= ltiplicative decrease to be triggered in the AIMD control loop of the TCPs = involved (or in the QUIC control loops whatever they may be).=0A=0ASo ideal= ly, a single very high-speed burst of noise packets sent over a period of s= ay 10msec. This would probably only cause a single round of Multiplicative = Decrease. These "noise packets" could be all duplicates of the same packet,= and would thus not require retransmission. But they'd trigger a brief mome= nt of congestion. ------=_20220901130848000000_75271 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Hi Sebastian -

=0A<= p style=3D"margin:0;padding:0;font-family: arial; font-size: 10pt; overflow= -wrap: break-word;">regarding slow start and video, I was just thinking and= I came up with a possible answer regarding the slow start issue compatible= with the Internet architecture. Read the whole piece below, I hope it make= s sense.

=0A

Since the "bottleneck link" in Starlink= will almost certainly be the downlink shared among multiple dishys, which = to me is like the CMTS in a cable modem system, I think the reasoning is mo= re general than Starlink.  Oddly, :-) the idea has to do with AQM and = TCP flow control. Our common interest.

=0A

 =0A

It's a concept, not a full implementation. It does= n't require massive rethinking of the whole Internet. Which means, unlike t= he QUIC project, it can be experimented with in just a part of the Internet= , but would require a video-server partner and a bottleneck-link operator p= artner.

=0A

 

=0A

Please= steal and improve if you like it.

=0A

 

=0A=

- David

=0A

 

=0A

> Date: Thu, 1 Sep 2022 09:58:16 +0200

=0A
=0A

> From: Sebastian Moeller <= moeller0@gmx.de>
> To: Ulrich Speidel <u.speidel@auckl= and.ac.nz>
> Cc: David Lang <david@lang.hm>, Ulrich Speide= l via Starlink
> <starlink@lists.bufferbloat.net>
> S= ubject: Re: [Starlink] Starlink "beam spread"

=0A

&n= bsp;

=0A

<snip>
> I am prepared to eat= crow on this in the future, but I am highly skeptical about
> CDNs= in space (in spite of it being a cool project from the technological side)= .
>

=0A

You and me both...
>
&= gt; *) As it looks slow start is getting a bad rep from multiple sides, but= I see not
> better alternative out there that solves the challenge= slow-start tackles in a
> better way. Namely gradual ramping and p= robing of sending rates/congestion windows
> to avoid collapse, thi= s in turn means that short flows will never reach capacity,
> the s= olution to which might well be, use longer flows then...

=0A=

I'm old enough to remember how the TCP slow start prob= lem was first dealt with in HTTP pretty well (good enough for the time, and= focused on the end-to-end picture of the WWW architecture (REST).

=0AMy friend Jim Gettys was involved, as were a couple of o= thers - I think Jeff Mogul, too.

=0A

 

=0AThe answer was HTTP 1.1. That is, using a single TCP flo= w for all HTTP requests to a server. That would increase the flow to get th= rough slow start more quickly (aggregating many flows) and holding the flow= open, assuming (almost always correctly) that between the user's clicks an= d the multiple parts of content, this would keep the flow out of slow start= . (HTTP 1.1's multiplexing of requests onto a single TCP flow had other adv= antages, too - OS Internet stacks couldn't handle many simultaneous HTTP co= nnections, adding more delay when a web page was sourced from many servers)= .

=0A

 

=0A

As far as "p= retty good" solution Akamai also solved the bulk of the Web's &nb= sp;problem - NOT by caching at all, but by letting ISPs *purchase* server c= apacity closer to the bulk of their consumers, and letting that *purchased*= capacity pay for Akamai's costs. This moved the function of what to cache = "out of the network" to the edge, and didn't require ANY changes to HTTP (i= n fact, it tended to concentrate more traffic into a single HTTP 1.1 flow, = to a shared Akamai server.

=0A

 

=0A

You may not see where I'm going with this yet - but actually i= t's an End-to-End Argument against putting "function into the network" when= the function is about the end points. With HTTP 1.1, a small change *at th= e endpoints of the Web* was easier and simpler than some hypothetical "inte= lligence" in the network that would obviate "slow start". With Akamai, a sm= all change that allowed heavily trafficed web services to "buy server capac= ity" in Akamai's server fabric, which was "at the endpoints of the Internet= " itself just distributed around the edges of the Internet was good enough.=

=0A

 

=0A

So, let's loo= k at video traffic. Well, first of all, any response time needs it has is m= ostly fixed by buffering at the receiver. Not needing any fixes "in the net= work" - you want to receive video, allocate a big buffer and fill it at the= start. (This is where slow start gets in the way, perhaps).

=0A

I'm pretty sure *most* video watching doesn't go over RTP anym= ore - it goes over TCP now. [Dropped frames are interpolated, but instead r= etransmitted, because the buffer is sufficient in all video watching gear a= t the viewing end. RTP is useful for two-way videoconferencing, but that's = because conferencing has the 100 msec. need for RTT, unless you are confere= ncing with Mars or the Moon.]

=0A

 

=0A

So "slow start" of some sort is the current solution that d= eals with the congestion burst that would disrupt others early in a video w= atching session.

=0A

 

=0A

But I think you are right - there are better ways to deal with the share= d bottleneck link that a new video session might encounter. - somethin= g like slow start is needed ONLY if the bottleneck is *shared* among multip= le users.

=0A

 

=0A

The = problem is the potential of "starvation" of other flows at the bottleneck l= ink along the video's path.

=0A

 

=0A

And this is a problem we now have one partial solution for, b= ut it's not deployed where it needs to be. That is, in a word - "fair" queu= ing plus aggressive packet dropping (and maybe aggressive ECN marking). Not= ice I say it needs to be at the *bottleneck link*, not the home router, whi= ch isn't the bottleneck in practice.

=0A

Now "fair" = queueing's most important property is that it eliminates starvation or (nea= r starvation) caused by one flow against another.

=0A

And the key about putting it in the bot= tleneck link (with packet drops being the default means of signalling incip= ient congestion) is that the bottleneck link is the only place in the netwo= rk where any computer knows what the competing traffic is. You need to some= how get both the new flow and the competing flows to negotiate a new "fair"= balance of usage, without building up unacceptable queueing delay.

=0A<= p style=3D"margin:0;padding:0;font-family: arial; font-size: 10pt; overflow= -wrap: break-word;"> 

=0A

So where is the "bott= leneck link" today?  Well, in cable Internet, it's the CMTS almost all= the time. That is, not at the premises of the watcher. This is where the c= ongestion needs to be signalled to ALL users from. Some need to slow down (= including the new video watching stream, which has to slow down relative to= what it is asking for which is to fill up the buffer as fast as possible w= hen you start watching).

=0A

 

=0A

The problem with video watching is that when you tune to a new= video, you want the buffer to fill "as fast as possible", but the others s= haring the Internet resources don't want to be "starved". They want "fair" = queuing (not Harrison Bergeron fairness - look that up), but a "best effort= s" (to use the Cerf and Kahn phrase) fairness, which is non-starvation.

= =0A

 

=0A

But fair queueing= only works if the sources of the traffic cut their rate as a result of con= gestion signalling.

=0A

The real "fairness" requires= pushing the problem to the endpoints. It's an end-to-end argument. That's = why we need to replace "slow start" gradually with something better.

=0A=

 

=0A

I don't know what th= e better thing is, but it needs to trigger the AQM mechanism more quickly t= han does slow start (including causing early signals to the sharers of the = bottleneck link).

=0A

If we assume that packet drops= are the only effective signals (because ECN just has failed to be sufficie= ntly widely deployed - a problem that really, really bothers me), that mean= s that during the initial phase of a TCP session you need some packets to g= et dropped between source and sink, and also some packets on the competing = flows to get dropped. But you don't want too much multiplicative decrease t= o be triggered in the AIMD control loop of the TCPs involved (or in the QUI= C control loops whatever they may be).

So ideally, a single very= high-speed burst of noise packets sent over a period of say 10msec. This w= ould probably only cause a single round of Multiplicative Decrease. These "= noise packets" could be all duplicates of the same packet, and would thus n= ot require retransmission. But they'd trigger a brief moment of congestion.=

=0A
 ------=_20220901130848000000_75271--