From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp75.iad3a.emailsrvr.com (smtp75.iad3a.emailsrvr.com [173.203.187.75]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id B863B3B29D for ; Fri, 29 Jul 2022 15:38:05 -0400 (EDT) Received: from app27.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by smtp34.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 28ABE239F7 for ; Fri, 29 Jul 2022 15:38:05 -0400 (EDT) Received: from deepplum.com (localhost.localdomain [127.0.0.1]) by app27.wa-webapps.iad3a (Postfix) with ESMTP id 0F5CF219E1 for ; Fri, 29 Jul 2022 15:38:05 -0400 (EDT) Received: by apps.rackspace.com (Authenticated sender: dpreed@deepplum.com, from: dpreed@deepplum.com) with HTTP; Fri, 29 Jul 2022 15:38:05 -0400 (EDT) X-Auth-ID: dpreed@deepplum.com Date: Fri, 29 Jul 2022 15:38:05 -0400 (EDT) From: "David P. Reed" To: starlink@lists.bufferbloat.net MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20220729153805000000_40084" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: References: X-Client-IP: 209.6.168.128 Message-ID: <1659123485.059828918@apps.rackspace.com> X-Mailer: webmail/19.0.17-RC X-Classification-ID: 65d9c637-a430-4d4c-b8a5-1d5bb7837e4c-1-1 Subject: Re: [Starlink] Finite-Buffer M/G/1 Queues with Time and Space Priorities X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jul 2022 19:38:05 -0000 ------=_20220729153805000000_40084 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0AFrom: "Bless, Roland (TM)" =0Amodels fromqueueing= theory is that they only work for load < 1, whereaswe are using the networ= k with load values ~1 (i.e., around one) due tocongestion control feedback = loops that drive the bottleneck linkto saturation (unless you consider appl= ication limited traffic sources).=0A =0ALet me remind people here that ther= e is some kind of really weird thinking going on here about what should be = typical behavior in the Intenet when it is working well.=0A =0ANo, the goal= of the Internet is not to saturate all bottlenecks at maximum capacity. Th= at is the opposite of the goal, and it is the opposite of a sane operating = point.=0A =0AEvery user seeks low response time, typically a response time = on the order of the unloaded delay in the network, for ALL traffic. (whethe= r it's the response to a file transfer or a voice frame or a WWW request). = * =0A =0AQueueing is always suboptimal, if you can achieve goodput without = introducing any queueing delay. Because a queue built up at any link delays= *all* traffic sharing that link, so the overall cost to all users goes up = radically when multiple streams share a link, because the queueing *delay* = gets multiplied by the number of flows affected!=0A =0ASo the most desirabl= e operating point (which Kleinrock and his students recently demonstrated w= ith his "power metric") is to have each queue in every link average < 1 pac= ket in length. (big or small packets, doesn't matter, actually).=0A =0ANow = the bigger issue is that this is unachievable when the flows in the network= are bursty. Poisson being the least bursty, and easiest to analyze of the = random processes generating flows. Typical Internet usage is incredibly bur= sty at all time scales, though - the burstiness is fractal when observed fo= r real (at least if you look at time scales from 1 ms. to 1 day as your uni= t of analysis). Fractal random processes of this sort are not Poisson at a= ll.=0A =0ASo what is the best one ought to try to do?=0A =0AWell, "keeping = utilization at 100%" is never what real network operators seek. Never, ever= . Instead, congestion control is focused on latency control, not optimizing= utilization.=0A =0AThe only folks who seem to focus on utilization is the = bean counting fraternity, because they seem to think the only cost is the w= ires, so you want the wires to be full. That, in my opinion, and even in mo= st accounting systems that consider the whole enterprise rather than the wi= res/fibers/airtime alone, is IGNORANT and STUPID.=0A =0AHowever, academics = and vendors of switches care nothing about latency at network scale. They f= ocus on wirespeed as the only metric.=0A =0AWell, in the old Bell Telephone= days, the metric of the Bell System that really mattered was not utilizati= on on every day. Instead it was avoiding outages due to peak load. That oft= en was "Mother's Day" - a few hours out of one day once a year. Because an = outage on Mother's day (busy signals) meant major frustration!=0A =0AWhy am= I talking about this?=0A =0ABecause I have been trying for decades (and I = am not alone) to apply a "Clue-by-Four" to the thick skulls of folks who do= n't think about the Internet at scale, or even won't think about an Enterpr= ise Internet at scale (or Starlink at scale). And it doesn't sink in.=0A = =0AAndrew Odlyzko, a brilliant mathematician at Bell Labs for most of his c= areer also tried to point out that the utilization of the "bottleneck links= " in any enterprise, up to the size of ATT in the old days, was typically t= uned to < 10% of saturation at almost any time. Why? Because the CEO freake= d out at the quality of service of this critical infrastructure (which mean= s completing tasks quickly, when load is unusual) and fired people.=0A =0AA= nd in fact, the wires are the cheapest resource - the computers and people = connected by those resources that can't do work while waiting for queueing = delay are vastly more expensive to leave idle. Networks don't do "work" tha= t matters. Queueing isn't "efficient". It's evil.=0A =0AWhich is why droppi= ng packets rather then queueing them is *good*, if the sender will slow dow= n and can resend them. Intentially dropped packets should be nonzero under = load, if an outsider is observing for measruing quality.=0A =0AI call this = brain-miswiring about optimizing throughput to fill a bottleneck link the H= otrodder Fallacy. That's the idea that one should optimize like a drag race= r optimizes his car - to burn up the tires and the engine to meet an irrele= vant metric for automobiles. A nice hobby that has never improved any actua= l vehicle. (Even F1 racing is far more realistic, given you want your cars = to last for the lifetime of the race).=0A =0AA problem with much of the "ne= twork research" community is that it never has actually looked at what netw= orks are used for and tried to solve those problems. Instead, they define i= rrelevant problems and encourage all students and professors to pursue irre= levancy.=0A =0ANow let's look at RRUL. While it nicely looks at latency for= small packets under load, it actually disregards the performance of the lo= ad streams, which are only used to "fill the pipe". Fortunately, they are T= CP, so they rate limit themselves by window adjustment. But they are speed = unlimited TCP streams that are meaningless.=0A =0AActual situations (like w= hat happens when someone starts using BitTorrent while another in the same = household is playing a twitch Multi-user FPS) don't actually look like RRUL= . Because in fact the big load is ALSO fractal. Bittorrent demand isn't con= stant over time - far from it. It's bursty.=0A =0AEverything is bursty at d= ifferent timescales in the Internet. There are no CBR flows.=0A =0ASo if we= want to address the real congestion problems, we need realistic thinking a= bout what the real problem is.=0A =0AUnfortunately this is not achieved by = the kind of thinking that created diffserv, sadly. Because everything is bu= rsty, just with different timescales in some cases. Even "flash override" p= riority traffic is incredibly bursty.=0A =0AComing back to Starlink - Starl= ink apparently is being designed by folks who really do not understand thes= e fundamental ideas. Instead, they probably all worked in researchy environ= ments where the practical realities of being part of a worldwide public Int= ernet were ignored.=0A (The FCC folks are almost as bad. I have found no-on= e at FCC engineering who understands fractal burstiness - even w.'t. the ol= d Bell System).=0A =0A =0A ------=_20220729153805000000_40084 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

From: "B= less, Roland (TM)" <roland.= bless@kit.edu>

=0A

models fro= m
queueing theory is that they only work for lo= ad < 1, whereas
we are using the network wit= h load values ~1 (i.e., around one) due to
cong= estion control feedback loops that drive the bottleneck link
to saturation (unless you consider application limited traffi= c sources).

=0A

 =0A

Let me remind people here that there is some kind of rea= lly weird thinking going on here about what should be typical behavior in t= he Intenet when it is working well.

=0A

 = ;

=0A

No, the goal of the Internet is not to saturate all = bottlenecks at maximum capacity. That is the opposite of the goal, and it i= s the opposite of a sane operating point.

=0A

 

=0A

Every user seeks low response time, typically = a response time on the order of the unloaded delay in the network, for ALL = traffic. (whether it's the response to a file transfer or a voice frame or = a WWW request). * 

=0A

 

=0A

Queueing is always suboptimal, if you can achieve goodput withou= t introducing any queueing delay. Because a queue built up at any link dela= ys *all* traffic sharing that link, so the overall cost to all users goes u= p radically when multiple streams share a link, because the queueing *delay= * gets multiplied by the number of flows affected!

=0A

 

=0A

So the most desirable operating point= (which Kleinrock and his students recently demonstrated with his "power me= tric") is to have each queue in every link average < 1 packet in length.= (big or small packets, doesn't matter, actually).

=0A

 

=0A

Now the bigger issue is that this is = unachievable when the flows in the network are bursty. Poisson being the le= ast bursty, and easiest to analyze of the random processes generating flows= . Typical Internet usage is incredibly bursty at all time scales, though - = the burstiness is fractal when observed for real (at least if you look at t= ime scales from 1 ms. to 1 day as your unit of analysis).  Fractal ran= dom processes of this sort are not Poisson at all.

=0A

 

=0A

So what is the best one ought to try = to do?

=0A

 

=0A

= Well, = "keeping utilization at 100%" is never what real network operators seek. Ne= ver, ever. Instead, congestion control is focused on latency control, not o= ptimizing utilization.

=0A

 

=0A

The only folks who seem to focus on utilization is the bean count= ing fraternity, because they seem to think the only cost is the wires, so y= ou want the wires to be full. That, in my opinion, and even in most account= ing systems that consider the whole enterprise rather than the wires/fibers= /airtime alone, is IGNORANT and STUPID.

=0A

&= nbsp;

=0A

However, academics and vendors of switches care = nothing about latency at network scale. They focus on wirespeed as the only= metric.

=0A

 

=0A

Well= , in the old Bell Telephone days, the metric of the Bell System that really= mattered was not utilization on every day. Instead it was avoiding outages= due to peak load. That often was "Mother's Day" - a few hours out of one d= ay once a year. Because an outage on Mother's day (busy signals) meant majo= r frustration!

=0A

 

=0A

 

= =0A

Because I have been trying for decades (and I am not alon= e) to apply a "Clue-by-Four" to the thick skulls of folks who don't think a= bout the Internet at scale, or even won't think about an Enterprise Interne= t at scale (or Starlink at scale). And it doesn't sink in.

=0A

 

=0A

Andrew Odlyzko, a brilliant m= athematician at Bell Labs for most of his career also tried to point out th= at the utilization of the "bottleneck links" in any enterprise, up to the s= ize of ATT in the old days, was typically tuned to < 10% of saturation a= t almost any time. Why? Because the CEO freaked out at the quality of servi= ce of this critical infrastructure (which means completing tasks quickly, w= hen load is unusual) and fired people.

=0A

&n= bsp;

=0A

And in fact, the wires are the cheapest resource = - the computers and people connected by those resources that can't do work = while waiting for queueing delay are vastly more expensive to leave idle. N= etworks don't do "work" that matters. Queueing isn't "efficient". It's evil= .

=0A

 

=0A

Which is wh= y dropping packets rather then queueing them is *good*, if the sender will = slow down and can resend them. Intentially dropped packets should be nonzer= o under load, if an outsider is observing for measruing quality.

= =0A

 

=0A

I call this brain-miswi= ring about optimizing throughput to fill a bottleneck link the Hotrodder Fa= llacy. That's the idea that one should optimize like a drag racer optimizes= his car - to burn up the tires and the engine to meet an irrelevant metric= for automobiles. A nice hobby that has never improved any actual vehicle. = (Even F1 racing is far more realistic, given you want your cars to last for= the lifetime of the race).

=0A

 

=0A=

A problem with much of the "network research" community is t= hat it never has actually looked at what networks are used for and tried to= solve those problems. Instead, they define irrelevant problems and encoura= ge all students and professors to pursue irrelevancy.

=0A

 

=0A

Now let's look at RRUL. While it n= icely looks at latency for small packets under load, it actually disregards= the performance of the load streams, which are only used to "fill the pipe= ". Fortunately, they are TCP, so they rate limit themselves by window adjus= tment. But they are speed unlimited TCP streams that are meaningless.

=0A

 

=0A

Actual situations= (like what happens when someone starts using BitTorrent while another in t= he same household is playing a twitch Multi-user FPS) don't actually look l= ike RRUL. Because in fact the big load is ALSO fractal. Bittorrent demand i= sn't constant over time - far from it. It's bursty.

=0A

 

=0A

Everything is bursty at different = timescales in the Internet. There are no CBR flows.

=0A

 

=0A

So if we want to address the real = congestion problems, we need realistic thinking about what the real problem= is.

=0A

 

=0A

Unfortun= ately this is not achieved by the kind of thinking that created diffserv, s= adly. Because everything is bursty, just with different timescales in some = cases. Even "flash override" priority traffic is incredibly bursty.<= /p>=0A

 

=0A

Coming back t= o Starlink - Starlink apparently is being designed by folks who really do n= ot understand these fundamental ideas. Instead, they probably all worked in= researchy environments where the practical realities of being part of a wo= rldwide public Internet were ignored.

=0A

 (The FCC folks are almost as bad. I have found no-one at FCC engi= neering who understands fractal burstiness - even w.'t. the old Bell System= ).

=0A

 

=0A

 

= =0A

 

 ------=_20220729153805000000_40084--