From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp98.iad3a.emailsrvr.com (smtp98.iad3a.emailsrvr.com [173.203.187.98]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id E5B843B29E for ; Sat, 15 Jun 2019 16:26:32 -0400 (EDT) Received: from smtp29.relay.iad3a.emailsrvr.com (localhost [127.0.0.1]) by smtp29.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id AED7123411; Sat, 15 Jun 2019 16:26:32 -0400 (EDT) X-SMTPDoctor-Processed: csmtpprox beta Received: from app1.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by smtp29.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 73702233EE; Sat, 15 Jun 2019 16:26:32 -0400 (EDT) X-Sender-Id: dpreed@deepplum.com Received: from app1.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by 0.0.0.0:25 (trex/5.7.12); Sat, 15 Jun 2019 16:26:32 -0400 Received: from deepplum.com (localhost.localdomain [127.0.0.1]) by app1.wa-webapps.iad3a (Postfix) with ESMTP id 5F34BE008B; Sat, 15 Jun 2019 16:26:32 -0400 (EDT) Received: by apps.rackspace.com (Authenticated sender: dpreed@deepplum.com, from: dpreed@deepplum.com) with HTTP; Sat, 15 Jun 2019 16:26:32 -0400 (EDT) X-Auth-ID: dpreed@deepplum.com Date: Sat, 15 Jun 2019 16:26:32 -0400 (EDT) From: "David P. Reed" To: "Dave Taht" Cc: "Luca Muscariello" , "ecn-sane@lists.bufferbloat.net" , "Bob Briscoe" , "tsvwg@ietf.org" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20190615162632000000_76806" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: <87sgsbkfk0.fsf@taht.net> References: <364514D5-07F2-4388-A2CD-35ED1AE38405@akamai.com> <87sgsbkfk0.fsf@taht.net> Message-ID: <1560630392.386610323@apps.rackspace.com> X-Mailer: webmail/16.4.5-RC Subject: Re: [Ecn-sane] [tsvwg] CoIt'smments on L4S drafts X-BeenThere: ecn-sane@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of explicit congestion notification's impact on the Internet List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Jun 2019 20:26:33 -0000 ------=_20190615162632000000_76806 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0AIt's essential that the normal state of the Internet, everywhere, is tha= t all queues, except at the ultimate source and destination, average to < 1= packet queued on the outgoing link.=0A =0AThat's for many interlocking rea= sons. Transient queue buildup MUST be drained back to less than 1 packet w= ith alacrity. All queue buildups must sit either at the entry to the shared= network or at the recipient node.=0A =0ANow it is straightforward algorith= mically to prioritize competing flows, basically by changing the packet adm= ission rates *at the entry* (through windowing or rate control). To do so r= equires that there be enough information reflected into each flow (by drops= or ECN or whatever) to cause the rates/windowing controls to selectively p= rioritize the scheduling of admission at the entry endpoint.=0A =0AThese ar= e in the nature of desirable invariants achieved by the interaction of the = distributed system of flows.=0A =0AThus, the task of scheduling packets at = every congestion point (whether there are priorities or not) must keep the = queues short.=0A =0AIMO, far too much focus is currently maintained on "wit= hin router" algorithmics. The problem of congestion is entirely a system-w= ide problem, not a router problem.=0A =0AUse a little queueing theory and c= ontrol theory to understand this. It's non-standard queueing and control th= eory, because the control is "decentralized" and "distributed".=0A =0AThe d= esign of the Internet *requires* no central controller. That's its design p= oint for very good reasons. That's why we (not you kids) built it.=0A =0AOn= e can imagine something called "5G" (not the Internet) that has a master wo= rld control center. It won't scale, but it is a fantasy of phone companies = like BT and suppliers like ALU. Feel free to design that thing. Just don't = think that would be an "Internet".=0A =0A =0A =0A =0AOn Friday, June 14, 20= 19 5:44pm, "Dave Taht" said:=0A=0A=0A=0A> =0A> This thread = using unconventional markers for it is hard to follow.=0A> =0A> Luca Muscar= iello writes:=0A> =0A> > On Fri, Jun 7, 2019 at 8:10= PM Bob Briscoe =0A> > wrote:=0A> >=0A> >=0A> >=0A> >= =0A> > I'm afraid there are not the same pressures to cause rapid=0A> > rol= l-out at all, cos it's flakey now, jam tomorrow. (Actually=0A> > ECN-DualQ-= SCE has a much greater problem - complete starvation of=0A> > SCE flows - b= ut we'll come on to that in Q4.)=0A> =0A> Answering that statement is the o= nly reason why I popped up here.=0A> more below.=0A> =0A> > I want to say a= t this point, that I really appreciate all the=0A> > effort you've been put= ting in, trying to find common ground.=0A> =0A> I am happy to see this thre= ad happen also, and I do plan to=0A> stay out of it.=0A> =0A> >=0A> > In tr= ying to find a compromise, you've taken the fire that is=0A> > really aimed= at the inadequacy of underlying SCE protocol - for=0A> > anything other th= an FQ.=0A> =0A> The SCE idea does, indeed work best with FQ, in a world of = widely=0A> varying congestion control ideas as explored in the recent paper= , 50=0A> shades of congestion control:=0A> =0A> https://arxiv.org/pdf/1903.= 03852.pdf=0A> =0A> > If the primary SCE proponents had=0A> > attempted to a= rticulate a way to use SCE in a single queue or a=0A> > dual queue, as you = have, that would have taken my fire.=0A> =0A> I have no faith in single or = dual queues with ECN either, due to=0A> how anyone can scribble on the rele= vant bits, however...=0A> =0A> >=0A> >=0A> > But regardless, the queue-buil= ding from classic ECN-capable endpoints=0A> that=0A> > only get 1 congestio= n signal per RTT is what I understand as the main=0A> > downside of the tra= deoff if we try to use ECN-capability as the dualq=0A> > classifier. Does t= hat match your understanding?=0A> >=0A> > This is indeed a major concern of= mine (not as major as the=0A> > starvation of SCE explained under Q4, but = we'll come to that).=0A> =0A> I think I missed a portion of this thread. St= arvation is impossible,=0A> you are reduced to no less than cwnd 2 (non-bbr= ), or cwnd 4 (bbr).=0A> =0A> Your own work points out a general problem wit= h needing sub-packet=0A> windows with too many flows that cause excessive m= arking using CE, which=0A> so far as I know remains an unsolved problem.=0A= > =0A> https://arxiv.org/pdf/1904.07598.pdf=0A> =0A> This is easily demonst= rated via experiment, also, and the primary reason=0A> why, even with FQ_co= del in the field, we generally have turned off ecn=0A> support at low bitra= tes until the first major release of sch_cake.=0A> =0A> I had an open quest= ion outstanding about the 10% figure for converting=0A> to drop sch_pie use= s that remains unresolved.=0A> =0A> As for what level of compatability with= classic transports in a single=0A> queue that is possible with a SCE capab= le receiver and sender, that=0A> remains to be seen. Only the bits have bee= n defined as yet. Two=0A> approaches are being tried in public, so far.=0A>= =0A> >=0A> > Fine-grained (DCTCP-like) and coarse-grained (Cubic-like)=0A>= > congestion controls need to be isolated, but I don't see how,=0A> > unle= ss their packets are tagged for separate queues. Without a=0A> > specific f= ine/coarse identifier, we're left with having to re-use=0A> > other identif= iers:=0A> >=0A> > * You've tried to use ECN vs Not-ECN. But that still lump= s two=0A> > large incompatible groups (fine ECN and coarse ECN) together.= =0A> > * The only alternative that would serve this purpose is the flow=0A>= > identifier at layer-4, because it isolates everything from=0A> > everyth= ing else. FQ is where SCE started, and that seems to be=0A> > as far as it = can go.=0A> =0A> Actually, I was seeking a solution (and had been, for goin= g on 5 years)=0A> to the "too many flows not getting out of slow start fast= enough",=0A> problem, which you can see from any congested airport, public= space,=0A> small office, or coffeeshop nowadays. The vast majority of traf= fic=0A> there does not consist of long duration high rate flows.=0A> =0A> E= ven if you eliminate the wireless retries and rate changes and put in a=0A>= good fq_codel aqm, the traffic in such a large shared environment is=0A> m= ostly flows lacking a need for congestion control at all (dns, voip,=0A> et= c), or in slow start, hammering away at ever increasing delays in=0A> those= environments until the user stops hitting the reload button.=0A> =0A> Othe= rs have different goals and outlooks in this project and I'm=0A> not really= part of that.=0A> =0A> I would rather like to see both approaches tried in= an environment=0A> that had a normal mix of traffic in a shared environmen= t like that.=0A> =0A> Some good potential solutions include reducing the sl= ower bits of the=0A> internet back to IW4 and/or using things like initial = spreading, both of=0A> which are good ideas and interact well with SCE's mo= re immediate=0A> response curve, paced chirping also.=0A> =0A> >=0A> > Shou= ld we burn the last unicorn for a capability needed on=0A> > "carrier-scale= " boxes, but which requires FQ to work? Perhaps yes=0A> > if there was no a= lternative. But there is: L4S.=0A> =0A> The core of the internet is simply = overprovisioned, with fairly short=0A> queues. DCTCP itself did not deploy = in very many places that I know of.=0A> =0A> could you define exactly what = carrier scale means?=0A> =0A> >=0A> >=0A> >=0A> > I have a problem to under= stand why all traffic ends up to be=0A> > classified as either Cubic-like o= r DCTCP-like.=0A> > If we know that this is not true today I fail to unders= tand why this=0A> > should be the case in the future.=0A> > It is also diff= icult to predict now how applications will change in=0A> > the future in te= rms of the traffic mix they'll generate.=0A> > I feel like we'd be moving t= owards more customized transport services=0A> > with less predictable patte= rns.=0A> >=0A> > I do not see for instance much discussion about the presen= ce of RTC=0A> > traffic and how the dualQ system behaves when the=0A> > inp= ut traffic does not respond as expected by the 2-types of sources=0A> > ass= umed by dualQ.=0A> >=0A> > If my application is using simulcast or multi-st= ream techniques I can=0A> > have several video streams in the same link, th= at, as far as I=0A> > understand,=0A> > will get significant latency in the= classic queue. Unless my app=0A> > starts cheating by marking packets to g= et into the priority queue.=0A> >=0A> > In both cases, i.e. my RTC app is c= heating or not, I do not understand=0A> > how the parametrization of the du= alQ scheduler=0A> > can cope with traffic that behaves in a different way t= o what is=0A> > assumed while tuning parameters.=0A> > For instance, in one= instantiation of dualQ based on WRR the weights=0A> > are set to 1:16. Thi= s has to necessarily=0A> > change when RTC traffic is present. How?=0A> >= =0A> > Is the assumption that a trusted marker is used as in typical diffse= rv=0A> > deployments=0A> > or that a policer identifies and punishes cheati= ng applications?=0A> >=0A> > BTW I'd love to understand how dualQ is suppos= ed to work under more=0A> > general traffic assumptions.=0A> >=0A> > Luca= =0A> _______________________________________________=0A> Ecn-sane mailing l= ist=0A> Ecn-sane@lists.bufferbloat.net=0A> https://lists.bufferbloat.net/li= stinfo/ecn-sane=0A> ------=_20190615162632000000_76806 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

It's essential that th= e normal state of the Internet, everywhere, is that all queues, except at t= he ultimate source and destination, average to < 1 packet queued on the = outgoing link.

=0A

 

=0A

That's for many interlocking reasons.  Transient queue buildup MUST b= e drained back to less than 1 packet with alacrity. All queue buildups must= sit either at the entry to the shared network or at the recipient node.=0A

 

=0A

Now it is straig= htforward algorithmically to prioritize competing flows, basically by chang= ing the packet admission rates *at the entry* (through windowing or rate co= ntrol). To do so requires that there be enough information reflected into e= ach flow (by drops or ECN or whatever) to cause the rates/windowing control= s to selectively prioritize the scheduling of admission at the entry endpoi= nt.

=0A

 

=0A

These are = in the nature of desirable invariants achieved by the interaction of the di= stributed system of flows.

=0A

 

=0A

Thus, the task of scheduling packets at every congestion point= (whether there are priorities or not) must keep the queues short.

=0A 

=0A

IMO, far too much focu= s is currently maintained on "within router" algorithmics.  The proble= m of congestion is entirely a system-wide problem, not a router problem.=0A

 

=0A

Use a little que= ueing theory and control theory to understand this. It's non-standard queue= ing and control theory, because the control is "decentralized" and "distrib= uted".

=0A

 

=0A

The des= ign of the Internet *requires* no central controller. That's its design poi= nt for very good reasons. That's why we (not you kids) built it.

=0A

 

=0A

One can imagine somethin= g called "5G" (not the Internet) that has a master world control center. It= won't scale, but it is a fantasy of phone companies like BT and suppliers = like ALU. Feel free to design that thing. Just don't think that would be an= "Internet".

=0A

 

=0A

&= nbsp;

=0A

 

=0A

 =0A

On Friday, June 14, 2019 5:44pm, "Dave Taht" <= dave@taht.net> said:

=0A
= =0A

>
> This thread using unconventional ma= rkers for it is hard to follow.
>
> Luca Muscariello <m= uscariello@ieee.org> writes:
>
> > On Fri, Jun 7, 20= 19 at 8:10 PM Bob Briscoe <ietf@bobbriscoe.net>
> > wrote:=
> >
> >
> >
> >
> >= I'm afraid there are not the same pressures to cause rapid
> > = roll-out at all, cos it's flakey now, jam tomorrow. (Actually
> >= ; ECN-DualQ-SCE has a much greater problem - complete starvation of
&g= t; > SCE flows - but we'll come on to that in Q4.)
>
> = Answering that statement is the only reason why I popped up here.
>= more below.
>
> > I want to say at this point, that I = really appreciate all the
> > effort you've been putting in, try= ing to find common ground.
>
> I am happy to see this thre= ad happen also, and I do plan to
> stay out of it.
>
= > >
> > In trying to find a compromise, you've taken the f= ire that is
> > really aimed at the inadequacy of underlying SCE= protocol - for
> > anything other than FQ.
>
>= The SCE idea does, indeed work best with FQ, in a world of widely
>= ; varying congestion control ideas as explored in the recent paper, 50
> shades of congestion control:
>
> https://arxiv.org/= pdf/1903.03852.pdf
>
> > If the primary SCE proponents = had
> > attempted to articulate a way to use SCE in a single que= ue or a
> > dual queue, as you have, that would have taken my fi= re.
>
> I have no faith in single or dual queues with ECN = either, due to
> how anyone can scribble on the relevant bits, howe= ver...
>
> >
> >
> > But regardle= ss, the queue-building from classic ECN-capable endpoints
> that> > only get 1 congestion signal per RTT is what I understand as t= he main
> > downside of the tradeoff if we try to use ECN-capabi= lity as the dualq
> > classifier. Does that match your understan= ding?
> >
> > This is indeed a major concern of mine = (not as major as the
> > starvation of SCE explained under Q4, b= ut we'll come to that).
>
> I think I missed a portion of = this thread. Starvation is impossible,
> you are reduced to no less= than cwnd 2 (non-bbr), or cwnd 4 (bbr).
>
> Your own work= points out a general problem with needing sub-packet
> windows wit= h too many flows that cause excessive marking using CE, which
> so = far as I know remains an unsolved problem.
>
> https://arx= iv.org/pdf/1904.07598.pdf
>
> This is easily demonstrated = via experiment, also, and the primary reason
> why, even with FQ_co= del in the field, we generally have turned off ecn
> support at low= bitrates until the first major release of sch_cake.
>
> I= had an open question outstanding about the 10% figure for converting
= > to drop sch_pie uses that remains unresolved.
>
> As = for what level of compatability with classic transports in a single
&g= t; queue that is possible with a SCE capable receiver and sender, that
> remains to be seen. Only the bits have been defined as yet. Two
= > approaches are being tried in public, so far.
>
> >= ;
> > Fine-grained (DCTCP-like) and coarse-grained (Cubic-like)<= br />> > congestion controls need to be isolated, but I don't see how= ,
> > unless their packets are tagged for separate queues. Witho= ut a
> > specific fine/coarse identifier, we're left with having= to re-use
> > other identifiers:
> >
> > = * You've tried to use ECN vs Not-ECN. But that still lumps two
> &g= t; large incompatible groups (fine ECN and coarse ECN) together.
> = > * The only alternative that would serve this purpose is the flow
= > > identifier at layer-4, because it isolates everything from
&= gt; > everything else. FQ is where SCE started, and that seems to be
> > as far as it can go.
>
> Actually, I was seeki= ng a solution (and had been, for going on 5 years)
> to the "too ma= ny flows not getting out of slow start fast enough",
> problem, whi= ch you can see from any congested airport, public space,
> small of= fice, or coffeeshop nowadays. The vast majority of traffic
> there = does not consist of long duration high rate flows.
>
> Eve= n if you eliminate the wireless retries and rate changes and put in a
= > good fq_codel aqm, the traffic in such a large shared environment is> mostly flows lacking a need for congestion control at all (dns, vo= ip,
> etc), or in slow start, hammering away at ever increasing del= ays in
> those environments until the user stops hitting the reload= button.
>
> Others have different goals and outlooks in t= his project and I'm
> not really part of that.
>
>= I would rather like to see both approaches tried in an environment
&g= t; that had a normal mix of traffic in a shared environment like that.
>
> Some good potential solutions include reducing the slower = bits of the
> internet back to IW4 and/or using things like initial= spreading, both of
> which are good ideas and interact well with S= CE's more immediate
> response curve, paced chirping also.
>= ;
> >
> > Should we burn the last unicorn for a capa= bility needed on
> > "carrier-scale" boxes, but which requires F= Q to work? Perhaps yes
> > if there was no alternative. But ther= e is: L4S.
>
> The core of the internet is simply overprov= isioned, with fairly short
> queues. DCTCP itself did not deploy in= very many places that I know of.
>
> could you define exa= ctly what carrier scale means?
>
> >
> >
> >
> > I have a problem to understand why all traffic e= nds up to be
> > classified as either Cubic-like or DCTCP-like.<= br />> > If we know that this is not true today I fail to understand = why this
> > should be the case in the future.
> > It= is also difficult to predict now how applications will change in
>= > the future in terms of the traffic mix they'll generate.
> &g= t; I feel like we'd be moving towards more customized transport services> > with less predictable patterns.
> >
> > = I do not see for instance much discussion about the presence of RTC
&g= t; > traffic and how the dualQ system behaves when the
> > in= put traffic does not respond as expected by the 2-types of sources
>= ; > assumed by dualQ.
> >
> > If my application is= using simulcast or multi-stream techniques I can
> > have sever= al video streams in the same link, that, as far as I
> > underst= and,
> > will get significant latency in the classic queue. Unle= ss my app
> > starts cheating by marking packets to get into the= priority queue.
> >
> > In both cases, i.e. my RTC a= pp is cheating or not, I do not understand
> > how the parametri= zation of the dualQ scheduler
> > can cope with traffic that beh= aves in a different way to what is
> > assumed while tuning para= meters.
> > For instance, in one instantiation of dualQ based on= WRR the weights
> > are set to 1:16. This has to necessarily> > change when RTC traffic is present. How?
> >
&= gt; > Is the assumption that a trusted marker is used as in typical diff= serv
> > deployments
> > or that a policer identifies= and punishes cheating applications?
> >
> > BTW I'd = love to understand how dualQ is supposed to work under more
> > = general traffic assumptions.
> >
> > Luca
> _= ______________________________________________
> Ecn-sane mailing l= ist
> Ecn-sane@lists.bufferbloat.net
> https://lists.buffer= bloat.net/listinfo/ecn-sane
>

=0A
 ------=_20190615162632000000_76806--