From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi1-x235.google.com (mail-oi1-x235.google.com [IPv6:2607:f8b0:4864:20::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id C2FB73B2A4 for ; Sat, 22 Jun 2019 18:30:29 -0400 (EDT) Received: by mail-oi1-x235.google.com with SMTP id e189so7095322oib.11 for ; Sat, 22 Jun 2019 15:30:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=t+XKFlNYnDboUUENjj6o0X/S4+2G4+QpY23GH9aiOYw=; b=klKDWUyNMhytdU4RttLRm6wdMfl0QZD451kfsX/LH6kpEuCy1azEsp2y+ThPVTg1vc xtysXgJ1UuMmAl0jILiwJbxYeUJsmYfTXRhgOOn6mDQKxWAeLf4ZhNnKeWZi4U4hDnMP 2i1glt+7fs9PdeWo1ZqgNEF8YxkplzlXVjy0uC4ns5yipZ2MLwSrJsmU1kJW7OERMWYw m12EXKA0mfxGu3o7WyBctNitolBJ4NUt2SZCbms1ZuIaa+8s74h5h8NpBNvwuTc73Hnx 8MBejJaxnA264mZPSsUOQgY9smz3bPM2BytRju3XWD4kv3BuEi5tTyGhGum1j6/37pF9 qTsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=t+XKFlNYnDboUUENjj6o0X/S4+2G4+QpY23GH9aiOYw=; b=JpWejcBGXQvr4m0c6Yn8UfZqDf00EfWjAD9Oz5OOewYNCOvgUTH+ULa33EP7N6sDba xvYVZpp3WJ81wC/5KUOR281+uqQ21Z8AvMGjjClbKNfi1bgf0j2ymBd9dYVmU2bQ3pON 4EFlYZfVIXIHPHOYPZiytJZjuLwP/GWnqobcm+O0zq4H/Tjs6kzpliLZyh4V6s4Ux4Vr XPj3ASYHPzBYlbpoYMhn0xsCFhYJzVr9MZ/lw37k+aikzEOaCc0NamR4l/aihMlSwt9Q 3Hm/lhFuRElnoBBcwyuax7VVXTAkFoXRRU2JpxO/9WSJu8o1PVOdAAaayKSuZth+VG7N XCag== X-Gm-Message-State: APjAAAUtm+xKXesAwnPm4TCMsh4u7tliJFID/L+Wg19XENEIf7qzwpu7 1yTVAaepA39eF46gDYauOY0WvHm/f7XLJ82DHMc= X-Google-Smtp-Source: APXvYqyXDaWF+Lm+TqXqkPZBWOaEA1YNENdQ+WV6sFqHKlJCTohFneiwy5z96Pq61XJVfMOkVuHMZSzxhgso7mhBWmw= X-Received: by 2002:aca:cdc8:: with SMTP id d191mr6498932oig.151.1561242628973; Sat, 22 Jun 2019 15:30:28 -0700 (PDT) MIME-Version: 1.0 References: <350f8dd5-65d4-d2f3-4d65-784c0379f58c@bobbriscoe.net> <46D1ABD8-715D-44D2-B7A0-12FE2A9263FE@gmx.de> <835b1fb3-e8d5-c58c-e2f8-03d2b886af38@gmail.com> <1561233009.95886420@apps.rackspace.com> <85397b28-4a7a-e125-40b9-9cfce574260a@gmail.com> <1561242313.903247453@apps.rackspace.com> In-Reply-To: <1561242313.903247453@apps.rackspace.com> From: Luca Muscariello Date: Sun, 23 Jun 2019 00:30:18 +0200 Message-ID: To: "David P. Reed" Cc: Brian E Carpenter , Sebastian Moeller , "ecn-sane@lists.bufferbloat.net" , tsvwg IETF list Content-Type: multipart/alternative; boundary="0000000000004ee785058bf120b7" Subject: Re: [Ecn-sane] [tsvwg] per-flow scheduling X-BeenThere: ecn-sane@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of explicit congestion notification's impact on the Internet List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Jun 2019 22:30:29 -0000 --0000000000004ee785058bf120b7 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks for the insights. On Sun 23 Jun 2019 at 00:25, David P. Reed wrote: > Given the complexity of my broader comments, let me be clear that I have > no problem with the broad concept of diffserv being compatible with the > end-to-end arguments. I was trying to lay out what I think is a useful wa= y > to think about these kinds of issues within the Internet context. > > > > Similarly, per-flow scheduling as an end-to-end concept (different flows > defined by address pairs being jointly managed as entities) makes great > sense, but it's really important to be clear that queue prioritization > within a single queue at entry to a bottleneck link is a special case > mechanism, and not a general end-to-end concept at the IP datagram level, > given the generality of IP as a network packet transport protocol. It's > really tied closely to routing, which isn't specified in any way by IP, > other than "best efforts", a term that has become much more well defined > over the years (including the notions of dropping rather than storing > packets, the idea that successive IP datagrams should traverse roughly th= e > same path in order to have stable congestion detection, ...). > > > > Per-flow scheduling seems to work quite well in the cases where it > applies, transparently below the IP datagram layer (that is, underneath t= he > hourglass neck). IP effectively defines "flows", and it is reasonable to = me > that "best efforts" as a concept could include some notion of network-wid= e > fairness among flows. Link-level "fairness" isn't a necessary preconditio= n > to network level fairness. > > > > On Saturday, June 22, 2019 5:10pm, "Brian E Carpenter" < > brian.e.carpenter@gmail.com> said: > > > Just three or four small comments: > > > > On 23-Jun-19 07:50, David P. Reed wrote: > > > Two points: > > > > > > > > > > > > - Jerry Saltzer and I were the primary authors of the End-to-end > argument > > paper, and the motivation was based *my* work on the original TCP and I= P > > protocols. Dave Clark got involved significantly later than all those > decisions, > > which were basically complete when he got involved. (Jerry was my thesi= s > > supervisor, I was his student, and I operated largely independently, > taking input > > from various others at MIT). I mention this because Dave understands th= e > > end-to-end arguments, but he understands (as we all did) that it was a > design > > *principle* and not a perfectly strict rule. That said, it's a rule tha= t > has a > > strong foundational argument from modularity and evolvability in a > context where > > the system has to work on a wide range of infrastructures (not all > knowable in > > advance) and support a wide range of usage/application-areas (not all > knowable in > > advance). Treating the paper as if it were "DDC" declaring a law is jus= t > wrong. He > > wasn't Moses and it is not written on tablets. Dave > > > did have some "power" in his role of trying to achieve interoperabili= ty > > across diverse implementations. But his focus was primarily on > interoperability, > > not other things. So ideas in the IP protocol like "TOS" which were > largely > > placeholders for not-completely-worked-out concepts deferred to the > future were > > left till later. > > > > Yes, well understood, but he was in fact the link between the e2e paper > and the > > differentiated services work. Although not a nominal author of the > "two-bit" RFC, > > he was heavily involved in it, which is why I mentioned him. And he was > very > > active in the IETF diffserv WG. > > > - It is clear (at least to me) that from the point of view of the > source of > > an IP datagram, the "handling" of that datagram within the network of > networks can > > vary, and so that is why there is a TOS field - to specify an > interoperable, > > meaningfully described per-packet indicator of differential handling. I= n > regards > > to the end-to-end argument, that handling choice is a network function, > *to the > > extent that it can completely be implemented in the network itself*. > > > > > > Congestion management, however, is not achievable entirely and only > within > > the network. That's completely obvious: congestion happens when the > > source-destination flows exceed the capacity of the network of networks > to satisfy > > all demands. > > > > > > The network can only implement *certain* general kinds of mechanisms > that may > > be used by the endpoints to resolve congestion: > > > > > > 1) admission controls. These are implemented at the interface between > the > > source entity and the network of networks. They tend to be impractical > in the > > Internet context, because there is, by a fundamental and irreversible > design > > choice made by Cerf and Kahn (and the rest of us), no central controlle= r > of the > > entire network of networks. This is to make evolvability and scalabilit= y > work. 5G > > (not an Internet system) implies a central controller, as does SNA, LTE= , > and many > > other networks. The Internet is an overlay on top of such networks. > > > > > > 2) signalling congestion to the endpoints, which will respond by > slowing > > their transmission rate (or explicitly re-routing transmission, or > compressing > > their content) through the network to match capacity. This response is > done > > *above* the IP layer, and has proven very practical. The function in th= e > network > > is reduced to "congestion signalling", in a universally understandable > meaningful > > mechanism: packet drops, ECN, packet-pair separation in arrival time, > ... > > This limited function is essential within the network, because it is th= e > state of > > the path(s) that is needed to implement the full function at the end > points. So > > congestion signalling, like ECN, is implemented according to the > end-to-end > > argument by carefully defining the network function to be the minimum > necessary > > mechanism so that endpoints can control their rates. > > > > > > 3) automatic selection of routes for flows. It's perfectly fine to > select > > different routes based on information in the IP header (the part that i= s > intended > > to be read and understood by the network of networks). Now this is > currently > > *rarely* done, due to the complexity of tracking more detailed routing > information > > at the router level. But we had expected that eventually the Internet > would be so > > well connected that there would be diverse routes with diverse > capabilities. For > > example, the "Interplanetary Internet" works with datagrams, that can b= e > > implemented with IP, but not using TCP, which requires very low > end-to-end > > latency. Thus, one would expect that TCP would not want any packets > transferred > > over a path via Mars, or for that matter a geosynchronous satellite, > even if the > > throughput would be higher. > > > > > > So one can imagine that eventually a "TOS" might say - send this pack= et > > preferably along a path that has at most 200 ms. RTT, *even if that > leads to > > congestion signalling*, while another TOS might say "send this path ove= r > the most > > "capacious" set of paths, ignoring RTT entirely. (these are just for > illustration, > > but obviously something like this woujld work). > > > > > > Note that TOS is really aimed at *route selection* preferences, and n= ot > > queueing management of individual routers. > > > > That may well have been the original intention, but it was hardly > mentioned at all > > in the diffserv WG (which I co-chaired), and "QOS-based routing" was in > very bad > > odour at that time. > > > > > > > > Queueing management to share a single queue on a path for multiple > priorities > > of traffic is not very compatible with "end-to-end arguments". There ar= e > any > > number of reasons why this doesn't work well. I can go into them. Mainl= y > these > > reasons are why "diffserv" has never been adopted - > > > > Oh, but it has, in lots of local deployments of voice over IP for > example. It's > > what I've taken to calling a limited domain protocol. What has not > happened is > > Internet-wide deployment, because... > > > > > it's NOT interoperable because the diversity of traffic between > endpoints is > > hard to specify in a way that translates into the network mechanisms. O= f > course > > any queue can be managed in some algorithmic way with parameters, but t= he > > endpoints that want to specify an end-to-end goal don't have a way to > understand > > the impact of those parameters on a specific queue that is currently > congested. > > > > Yes. And thanks for your insights. > > > > Brian > > > > > > > > > > > > > > Instead, the history of the Internet (and for that matter *all* > networks, > > even Bell's voice systems) has focused on minimizing queueing delay to > near zero > > throughout the network by whatever means it has at the endpoints or in > the design. > > This is why we have AIMD's MD as a response to detection of congestion. > > > > > > > > > > > > Pragmatic networks (those that operate in the real world) do not > choose to > > operate with shared links in a saturated state. That's known in the > phone business > > as the Mother's Day problem. You want to have enough capacity for the > rare > > near-overload to never result in congestion. Which means that the norm= al > > state of the network is very lightly loaded indeed, in order to minimiz= e > RTT. > > Consequently, focusing on somehow trying to optimize the utilization of > the > > network to 100% is just a purely academic exercise. Since "priority" at > the packet > > level within a queue only improves that case, it's just a focus of (bad= ) > Ph.D. > > theses. (Good Ph.D. theses focus on actual real problems like getting > the queues > > down to 1 packet or less by signalling the endpoints with information > that allows > > them to do their job). > > > > > > > > > > > > So, in considering what goes in the IP layer, both its header and the > > mechanics of the network of networks, it is those things that actually > have > > implementable meaning in the network of networks when processing the IP > datagram. > > The rest is "content" because the network of networks doesn't need to > see it. > > > > > > > > > > > > Thus, don't put anything in the IP header that belongs in the > "content" part, > > just being a signal between end points. Some information used in the > network of > > networks is also logically carried between endpoints. > > > > > > > > > > > > > > > > > > On Friday, June 21, 2019 4:37pm, "Brian E Carpenter" > > said: > > > > > >> Below... > > >> On 21-Jun-19 21:33, Luca Muscariello wrote: > > >> > + David Reed, as I'm not sure he's on the ecn-sane list. > > >> > > > >> > To me, it seems like a very religious position against per-flow > > >> queueing. > > >> > BTW, I fail to see how this would violate (in a "profound" way ) t= he > > e2e > > >> principle. > > >> > > > >> > When I read it (the e2e principle) > > >> > > > >> > Saltzer, J. H., D. P. Reed, and D. D. Clark (1981) "End-to-End > > Arguments in > > >> System Design". > > >> > In: Proceedings of the Second International Conference on > > Distributed > > >> Computing Systems. Paris, France. > > >> > April 8=E2=80=9310, 1981. IEEE Computer Society, pp. 509-512. > > >> > (available on line for free). > > >> > > > >> > It seems very much like the application of the Occam's razor to > > function > > >> placement in communication networks back in the 80s. > > >> > I see no conflict between what is written in that paper and per-fl= ow > > queueing > > >> today, even after almost 40 years. > > >> > > > >> > If that was the case, then all service differentiation techniques > > would > > >> violate the e2e principle in a "profound" way too, > > >> > and dualQ too. A policer? A shaper? A priority queue? > > >> > > > >> > Luca > > >> > > >> Quoting RFC2638 (the "two-bit" RFC): > > >> > > >> >>> Both these > > >> >>> proposals seek to define a single common mechanism that is > > used > > >> by > > >> >>> interior network routers, pushing most of the complexity and > > state > > >> of > > >> >>> differentiated services to the network edges. > > >> > > >> I can't help thinking that if DDC had felt this was against the E2E > > principle, > > >> he would have kicked up a fuss when it was written. > > >> > > >> Bob's right, however, that there might be a tussle here. If end-poin= ts > > are > > >> attempting to pace their packets to suit their own needs, and the > network > > is > > >> policing packets to support both service differentiation and fairnes= s, > > >> these may well be competing rather than collaborating behaviours. An= d > > there > > >> probably isn't anything we can do about it by twiddling with > algorithms. > > >> > > >> Brian > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > On Fri, Jun 21, 2019 at 9:00 AM Sebastian Moeller > > > >> > wrote: > > >> > > > >> > > > >> > > > >> > > On Jun 19, 2019, at 16:12, Bob Briscoe > >> > wrote: > > >> > > > > >> > > Jake, all, > > >> > > > > >> > > You may not be aware of my long history of concern about how > > >> per-flow scheduling within endpoints and networks will limit the > Internet > > in > > >> future. I find per-flow scheduling a violation of the e2e principle = in > > such a > > >> profound way - the dynamic choice of the spacing between packets - > that > > most > > >> people don't even associate it with the e2e principle. > > >> > > > >> > Maybe because it is not a violation of the e2e principle at all? M= y > > point > > >> is that with shared resources between the endpoints, the endpoints > simply > > should > > >> have no expectancy that their choice of spacing between packets will > be > > conserved. > > >> For the simple reason that it seems generally impossible to guarante= e > > that > > >> inter-packet spacing is conserved (think "cross-traffic" at the > > bottleneck hop > > >> along the path and general bunching up of packets in the queue of a > fast > > to slow > > >> transition*). I also would claim that the way L4S works (if it works= ) > is > > to > > >> synchronize all active flows at the bottleneck which in tirn means > each > > sender has > > >> only a very small timewindow in which to transmit a packet for it to > hits > > its > > >> "slot" in the bottleneck L4S scheduler, otherwise, L4S's low queuein= g > > delay > > >> guarantees will not work. In other words the senders have basically = no > > say in the > > >> "spacing between packets", I fail to see how L4S improves upon FQ in > that > > regard. > > >> > > > >> > > > >> > IMHO having per-flow fairness as the defaults seems quite > > >> reasonable, endpoints can still throttle flows to their liking. Now > > per-flow > > >> fairness still can be "abused", so by itself it might not be > sufficient, > > but > > >> neither is L4S as it has at best stochastic guarantees, as a single > queue > > AQM > > >> (let's ignore the RFC3168 part of the AQM) there is the probability = to > > send a > > >> throtteling signal to a low bandwidth flow (fair enough, it is only = a > > mild > > >> throtteling signal, but still). > > >> > But enough about my opinion, what is the ideal fairness measure in > > your > > >> mind, and what is realistically achievable over the internet? > > >> > > > >> > > > >> > Best Regards > > >> > Sebastian > > >> > > > >> > > > >> > > > >> > > > >> > > > > >> > > I detected that you were talking about FQ in a way that might > > have > > >> assumed my concern with it was just about implementation complexity. > If > > you (or > > >> anyone watching) is not aware of the architectural concerns with > > per-flow > > >> scheduling, I can enumerate them. > > >> > > > > >> > > I originally started working on what became L4S to prove that > > it was > > >> possible to separate out reducing queuing delay from throughput > > scheduling. When > > >> Koen and I started working together on this, we discovered we had > > identical > > >> concerns on this. > > >> > > > > >> > > > > >> > > > > >> > > Bob > > >> > > > > >> > > > > >> > > -- > > >> > > > > ________________________________________________________________ > > >> > > Bob Briscoe > > > > >> > > http://bobbriscoe.net/ > > >> > > > > >> > > _______________________________________________ > > >> > > Ecn-sane mailing list > > >> > > Ecn-sane@lists.bufferbloat.net > > >> > > >> > > https://lists.bufferbloat.net/listinfo/ecn-sane > > >> > > > >> > _______________________________________________ > > >> > Ecn-sane mailing list > > >> > Ecn-sane@lists.bufferbloat.net > > >> > > >> > https://lists.bufferbloat.net/listinfo/ecn-sane > > >> > > > >> > > >> > > > > > > > > --0000000000004ee785058bf120b7 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks for the insights.

On Sun 23 Jun 201= 9 at 00:25, David P. Reed <dpreed= @deepplum.com> wrote:

Given the complexity of my broader comments,=C2=A0let me be= clear that I have no problem with the broad concept of diffserv being comp= atible with the end-to-end arguments. I was trying to lay out what I think = is a useful way to think about these kinds of issues within the Internet co= ntext.

=C2=A0

Similarly,= per-flow scheduling as an end-to-end concept (different flows defined by a= ddress pairs being jointly managed as entities) makes great sense, but it&#= 39;s really important to be clear that queue prioritization within a single= queue at entry to a bottleneck link is a special case mechanism, and not a= general end-to-end concept at the IP datagram level, given the generality = of IP as a network packet transport protocol. It's really tied closely = to routing, which isn't specified in any way by IP, other than "be= st efforts", a term that has become much more well defined over the ye= ars (including the notions of dropping rather than storing packets, the ide= a that successive IP datagrams should traverse roughly the same path in ord= er to have stable congestion detection, ...).

=C2=A0

Per-flow s= cheduling seems to work quite well in the cases where it applies, transpare= ntly below the IP datagram layer (that is, underneath the hourglass neck). = IP effectively defines "flows", and it is reasonable to me that &= quot;best efforts" as a concept could include some notion of network-w= ide fairness among flows. Link-level "fairness" isn't a neces= sary precondition to network level fairness.

=C2=A0

On Saturda= y, June 22, 2019 5:10pm, "Brian E Carpenter" <brian.e.carpenter@gmail.co= m> said:

> Just = three or four small comments:
>
> On 23-Jun-19 07:50, David P.= Reed wrote:
> > Two points:
> >
> > =C2=A0
&= gt; >
> > - Jerry Saltzer and I were the primary authors of the= End-to-end argument
> paper, and the motivation was based *my* work = on the original TCP and IP
> protocols. Dave Clark got involved signi= ficantly later than all those decisions,
> which were basically compl= ete when he got involved. (Jerry was my thesis
> supervisor, I was hi= s student, and I operated largely independently, taking input
> from = various others at MIT). I mention this because Dave understands the
>= end-to-end arguments, but he understands (as we all did) that it was a des= ign
> *principle* and not a perfectly strict rule. That said, it'= s a rule that has a
> strong foundational argument from modularity an= d evolvability in a context where
> the system has to work on a wide = range of infrastructures (not all knowable in
> advance) and support = a wide range of usage/application-areas (not all knowable in
> advanc= e). Treating the paper as if it were "DDC" declaring a law is jus= t wrong. He
> wasn't Moses and it is not written on tablets. Dave=
> > did have some "power" in his role of trying to achi= eve interoperability
> across diverse implementations. But his focus = was primarily on interoperability,
> not other things. So ideas in th= e IP protocol like "TOS" which were largely
> placeholders = for not-completely-worked-out concepts deferred to the future were
> = left till later.
>
> Yes, well understood, but he was in fact = the link between the e2e paper and the
> differentiated services work= . Although not a nominal author of the "two-bit" RFC,
> he = was heavily involved in it, which is why I mentioned him. And he was very> active in the IETF diffserv WG.
> > - It is clear (at least= to me) that from the point of view of the source of
> an IP datagram= , the "handling" of that datagram within the network of networks = can
> vary, and so that is why there is a TOS field - to specify an i= nteroperable,
> meaningfully described per-packet indicator of differ= ential handling. In regards
> to the end-to-end argument, that handli= ng choice is a network function, *to the
> extent that it can complet= ely be implemented in the network itself*.
> >
> > Conges= tion management, however, is not achievable entirely and only within
>= ; the network. That's completely obvious: congestion happens when the> source-destination flows exceed the capacity of the network of netwo= rks to satisfy
> all demands.
> >
> > The network c= an only implement *certain* general kinds of mechanisms that may
> be= used by the endpoints to resolve congestion:
> >
> > 1) = admission controls. These are implemented at the interface between the
&= gt; source entity and the network of networks. They tend to be impractical = in the
> Internet context, because there is, by a fundamental and irr= eversible design
> choice made by Cerf and Kahn (and the rest of us),= no central controller of the
> entire network of networks. This is t= o make evolvability and scalability work. 5G
> (not an Internet syste= m) implies a central controller, as does SNA, LTE, and many
> other n= etworks. The Internet is an overlay on top of such networks.
> >> > 2) signalling congestion to the endpoints, which will respond b= y slowing
> their transmission rate (or explicitly re-routing transmi= ssion, or compressing
> their content) through the network to match c= apacity. This response is done
> *above* the IP layer, and has proven= very practical. The function in the network
> is reduced to "co= ngestion signalling", in a universally understandable meaningful
&g= t; mechanism: packet drops, ECN, packet-pair separation in arrival time, ..= .=C2=A0
> This limited function is essential within the network, beca= use it is the state of
> the path(s) that is needed to implement the = full function at the end points. So
> congestion signalling, like ECN= , is implemented according to the end-to-end
> argument by carefully = defining the network function to be the minimum necessary
> mechanism= so that endpoints can control their rates.
> >
> > 3) au= tomatic selection of routes for flows. It's perfectly fine to select> different routes based on information in the IP header (the part that= is intended
> to be read and understood by the network of networks).= Now this is currently
> *rarely* done, due to the complexity of trac= king more detailed routing information
> at the router level. But we = had expected that eventually the Internet would be so
> well connecte= d that there would be diverse routes with diverse capabilities. For
>= example, the "Interplanetary Internet" works with datagrams, tha= t can be
> implemented with IP, but not using TCP, which requires ver= y low end-to-end
> latency. Thus, one would expect that TCP would not= want any packets transferred
> over a path via Mars, or for that mat= ter a geosynchronous satellite, even if the
> throughput would be hig= her.
> >
> > So one can imagine that eventually a "T= OS" might say - send this packet
> preferably along a path that = has at most 200 ms. RTT, *even if that leads to
> congestion signalli= ng*, while another TOS might say "send this path over the most
>= "capacious" set of paths, ignoring RTT entirely. (these are just= for illustration,
> but obviously something like this woujld work).<= br>> >
> > Note that TOS is really aimed at *route selection= * preferences, and not
> queueing management of individual routers.>
> That may well have been the original intention, but it was = hardly mentioned at all
> in the diffserv WG (which I co-chaired), an= d "QOS-based routing" was in very bad
> odour at that time.=
> =C2=A0
> >
> > Queueing management to share a si= ngle queue on a path for multiple priorities
> of traffic is not very= compatible with "end-to-end arguments". There are any
> nu= mber of reasons why this doesn't work well. I can go into them. Mainly = these
> reasons are why "diffserv" has never been adopted -=
>
> Oh, but it has, in lots of local deployments of voice ove= r IP for example. It's
> what I've taken to calling a limited= domain protocol. What has not happened is
> Internet-wide deployment= , because...
>
> > it's NOT interoperable because the d= iversity of traffic between endpoints is
> hard to specify in a way t= hat translates into the network mechanisms. Of course
> any queue can= be managed in some algorithmic way with parameters, but the
> endpoi= nts that want to specify an end-to-end goal don't have a way to underst= and
> the impact of those parameters on a specific queue that is curr= ently congested.
>
> Yes. And thanks for your insights.
>= ;
> Brian
>
> >
> > =C2=A0
> >
= > > Instead, the history of the Internet (and for that matter *all* n= etworks,
> even Bell's voice systems) has focused on minimizing q= ueueing delay to near zero
> throughout the network by whatever means= it has at the endpoints or in the design.
> This is why we have AIMD= 's MD as a response to detection of congestion.
> >
> &g= t; =C2=A0
> >
> > Pragmatic networks (those that operate = in the real world) do not choose to
> operate with shared links in a = saturated state. That's known in the phone business
> as the Moth= er's Day problem. You want to have enough capacity for the rare
>= near-overload to never result in congestion.=C2=A0 Which means that the no= rmal
> state of the network is very lightly loaded indeed, in order t= o minimize RTT.
> Consequently, focusing on somehow trying to optimiz= e the utilization of the
> network to 100% is just a purely academic = exercise. Since "priority" at the packet
> level within a q= ueue only improves that case, it's just a focus of (bad) Ph.D.
> = theses. (Good Ph.D. theses focus on actual real problems like getting the q= ueues
> down to 1 packet or less by signalling the endpoints with inf= ormation that allows
> them to do their job).
> >
> &g= t; =C2=A0
> >
> > So, in considering what goes in the IP = layer, both its header and the
> mechanics of the network of networks= , it is those things that actually have
> implementable meaning in th= e network of networks when processing the IP datagram.
> The rest is = "content" because the network of networks doesn't need to see= it.
> >
> > =C2=A0
> >
> > Thus, don&#= 39;t put anything in the IP header that belongs in the "content" = part,
> just being a signal between end points. Some information used= in the network of
> networks is also logically carried between endpo= ints.
> >
> > =C2=A0
> >
> > =C2=A0
= > >
> > On Friday, June 21, 2019 4:37pm, "Brian E Carpe= nter"
> <brian.e.carpenter@gmail.com> said:
> >
>= >> Below...
> >> On 21-Jun-19 21:33, Luca Muscariello wr= ote:
> >> > + David Reed, as I'm not sure he's on th= e ecn-sane list.
> >> >
> >> > To me, it seem= s like a very religious position against per-flow
> >> queueing= .=C2=A0
> >> > BTW, I fail to see how this would violate (in= a "profound" way ) the
> e2e
> >> principle.> >> >
> >> > When I read it (the e2e principl= e)
> >> >
> >> > Saltzer, J. H., D. P. Reed, = and D. D. Clark (1981) "End-to-End
> Arguments in
> >&g= t; System Design".=C2=A0
> >> > In: Proceedings of the = Second International Conference on
> Distributed
> >> Com= puting Systems. Paris, France.=C2=A0
> >> > April 8=E2=80=93= 10, 1981. IEEE Computer Society, pp. 509-512.
> >> > (availa= ble on line for free).
> >> >
> >> > It seems= very much like the application of the Occam's razor to
> functio= n
> >> placement in communication networks back in the 80s.
= > >> > I see no conflict between what is written in that paper = and per-flow
> queueing
> >> today, even after almost 40 = years.
> >> >
> >> > If that was the case, th= en all service differentiation techniques
> would
> >> vi= olate the e2e principle in a "profound" way too,
> >>= > and dualQ too. A policer? A shaper? A priority queue?
> >>= ; >
> >> > Luca
> >>
> >> Quoting= RFC2638 (the "two-bit" RFC):
> >>
> >> &= gt;>> Both these
> >> >>> proposals seek to defi= ne a single common mechanism that is
> used
> >> by
&g= t; >> >>> interior network routers, pushing most of the comp= lexity and
> state
> >> of
> >> >>> = differentiated services to the network edges.
> >>
> >= > I can't help thinking that if DDC had felt this was against the E2= E
> principle,
> >> he would have kicked up a fuss when i= t was written.
> >>
> >> Bob's right, however, = that there might be a tussle here. If end-points
> are
> >&g= t; attempting to pace their packets to suit their own needs, and the networ= k
> is
> >> policing packets to support both service diff= erentiation and fairness,
> >> these may well be competing rath= er than collaborating behaviours. And
> there
> >> probab= ly isn't anything we can do about it by twiddling with algorithms.
&= gt; >>
> >> Brian
> >>
> >>
&g= t; >>
> >>
> >>
> >>
> >= >
> >> >
> >> >
> >> >
&= gt; >> >
> >> >
> >> >
> >&= gt; > =C2=A0
> >> >
> >> > On Fri, Jun 21,= 2019 at 9:00 AM Sebastian Moeller
> <moeller0@gmx.de
> >> <mailto:moeller0@gmx.de>&= gt; wrote:
> >> >
> >> >
> >> >= ;
> >> > > On Jun 19, 2019, at 16:12, Bob Briscoe <ietf@bobbriscoe.net
> >> <mailto:ietf@bobbriscoe.net>> wrote:
> >> > &= gt;
> >> > > Jake, all,
> >> > >
>= ; >> > > You may not be aware of my long history of concern abo= ut how
> >> per-flow scheduling within endpoints and networks w= ill limit the Internet
> in
> >> future. I find per-flow = scheduling a violation of the e2e principle in
> such a
> >&= gt; profound way - the dynamic choice of the spacing between packets - that=
> most
> >> people don't even associate it with the = e2e principle.
> >> >
> >> > Maybe because it= is not a violation of the e2e principle at all? My
> point
> &= gt;> is that with shared resources between the endpoints, the endpoints = simply
> should
> >> have no expectancy that their choice= of spacing between packets will be
> conserved.
> >> For= the simple reason that it seems generally impossible to guarantee
> = that
> >> inter-packet spacing is conserved (think "cross-= traffic" at the
> bottleneck hop
> >> along the path= and general bunching up of packets in the queue of a fast
> to slow<= br>> >> transition*). I also would claim that the way L4S works (i= f it works) is
> to
> >> synchronize all active flows at = the bottleneck which in tirn means each
> sender has
> >>= only a very small timewindow in which to transmit a packet for it to hits<= br>> its
> >> "slot" in the bottleneck L4S schedul= er, otherwise, L4S's low queueing
> delay
> >> guaran= tees will not work. In other words the senders have basically no
> sa= y in the
> >> "spacing between packets", I fail to se= e how L4S improves upon FQ in that
> regard.
> >> >> >> >
> >> > =C2=A0IMHO having per-flow fairne= ss as the defaults seems quite
> >> reasonable, endpoints can s= till throttle flows to their liking. Now
> per-flow
> >> = fairness still can be "abused", so by itself it might not be suff= icient,
> but
> >> neither is L4S as it has at best stoch= astic guarantees, as a single queue
> AQM
> >> (let's= ignore the RFC3168 part of the AQM) there is the probability to
> se= nd a
> >> throtteling signal to a low bandwidth flow (fair enou= gh, it is only a
> mild
> >> throtteling signal, but stil= l).
> >> > But enough about my opinion, what is the ideal fa= irness measure in
> your
> >> mind, and what is realistic= ally achievable over the internet?
> >> >
> >> &= gt;
> >> > Best Regards
> >> > =C2=A0 =C2=A0 = =C2=A0 =C2=A0 Sebastian
> >> >
> >> >
>= >> >
> >> >
> >> > >
> >= ;> > > I detected that you were talking about FQ in a way that mig= ht
> have
> >> assumed my concern with it was just about = implementation complexity. If
> you (or
> >> anyone watch= ing) is not aware of the architectural concerns with
> per-flow
&g= t; >> scheduling, I can enumerate them.
> >> > >> >> > > I originally started working on what became L4S to= prove that
> it was
> >> possible to separate out reduci= ng queuing delay from throughput
> scheduling. When
> >> = Koen and I started working together on this, we discovered we had
> i= dentical
> >> concerns on this.
> >> > >
&= gt; >> > >
> >> > >
> >> > >= ; Bob
> >> > >
> >> > >
> >>= ; > > --
> >> > >
> _________________________= _______________________________________
> >> > > Bob Bris= coe=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0
> =C2=A0
> = >> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0
> =C2=A0http://bobbriscoe.net/
> >> > >
> >> > > ___________________= ____________________________
> >> > > Ecn-sane mailing li= st
> >> > > Ecn-sane@lists.bufferbloat.net
> >> = <mailto:Ecn-sane@lists.bufferbloat.net>
> >> > > ht= tps://lists.bufferbloat.net/listinfo/ecn-sane
> >> >
= > >> > _______________________________________________
> = >> > Ecn-sane mailing list
> >> > Ecn-sane@lists.bufferblo= at.net
> >> <mailto:Ecn-sane@lists.bufferbloat.net>
&= gt; >> > https://lists.bufferbloat.net/listinfo/ecn-sane> >> >
> >>
> >>
> >
> =
>

--0000000000004ee785058bf120b7--