[Ecn-sane] [tsvwg] Comments on L4S drafts

Sun Jul 21 12:43:06 EDT 2019

Bob,

Pulling relevant text to the top ...

> > As you know, we have been at pains to address every concern about L4S
> > that has come up over the years, and I thought we had addressed this
> > one to your satisfaction.

Truth be told, "acquiescence" would be a more accurate word than "satisfaction."  I can live with the current plans, but I would not describe myself as satisfied with them.

> > The reliable transports you are concerned about require ordered
> > delivery by the underlying fabric, so they can only ever exist in a
> > controlled environment. In such a controlled environment, your
> > ECT1+DSCP idea (below) could be used to isolate the L4S experiment
> > from these transports and their firmware/hardware constraints.

There appears to be a lack of understanding here.  The protocols in question, RoCEv2 in particular have some reordering tolerance, but not as good as TCP's.  Current requirements for ordered delivery are in the same general area as TCP with 3DupACK, which is not constrained to controlled environments.

> > On the public Internet, the DSCP commonly gets wiped at the first hop.
> > So requiring a DSCP as well as ECT1 to separate off L4S would serve no
> > useful purpose: it would still lead to ECT1 packets without the DSCP
> > sent from a scalable congestion controls (which is behind Jonathan's
> > concern in response to you).

We're on the same page here, as I also wrote the following (although a stronger word that "subtleties" would have been better in 20/20 hindsight):

> >> traffic (there are some subtleties here, e.g., interaction
> >> with operator bleaching of DSCPs to zero at network boundaries).

On to the two requests.

> > Please confirm:
> > a) that your RACK concern only applies in controlled environments, and
> > ECT1+DSCP resolves it

No, twice.  I hope that’s clearer now between what Gorry, Michael, and myself have posted.

As stated in the past, and moreover in this email thread, I can accept some sort controlled environment text as a compromise means of moving the experiment forward:

> >> Process wise, I'm ok with addressing this objection via some sort of
> >> "controlled environment" escape clause text that makes this RACK-like
> >> requirement inapplicable in a "controlled environment" that does not
> >> need that behavior (e.g., where 3DupACK does not cause problems and
> >> is not expected to cause problems).

Moving on to the next topic:

> > b) on the public Internet, we currently have one issue to address:
> > single-queue RFC3168 AQMs,
> > and if we can resolve that, ECT1 alone would be acceptable as an L4S
> > identifier.

In addition to a), there is now the desire of SCE to use ECT(1) at similar scope.

Thanks, --David

> -----Original Message-----
> From: Gorry Fairhurst <gorry at erg.abdn.ac.uk>
> Sent: Sunday, July 21, 2019 11:10 AM
> To: Bob Briscoe
> Cc: Black, David; Wesley Eddy; Dave Taht; De Schepper, Koen (Nokia -
> BE/Antwerp); ecn-sane at lists.bufferbloat.net; tsvwg at ietf.org
> Subject: Re: [tsvwg] [Ecn-sane] Comments on L4S drafts
> 
> 
> [EXTERNAL EMAIL]
> 
> I'd like to add to this what I understand as an individual ... see inline.
> 
> On 21/07/2019, 08:30, Bob Briscoe wrote:
> > David,
> >
> > On 19/07/2019 21:06, Black, David wrote:
> >> Two comments as an individual, not as a WG chair:
> >>
> >>> Mostly, they're things that an end-host algorithm needs
> >>> to do in order to behave nicely, that might be good things anyways
> >>> without regard to L4S in the network (coexist w/ Reno, avoid RTT bias,
> >>> work well w/ small RTT, be robust to reordering).  I am curious which
> >>> ones you think are too rigid ... maybe they can be loosened?
> >> [1] I have profoundly objected to L4S's RACK-like requirement (use
> >> time to detect loss, and in particular do not use 3DupACK) in public
> >> on multiple occasions, because in reliable transport space, that
> >> forces use of TCP Prague, a protocol with which we have little to no
> >> deployment or operational experience.  Moreover, that requirement
> >> raises the bar for other protocols in a fashion that impacts endpoint
> >> firmware, and possibly hardware in some important (IMHO) environments
> >> where investing in those changes delivers little to no benefit.  The
> >> environments that I have in mind include a lot of data centers.
> >> Process wise, I'm ok with addressing this objection via some sort of
> >> "controlled environment" escape clause text that makes this RACK-like
> >> requirement inapplicable in a "controlled environment" that does not
> >> need that behavior (e.g., where 3DupACK does not cause problems and
> >> is not expected to cause problems).
> >>
> >> For clarity, I understand the multi-lane link design rationale behind
> >> the RACK-like requirement and would agree with that requirement in a
> >> perfect world ... BUT ... this world is not perfect ... e.g., 3DupACK
> >> will not vanish from "running code" anytime soon.
> > As you know, we have been at pains to address every concern about L4S
> > that has come up over the years, and I thought we had addressed this
> > one to your satisfaction.
> >
> > The reliable transports you are are concerned about require ordered
> > delivery by the underlying fabric, so they can only ever exist in a
> > controlled environment. In such a controlled environment, your
> > ECT1+DSCP idea (below) could be used to isolate the L4S experiment
> > from these transports and their firmware/hardware constraints.
> >
> > On the public Internet, the DSCP commonly gets wiped at the first hop.
> > So requiring a DSCP as well as ECT1 to separate off L4S would serve no
> > useful purpose: it would still lead to ECT1 packets without the DSCP
> > sent from a scalable congestion controls (which is behind Jonathan's
> > concern in response to you).
> >
> >
> It would always be possible to have taken an approach that required a
> DSCP to use the "alternantive ECN semantic" . This option was debated
> when L4S was first discussed. The WG draft decided against that
> approach, and instead chose to use an ECT(1) codepoint. That I recall
> was analysed in depth.
> 
> This does not preclude someone from classifying on a DSCP (such as the
> suggested NQB) to also choose which ECN treatment to use (should that be
> useful for some reason, e.g. because the traffic is low rate). To me, at
> least, it important to allow traffic with DSCP markings to utilise the
> AQM ECN treatments.
> 
> >>>> So to me, it goes back to slamming the door shut, or not, on L4S's
> >>>> usage
> >>>> of ect(1) as a too easily gamed e2e identifier. As I don't think it
> >>>> and
> >>>> all the dependent code and algorithms can possibly scale past a single
> >>>> physical layer tech, I'd like to see it move to a DSCP codepoint,
> >>>> worst
> >>>> case... and certainly remain "experimental" in scope until anyone
> >>>> independent cn attempt to evaluate it.
> >>> That seems good to discuss in regard to the L4S ID draft.  There is a
> >>> section (5.2) there already discussing DSCP, and why it alone isn't
> >>> feasible.  There's also more detailed description of the relation and
> >>> interworking in
> >>> https://tools.ietf.org/html/draft-briscoe-tsvwg-l4s-diffserv-02
> >> [2] We probably should pay more attention to that draft.  One of the
> >> things that I think is important in that draft is a requirement that
> >> operators can enable/disable L4S behavior of ECT(1) on a per-DSCP
> >> basis - the rationale for that functionality starts with incremental
> >> deployment.   This technique may also have the potential to provide a
> >> means for L4S and SCE to coexist via use of different DSCPs for L4S
> >> vs. SCE traffic (there are some subtleties here, e.g., interaction
> >> with operator bleaching of DSCPs to zero at network boundaries).
> >>
> >> To be clear on what I have in mind:
> >>     o Unacceptable: All traffic marked with ECT(1) goes into the L4S
> >> queue, independent of what DSCP it is marked with.
> That is what has been described in the WG drafts since they entered
> TSVWG. I don't recall any suggested change to that decision until just now.
> >>     o Acceptable:  There's an operator-configurable list of DSCPs
> >> that support an L4S service - traffic marked with ECT(1) goes into
> >> the L4S queue if and only if that traffic is also marked with a DSCP
> >> that is on the operator's DSCPs-for-L4S list.
> That was always possible under the "alternative ECN markings", but I
> understood the purpose was to facilitate an Internet experiment.
> > Please confirm:
> > a) that your RACK concern only applies in controlled environments, and
> > ECT1+DSCP resolves it
> That seems more than obviously needed to me. There is a lot of traffic
> that uses some notion of timeliness for retransmission. Designing such a
> transport to be robust is tricky, but we're alreday exploring that for
> TCP and QUIC.
> 
> On the other hand, I have many times urged caution in creating
> assumptions that wit would be OK for Internet paths to somehow now allow
> more reordering. I'd like to see that happen - but I don't this
> recommendation is appropriuate.
> > b) on the public Internet, we currently have one issue to address:
> > single-queue RFC3168 AQMs,
> > and if we can resolve that, ECT1 alone would be acceptable as an L4S
> > identifier.
> >
> > I am trying to focus the issues list, which I would hope you would
> > support, even without your chair hat on.
> >
> >
> >
> > Bob
> >
> Gorry
> >>
> >> Reminder: This entire message is posted as an individual, not as a WG
> >> chair.
> >>
> >> Thanks, --David
> >>
> >>> -----Original Message-----
> >>> From: tsvwg <tsvwg-bounces at ietf.org> On Behalf Of Wesley Eddy
> >>> Sent: Friday, July 19, 2019 2:34 PM
> >>> To: Dave Taht; De Schepper, Koen (Nokia - BE/Antwerp)
> >>> Cc: ecn-sane at lists.bufferbloat.net; tsvwg at ietf.org
> >>> Subject: Re: [tsvwg] [Ecn-sane] Comments on L4S drafts
> >>>
> >>>
> >>> [EXTERNAL EMAIL]
> >>>
> >>> On 7/19/2019 11:37 AM, Dave Taht wrote:
> >>>> It's the common-q with AQM **+ ECN** that's the sticking point. I'm
> >>>> perfectly satisfied with the behavior of every ietf approved single
> >>>> queued AQM without ecn enabled. Let's deploy more of those!
> >>> Hi Dave, I'm just trying to make sure I'm reading into your message
> >>> correctly ... if I'm understanding it, then you're not in favor of
> >>> either SCE or L4S at all?  With small queues and without ECN, loss
> >>> becomes the only congestion signal, which is not desirable, IMHO, or am
> >>> I totally misunderstanding something?
> >>>
> >>>
> >>>> If we could somehow create a neutral poll in the general networking
> >>>> community outside the ietf (nanog, bsd, linux, dcs, bigcos, routercos,
> >>>> ISPs small and large) , and do it much like your classic "vote for a
> >>>> political measure" thing, with a single point/counterpoint section,
> >>>> maybe we'd get somewhere.
> >>> While I agree that would be really useful, it's kind of an "I want a
> >>> pony" statement.  As a TSVWG chair where we're doing this work, we've
> >>> been getting inputs from people that have a foot in many of the
> >>> communities you mention, but always looking for more.
> >>>
> >>>
> >>>> In particular conflating "low latency" really confounds the subject
> >>>> matter, and has for years. FQ gives "low latency" for the vast
> >>>> majority of flows running below their fair share. L4S promises "low
> >>>> latency" for a rigidly defined set of congestion controls in a
> >>>> specialized queue, and otherwise tosses all flows into a higher
> >>>> latency
> >>>> queue when one flow is greedy.
> >>> I don't think this is a correct statement.  Packets have to be from a
> >>> "scalable congestion control" to get access to the L4S queue.  There
> >>> are
> >>> some draft requirements for using the L4S ID, but they seem pretty
> >>> flexible to me.  Mostly, they're things that an end-host algorithm
> >>> needs
> >>> to do in order to behave nicely, that might be good things anyways
> >>> without regard to L4S in the network (coexist w/ Reno, avoid RTT bias,
> >>> work well w/ small RTT, be robust to reordering).  I am curious which
> >>> ones you think are too rigid ... maybe they can be loosened?
> >>>
> >>> Also, I don't think the "tosses all flows into a higher latency queue
> >>> when one flow is greedy" characterization is correct.  The other queue
> >>> is for classic/non-scalable traffic, and not necessarily higher latency
> >>> for a given flow, nor is winding up there related to whether another
> >>> flow is greedy.
> >>>
> >>>
> >>>> So to me, it goes back to slamming the door shut, or not, on L4S's
> >>>> usage
> >>>> of ect(1) as a too easily gamed e2e identifier. As I don't think it
> >>>> and
> >>>> all the dependent code and algorithms can possibly scale past a single
> >>>> physical layer tech, I'd like to see it move to a DSCP codepoint,
> >>>> worst
> >>>> case... and certainly remain "experimental" in scope until anyone
> >>>> independent can attempt to evaluate it.
> >>> That seems good to discuss in regard to the L4S ID draft.  There is a
> >>> section (5.2) there already discussing DSCP, and why it alone isn't
> >>> feasible.  There's also more detailed description of the relation and
> >>> interworking in
> >>> https://tools.ietf.org/html/draft-briscoe-tsvwg-l4s-diffserv-02
> >>>
> >>>
> >>>> I'd really all the tcp-go-fast-at-any-cost people to take a year
> >>>> off to
> >>>> dogfood their designs, and go live somewhere with a congested network
> >>> to
> >>>> deal with daily, like a railway or airport, or on 3G network on a
> >>>> sailboat or beach somewhere. It's not a bad life... REALLY.
> >>>>
> >>> Fortunately, at least in the IETF, I don't think there have been
> >>> initiatives in the direction of going fast at any cost in recent
> >>> history, and they would be unlikely to be well accepted if there were!
> >>> That is at least one place that there seems to be strong consensus.
> >>>
> >> _______________________________________________
> >> Ecn-sane mailing list
> >> Ecn-sane at lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/ecn-sane
> >