[Ecn-sane] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Ecn-sane] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 1368 bytes --]

I would love to have some fresh eyeballs on a new IETF draft for the
TSVWG we intend to submit tonight.

I've attached the html for easy to read purposes, but I would prefer
that folk referred back to the github repository for the most current
version, which is here:

https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-SCE.txt

and in open source tradition, discuss here, or file bugs, and submit
pull requests to the gitub.

The first draft (of 3 or more pending), is creating the SCE codepoint
and defining the state machine, is pretty short, and we think the
basic concept solves a zillion problems with ECN in one stroke. It's
easy to implement (one line of code in codel), backward compatible
with all the RFCs, and somewhat incompatible with the stalled out TCP
Prague/dualpi effort in the IETF.

We have several other drafts in progress which I increasingly doubt
we'll finish today, but I think only this one is required to get an
audience in the tsvwg at the coming IETF meeting.

If ya have any comments and spare time today, I'd like to get the
first draft in tonight, and the filing deadline for final drafts is
sometime tomorrow. It may help for context to review some of the other
work in the github repo.

THX!

-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

[-- Attachment #2: draft-morton-taht-SCE.html --]
[-- Type: text/html, Size: 27736 bytes --]

Re: [Ecn-sane] [Cake] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Sebastian Moeller]

[-- Attachment #1: Type: text/plain, Size: 1907 bytes --]

Hi Dave, hi Jonathan,

I like it! Outside of my area of expertise, sure, but still short, sweet and focussed. As I commented on github, it might halp if you could make a point how this could actually help core-routers to soften their load, as it is those devices that need to learn to use SCE for the proposed endp-point methods to get anywhere in real life.

Best Regards
	Sebastian


> On Mar 10, 2019, at 19:04, Dave Taht <dave.taht@gmail.com> wrote:
> 
> I would love to have some fresh eyeballs on a new IETF draft for the
> TSVWG we intend to submit tonight.
> 
> I've attached the html for easy to read purposes, but I would prefer
> that folk referred back to the github repository for the most current
> version, which is here:
> 
> https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-SCE.txt
> 
> and in open source tradition, discuss here, or file bugs, and submit
> pull requests to the gitub.
> 
> The first draft (of 3 or more pending), is creating the SCE codepoint
> and defining the state machine, is pretty short, and we think the
> basic concept solves a zillion problems with ECN in one stroke. It's
> easy to implement (one line of code in codel), backward compatible
> with all the RFCs, and somewhat incompatible with the stalled out TCP
> Prague/dualpi effort in the IETF.
> 
> We have several other drafts in progress which I increasingly doubt
> we'll finish today, but I think only this one is required to get an
> audience in the tsvwg at the coming IETF meeting.
> 
> If ya have any comments and spare time today, I'd like to get the
> first draft in tonight, and the filing deadline for final drafts is
> sometime tomorrow. It may help for context to review some of the other
> work in the github repo.
> 
> THX!
> 
> -- 
> 
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-205-9740

[-- Attachment #2: draft-morton-taht-SCE.html --]
[-- Type: text/html, Size: 27736 bytes --]

[-- Attachment #3: Type: text/plain, Size: 146 bytes --]

> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Holland, Jake]

Hi Dave,

You and John have my enthusiastic +1.

It's a frank relief to read this draft after trying to figure out L4S,
and I think the basic core concept upon which to build the actual
response systems is very well separated and very well framed here.

Please submit this and present, I humbly beg you.  It seems to me a
strictly better use of ECT(1), even though there's still probably a
few hundred pages' worth of catching up to do on draft-writing to
nail down details.

I have a few minor comments for your consideration, but please don't
let them stop you from posting before deadline, if any are hard to
integrate.  It would be better to ignore them all and post as-is than
to get hung up on these:

1.
"Some" in "Some Congestion Experienced" is maybe misleading, and
arguably has the same meaning as "Congestion Experienced".

I was thinking maybe "Pre-Congestion Experienced" or "Queue
Utilization Observed", or if you want to preserve "SCE" and the
link to CE (which I do agree is nice), maybe "Slight" or "Sub"
instead of "Some", just to try to make it more obvious it's
flagging a lesser situation than "there is some congestion".

2.
It's easy to accidently read section 5 as underspecified concrete
proposals instead of rough sketches for future direction that might
be worth investigating.  I'll offer an attempt at some language,
feel free to edit (or ignore if you think the intro is enough to
make the scope sufficiently clear already):


The scope of this document is limited to the definition of the
SCE codepoint.  However, for illustration purposes, a few possible
future usage scenarios are outlined here.  These examples are non
normative.

3.
Similarly, I would lower-case the "MAY" and "SHOULD" in section
5.2 for receiver-side handling in TCP--it's not clear this will
ever be a good idea to do without more explicit signaling thru
new opts or whatever, and granting permission here seems like
asking for trouble that's just not necessary.


And a few that I'd defer if I were you, but I'd like to see
sometime in at least a post-Prague version or list discussion:

4.
an informative reference or 2 would be a welcome addition in Section 3:

   Research has shown that the ECT(1) codepoint goes essentially unused,
   with the "Nonce Sum" extension to ECN having not been implemented in

5.
Should this must be MUST in Section 4?  If not, why not?

   New SCE-aware receivers and transport protocols must continue to


Thanks guys, nice work and good luck!

Cheers,
Jake


On 2019-03-10, 11:07, "Dave Taht" <dave.taht@gmail.com> wrote:

    I would love to have some fresh eyeballs on a new IETF draft for the
    TSVWG we intend to submit tonight.
    
    I've attached the html for easy to read purposes, but I would prefer
    that folk referred back to the github repository for the most current
    version, which is here:
    
    https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-SCE.txt
    
    and in open source tradition, discuss here, or file bugs, and submit
    pull requests to the gitub.
    
    The first draft (of 3 or more pending), is creating the SCE codepoint
    and defining the state machine, is pretty short, and we think the
    basic concept solves a zillion problems with ECN in one stroke. It's
    easy to implement (one line of code in codel), backward compatible
    with all the RFCs, and somewhat incompatible with the stalled out TCP
    Prague/dualpi effort in the IETF.
    
    We have several other drafts in progress which I increasingly doubt
    we'll finish today, but I think only this one is required to get an
    audience in the tsvwg at the coming IETF meeting.
    
    If ya have any comments and spare time today, I'd like to get the
    first draft in tonight, and the filing deadline for final drafts is
    sometime tomorrow. It may help for context to review some of the other
    work in the github repo.
    
    THX!
    
    -- 
    
    Dave Täht
    CTO, TekLibre, LLC
    http://www.teklibre.com
    Tel: 1-831-205-9740
    

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Jonathan Morton]

> On 10 Mar, 2019, at 9:08 pm, Holland, Jake <jholland@akamai.com> wrote:
> 
> It's easy to accidently read section 5 as underspecified concrete
> proposals instead of rough sketches for future direction that might
> be worth investigating.

This is something I noticed as well, and have edited to match the intended tone of the document (which is to leave specific implementation details to other documents, and to focus on the information conveyed).

> New SCE-aware receivers and transport protocols must continue to


I had already changed this to SHOULD, among other edits.

> "Some" in "Some Congestion Experienced" is maybe misleading, and
> arguably has the same meaning as "Congestion Experienced".
> 
> I was thinking maybe "Pre-Congestion Experienced" or "Queue
> Utilization Observed", or if you want to preserve "SCE" and the
> link to CE (which I do agree is nice), maybe "Slight" or "Sub"
> instead of "Some", just to try to make it more obvious it's
> flagging a lesser situation than "there is some congestion".

An interesting idea, but SCE marks will appear even when there's a lot of congestion (at high rates, ie. probably every packet that doesn't carry CE), as well as showing up at low frequency when the level of congestion only warrants reducing the growth rate.  I think the word "Some" is sufficiently descriptive, while "Slight" might cause people to ignore it completely.

I think other points are less urgent to address in the first submission.

 - Jonathan Morton

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Dave Taht]

On Sun, Mar 10, 2019 at 12:08 PM Holland, Jake <jholland@akamai.com> wrote:
>
> Hi Dave,
>
> You and John have my enthusiastic +1.
>
> It's a frank relief to read this draft after trying to figure out L4S,
> and I think the basic core concept upon which to build the actual
> response systems is very well separated and very well framed here.
>
> Please submit this and present, I humbly beg you.  It seems to me a
> strictly better use of ECT(1), even though there's still probably a
> few hundred pages' worth of catching up to do on draft-writing to
> nail down details.
>
> I have a few minor comments for your consideration, but please don't
> let them stop you from posting before deadline, if any are hard to
> integrate.  It would be better to ignore them all and post as-is than
> to get hung up on these:
>
> 1.
> "Some" in "Some Congestion Experienced" is maybe misleading, and
> arguably has the same meaning as "Congestion Experienced".
>
> I was thinking maybe "Pre-Congestion Experienced" or "Queue
> Utilization Observed", or if you want to preserve "SCE" and the
> link to CE (which I do agree is nice), maybe "Slight" or "Sub"
> instead of "Some", just to try to make it more obvious it's
> flagging a lesser situation than "there is some congestion".
>
> 2.
> It's easy to accidently read section 5 as underspecified concrete
> proposals instead of rough sketches for future direction that might
> be worth investigating.  I'll offer an attempt at some language,
> feel free to edit (or ignore if you think the intro is enough to
> make the scope sufficiently clear already):
>
>
> The scope of this document is limited to the definition of the
> SCE codepoint.  However, for illustration purposes, a few possible
> future usage scenarios are outlined here.  These examples are non
> normative.
>
> 3.
> Similarly, I would lower-case the "MAY" and "SHOULD" in section
> 5.2 for receiver-side handling in TCP--it's not clear this will
> ever be a good idea to do without more explicit signaling thru
> new opts or whatever, and granting permission here seems like
> asking for trouble that's just not necessary.
>
>
> And a few that I'd defer if I were you, but I'd like to see
> sometime in at least a post-Prague version or list discussion:
>
> 4.
> an informative reference or 2 would be a welcome addition in Section 3:
>
>    Research has shown that the ECT(1) codepoint goes essentially unused,
>    with the "Nonce Sum" extension to ECN having not been implemented in

I have been trying to find that presentation or paper or talk or rfc
for days now. I *know* it went by sometime in the past 2 years, I had
my aha! moment - and I keep drawing a blank. Might have been something
brian trammel did, I simply cannot remember.

I definately want to cite that, and I sure hope I'm not delusional.

> 5.
> Should this must be MUST in Section 4?  If not, why not?
>
>    New SCE-aware receivers and transport protocols must continue to
>
>
> Thanks guys, nice work and good luck!
>
> Cheers,
> Jake
>
>
> On 2019-03-10, 11:07, "Dave Taht" <dave.taht@gmail.com> wrote:
>
>     I would love to have some fresh eyeballs on a new IETF draft for the
>     TSVWG we intend to submit tonight.
>
>     I've attached the html for easy to read purposes, but I would prefer
>     that folk referred back to the github repository for the most current
>     version, which is here:
>
>     https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-SCE.txt
>
>     and in open source tradition, discuss here, or file bugs, and submit
>     pull requests to the gitub.
>
>     The first draft (of 3 or more pending), is creating the SCE codepoint
>     and defining the state machine, is pretty short, and we think the
>     basic concept solves a zillion problems with ECN in one stroke. It's
>     easy to implement (one line of code in codel), backward compatible
>     with all the RFCs, and somewhat incompatible with the stalled out TCP
>     Prague/dualpi effort in the IETF.
>
>     We have several other drafts in progress which I increasingly doubt
>     we'll finish today, but I think only this one is required to get an
>     audience in the tsvwg at the coming IETF meeting.
>
>     If ya have any comments and spare time today, I'd like to get the
>     first draft in tonight, and the filing deadline for final drafts is
>     sometime tomorrow. It may help for context to review some of the other
>     work in the github repo.
>
>     THX!
>
>     --
>
>     Dave Täht
>     CTO, TekLibre, LLC
>     http://www.teklibre.com
>     Tel: 1-831-205-9740
>
>


-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Dave Taht]

AHA!

http://www.hjp.at/doc/rfc/rfc8311.html#sec_3

While the ECN nonce works as specified, and has been deployed in
   limited environments, widespread usage in the Internet has not
   materialized.  A study of the ECN behavior of the top one million web
   servers using 2014 data [Trammell15] found that after ECN was
   negotiated, none of the 581,711 IPv4 servers tested were using both
   ECT codepoints, which would have been a possible sign of ECN nonce
   usage.  Of the 17,028 IPv6 servers tested, four set both ECT(0) and
   ECT(1) on data packets.  This might have been evidence of use of the
   ECN nonce by these four servers, but it might equally have been due
   to erroneous re-marking of the ECN field by a middlebox or router.

I'm not sure if I should cite 8311 or not here, rather than the
original research.

I can't seem to coax our tool to output RFC3168 as normative.



On Sun, Mar 10, 2019 at 2:11 PM Dave Taht <dave.taht@gmail.com> wrote:
>
> On Sun, Mar 10, 2019 at 12:08 PM Holland, Jake <jholland@akamai.com> wrote:
> >
> > Hi Dave,
> >
> > You and John have my enthusiastic +1.
> >
> > It's a frank relief to read this draft after trying to figure out L4S,
> > and I think the basic core concept upon which to build the actual
> > response systems is very well separated and very well framed here.
> >
> > Please submit this and present, I humbly beg you.  It seems to me a
> > strictly better use of ECT(1), even though there's still probably a
> > few hundred pages' worth of catching up to do on draft-writing to
> > nail down details.
> >
> > I have a few minor comments for your consideration, but please don't
> > let them stop you from posting before deadline, if any are hard to
> > integrate.  It would be better to ignore them all and post as-is than
> > to get hung up on these:
> >
> > 1.
> > "Some" in "Some Congestion Experienced" is maybe misleading, and
> > arguably has the same meaning as "Congestion Experienced".
> >
> > I was thinking maybe "Pre-Congestion Experienced" or "Queue
> > Utilization Observed", or if you want to preserve "SCE" and the
> > link to CE (which I do agree is nice), maybe "Slight" or "Sub"
> > instead of "Some", just to try to make it more obvious it's
> > flagging a lesser situation than "there is some congestion".
> >
> > 2.
> > It's easy to accidently read section 5 as underspecified concrete
> > proposals instead of rough sketches for future direction that might
> > be worth investigating.  I'll offer an attempt at some language,
> > feel free to edit (or ignore if you think the intro is enough to
> > make the scope sufficiently clear already):
> >
> >
> > The scope of this document is limited to the definition of the
> > SCE codepoint.  However, for illustration purposes, a few possible
> > future usage scenarios are outlined here.  These examples are non
> > normative.
> >
> > 3.
> > Similarly, I would lower-case the "MAY" and "SHOULD" in section
> > 5.2 for receiver-side handling in TCP--it's not clear this will
> > ever be a good idea to do without more explicit signaling thru
> > new opts or whatever, and granting permission here seems like
> > asking for trouble that's just not necessary.
> >
> >
> > And a few that I'd defer if I were you, but I'd like to see
> > sometime in at least a post-Prague version or list discussion:
> >
> > 4.
> > an informative reference or 2 would be a welcome addition in Section 3:
> >
> >    Research has shown that the ECT(1) codepoint goes essentially unused,
> >    with the "Nonce Sum" extension to ECN having not been implemented in
>
> I have been trying to find that presentation or paper or talk or rfc
> for days now. I *know* it went by sometime in the past 2 years, I had
> my aha! moment - and I keep drawing a blank. Might have been something
> brian trammel did, I simply cannot remember.
>
> I definately want to cite that, and I sure hope I'm not delusional.
>
> > 5.
> > Should this must be MUST in Section 4?  If not, why not?
> >
> >    New SCE-aware receivers and transport protocols must continue to
> >
> >
> > Thanks guys, nice work and good luck!
> >
> > Cheers,
> > Jake
> >
> >
> > On 2019-03-10, 11:07, "Dave Taht" <dave.taht@gmail.com> wrote:
> >
> >     I would love to have some fresh eyeballs on a new IETF draft for the
> >     TSVWG we intend to submit tonight.
> >
> >     I've attached the html for easy to read purposes, but I would prefer
> >     that folk referred back to the github repository for the most current
> >     version, which is here:
> >
> >     https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-SCE.txt
> >
> >     and in open source tradition, discuss here, or file bugs, and submit
> >     pull requests to the gitub.
> >
> >     The first draft (of 3 or more pending), is creating the SCE codepoint
> >     and defining the state machine, is pretty short, and we think the
> >     basic concept solves a zillion problems with ECN in one stroke. It's
> >     easy to implement (one line of code in codel), backward compatible
> >     with all the RFCs, and somewhat incompatible with the stalled out TCP
> >     Prague/dualpi effort in the IETF.
> >
> >     We have several other drafts in progress which I increasingly doubt
> >     we'll finish today, but I think only this one is required to get an
> >     audience in the tsvwg at the coming IETF meeting.
> >
> >     If ya have any comments and spare time today, I'd like to get the
> >     first draft in tonight, and the filing deadline for final drafts is
> >     sometime tomorrow. It may help for context to review some of the other
> >     work in the github repo.
> >
> >     THX!
> >
> >     --
> >
> >     Dave Täht
> >     CTO, TekLibre, LLC
> >     http://www.teklibre.com
> >     Tel: 1-831-205-9740
> >
> >
>
>
> --
>
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-205-9740



--

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Dave Taht]

the SCE draft is now  submitted to IETF! I'm too tired to figure out
what timezone conversion the UTC deadline is in PDT right now, and I
guess the chairs need to move it into the tsvwg working group instead
of individual submissions... and I have no idea what else I missed in
the ietf processes. Ironically the ietf tools do not take upper case,
so I had to rename the draft in the github repo. this is where it lies
now:

https://datatracker.ietf.org/doc/draft-morton-taht-sce/


On Sun, Mar 10, 2019 at 2:28 PM Dave Taht <dave.taht@gmail.com> wrote:
>
> AHA!
>
> http://www.hjp.at/doc/rfc/rfc8311.html#sec_3
>
> While the ECN nonce works as specified, and has been deployed in
>    limited environments, widespread usage in the Internet has not
>    materialized.  A study of the ECN behavior of the top one million web
>    servers using 2014 data [Trammell15] found that after ECN was
>    negotiated, none of the 581,711 IPv4 servers tested were using both
>    ECT codepoints, which would have been a possible sign of ECN nonce
>    usage.  Of the 17,028 IPv6 servers tested, four set both ECT(0) and
>    ECT(1) on data packets.  This might have been evidence of use of the
>    ECN nonce by these four servers, but it might equally have been due
>    to erroneous re-marking of the ECN field by a middlebox or router.
>
> I'm not sure if I should cite 8311 or not here, rather than the
> original research.
>
> I can't seem to coax our tool to output RFC3168 as normative.
>
>
>
> On Sun, Mar 10, 2019 at 2:11 PM Dave Taht <dave.taht@gmail.com> wrote:
> >
> > On Sun, Mar 10, 2019 at 12:08 PM Holland, Jake <jholland@akamai.com> wrote:
> > >
> > > Hi Dave,
> > >
> > > You and John have my enthusiastic +1.
> > >
> > > It's a frank relief to read this draft after trying to figure out L4S,
> > > and I think the basic core concept upon which to build the actual
> > > response systems is very well separated and very well framed here.
> > >
> > > Please submit this and present, I humbly beg you.  It seems to me a
> > > strictly better use of ECT(1), even though there's still probably a
> > > few hundred pages' worth of catching up to do on draft-writing to
> > > nail down details.
> > >
> > > I have a few minor comments for your consideration, but please don't
> > > let them stop you from posting before deadline, if any are hard to
> > > integrate.  It would be better to ignore them all and post as-is than
> > > to get hung up on these:
> > >
> > > 1.
> > > "Some" in "Some Congestion Experienced" is maybe misleading, and
> > > arguably has the same meaning as "Congestion Experienced".
> > >
> > > I was thinking maybe "Pre-Congestion Experienced" or "Queue
> > > Utilization Observed", or if you want to preserve "SCE" and the
> > > link to CE (which I do agree is nice), maybe "Slight" or "Sub"
> > > instead of "Some", just to try to make it more obvious it's
> > > flagging a lesser situation than "there is some congestion".
> > >
> > > 2.
> > > It's easy to accidently read section 5 as underspecified concrete
> > > proposals instead of rough sketches for future direction that might
> > > be worth investigating.  I'll offer an attempt at some language,
> > > feel free to edit (or ignore if you think the intro is enough to
> > > make the scope sufficiently clear already):
> > >
> > >
> > > The scope of this document is limited to the definition of the
> > > SCE codepoint.  However, for illustration purposes, a few possible
> > > future usage scenarios are outlined here.  These examples are non
> > > normative.
> > >
> > > 3.
> > > Similarly, I would lower-case the "MAY" and "SHOULD" in section
> > > 5.2 for receiver-side handling in TCP--it's not clear this will
> > > ever be a good idea to do without more explicit signaling thru
> > > new opts or whatever, and granting permission here seems like
> > > asking for trouble that's just not necessary.
> > >
> > >
> > > And a few that I'd defer if I were you, but I'd like to see
> > > sometime in at least a post-Prague version or list discussion:
> > >
> > > 4.
> > > an informative reference or 2 would be a welcome addition in Section 3:
> > >
> > >    Research has shown that the ECT(1) codepoint goes essentially unused,
> > >    with the "Nonce Sum" extension to ECN having not been implemented in
> >
> > I have been trying to find that presentation or paper or talk or rfc
> > for days now. I *know* it went by sometime in the past 2 years, I had
> > my aha! moment - and I keep drawing a blank. Might have been something
> > brian trammel did, I simply cannot remember.
> >
> > I definately want to cite that, and I sure hope I'm not delusional.
> >
> > > 5.
> > > Should this must be MUST in Section 4?  If not, why not?
> > >
> > >    New SCE-aware receivers and transport protocols must continue to
> > >
> > >
> > > Thanks guys, nice work and good luck!
> > >
> > > Cheers,
> > > Jake
> > >
> > >
> > > On 2019-03-10, 11:07, "Dave Taht" <dave.taht@gmail.com> wrote:
> > >
> > >     I would love to have some fresh eyeballs on a new IETF draft for the
> > >     TSVWG we intend to submit tonight.
> > >
> > >     I've attached the html for easy to read purposes, but I would prefer
> > >     that folk referred back to the github repository for the most current
> > >     version, which is here:
> > >
> > >     https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-SCE.txt
> > >
> > >     and in open source tradition, discuss here, or file bugs, and submit
> > >     pull requests to the gitub.
> > >
> > >     The first draft (of 3 or more pending), is creating the SCE codepoint
> > >     and defining the state machine, is pretty short, and we think the
> > >     basic concept solves a zillion problems with ECN in one stroke. It's
> > >     easy to implement (one line of code in codel), backward compatible
> > >     with all the RFCs, and somewhat incompatible with the stalled out TCP
> > >     Prague/dualpi effort in the IETF.
> > >
> > >     We have several other drafts in progress which I increasingly doubt
> > >     we'll finish today, but I think only this one is required to get an
> > >     audience in the tsvwg at the coming IETF meeting.
> > >
> > >     If ya have any comments and spare time today, I'd like to get the
> > >     first draft in tonight, and the filing deadline for final drafts is
> > >     sometime tomorrow. It may help for context to review some of the other
> > >     work in the github repo.
> > >
> > >     THX!
> > >
> > >     --
> > >
> > >     Dave Täht
> > >     CTO, TekLibre, LLC
> > >     http://www.teklibre.com
> > >     Tel: 1-831-205-9740
> > >
> > >
> >
> >
> > --
> >
> > Dave Täht
> > CTO, TekLibre, LLC
> > http://www.teklibre.com
> > Tel: 1-831-205-9740
>
>
>
> --
>
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-205-9740



-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Toke Høiland-Jørgensen]

Dave Taht <dave.taht@gmail.com> writes:

> the SCE draft is now  submitted to IETF! I'm too tired to figure out
> what timezone conversion the UTC deadline is in PDT right now, and I
> guess the chairs need to move it into the tsvwg working group instead
> of individual submissions... and I have no idea what else I missed in
> the ietf processes. Ironically the ietf tools do not take upper case,
> so I had to rename the draft in the github repo. this is where it lies
> now:
>
> https://datatracker.ietf.org/doc/draft-morton-taht-sce/

Nice. Too bad I can't make IETF, would have loved to see how this plays
out at TSVWG :)

-Toke

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Michael Richardson]

[-- Attachment #1: Type: text/plain, Size: 698 bytes --]


Holland, Jake <jholland@akamai.com> wrote:
    > 1.
    > "Some" in "Some Congestion Experienced" is maybe misleading, and
    > arguably has the same meaning as "Congestion Experienced".

    > I was thinking maybe "Pre-Congestion Experienced" or "Queue
    > Utilization Observed", or if you want to preserve "SCE" and the

I'll combine them:
     "Some Congestion Observed"

(I image a packet rubber-necking to observe some collision on the exressway)

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     mcr@sandelman.ca  http://www.sandelman.ca/        |   ruby on rails    [


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Dave Taht]

On Sun, Mar 10, 2019 at 8:23 PM Michael Richardson <mcr@sandelman.ca> wrote:
>
>
> Holland, Jake <jholland@akamai.com> wrote:
>     > 1.
>     > "Some" in "Some Congestion Experienced" is maybe misleading, and
>     > arguably has the same meaning as "Congestion Experienced".
>
>     > I was thinking maybe "Pre-Congestion Experienced" or "Queue
>     > Utilization Observed", or if you want to preserve "SCE" and the
>
> I'll combine them:
>      "Some Congestion Observed"

Hah.

Ironically as hell, I spent the first 1/4th of my career working for SCO.

>
> (I image a packet rubber-necking to observe some collision on the exressway)
>
> --
> ]               Never tell me the odds!                 | ipv6 mesh networks [
> ]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
> ]     mcr@sandelman.ca  http://www.sandelman.ca/        |   ruby on rails    [
>


-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Richard Scheffenegger]

I can remember reading quite a few papers where a similar scheme for ect(1) was adopted - often with additional changes on both ends to make use of this signal. Including schemes that encoded complex information in the stream of ect0/ect1...

Where can one find simulations of the interaction between legacy and l4s flows when using this?

I think the l4s use of dctcp, to allow coupled queue selection based on the transports expectations, is a more useful case for ect(1)

Richard

Gesendet mit der GMX iPhone App

Am 11.03.19 um 08:08 schrieb Mikael Abrahamsson

> On Sun, 10 Mar 2019, Jonathan Morton wrote:
> 
> > An interesting idea, but SCE marks will appear even when there's a lot 
> > of congestion (at high rates, ie. probably every packet that doesn't 
> > carry CE), as well as showing up at low frequency when the level of 
> > congestion only warrants reducing the growth rate.  I think the word 
> > "Some" is sufficiently descriptive, while "Slight" might cause people to 
> > ignore it completely.
> 
> One way to handle this would be "buffering experienced" or something like 
> that. Ie if this packet is being enqueued into a buffer with non-trivial 
> number of packets in it, mark it.
> 
> The L4S proposal also has the property that their use of this last code 
> point combination in the entire packet header (and this is a big thing, 
> this is the last unicorn) also meant the packet was allowed to be 
> re-ordered. I thought this was a big and nice thing, for other areas. This 
> new proposal removes that property.
> 
> From what I can see, L4S actually is quite novel and has the chance to 
> seriously change the way queueing is done. This proposal seems more like 
> "a little more of what we had before" which I do not think warrants 
> claiming this last unicorn codepoint. I'd like its use to be truly novel 
> and be more than a tweak.
> 
> -- 
> Mikael Abrahamsson    email: swmike@swm.pp.se
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Ecn-sane] [Cake] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Sebastian Moeller]

Hi Mikael,


> On Mar 11, 2019, at 08:08, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
> 
> On Sun, 10 Mar 2019, Jonathan Morton wrote:
> 
>> An interesting idea, but SCE marks will appear even when there's a lot of congestion (at high rates, ie. probably every packet that doesn't carry CE), as well as showing up at low frequency when the level of congestion only warrants reducing the growth rate.  I think the word "Some" is sufficiently descriptive, while "Slight" might cause people to ignore it completely.
> 
> One way to handle this would be "buffering experienced" or something like that. Ie if this packet is being enqueued into a buffer with non-trivial number of packets in it, mark it.
> 
> The L4S proposal also has the property that their use of this last code point combination in the entire packet header (and this is a big thing, this is the last unicorn) also meant the packet was allowed to be re-ordered.

	How is packet reordering for anybody but the folks responsible for operating the "conduits" in any way attractive? 




> I thought this was a big and nice thing, for other areas. This new proposal removes that property.
> 
> From what I can see, L4S actually is quite novel and has the chance to seriously change the way queueing is done. This proposal seems more like "a little more of what we had before" which I do not think warrants claiming this last unicorn codepoint.

	I would argue that evolutionary changes tend to look mostly like "more of the old, just a little different" sothis might be considered a plus for this proposal ;)

Best Regards
	Sebastian


> I'd like its use to be truly novel and be more than a tweak.
> 
> -- 
> Mikael Abrahamsson    email: swmike@swm.pp.se
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Sebastian Moeller]

Dear Richard,

I am an interested layman here, so do not take my questions too seriously...

> On Mar 11, 2019, at 08:35, Richard Scheffenegger <rscheff@gmx.at> wrote:
> 
> I can remember reading quite a few papers where a similar scheme for ect(1) was adopted - often with additional changes on both ends to make use of this signal. Including schemes that encoded complex information in the stream of ect0/ect1...
> 
> Where can one find simulations of the interaction between legacy and l4s flows when using this?

	Given that l4s is not deployed, I would argue that until it is anything new will only need to play nice with what you call legacy flows (I would call them normal flows).

> 
> I think the l4s use of dctcp, to allow coupled queue selection based on the transports expectations, is a more useful case for ect(1)

	But L4s, at least from a glance at draft-briscoe-tsvwg-l4s-arch-01 seems ti merit the unicorn-qualifier that Mikael used: since it seems to imply that the l4s path will not encounter drops just marks to achieve its goal of "keep[ing] congestion loss to zero". This probably is true for a rather strict definition of "congestion loss", but it flies into the face of the naive insight that once under sufficient load a router is going to drop packets to keep its queues under control, and if each flow uses up less of the queue (a worthy goal) it will just take more concurrent flow to push the router into the dropping regime, but that is a quantitative difference not a qualitative one. I guess this shows the lack of understanding from my side more than short-comings of l4s...

Best Regards
	Sebastian

> 
> Richard
> 
> Gesendet mit der GMX iPhone App
> 
> Am 11.03.19 um 08:08 schrieb Mikael Abrahamsson
> 
>> On Sun, 10 Mar 2019, Jonathan Morton wrote:
>> 
>>> An interesting idea, but SCE marks will appear even when there's a lot 
>>> of congestion (at high rates, ie. probably every packet that doesn't 
>>> carry CE), as well as showing up at low frequency when the level of 
>>> congestion only warrants reducing the growth rate.  I think the word 
>>> "Some" is sufficiently descriptive, while "Slight" might cause people to 
>>> ignore it completely.
>> 
>> One way to handle this would be "buffering experienced" or something like 
>> that. Ie if this packet is being enqueued into a buffer with non-trivial 
>> number of packets in it, mark it.
>> 
>> The L4S proposal also has the property that their use of this last code 
>> point combination in the entire packet header (and this is a big thing, 
>> this is the last unicorn) also meant the packet was allowed to be 
>> re-ordered. I thought this was a big and nice thing, for other areas. This 
>> new proposal removes that property.
>> 
>> From what I can see, L4S actually is quite novel and has the chance to 
>> seriously change the way queueing is done. This proposal seems more like 
>> "a little more of what we had before" which I do not think warrants 
>> claiming this last unicorn codepoint. I'd like its use to be truly novel 
>> and be more than a tweak.
>> 
>> -- 
>> Mikael Abrahamsson    email: swmike@swm.pp.se
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Jonathan Morton]

> On 11 Mar, 2019, at 9:08 am, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
> 
> …also meant the packet was allowed to be re-ordered. I thought this was a big and nice thing…

Seriously?  I had to dig in the specs to find any mention of that, and… it's all about better supporting bonded links.  Which can already be done by implementing RACK at the sender, and all you propose is that when L4S is in use, the extra buffering at the link layer is dropped.  This is absolutely useless for ordinary Internet users, who are unlikely to have consecutive packets sufficiently closely traversing such a link for this reordering to exceed the 3-dupack threshold in any case - so you might as well delete that reordering buffer anyway, and let the endpoints handle it.  You don't need L4S for that.

At bottom L4S is also a very simple scheme: when ECT(1) is used, a higher CE marking rate is expected of middleboxes, and a less aggressive backoff in response to CE is expected of endpoints.  This is, incidentally, incompatible with Codel-like AQMs on the one hand and existing TCPs on the other hand, and so requires negotiation between the endpoints  (eg. using AccECN) to discover whether setting ECT(1) at the sender is legal.  SCE does not require such negotiation (ie. a transport could implement it entirely at the receiver, manipulating the send rate via the already-standardised receive window), so should be easier to specify and deploy successfully.

> On 11 Mar, 2019, at 9:35 am, Richard Scheffenegger <rscheff@gmx.at> wrote:
> 
> I can remember reading quite a few papers where a similar scheme for ect(1) was adopted - often with additional changes on both ends to make use of this signal. Including schemes that encoded complex information in the stream of ect0/ect1...
> 
> Where can one find simulations of the interaction between legacy and l4s flows when using this?

If by "between legacy and l4s" you mean "between existing SCE-ignorant and new SCE-aware" - because SCE is not L4S…

We haven't yet drilled down to that level of detail, but we plan to do so in the very near future.  SCE itself is just a means of communicating network state from AQMs to receivers, hence the brevity and simplicity of the I-D.

The encoding is not complex, just a ratio of ECT to SCE markings which is ignored by existing receivers and not produced by existing middleboxes, *and* continued use of CE as presently standardised.  This naturally ensures backwards compatibility and incremental deployability.  The stable ratio of ECT to SCE is defined as 1:1, allowing more flexibility in signalling how much unused link capacity remains than DCTCP's "2 CEs per RTT" stability condition permits, and potentially simplifying implementations.

The question of whether SCE-aware transports will compete fairly with SCE-ignorant transports, given a single-queue AQM implementing SCE, is an interesting and important one.  Intuitively, one can see that the SCE-ignorant transport will drive the AQM into the CE-marking regime, which may cause both flows to drop back, but there will be periods when the SCE-aware flow has inhibited growth while the SCE-ignorant flow has not, and so the SCE-aware flow might have overall lower throughput.

On the other hand, a Codel-like AQM (as opposed to a RED-like AQM) has a higher probability of signalling CE to a flow that sends more packets, which may naturally compensate for this.  And on the gripping hand, an SCE-aware flow operating alone should have (slightly) higher goodput than an SCE-ignorant flow operating alone, which illustrates the case for flow-isolating AQMs.

Two SCE-aware transports might interact strangely on a single-queue AQM, depending on how their response to SCE is implemented.  Eliminating the present sawtooth behaviour entirely is a desirable goal, but AIMD is what allows flows competing in a single queue to converge on fair throughput.  This will require careful attention when specifying transport behaviour.

One exciting possibility is finally solving the slow-start problem, by beginning SCE signalling (at a low rate which still permits window growth) when the link is half-full rather than when queuing begins.  That would cause the exponential growth phase of typical TCPs to exit one RTT later, when the link is just fully utilised, instead of massively overshooting and being cut back to nothing as presently occurs.  There is a range of flow lengths which could see a concrete performance benefit from that.

That requires that the AQM is able to determine when the link reaches 50% utilisation, which is not always true (Cake and hardware AQMs could do it, being aware of the link capacity) - but SCE does not mandate this behaviour, it's just a good idea.  I'm not aware of any possibility for L4S to achieve the same result; however much it redefines the meaning of CE, it won't be asserted until the link is full.

SCE-aware AQMs and TCPs are not difficult to describe and should not be very difficult to implement.  Though not described explicitly in the SCE I-D, I'm working on descriptions of how to do so, which should become additional I-Ds.  We should then be able to write working code and run simulations in ns-3, and maybe also in Linux.

 - Jonathan Morton

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Mikael Abrahamsson]

[-- Attachment #1: Type: text/plain, Size: 1641 bytes --]

On Mon, 11 Mar 2019, Jonathan Morton wrote:

> Seriously?  I had to dig in the specs to find any mention of that, and… 
> it's all about better supporting bonded links.  Which can already be

It doesn't stop there. Right now DOCSIS, 3GPP networks, Wifi etc all do 
ordering guarantees, so they will hold up any delivery of packets until it 
can assure that none of them are delivered out of order.

Allowing these transports to re-order the packets mean they can do a 
better job than they do today. You do not want to ask them to drop their 
packets either because the drop rate is potentially way higher than most 
transports would feel comfortable with.

> done by implementing RACK at the sender, and all you propose is that 
> when L4S is in use, the extra buffering at the link layer is dropped.

I did?

> This is absolutely useless for ordinary Internet users, who are unlikely 
> to have consecutive packets sufficiently closely traversing such a link 
> for this reordering to exceed the 3-dupack threshold in any case - so 
> you might as well delete that reordering buffer anyway, and let the 
> endpoints handle it.  You don't need L4S for that.

That's not my experience with wifi and how it behaves at the edge.

> endpoints (eg. using AccECN) to discover whether setting ECT(1) at the 
> sender is legal.  SCE does not require such negotiation (ie. a transport 
> could implement it entirely at the receiver, manipulating the send rate 
> via the already-standardised receive window), so should be easier to 
> specify and deploy successfully.

Well, I am not convinced blowing the last codepoint on SCE has enough 
merit.

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Jonathan Morton]

> On 11 Mar, 2019, at 11:07 am, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
> 
> Well, I am not convinced blowing the last codepoint on SCE has enough merit.

I will make a stronger statement: I am convinced that blowing the last codepoint on L4S does *not* have enough merit.

Meanwhile, work continues.

 - Jonathan Morton

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Mikael Abrahamsson]

On Mon, 11 Mar 2019, Jonathan Morton wrote:

>> On 11 Mar, 2019, at 11:07 am, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
>>
>> Well, I am not convinced blowing the last codepoint on SCE has enough merit.
>
> I will make a stronger statement: I am convinced that blowing the last codepoint on L4S does *not* have enough merit.

... and I believe that blowing it on SCE doesn't have merit either.

That's the entire thing why I am opposing the use of ECT(1) unless we're 
*really* *really* *really* sure there is something we want to blow it on.

Using it for SCE for me is marginal benefit compared to what ECT(1) could 
be used for either. I think L4S is proposing enough novelty that it could 
be used for that, but I'm open for other suggestions. SCE isn't enough.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Ecn-sane] [Codel] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Dave Taht]

Everybody, calm down. I put this out merely to get comment before we
submitted the first of several drafts. That draft is now submitted and
we've asked for a talk slot in the tsvwg for it. I cc'd the world to
get quick initial feedback, and I want to shut this overbroad
conversation down and move it to just the ecn-sane mailing list.

The l4s mailing list is dead, and the debates on the AQM mailing list and here,
unhelpful - for decades. So, back in august I started a new working
group here, under house rules that I thought would be more productive,
and asked that people that wanted to debate ecn more sanely, join. few
did.

And jon and I have been working for months (and largely not on the
list) to try and create a compromise proposal of which y'all just saw
the first output. There's more in the bufferbloat-rfcs repo.

The rules for joining the ecn-sane list are simple - take the time to
step back and write a write a short position paper, and join (or
create) a team. You needn't do that immediately. If you disagree with
the rules of operation of the ecn-sane working group, submit a pull
request or file a bug on the web site. where we can discuss it.

Ironically our ssl cert just expired and I don't remember how to fix it.

Please join the ecn-sane mailing list for discussing this stuff and
stop cc-ing the whole bufferbloat.net  world on it, please.

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Holland, Jake]

Hi,

It's a fair question which is the best use of the last IP codepoint,
and thanks Mikael for raising some good questions.

There was a great example posted recently of why the wifi,docsis,etc.
ordering attempts aren't good enough to guarantee ordering (namely ECMP):
https://mailarchive.ietf.org/arch/msg/tsvwg/WR-UXtj9jXiYNQQIALfXQDikAjw
<h.t. Richard Scheffenegger>

I also was thinking something like RACK is a better solution in the long
term.  If >80% of flows start responding better to reordering, future switches
can safely drop the reordering requirements they've adopted to work around
TCP's problems.  (It's a shame that the switches ended up having to work
around that, but there was never an ordering requirement on the IP
packets, just a performance improvement to the extent you managed to
provide one.)

I think both proposals are doing almost the same thing from the point of
view of the endpoints (that is: they're providing a fine-grained congestion
signal so that sender can back off earlier and less aggressively than
multiplicative decrease).

The key difference to me is that L4S doesn't work without a dual queue.
It embeds a major design decision ("how many queues should there be at
a middle box?") into the codepoint, and comes with very narrow requirements
on the sender's congestion control.

SCE by contrast can have different responses for different congestion
controls, with more room for experimenting and fine tuning.  It fits much
better with concepts like fq, which can maintain something like fairness
even when different senders are behaving differently.

Along the same lines, it's less open to abuse, because there's not an
opt-in fast lane, like with L4S.  I'm worried L4S just sticks us in the
same arms race with a different queue, in the end, where everybody does
better for them and worse for the network if they can find a way to cheat
a little.  So I view good fq support as more useful in the end.  (And I
agree that wifi/docsis/etc. shouldn't be trying to maintain ordering
guarantees.)

I do think SCE is going to end up needing TCP options to communicate the
congestion signal, but I think this is a good first step down a long
road, and I look forward to seeing experiments that can demonstrate the
advantages, if they turn out to be what I expect here.

Cheers,
Jake

On 2019-03-11, 02:07, "Mikael Abrahamsson" <swmike@swm.pp.se> wrote:

    On Mon, 11 Mar 2019, Jonathan Morton wrote:
    
    > Seriously?  I had to dig in the specs to find any mention of that, and… 
    > it's all about better supporting bonded links.  Which can already be
    
    It doesn't stop there. Right now DOCSIS, 3GPP networks, Wifi etc all do 
    ordering guarantees, so they will hold up any delivery of packets until it 
    can assure that none of them are delivered out of order.
    
    Allowing these transports to re-order the packets mean they can do a 
    better job than they do today. You do not want to ask them to drop their 
    packets either because the drop rate is potentially way higher than most 
    transports would feel comfortable with.
    
    > done by implementing RACK at the sender, and all you propose is that 
    > when L4S is in use, the extra buffering at the link layer is dropped.
    
    I did?
    
    > This is absolutely useless for ordinary Internet users, who are unlikely 
    > to have consecutive packets sufficiently closely traversing such a link 
    > for this reordering to exceed the 3-dupack threshold in any case - so 
    > you might as well delete that reordering buffer anyway, and let the 
    > endpoints handle it.  You don't need L4S for that.
    
    That's not my experience with wifi and how it behaves at the edge.
    
    > endpoints (eg. using AccECN) to discover whether setting ECT(1) at the 
    > sender is legal.  SCE does not require such negotiation (ie. a transport 
    > could implement it entirely at the receiver, manipulating the send rate 
    > via the already-standardised receive window), so should be easier to 
    > specify and deploy successfully.
    
    Well, I am not convinced blowing the last codepoint on SCE has enough 
    merit.
    

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Jonathan Morton]

> On 11 Mar, 2019, at 5:29 pm, Holland, Jake <jholland@akamai.com> wrote:
> 
> The key difference to me is that L4S doesn't work without a dual queue.
> It embeds a major design decision ("how many queues should there be at
> a middle box?") into the codepoint, and comes with very narrow requirements
> on the sender's congestion control.

It's certainly unclear to me what happens when an L4S flow passes through a Classic ECN middlebox, especially one with only a single queue.  I get the impression that a DCTCP-like congestion response would tend to starve out conventional flows competing with it.  Unless you have data showing otherwise?

That has serious implications for incremental deployability, because you can't safely deploy L4S endpoints until single-queue Classic ECN middleboxes have all been replaced with L4S or flow-isolating types.  And without L4S endpoints in use, where's the impetus to do that?  Chicken and egg.

Conversely, an SCE-aware flow passing through a Classic ECN middlebox behaves just like a Classic ECN flow.  The only question arises when a single-queue AQM is made SCE-aware, and in that case the SCE-aware flows are the ones that might get outcompeted (ie. they are friendlier, not more aggressive).  If necessary, it would be easy to specify that single-queue AQMs "should not" produce SCE marks, only the flow-isolating types - which in any case are easier to deploy at edge devices where statistical multiplexing works less well.

Incremental deployability is very important when tackling a project as big as this.  SCE takes it as a central tenet. L4S appears, in practice, to have overlooked it.  That's my objection to L4S.

 - Jonathan Morton

Re: [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Holland, Jake]

+1, I agree SCE on its own isn't enough.

Before I support adoption as a proposed standard I'd want real-world tests
demonstrating the value.  I believe SCE has potential similar to L4S by
providing a similar fine-grained congestion signal, and that it does so
in a much cleaner way.

But there's a big gap between "has potential" and "has proven benefit" that
I'd want to see filled before it's an RFC.

That said, I would like to see experiments go forward, and I would like to
see this become an active draft that a wg owns, so that if the experiments
do prove there's utility that can be captured, it has a good path forward.

I have concerns about L4S, and so my relief is about seeing a cleaner
(and backward compatible!) proposal that does something that to me looks
like a very similar effect.  And I would very much like to see a bakeoff
of some sort before committing ECT(1) to the use of L4S, since it seems
to me there are some ugly problems down that road.

Cheers,
Jake

On 2019-03-11, 02:47, "Mikael Abrahamsson" <swmike@swm.pp.se> wrote:

    On Mon, 11 Mar 2019, Jonathan Morton wrote:
    
    >> On 11 Mar, 2019, at 11:07 am, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
    >>
    >> Well, I am not convinced blowing the last codepoint on SCE has enough merit.
    >
    > I will make a stronger statement: I am convinced that blowing the last codepoint on L4S does *not* have enough merit.
    
    ... and I believe that blowing it on SCE doesn't have merit either.
    
    That's the entire thing why I am opposing the use of ECT(1) unless we're 
    *really* *really* *really* sure there is something we want to blow it on.
    
    Using it for SCE for me is marginal benefit compared to what ECT(1) could 
    be used for either. I think L4S is proposing enough novelty that it could 
    be used for that, but I'm open for other suggestions. SCE isn't enough.
    
    -- 
    Mikael Abrahamsson    email: swmike@swm.pp.se
    _______________________________________________
    Bloat mailing list
    Bloat@lists.bufferbloat.net
    https://lists.bufferbloat.net/listinfo/bloat
    

Re: [Ecn-sane] [Bloat] [Codel] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Dave Taht]

On Mon, Mar 11, 2019 at 7:14 AM Bob Briscoe <research@bobbriscoe.net> wrote:
>
> Dave,
>
> L4S is far from dead. It's merely been working differently from how you're used to. Those working on an L4S AQM (at least those in the cable industry) had to have a private WG for the last ~18 months, but now we're allowed to publish and talk openly again. Similarly, there's work under the covers on an L4S AQM in switch hardware.

After traffic essentially vanished from the various ietf mailing
lists, and specs kept dropping, I assumed there was work being done
somewhere. I'm very glad that it's now more open

We announced the ecn-sane project and it's goals last august. If you
and/or you group cannot participate under those house rules please
suggest changes. And/or resurrect an appropriate list within the ietf.

https://www.bufferbloat.net/projects/ecn-sane/wiki/

>And I see external signs of work under covers on DSL access equipment (covers that I am not under any longer).

It has certainly been my hope that the DSL folk would at least wake up
and implement BQL in their lowest level firmware for about 8 years
now. Free.fr basically did that in 2012 when they shipped fq_codel on
their revolution series of modems.

>
> Nonetheless, I think you will see updated Linux code for an L4S DualQ Coupled AQM built against the mainline tree appear on netdev list today.

I am beyond delighted to finally have a chance to evaluate this. Have
you run any flent related tests through it yet?

Regrettably since this code posting is so close to netdevconf it's
going to be very difficult for me to do a comprehensive evaluation in
time to form an objective opinion. I'm busy on something else.

> ==In summary==
>
> The problem that the SCE draft identifies with TCP's sharp multiplicative decrease is also the primary problem that L4S identified.

Yes, we have long been in agreement that some congestion signal should
be an earlier notification than drop.

> Like L4S, SCE requires changes to network, sender and receiver (see comment later about the rcv-window approach hinted at in the SCE draft). But SCE is just starting on its journey. Having to change end systems and network together is really tough and takes many years.

Not really, the AQM portion of SCE could roll out very quickly across
all of the linux and freebsd universe, if consensus is achieved that
it's a good idea.

>
> It seems you're trying to do the same thing as L4S, but by slightly different means. Before splitting the people involved in this into two factions, can you say what you didn't like about the L4S approach in the first place? We've been very careful to specify L4S broadly enough so that it can encompass many different approaches within it.

I've read the 100+ of spec now multiple times over the years (and all
your work on ecn in general), and I hope, that once we get a bit of
time, we can do a detailed comparison of the two approaches.

But, honestly, based on the total inactivity on the tcp prague mailing
list that it had died, until recently.

>
> The only thing stated against L4S I can find is that it's taking a long time. Starting an identically difficult approach now is going to restart the clock, and take a lot lot longer.

SCE and the modifications to the relevant already IETF approved AQMs
are extremely straightforward, backward compatible approaches to
extending RFC3168 for all existing transports.

It's not an identically difficult approach at all.

Aside from some more detailed analysis of transport effects and the
inevitable debate in netdevconf and ietf, rolling out SCE, at least in
openwrt and linux, could be happen by about.... june.  Particularly
with the now more readily available source code to compare for the two
approaches, independent experts should be able to leap in and provide
feedback.

> ==2 output values vs. 2 input values.==
>
> We considered schemes where the AQM can use a second marking as a lower strength /output/ (like VCP, my own QV and now SCE). But we worked out that there were a wider range of advantages and much more significant performance improvements from the sender using a second marking to distinguish how it will behave (i.e. a second /input/ to the classifier in front of the AQM).
>
> Don't get me wrong. It's useful that you guys are putting the work in on SCE. Then everyone can compare the two approaches (again), as a check on whether that decision was correct. That's important, cos ECT(1) is the last useful codepoint in the IP header. See: "Notification of Less Severe Congestion than CE" at https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-05#appendix-C.2 where we've written:

Yep. I am glad that both of our cards are on the table.

>
>    Before assigning ECT(1) as an identifer for L4S, we must carefully
>    consider whether it might be better to hold ECT(1) in reserve for
>    future standardisation of rapid flow acceleration, which is an
>    important and enduring problem [RFC6077].
>
>
> ==FQ-only vs. FQ or DualQ==
>
> One of the problems with the 2 outputs approach (SCE etc.) is that it is only possible with per-flow queuing. I doubt you'll get the last useful codepoint in the IP header for just that. It's sort-of obvious that, if you try to implement SCE in a FIFO, you can only have one queue length for all the flows. Then legacy TCP flows that don't understand SCE would push the queue deeper to the CE threshold, ruining it for the flows that support SCE.
>
> We worked out that the 2 inputs approach (L4S) is more generic - ie. it can be used with FQ or DualQ (multiple or just 2 queues).
>
> For instance, you can modify fq_CoDel for senders that uses ECT(1) to indicate that they support a small multiplicative decrease (L4S senders). You only need the following: Include the last bit of the ECN field with the flow ID when you do the classification for sfq. Then in the queues with ECN==X1, you instantiate a shallow threshold ECN AQM. This could be CoDel with a shallow 'target', but you also want it to respond immediately (zero 'interval'), so even a simple step at about 1ms will work, but a random RED-like ramp on the /instantaneous/ queue is much better.
>
> ==Re-purposing the Receive Window?===
>
> Receiver congestion control using the receive window may seem like a useful stop-gap, but it runs counter to how nearly all today's transport protocols are intended to work (except, I know of a LEDBAT-like example from Microsoft Research). So you will have your work cut out proving that it is stable and that the two ends don't fight, etc. if you think L4S is taking years, you will find that takes longer. There is current research on this that I can point you to, if you want.

I kind of wish we'd cut that from the draft. My proposal was
considerably different, and a bit longer in the prior paragraph.

> That's why we chose an approach that had a pre-existing widely deployed existence proof (DCTCP) to start from.

I have never had a good number for DCTCP's actual deployment. In the
cloud services I use, it's all (paced) cubic and BBR nowadays.

>
> IETF groups like rmcat explicitly decided early on to require the approach where the receiver is a dumb reflector, then new sender congestion control algorithms can be deployed unilaterally. The argument was that the feedback function can be thought of as a sub-layer below the congestion control function. The ongoing addition of accurate ECN feedback to TCP and to QUIC also take the dumb reflector approach. And RTCP already does it that way.

I do think there is more work to do here.

> ==ECN feedback problems===
>
> Over the last decades, we've made sure that the ECN feedback schemes for TCP, QUIC, RTP (but not SCTP yet) can all feed back ECT(1) as well as CE, in case a scheme like SCE came along.
>
> However, the solution in the TCP case [draft-ietf-tcpm-accurate-ecn] is still problematic for SCE if you're impatient. The base scheme overloads 3 bits in the TCP header, which it uses to feed back CE only. To feed back ECT(1) we had to add a TCP option. That's not going to get through middleboxes for many years. The TCP option is also optional to implement. Two of the main TCP developers are currently saying they will probably not implement it, at least not initially.

One thing I've not understood about L4S is why didn't you just pick a
new NNN codepoint for it and not fiddled with the ECT(1) at all.

> ==Tunnels and lower layers==
>
> Over the years I've maintained a fairly lonely activity to make sure that the ECN propagation behaviour of tunnels and layer 2 protocols will treat ECT(1) as either a stronger output signal (as in SCE) or as an alternative input signal to an AQM (as in L4S). Theoretically, this allows either the SCE or the L4S approach.
>
> HOWEVER, you would probably not be surprised at how many people read the spec [RFC6040], and say "Ah, no router alters ECT(0) to ECT(1) today, so I'm not going to implement that unnecessary extra line of code in my tunnel decap."

For the record I have made sure your efforts to do ecn as right as
possible made it into new things - notably wireguard "does it right",
now - and mosh has a wonderful response to ecn that we put in there
years ago.

I am grateful for your work on clarifying these things.
>
> ==Wider benefit: Relaxing link ordering==
>
> By overloading the ECT(1) marking to mean "the sender uses time for loss detection" a link can relax the reordering requirement on ECT(1) packets today. You can do that with L4S, cos the sender is selecting the marking. You can't do that when the AQM is selecting the marking (as with SCE).
>
> If transport protocols detect loss in time units without tying it to any marking (as in RACK on its own), a link cannot use this to relax the ordering requirement until it is sure that all the legacy non-RACK transports have decayed out of the network. That would be measured in decades.

There's an awful lot here worthy of discussion that I cannot respond
to today, and I've got this other, open, mailing list for it. You are
welcome to join.

> HTH
>
>
>
> Bob
>
> On 11/03/2019 10:11, Dave Taht wrote:
>
> Everybody, calm down. I put this out merely to get comment before we
> submitted the first of several drafts. That draft is now submitted and
> we've asked for a talk slot in the tsvwg for it. I cc'd the world to
> get quick initial feedback, and I want to shut this overbroad
> conversation down and move it to just the ecn-sane mailing list.
>
> The l4s mailing list is dead, and the debates on the AQM mailing list and here,
> unhelpful - for decades. So, back in august I started a new working
> group here, under house rules that I thought would be more productive,
> and asked that people that wanted to debate ecn more sanely, join. few
> did.
>
> And jon and I have been working for months (and largely not on the
> list) to try and create a compromise proposal of which y'all just saw
> the first output. There's more in the bufferbloat-rfcs repo.
>
> The rules for joining the ecn-sane list are simple - take the time to
> step back and write a write a short position paper, and join (or
> create) a team. You needn't do that immediately. If you disagree with
> the rules of operation of the ecn-sane working group, submit a pull
> request or file a bug on the web site. where we can discuss it.
>
> Ironically our ssl cert just expired and I don't remember how to fix it.
>
> Please join the ecn-sane mailing list for discussing this stuff and
> stop cc-ing the whole bufferbloat.net  world on it, please.
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
>
> --
> ________________________________________________________________
> Bob Briscoe                               http://bobbriscoe.net/



-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: [Ecn-sane] [Cake] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[David Lang]

On Mon, 11 Mar 2019, Sebastian Moeller wrote:

> How is packet reordering for anybody but the folks responsible for operating 
> the "conduits" in any way attractive?

It's more that not worrying about maintaining the order, and just moving the 
packets as fast as possible reduces the overhead.

The majority of the time, packets will be in order, but race conditions and 
corner cases are allowed to forward packets out of order rather than having the 
delay some packets to maintain the order.

David Lang