[Codel] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Codel] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 1368 bytes --]

I would love to have some fresh eyeballs on a new IETF draft for the
TSVWG we intend to submit tonight.

I've attached the html for easy to read purposes, but I would prefer
that folk referred back to the github repository for the most current
version, which is here:

https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-SCE.txt

and in open source tradition, discuss here, or file bugs, and submit
pull requests to the gitub.

The first draft (of 3 or more pending), is creating the SCE codepoint
and defining the state machine, is pretty short, and we think the
basic concept solves a zillion problems with ECN in one stroke. It's
easy to implement (one line of code in codel), backward compatible
with all the RFCs, and somewhat incompatible with the stalled out TCP
Prague/dualpi effort in the IETF.

We have several other drafts in progress which I increasingly doubt
we'll finish today, but I think only this one is required to get an
audience in the tsvwg at the coming IETF meeting.

If ya have any comments and spare time today, I'd like to get the
first draft in tonight, and the filing deadline for final drafts is
sometime tomorrow. It may help for context to review some of the other
work in the github repo.

THX!

-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

[-- Attachment #2: draft-morton-taht-SCE.html --]
[-- Type: text/html, Size: 27736 bytes --]

Re: [Codel] [Cake] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Sebastian Moeller]

[-- Attachment #1: Type: text/plain, Size: 1907 bytes --]

Hi Dave, hi Jonathan,

I like it! Outside of my area of expertise, sure, but still short, sweet and focussed. As I commented on github, it might halp if you could make a point how this could actually help core-routers to soften their load, as it is those devices that need to learn to use SCE for the proposed endp-point methods to get anywhere in real life.

Best Regards
	Sebastian


> On Mar 10, 2019, at 19:04, Dave Taht <dave.taht@gmail.com> wrote:
> 
> I would love to have some fresh eyeballs on a new IETF draft for the
> TSVWG we intend to submit tonight.
> 
> I've attached the html for easy to read purposes, but I would prefer
> that folk referred back to the github repository for the most current
> version, which is here:
> 
> https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-SCE.txt
> 
> and in open source tradition, discuss here, or file bugs, and submit
> pull requests to the gitub.
> 
> The first draft (of 3 or more pending), is creating the SCE codepoint
> and defining the state machine, is pretty short, and we think the
> basic concept solves a zillion problems with ECN in one stroke. It's
> easy to implement (one line of code in codel), backward compatible
> with all the RFCs, and somewhat incompatible with the stalled out TCP
> Prague/dualpi effort in the IETF.
> 
> We have several other drafts in progress which I increasingly doubt
> we'll finish today, but I think only this one is required to get an
> audience in the tsvwg at the coming IETF meeting.
> 
> If ya have any comments and spare time today, I'd like to get the
> first draft in tonight, and the filing deadline for final drafts is
> sometime tomorrow. It may help for context to review some of the other
> work in the github repo.
> 
> THX!
> 
> -- 
> 
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-205-9740

[-- Attachment #2: draft-morton-taht-SCE.html --]
[-- Type: text/html, Size: 27736 bytes --]

[-- Attachment #3: Type: text/plain, Size: 146 bytes --]

> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Codel] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Jonathan Morton]

> On 10 Mar, 2019, at 9:08 pm, Holland, Jake <jholland@akamai.com> wrote:
> 
> It's easy to accidently read section 5 as underspecified concrete
> proposals instead of rough sketches for future direction that might
> be worth investigating.

This is something I noticed as well, and have edited to match the intended tone of the document (which is to leave specific implementation details to other documents, and to focus on the information conveyed).

> New SCE-aware receivers and transport protocols must continue to


I had already changed this to SHOULD, among other edits.

> "Some" in "Some Congestion Experienced" is maybe misleading, and
> arguably has the same meaning as "Congestion Experienced".
> 
> I was thinking maybe "Pre-Congestion Experienced" or "Queue
> Utilization Observed", or if you want to preserve "SCE" and the
> link to CE (which I do agree is nice), maybe "Slight" or "Sub"
> instead of "Some", just to try to make it more obvious it's
> flagging a lesser situation than "there is some congestion".

An interesting idea, but SCE marks will appear even when there's a lot of congestion (at high rates, ie. probably every packet that doesn't carry CE), as well as showing up at low frequency when the level of congestion only warrants reducing the growth rate.  I think the word "Some" is sufficiently descriptive, while "Slight" might cause people to ignore it completely.

I think other points are less urgent to address in the first submission.

 - Jonathan Morton

Re: [Codel] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Dave Taht]

On Sun, Mar 10, 2019 at 12:08 PM Holland, Jake <jholland@akamai.com> wrote:
>
> Hi Dave,
>
> You and John have my enthusiastic +1.
>
> It's a frank relief to read this draft after trying to figure out L4S,
> and I think the basic core concept upon which to build the actual
> response systems is very well separated and very well framed here.
>
> Please submit this and present, I humbly beg you.  It seems to me a
> strictly better use of ECT(1), even though there's still probably a
> few hundred pages' worth of catching up to do on draft-writing to
> nail down details.
>
> I have a few minor comments for your consideration, but please don't
> let them stop you from posting before deadline, if any are hard to
> integrate.  It would be better to ignore them all and post as-is than
> to get hung up on these:
>
> 1.
> "Some" in "Some Congestion Experienced" is maybe misleading, and
> arguably has the same meaning as "Congestion Experienced".
>
> I was thinking maybe "Pre-Congestion Experienced" or "Queue
> Utilization Observed", or if you want to preserve "SCE" and the
> link to CE (which I do agree is nice), maybe "Slight" or "Sub"
> instead of "Some", just to try to make it more obvious it's
> flagging a lesser situation than "there is some congestion".
>
> 2.
> It's easy to accidently read section 5 as underspecified concrete
> proposals instead of rough sketches for future direction that might
> be worth investigating.  I'll offer an attempt at some language,
> feel free to edit (or ignore if you think the intro is enough to
> make the scope sufficiently clear already):
>
>
> The scope of this document is limited to the definition of the
> SCE codepoint.  However, for illustration purposes, a few possible
> future usage scenarios are outlined here.  These examples are non
> normative.
>
> 3.
> Similarly, I would lower-case the "MAY" and "SHOULD" in section
> 5.2 for receiver-side handling in TCP--it's not clear this will
> ever be a good idea to do without more explicit signaling thru
> new opts or whatever, and granting permission here seems like
> asking for trouble that's just not necessary.
>
>
> And a few that I'd defer if I were you, but I'd like to see
> sometime in at least a post-Prague version or list discussion:
>
> 4.
> an informative reference or 2 would be a welcome addition in Section 3:
>
>    Research has shown that the ECT(1) codepoint goes essentially unused,
>    with the "Nonce Sum" extension to ECN having not been implemented in

I have been trying to find that presentation or paper or talk or rfc
for days now. I *know* it went by sometime in the past 2 years, I had
my aha! moment - and I keep drawing a blank. Might have been something
brian trammel did, I simply cannot remember.

I definately want to cite that, and I sure hope I'm not delusional.

> 5.
> Should this must be MUST in Section 4?  If not, why not?
>
>    New SCE-aware receivers and transport protocols must continue to
>
>
> Thanks guys, nice work and good luck!
>
> Cheers,
> Jake
>
>
> On 2019-03-10, 11:07, "Dave Taht" <dave.taht@gmail.com> wrote:
>
>     I would love to have some fresh eyeballs on a new IETF draft for the
>     TSVWG we intend to submit tonight.
>
>     I've attached the html for easy to read purposes, but I would prefer
>     that folk referred back to the github repository for the most current
>     version, which is here:
>
>     https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-SCE.txt
>
>     and in open source tradition, discuss here, or file bugs, and submit
>     pull requests to the gitub.
>
>     The first draft (of 3 or more pending), is creating the SCE codepoint
>     and defining the state machine, is pretty short, and we think the
>     basic concept solves a zillion problems with ECN in one stroke. It's
>     easy to implement (one line of code in codel), backward compatible
>     with all the RFCs, and somewhat incompatible with the stalled out TCP
>     Prague/dualpi effort in the IETF.
>
>     We have several other drafts in progress which I increasingly doubt
>     we'll finish today, but I think only this one is required to get an
>     audience in the tsvwg at the coming IETF meeting.
>
>     If ya have any comments and spare time today, I'd like to get the
>     first draft in tonight, and the filing deadline for final drafts is
>     sometime tomorrow. It may help for context to review some of the other
>     work in the github repo.
>
>     THX!
>
>     --
>
>     Dave Täht
>     CTO, TekLibre, LLC
>     http://www.teklibre.com
>     Tel: 1-831-205-9740
>
>


-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: [Codel] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Dave Taht]

AHA!

http://www.hjp.at/doc/rfc/rfc8311.html#sec_3

While the ECN nonce works as specified, and has been deployed in
   limited environments, widespread usage in the Internet has not
   materialized.  A study of the ECN behavior of the top one million web
   servers using 2014 data [Trammell15] found that after ECN was
   negotiated, none of the 581,711 IPv4 servers tested were using both
   ECT codepoints, which would have been a possible sign of ECN nonce
   usage.  Of the 17,028 IPv6 servers tested, four set both ECT(0) and
   ECT(1) on data packets.  This might have been evidence of use of the
   ECN nonce by these four servers, but it might equally have been due
   to erroneous re-marking of the ECN field by a middlebox or router.

I'm not sure if I should cite 8311 or not here, rather than the
original research.

I can't seem to coax our tool to output RFC3168 as normative.



On Sun, Mar 10, 2019 at 2:11 PM Dave Taht <dave.taht@gmail.com> wrote:
>
> On Sun, Mar 10, 2019 at 12:08 PM Holland, Jake <jholland@akamai.com> wrote:
> >
> > Hi Dave,
> >
> > You and John have my enthusiastic +1.
> >
> > It's a frank relief to read this draft after trying to figure out L4S,
> > and I think the basic core concept upon which to build the actual
> > response systems is very well separated and very well framed here.
> >
> > Please submit this and present, I humbly beg you.  It seems to me a
> > strictly better use of ECT(1), even though there's still probably a
> > few hundred pages' worth of catching up to do on draft-writing to
> > nail down details.
> >
> > I have a few minor comments for your consideration, but please don't
> > let them stop you from posting before deadline, if any are hard to
> > integrate.  It would be better to ignore them all and post as-is than
> > to get hung up on these:
> >
> > 1.
> > "Some" in "Some Congestion Experienced" is maybe misleading, and
> > arguably has the same meaning as "Congestion Experienced".
> >
> > I was thinking maybe "Pre-Congestion Experienced" or "Queue
> > Utilization Observed", or if you want to preserve "SCE" and the
> > link to CE (which I do agree is nice), maybe "Slight" or "Sub"
> > instead of "Some", just to try to make it more obvious it's
> > flagging a lesser situation than "there is some congestion".
> >
> > 2.
> > It's easy to accidently read section 5 as underspecified concrete
> > proposals instead of rough sketches for future direction that might
> > be worth investigating.  I'll offer an attempt at some language,
> > feel free to edit (or ignore if you think the intro is enough to
> > make the scope sufficiently clear already):
> >
> >
> > The scope of this document is limited to the definition of the
> > SCE codepoint.  However, for illustration purposes, a few possible
> > future usage scenarios are outlined here.  These examples are non
> > normative.
> >
> > 3.
> > Similarly, I would lower-case the "MAY" and "SHOULD" in section
> > 5.2 for receiver-side handling in TCP--it's not clear this will
> > ever be a good idea to do without more explicit signaling thru
> > new opts or whatever, and granting permission here seems like
> > asking for trouble that's just not necessary.
> >
> >
> > And a few that I'd defer if I were you, but I'd like to see
> > sometime in at least a post-Prague version or list discussion:
> >
> > 4.
> > an informative reference or 2 would be a welcome addition in Section 3:
> >
> >    Research has shown that the ECT(1) codepoint goes essentially unused,
> >    with the "Nonce Sum" extension to ECN having not been implemented in
>
> I have been trying to find that presentation or paper or talk or rfc
> for days now. I *know* it went by sometime in the past 2 years, I had
> my aha! moment - and I keep drawing a blank. Might have been something
> brian trammel did, I simply cannot remember.
>
> I definately want to cite that, and I sure hope I'm not delusional.
>
> > 5.
> > Should this must be MUST in Section 4?  If not, why not?
> >
> >    New SCE-aware receivers and transport protocols must continue to
> >
> >
> > Thanks guys, nice work and good luck!
> >
> > Cheers,
> > Jake
> >
> >
> > On 2019-03-10, 11:07, "Dave Taht" <dave.taht@gmail.com> wrote:
> >
> >     I would love to have some fresh eyeballs on a new IETF draft for the
> >     TSVWG we intend to submit tonight.
> >
> >     I've attached the html for easy to read purposes, but I would prefer
> >     that folk referred back to the github repository for the most current
> >     version, which is here:
> >
> >     https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-SCE.txt
> >
> >     and in open source tradition, discuss here, or file bugs, and submit
> >     pull requests to the gitub.
> >
> >     The first draft (of 3 or more pending), is creating the SCE codepoint
> >     and defining the state machine, is pretty short, and we think the
> >     basic concept solves a zillion problems with ECN in one stroke. It's
> >     easy to implement (one line of code in codel), backward compatible
> >     with all the RFCs, and somewhat incompatible with the stalled out TCP
> >     Prague/dualpi effort in the IETF.
> >
> >     We have several other drafts in progress which I increasingly doubt
> >     we'll finish today, but I think only this one is required to get an
> >     audience in the tsvwg at the coming IETF meeting.
> >
> >     If ya have any comments and spare time today, I'd like to get the
> >     first draft in tonight, and the filing deadline for final drafts is
> >     sometime tomorrow. It may help for context to review some of the other
> >     work in the github repo.
> >
> >     THX!
> >
> >     --
> >
> >     Dave Täht
> >     CTO, TekLibre, LLC
> >     http://www.teklibre.com
> >     Tel: 1-831-205-9740
> >
> >
>
>
> --
>
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-205-9740



--

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: [Codel] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Dave Taht]

the SCE draft is now  submitted to IETF! I'm too tired to figure out
what timezone conversion the UTC deadline is in PDT right now, and I
guess the chairs need to move it into the tsvwg working group instead
of individual submissions... and I have no idea what else I missed in
the ietf processes. Ironically the ietf tools do not take upper case,
so I had to rename the draft in the github repo. this is where it lies
now:

https://datatracker.ietf.org/doc/draft-morton-taht-sce/


On Sun, Mar 10, 2019 at 2:28 PM Dave Taht <dave.taht@gmail.com> wrote:
>
> AHA!
>
> http://www.hjp.at/doc/rfc/rfc8311.html#sec_3
>
> While the ECN nonce works as specified, and has been deployed in
>    limited environments, widespread usage in the Internet has not
>    materialized.  A study of the ECN behavior of the top one million web
>    servers using 2014 data [Trammell15] found that after ECN was
>    negotiated, none of the 581,711 IPv4 servers tested were using both
>    ECT codepoints, which would have been a possible sign of ECN nonce
>    usage.  Of the 17,028 IPv6 servers tested, four set both ECT(0) and
>    ECT(1) on data packets.  This might have been evidence of use of the
>    ECN nonce by these four servers, but it might equally have been due
>    to erroneous re-marking of the ECN field by a middlebox or router.
>
> I'm not sure if I should cite 8311 or not here, rather than the
> original research.
>
> I can't seem to coax our tool to output RFC3168 as normative.
>
>
>
> On Sun, Mar 10, 2019 at 2:11 PM Dave Taht <dave.taht@gmail.com> wrote:
> >
> > On Sun, Mar 10, 2019 at 12:08 PM Holland, Jake <jholland@akamai.com> wrote:
> > >
> > > Hi Dave,
> > >
> > > You and John have my enthusiastic +1.
> > >
> > > It's a frank relief to read this draft after trying to figure out L4S,
> > > and I think the basic core concept upon which to build the actual
> > > response systems is very well separated and very well framed here.
> > >
> > > Please submit this and present, I humbly beg you.  It seems to me a
> > > strictly better use of ECT(1), even though there's still probably a
> > > few hundred pages' worth of catching up to do on draft-writing to
> > > nail down details.
> > >
> > > I have a few minor comments for your consideration, but please don't
> > > let them stop you from posting before deadline, if any are hard to
> > > integrate.  It would be better to ignore them all and post as-is than
> > > to get hung up on these:
> > >
> > > 1.
> > > "Some" in "Some Congestion Experienced" is maybe misleading, and
> > > arguably has the same meaning as "Congestion Experienced".
> > >
> > > I was thinking maybe "Pre-Congestion Experienced" or "Queue
> > > Utilization Observed", or if you want to preserve "SCE" and the
> > > link to CE (which I do agree is nice), maybe "Slight" or "Sub"
> > > instead of "Some", just to try to make it more obvious it's
> > > flagging a lesser situation than "there is some congestion".
> > >
> > > 2.
> > > It's easy to accidently read section 5 as underspecified concrete
> > > proposals instead of rough sketches for future direction that might
> > > be worth investigating.  I'll offer an attempt at some language,
> > > feel free to edit (or ignore if you think the intro is enough to
> > > make the scope sufficiently clear already):
> > >
> > >
> > > The scope of this document is limited to the definition of the
> > > SCE codepoint.  However, for illustration purposes, a few possible
> > > future usage scenarios are outlined here.  These examples are non
> > > normative.
> > >
> > > 3.
> > > Similarly, I would lower-case the "MAY" and "SHOULD" in section
> > > 5.2 for receiver-side handling in TCP--it's not clear this will
> > > ever be a good idea to do without more explicit signaling thru
> > > new opts or whatever, and granting permission here seems like
> > > asking for trouble that's just not necessary.
> > >
> > >
> > > And a few that I'd defer if I were you, but I'd like to see
> > > sometime in at least a post-Prague version or list discussion:
> > >
> > > 4.
> > > an informative reference or 2 would be a welcome addition in Section 3:
> > >
> > >    Research has shown that the ECT(1) codepoint goes essentially unused,
> > >    with the "Nonce Sum" extension to ECN having not been implemented in
> >
> > I have been trying to find that presentation or paper or talk or rfc
> > for days now. I *know* it went by sometime in the past 2 years, I had
> > my aha! moment - and I keep drawing a blank. Might have been something
> > brian trammel did, I simply cannot remember.
> >
> > I definately want to cite that, and I sure hope I'm not delusional.
> >
> > > 5.
> > > Should this must be MUST in Section 4?  If not, why not?
> > >
> > >    New SCE-aware receivers and transport protocols must continue to
> > >
> > >
> > > Thanks guys, nice work and good luck!
> > >
> > > Cheers,
> > > Jake
> > >
> > >
> > > On 2019-03-10, 11:07, "Dave Taht" <dave.taht@gmail.com> wrote:
> > >
> > >     I would love to have some fresh eyeballs on a new IETF draft for the
> > >     TSVWG we intend to submit tonight.
> > >
> > >     I've attached the html for easy to read purposes, but I would prefer
> > >     that folk referred back to the github repository for the most current
> > >     version, which is here:
> > >
> > >     https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-SCE.txt
> > >
> > >     and in open source tradition, discuss here, or file bugs, and submit
> > >     pull requests to the gitub.
> > >
> > >     The first draft (of 3 or more pending), is creating the SCE codepoint
> > >     and defining the state machine, is pretty short, and we think the
> > >     basic concept solves a zillion problems with ECN in one stroke. It's
> > >     easy to implement (one line of code in codel), backward compatible
> > >     with all the RFCs, and somewhat incompatible with the stalled out TCP
> > >     Prague/dualpi effort in the IETF.
> > >
> > >     We have several other drafts in progress which I increasingly doubt
> > >     we'll finish today, but I think only this one is required to get an
> > >     audience in the tsvwg at the coming IETF meeting.
> > >
> > >     If ya have any comments and spare time today, I'd like to get the
> > >     first draft in tonight, and the filing deadline for final drafts is
> > >     sometime tomorrow. It may help for context to review some of the other
> > >     work in the github repo.
> > >
> > >     THX!
> > >
> > >     --
> > >
> > >     Dave Täht
> > >     CTO, TekLibre, LLC
> > >     http://www.teklibre.com
> > >     Tel: 1-831-205-9740
> > >
> > >
> >
> >
> > --
> >
> > Dave Täht
> > CTO, TekLibre, LLC
> > http://www.teklibre.com
> > Tel: 1-831-205-9740
>
>
>
> --
>
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-205-9740



-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: [Codel] [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Toke Høiland-Jørgensen]

Dave Taht <dave.taht@gmail.com> writes:

> the SCE draft is now  submitted to IETF! I'm too tired to figure out
> what timezone conversion the UTC deadline is in PDT right now, and I
> guess the chairs need to move it into the tsvwg working group instead
> of individual submissions... and I have no idea what else I missed in
> the ietf processes. Ironically the ietf tools do not take upper case,
> so I had to rename the draft in the github repo. this is where it lies
> now:
>
> https://datatracker.ietf.org/doc/draft-morton-taht-sce/

Nice. Too bad I can't make IETF, would have loved to see how this plays
out at TSVWG :)

-Toke

Re: [Codel] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Michael Richardson]

[-- Attachment #1: Type: text/plain, Size: 698 bytes --]


Holland, Jake <jholland@akamai.com> wrote:
    > 1.
    > "Some" in "Some Congestion Experienced" is maybe misleading, and
    > arguably has the same meaning as "Congestion Experienced".

    > I was thinking maybe "Pre-Congestion Experienced" or "Queue
    > Utilization Observed", or if you want to preserve "SCE" and the

I'll combine them:
     "Some Congestion Observed"

(I image a packet rubber-necking to observe some collision on the exressway)

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     mcr@sandelman.ca  http://www.sandelman.ca/        |   ruby on rails    [


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [Codel] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Dave Taht]

On Sun, Mar 10, 2019 at 8:23 PM Michael Richardson <mcr@sandelman.ca> wrote:
>
>
> Holland, Jake <jholland@akamai.com> wrote:
>     > 1.
>     > "Some" in "Some Congestion Experienced" is maybe misleading, and
>     > arguably has the same meaning as "Congestion Experienced".
>
>     > I was thinking maybe "Pre-Congestion Experienced" or "Queue
>     > Utilization Observed", or if you want to preserve "SCE" and the
>
> I'll combine them:
>      "Some Congestion Observed"

Hah.

Ironically as hell, I spent the first 1/4th of my career working for SCO.

>
> (I image a packet rubber-necking to observe some collision on the exressway)
>
> --
> ]               Never tell me the odds!                 | ipv6 mesh networks [
> ]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
> ]     mcr@sandelman.ca  http://www.sandelman.ca/        |   ruby on rails    [
>


-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: [Codel] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Richard Scheffenegger]

I can remember reading quite a few papers where a similar scheme for ect(1) was adopted - often with additional changes on both ends to make use of this signal. Including schemes that encoded complex information in the stream of ect0/ect1...

Where can one find simulations of the interaction between legacy and l4s flows when using this?

I think the l4s use of dctcp, to allow coupled queue selection based on the transports expectations, is a more useful case for ect(1)

Richard

Gesendet mit der GMX iPhone App

Am 11.03.19 um 08:08 schrieb Mikael Abrahamsson

> On Sun, 10 Mar 2019, Jonathan Morton wrote:
> 
> > An interesting idea, but SCE marks will appear even when there's a lot 
> > of congestion (at high rates, ie. probably every packet that doesn't 
> > carry CE), as well as showing up at low frequency when the level of 
> > congestion only warrants reducing the growth rate.  I think the word 
> > "Some" is sufficiently descriptive, while "Slight" might cause people to 
> > ignore it completely.
> 
> One way to handle this would be "buffering experienced" or something like 
> that. Ie if this packet is being enqueued into a buffer with non-trivial 
> number of packets in it, mark it.
> 
> The L4S proposal also has the property that their use of this last code 
> point combination in the entire packet header (and this is a big thing, 
> this is the last unicorn) also meant the packet was allowed to be 
> re-ordered. I thought this was a big and nice thing, for other areas. This 
> new proposal removes that property.
> 
> From what I can see, L4S actually is quite novel and has the chance to 
> seriously change the way queueing is done. This proposal seems more like 
> "a little more of what we had before" which I do not think warrants 
> claiming this last unicorn codepoint. I'd like its use to be truly novel 
> and be more than a tweak.
> 
> -- 
> Mikael Abrahamsson    email: swmike@swm.pp.se
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Codel] [Cake] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Sebastian Moeller]

Hi Mikael,


> On Mar 11, 2019, at 08:08, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
> 
> On Sun, 10 Mar 2019, Jonathan Morton wrote:
> 
>> An interesting idea, but SCE marks will appear even when there's a lot of congestion (at high rates, ie. probably every packet that doesn't carry CE), as well as showing up at low frequency when the level of congestion only warrants reducing the growth rate.  I think the word "Some" is sufficiently descriptive, while "Slight" might cause people to ignore it completely.
> 
> One way to handle this would be "buffering experienced" or something like that. Ie if this packet is being enqueued into a buffer with non-trivial number of packets in it, mark it.
> 
> The L4S proposal also has the property that their use of this last code point combination in the entire packet header (and this is a big thing, this is the last unicorn) also meant the packet was allowed to be re-ordered.

	How is packet reordering for anybody but the folks responsible for operating the "conduits" in any way attractive? 




> I thought this was a big and nice thing, for other areas. This new proposal removes that property.
> 
> From what I can see, L4S actually is quite novel and has the chance to seriously change the way queueing is done. This proposal seems more like "a little more of what we had before" which I do not think warrants claiming this last unicorn codepoint.

	I would argue that evolutionary changes tend to look mostly like "more of the old, just a little different" sothis might be considered a plus for this proposal ;)

Best Regards
	Sebastian


> I'd like its use to be truly novel and be more than a tweak.
> 
> -- 
> Mikael Abrahamsson    email: swmike@swm.pp.se
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Codel] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Sebastian Moeller]

Dear Richard,

I am an interested layman here, so do not take my questions too seriously...

> On Mar 11, 2019, at 08:35, Richard Scheffenegger <rscheff@gmx.at> wrote:
> 
> I can remember reading quite a few papers where a similar scheme for ect(1) was adopted - often with additional changes on both ends to make use of this signal. Including schemes that encoded complex information in the stream of ect0/ect1...
> 
> Where can one find simulations of the interaction between legacy and l4s flows when using this?

	Given that l4s is not deployed, I would argue that until it is anything new will only need to play nice with what you call legacy flows (I would call them normal flows).

> 
> I think the l4s use of dctcp, to allow coupled queue selection based on the transports expectations, is a more useful case for ect(1)

	But L4s, at least from a glance at draft-briscoe-tsvwg-l4s-arch-01 seems ti merit the unicorn-qualifier that Mikael used: since it seems to imply that the l4s path will not encounter drops just marks to achieve its goal of "keep[ing] congestion loss to zero". This probably is true for a rather strict definition of "congestion loss", but it flies into the face of the naive insight that once under sufficient load a router is going to drop packets to keep its queues under control, and if each flow uses up less of the queue (a worthy goal) it will just take more concurrent flow to push the router into the dropping regime, but that is a quantitative difference not a qualitative one. I guess this shows the lack of understanding from my side more than short-comings of l4s...

Best Regards
	Sebastian

> 
> Richard
> 
> Gesendet mit der GMX iPhone App
> 
> Am 11.03.19 um 08:08 schrieb Mikael Abrahamsson
> 
>> On Sun, 10 Mar 2019, Jonathan Morton wrote:
>> 
>>> An interesting idea, but SCE marks will appear even when there's a lot 
>>> of congestion (at high rates, ie. probably every packet that doesn't 
>>> carry CE), as well as showing up at low frequency when the level of 
>>> congestion only warrants reducing the growth rate.  I think the word 
>>> "Some" is sufficiently descriptive, while "Slight" might cause people to 
>>> ignore it completely.
>> 
>> One way to handle this would be "buffering experienced" or something like 
>> that. Ie if this packet is being enqueued into a buffer with non-trivial 
>> number of packets in it, mark it.
>> 
>> The L4S proposal also has the property that their use of this last code 
>> point combination in the entire packet header (and this is a big thing, 
>> this is the last unicorn) also meant the packet was allowed to be 
>> re-ordered. I thought this was a big and nice thing, for other areas. This 
>> new proposal removes that property.
>> 
>> From what I can see, L4S actually is quite novel and has the chance to 
>> seriously change the way queueing is done. This proposal seems more like 
>> "a little more of what we had before" which I do not think warrants 
>> claiming this last unicorn codepoint. I'd like its use to be truly novel 
>> and be more than a tweak.
>> 
>> -- 
>> Mikael Abrahamsson    email: swmike@swm.pp.se
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Codel] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Jonathan Morton]

> On 11 Mar, 2019, at 9:08 am, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
> 
> …also meant the packet was allowed to be re-ordered. I thought this was a big and nice thing…

Seriously?  I had to dig in the specs to find any mention of that, and… it's all about better supporting bonded links.  Which can already be done by implementing RACK at the sender, and all you propose is that when L4S is in use, the extra buffering at the link layer is dropped.  This is absolutely useless for ordinary Internet users, who are unlikely to have consecutive packets sufficiently closely traversing such a link for this reordering to exceed the 3-dupack threshold in any case - so you might as well delete that reordering buffer anyway, and let the endpoints handle it.  You don't need L4S for that.

At bottom L4S is also a very simple scheme: when ECT(1) is used, a higher CE marking rate is expected of middleboxes, and a less aggressive backoff in response to CE is expected of endpoints.  This is, incidentally, incompatible with Codel-like AQMs on the one hand and existing TCPs on the other hand, and so requires negotiation between the endpoints  (eg. using AccECN) to discover whether setting ECT(1) at the sender is legal.  SCE does not require such negotiation (ie. a transport could implement it entirely at the receiver, manipulating the send rate via the already-standardised receive window), so should be easier to specify and deploy successfully.

> On 11 Mar, 2019, at 9:35 am, Richard Scheffenegger <rscheff@gmx.at> wrote:
> 
> I can remember reading quite a few papers where a similar scheme for ect(1) was adopted - often with additional changes on both ends to make use of this signal. Including schemes that encoded complex information in the stream of ect0/ect1...
> 
> Where can one find simulations of the interaction between legacy and l4s flows when using this?

If by "between legacy and l4s" you mean "between existing SCE-ignorant and new SCE-aware" - because SCE is not L4S…

We haven't yet drilled down to that level of detail, but we plan to do so in the very near future.  SCE itself is just a means of communicating network state from AQMs to receivers, hence the brevity and simplicity of the I-D.

The encoding is not complex, just a ratio of ECT to SCE markings which is ignored by existing receivers and not produced by existing middleboxes, *and* continued use of CE as presently standardised.  This naturally ensures backwards compatibility and incremental deployability.  The stable ratio of ECT to SCE is defined as 1:1, allowing more flexibility in signalling how much unused link capacity remains than DCTCP's "2 CEs per RTT" stability condition permits, and potentially simplifying implementations.

The question of whether SCE-aware transports will compete fairly with SCE-ignorant transports, given a single-queue AQM implementing SCE, is an interesting and important one.  Intuitively, one can see that the SCE-ignorant transport will drive the AQM into the CE-marking regime, which may cause both flows to drop back, but there will be periods when the SCE-aware flow has inhibited growth while the SCE-ignorant flow has not, and so the SCE-aware flow might have overall lower throughput.

On the other hand, a Codel-like AQM (as opposed to a RED-like AQM) has a higher probability of signalling CE to a flow that sends more packets, which may naturally compensate for this.  And on the gripping hand, an SCE-aware flow operating alone should have (slightly) higher goodput than an SCE-ignorant flow operating alone, which illustrates the case for flow-isolating AQMs.

Two SCE-aware transports might interact strangely on a single-queue AQM, depending on how their response to SCE is implemented.  Eliminating the present sawtooth behaviour entirely is a desirable goal, but AIMD is what allows flows competing in a single queue to converge on fair throughput.  This will require careful attention when specifying transport behaviour.

One exciting possibility is finally solving the slow-start problem, by beginning SCE signalling (at a low rate which still permits window growth) when the link is half-full rather than when queuing begins.  That would cause the exponential growth phase of typical TCPs to exit one RTT later, when the link is just fully utilised, instead of massively overshooting and being cut back to nothing as presently occurs.  There is a range of flow lengths which could see a concrete performance benefit from that.

That requires that the AQM is able to determine when the link reaches 50% utilisation, which is not always true (Cake and hardware AQMs could do it, being aware of the link capacity) - but SCE does not mandate this behaviour, it's just a good idea.  I'm not aware of any possibility for L4S to achieve the same result; however much it redefines the meaning of CE, it won't be asserted until the link is full.

SCE-aware AQMs and TCPs are not difficult to describe and should not be very difficult to implement.  Though not described explicitly in the SCE I-D, I'm working on descriptions of how to do so, which should become additional I-Ds.  We should then be able to write working code and run simulations in ns-3, and maybe also in Linux.

 - Jonathan Morton

Re: [Codel] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Mikael Abrahamsson]

[-- Attachment #1: Type: text/plain, Size: 1641 bytes --]

On Mon, 11 Mar 2019, Jonathan Morton wrote:

> Seriously?  I had to dig in the specs to find any mention of that, and… 
> it's all about better supporting bonded links.  Which can already be

It doesn't stop there. Right now DOCSIS, 3GPP networks, Wifi etc all do 
ordering guarantees, so they will hold up any delivery of packets until it 
can assure that none of them are delivered out of order.

Allowing these transports to re-order the packets mean they can do a 
better job than they do today. You do not want to ask them to drop their 
packets either because the drop rate is potentially way higher than most 
transports would feel comfortable with.

> done by implementing RACK at the sender, and all you propose is that 
> when L4S is in use, the extra buffering at the link layer is dropped.

I did?

> This is absolutely useless for ordinary Internet users, who are unlikely 
> to have consecutive packets sufficiently closely traversing such a link 
> for this reordering to exceed the 3-dupack threshold in any case - so 
> you might as well delete that reordering buffer anyway, and let the 
> endpoints handle it.  You don't need L4S for that.

That's not my experience with wifi and how it behaves at the edge.

> endpoints (eg. using AccECN) to discover whether setting ECT(1) at the 
> sender is legal.  SCE does not require such negotiation (ie. a transport 
> could implement it entirely at the receiver, manipulating the send rate 
> via the already-standardised receive window), so should be easier to 
> specify and deploy successfully.

Well, I am not convinced blowing the last codepoint on SCE has enough 
merit.

Re: [Codel] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Jonathan Morton]

> On 11 Mar, 2019, at 11:07 am, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
> 
> Well, I am not convinced blowing the last codepoint on SCE has enough merit.

I will make a stronger statement: I am convinced that blowing the last codepoint on L4S does *not* have enough merit.

Meanwhile, work continues.

 - Jonathan Morton

Re: [Codel] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Mikael Abrahamsson]

On Mon, 11 Mar 2019, Jonathan Morton wrote:

>> On 11 Mar, 2019, at 11:07 am, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
>>
>> Well, I am not convinced blowing the last codepoint on SCE has enough merit.
>
> I will make a stronger statement: I am convinced that blowing the last codepoint on L4S does *not* have enough merit.

... and I believe that blowing it on SCE doesn't have merit either.

That's the entire thing why I am opposing the use of ECT(1) unless we're 
*really* *really* *really* sure there is something we want to blow it on.

Using it for SCE for me is marginal benefit compared to what ECT(1) could 
be used for either. I think L4S is proposing enough novelty that it could 
be used for that, but I'm open for other suggestions. SCE isn't enough.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Codel] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Dave Taht]

Everybody, calm down. I put this out merely to get comment before we
submitted the first of several drafts. That draft is now submitted and
we've asked for a talk slot in the tsvwg for it. I cc'd the world to
get quick initial feedback, and I want to shut this overbroad
conversation down and move it to just the ecn-sane mailing list.

The l4s mailing list is dead, and the debates on the AQM mailing list and here,
unhelpful - for decades. So, back in august I started a new working
group here, under house rules that I thought would be more productive,
and asked that people that wanted to debate ecn more sanely, join. few
did.

And jon and I have been working for months (and largely not on the
list) to try and create a compromise proposal of which y'all just saw
the first output. There's more in the bufferbloat-rfcs repo.

The rules for joining the ecn-sane list are simple - take the time to
step back and write a write a short position paper, and join (or
create) a team. You needn't do that immediately. If you disagree with
the rules of operation of the ecn-sane working group, submit a pull
request or file a bug on the web site. where we can discuss it.

Ironically our ssl cert just expired and I don't remember how to fix it.

Please join the ecn-sane mailing list for discussing this stuff and
stop cc-ing the whole bufferbloat.net  world on it, please.

Re: [Codel] [Ecn-sane] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[Jonathan Morton]

> On 11 Mar, 2019, at 5:29 pm, Holland, Jake <jholland@akamai.com> wrote:
> 
> The key difference to me is that L4S doesn't work without a dual queue.
> It embeds a major design decision ("how many queues should there be at
> a middle box?") into the codepoint, and comes with very narrow requirements
> on the sender's congestion control.

It's certainly unclear to me what happens when an L4S flow passes through a Classic ECN middlebox, especially one with only a single queue.  I get the impression that a DCTCP-like congestion response would tend to starve out conventional flows competing with it.  Unless you have data showing otherwise?

That has serious implications for incremental deployability, because you can't safely deploy L4S endpoints until single-queue Classic ECN middleboxes have all been replaced with L4S or flow-isolating types.  And without L4S endpoints in use, where's the impetus to do that?  Chicken and egg.

Conversely, an SCE-aware flow passing through a Classic ECN middlebox behaves just like a Classic ECN flow.  The only question arises when a single-queue AQM is made SCE-aware, and in that case the SCE-aware flows are the ones that might get outcompeted (ie. they are friendlier, not more aggressive).  If necessary, it would be easy to specify that single-queue AQMs "should not" produce SCE marks, only the flow-isolating types - which in any case are easier to deploy at edge devices where statistical multiplexing works less well.

Incremental deployability is very important when tackling a project as big as this.  SCE takes it as a central tenet. L4S appears, in practice, to have overlooked it.  That's my objection to L4S.

 - Jonathan Morton

Re: [Codel] [Cake] [Bloat] The "Some Congestion Experienced" ECN codepoint - a new internet draft -

[David Lang]

On Mon, 11 Mar 2019, Sebastian Moeller wrote:

> How is packet reordering for anybody but the folks responsible for operating 
> the "conduits" in any way attractive?

It's more that not worrying about maintaining the order, and just moving the 
packets as fast as possible reduces the overhead.

The majority of the time, packets will be in order, but race conditions and 
corner cases are allowed to forward packets out of order rather than having the 
delay some packets to maintain the order.

David Lang