[Ecn-sane] BBRv3 ecn_low per-route feature and the CE confusion

[Ecn-sane] BBRv3 ecn_low per-route feature and the CE confusion

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 1184 bytes --]

Regarding the per-route ecn_low feature in BBRv3:
https://github.com/google/bbr/blob/v3/README.md#introducing-the-ecn_low-per-route-feature

When set, any incoming CE marks from an ecn_low flagged route will be
treated as an L4S/RFC9331 CE, while CE marks from routes without that
flag will be treated as a regular RFC3168 CE. Obviously this is
problematic, as the two CE marks mean very different things, and
outside of closed environments, there's no reliable way to know which
type of CE you're receiving. The consequences of confusing them range
from massive self-induced bloat, to driving competing traffic in the
same queue down to minimum cwnd. The solution here is to just punt, so
at least tests and demos can be made to work.

This isn't a problem with BBR. The problem is, we've started an
experiment [RFC9331] that redefines the meaning of CE in a way that's
incompatible with existing RFC3168 middleboxes. This feature in BBR is
just a reflection of that. CCA developers are now tasked with somehow
deciding which type of CE they're seeing.

Speaking of ecn "sane", does anyone else see this as not? :) and as a
problem that needs solving?

Pete


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [Ecn-sane] BBRv3 ecn_low per-route feature and the CE confusion

[Jonathan Morton]

> On 28 Nov, 2023, at 2:57 pm, Pete Heist via Ecn-sane <ecn-sane@lists.bufferbloat.net> wrote:
> 
> This isn't a problem with BBR. The problem is, we've started an
> experiment [RFC9331] that redefines the meaning of CE in a way that's
> incompatible with existing RFC3168 middleboxes. This feature in BBR is
> just a reflection of that. CCA developers are now tasked with somehow
> deciding which type of CE they're seeing.
> 
> Speaking of ecn "sane", does anyone else see this as not? :) and as a
> problem that needs solving?

Count me in the "not sane" camp, at least.  This is exactly the kind of ambiguity that SCE is designed to avoid.

 - Jonathan Morton

Re: [Ecn-sane] BBRv3 ecn_low per-route feature and the CE confusion

[Sebastian Moeller]

Hi Pete,


> On Nov 28, 2023, at 13:57, Pete Heist via Ecn-sane <ecn-sane@lists.bufferbloat.net> wrote:
> 
> Regarding the per-route ecn_low feature in BBRv3:
> https://github.com/google/bbr/blob/v3/README.md#introducing-the-ecn_low-per-route-feature
> 
> When set, any incoming CE marks from an ecn_low flagged route will be
> treated as an L4S/RFC9331 CE, while CE marks from routes without that
> flag will be treated as a regular RFC3168 CE.

	I am probably blind, but looking at https://github.com/google/bbr/blob/v3/net/ipv4/tcp_bbr.c
I do not see this fall back to a rfc3168 compliant response, as far as I read this this is DCTCP or nothing... 
Or did you just mean that BBRv3 will treat CE just like BBRv1 and silently ignore it?



> Obviously this is
> problematic, as the two CE marks mean very different things, and
> outside of closed environments, there's no reliable way to know which
> type of CE you're receiving. The consequences of confusing them range
> from massive self-induced bloat, to driving competing traffic in the
> same queue down to minimum cwnd. The solution here is to just punt, so
> at least tests and demos can be made to work.

	The per-route feature looks like a desirable safety... as does the default 5ms rtt "low-pass", where IIUC ECN will only be evaluated if the RTT is below 5ms...


> This isn't a problem with BBR. The problem is, we've started an
> experiment [RFC9331] that redefines the meaning of CE in a way that's
> incompatible with existing RFC3168 middleboxes. This feature in BBR is
> just a reflection of that. CCA developers are now tasked with somehow
> deciding which type of CE they're seeing.
> 
> Speaking of ecn "sane", does anyone else see this as not? :) and as a
> problem that needs solving?

	Yes, however I assume this will solve it self, as is this usage of CE is over-promising and under-delivering, so I do not see this "winning the internet" in spite of the amount of thrust is put behind it...

Regards
	Sebastian


> 
> Pete
> 
> _______________________________________________
> Ecn-sane mailing list
> Ecn-sane@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/ecn-sane

Re: [Ecn-sane] BBRv3 ecn_low per-route feature and the CE confusion

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 2877 bytes --]

On Tue, 2023-11-28 at 16:21 +0100, Sebastian Moeller wrote:
> Hi Pete,
> 
> 
> > On Nov 28, 2023, at 13:57, Pete Heist via Ecn-sane
> > <ecn-sane@lists.bufferbloat.net> wrote:
> > 
> > Regarding the per-route ecn_low feature in BBRv3:
> > https://github.com/google/bbr/blob/v3/README.md#introducing-the-ecn_low-per-route-feature
> > 
> > When set, any incoming CE marks from an ecn_low flagged route will
> > be
> > treated as an L4S/RFC9331 CE, while CE marks from routes without
> > that
> > flag will be treated as a regular RFC3168 CE.
> 
> 	I am probably blind, but looking at
> https://github.com/google/bbr/blob/v3/net/ipv4/tcp_bbr.c
> I do not see this fall back to a rfc3168 compliant response, as far
> as I read this this is DCTCP or nothing... 
> Or did you just mean that BBRv3 will treat CE just like BBRv1 and
> silently ignore it?

TBH, I hadn't read the code but I think you're right. The doc makes it
sound like enabling ECN support could be done with or without the
ecn_low feature, implying that it would react properly to either type
of CE. It appears to still ignore CE though unless ecn_low is flagged,
and other conditions are met. So yes, L4S/DCTCP response or bust.

Same result though, this confusion is harmful to the ECN field.

Pete

> > Obviously this is
> > problematic, as the two CE marks mean very different things, and
> > outside of closed environments, there's no reliable way to know
> > which
> > type of CE you're receiving. The consequences of confusing them
> > range
> > from massive self-induced bloat, to driving competing traffic in
> > the
> > same queue down to minimum cwnd. The solution here is to just punt,
> > so
> > at least tests and demos can be made to work.
> 
> 	The per-route feature looks like a desirable safety... as
> does the default 5ms rtt "low-pass", where IIUC ECN will only be
> evaluated if the RTT is below 5ms...
> 
> 
> > This isn't a problem with BBR. The problem is, we've started an
> > experiment [RFC9331] that redefines the meaning of CE in a way
> > that's
> > incompatible with existing RFC3168 middleboxes. This feature in BBR
> > is
> > just a reflection of that. CCA developers are now tasked with
> > somehow
> > deciding which type of CE they're seeing.
> > 
> > Speaking of ecn "sane", does anyone else see this as not? :) and as
> > a
> > problem that needs solving?
> 
> 	Yes, however I assume this will solve it self, as is this
> usage of CE is over-promising and under-delivering, so I do not see
> this "winning the internet" in spite of the amount of thrust is put
> behind it...
> 
> Regards
> 	Sebastian
> 
> 
> > 
> > Pete
> > 
> > _______________________________________________
> > Ecn-sane mailing list
> > Ecn-sane@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/ecn-sane
> 


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 659 bytes --]