[Ecn-sane] ect(1) queue selector question

[Ecn-sane] ect(1) queue selector question

[Dave Taht]

I note that I have done my best to stay out of this for a while, so
long that I (thankfully) mis-remember various aspects of the debate.
Today I have a question about l4s vs SCE as it's come up again.

l4s uses both a dscp codepoint AND ect(1) to indicate dctcp style
congestion control
is in use, and also can dump other protocols into that queue lacking
any ecn markings.

SCE proposes to use ect(1) as an indicator of some congestion and does
not explictly
require a dscp codepoint in a FQ'd implementation.

Do I have that right? Now, my question was, simply, in MPLS or X-G are
they out of bits, and
that's why they want to use up this one in L4S?

-- 
"For a successful technology, reality must take precedence over public
relations, for Mother Nature cannot be fooled" - Richard Feynman

dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

Re: [Ecn-sane] ect(1) queue selector question

[Sebastian Moeller]

Hi Dave,


> On Feb 20, 2021, at 20:27, Dave Taht <dave.taht@gmail.com> wrote:
> 
> I note that I have done my best to stay out of this for a while, so
> long that I (thankfully) mis-remember various aspects of the debate.
> Today I have a question about l4s vs SCE as it's come up again.
> 
> l4s uses both a dscp codepoint AND ect(1) to indicate dctcp style
> congestion control
> is in use, and also can dump other protocols into that queue lacking
> any ecn markings.

	Mmmh, according to the L4S internet drafts, L4S does not want to use a DSCP at all. Interestingly enough, Greg White is proposing a related internet draft about the NQB PHB and DSCP that sounds awfully like the missing DSCP in the L4S drafts. IMHO if the whole thing would be guarded behind a DSCP I would be less annoyed by the design and process of L4S....

> 
> SCE proposes to use ect(1) as an indicator of some congestion and does
> not explictly
> require a dscp codepoint in a FQ'd implementation.

	Pretty much. I do think that a demonstration using an additional DSCP to create a similar HOV lane for SCE would have gone miles in convincing people in the WG that L4S might really not be as swell as its proponents argue, IMHO it won the day more with its attractive promise of low latency for all instead of what it delivers.

> 
> Do I have that right? Now, my question was, simply, in MPLS or X-G are
> they out of bits, and
> that's why they want to use up this one in L4S?

	I do not think that MPLS folks are a driver in all of this, no? No idea what X-G is. 

> 
> -- 
> "For a successful technology, reality must take precedence over public
> relations, for Mother Nature cannot be fooled" - Richard Feynman
> 
> dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729
> _______________________________________________
> Ecn-sane mailing list
> Ecn-sane@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/ecn-sane

Re: [Ecn-sane] ect(1) queue selector question

[Rodney W. Grimes]

Hello Dave, and Sebastian,

> Hi Dave,
> 
> 
> > On Feb 20, 2021, at 20:27, Dave Taht <dave.taht@gmail.com> wrote:
> > 
> > I note that I have done my best to stay out of this for a while, so
> > long that I (thankfully) mis-remember various aspects of the debate.
> > Today I have a question about l4s vs SCE as it's come up again.
> > 
> > l4s uses both a dscp codepoint AND ect(1) to indicate dctcp style
> > congestion control
> > is in use, and also can dump other protocols into that queue lacking
> > any ecn markings.
> 
> 	Mmmh, according to the L4S internet drafts, L4S does not want to use a DSCP at all. Interestingly enough, Greg White is proposing a related internet draft about the NQB PHB and DSCP that sounds awfully like the missing DSCP in the L4S drafts. IMHO if the whole thing would be guarded behind a DSCP I would be less annoyed by the design and process of L4S....

That is correct, the L4S proponents absolutely do not want anything to do with a DSCP and L4S.
If they would of simply agreed that this was actually a good idea they could propbably be at deployment state now.... but they insist it is pointless to take this route due to the traveral problem.

> 
> > 
> > SCE proposes to use ect(1) as an indicator of some congestion and does
> > not explictly
> > require a dscp codepoint in a FQ'd implementation.
> 
> 	Pretty much. I do think that a demonstration using an additional DSCP to create a similar HOV lane for SCE would have gone miles in convincing people in the WG that L4S might really not be as swell as its proponents argue, IMHO it won the day more with its attractive promise of low latency for all instead of what it delivers.

The original SCE design was without any mention of DSCP.  Since that pretty much stahled us in the battle with L4S there actually *IS* an SCE design that uses one or more DSCP.  This technically allows SCE to proceed forward on its own without any blessing from IETF so that we may gain some additional experience and data.  This *IS NOT* the ideal design, but I feel by taking this road we might gain ground.

> > Do I have that right? Now, my question was, simply, in MPLS or X-G are
> > they out of bits, and
> > that's why they want to use up this one in L4S?
> 
> 	I do not think that MPLS folks are a driver in all of this, no? No idea what X-G is. 

We do not know fully all of there motivations for being so very insistant on doing it this way, only this way, and with no changes of any kind.  But they do seem very stuck on there design and unflexiable in any and all suggestions of change.

> > -- 
> > "For a successful technology, reality must take precedence over public
> > relations, for Mother Nature cannot be fooled" - Richard Feynman
> > 
> > dave@taht.net <Dave T?ht> CTO, TekLibre, LLC Tel: 1-831-435-0729

Regards,
-- 
Rod Grimes                                                 rgrimes@freebsd.org

Re: [Ecn-sane] ect(1) queue selector question

[Dave Taht]

Thx for the clarifications. What I meant by X-G was 3G, 4G, 5G.

On Sun, Feb 21, 2021 at 9:14 AM Rodney W. Grimes
<4bone@gndrsh.dnsmgr.net> wrote:
>
> Hello Dave, and Sebastian,
>
> > Hi Dave,
> >
> >
> > > On Feb 20, 2021, at 20:27, Dave Taht <dave.taht@gmail.com> wrote:
> > >
> > > I note that I have done my best to stay out of this for a while, so
> > > long that I (thankfully) mis-remember various aspects of the debate.
> > > Today I have a question about l4s vs SCE as it's come up again.
> > >
> > > l4s uses both a dscp codepoint AND ect(1) to indicate dctcp style
> > > congestion control
> > > is in use, and also can dump other protocols into that queue lacking
> > > any ecn markings.
> >
> >       Mmmh, according to the L4S internet drafts, L4S does not want to use a DSCP at all. Interestingly enough, Greg White is proposing a related internet draft about the NQB PHB and DSCP that sounds awfully like the missing DSCP in the L4S drafts. IMHO if the whole thing would be guarded behind a DSCP I would be less annoyed by the design and process of L4S....
>
> That is correct, the L4S proponents absolutely do not want anything to do with a DSCP and L4S.
> If they would of simply agreed that this was actually a good idea they could propbably be at deployment state now.... but they insist it is pointless to take this route due to the traveral problem.
>
> >
> > >
> > > SCE proposes to use ect(1) as an indicator of some congestion and does
> > > not explictly
> > > require a dscp codepoint in a FQ'd implementation.
> >
> >       Pretty much. I do think that a demonstration using an additional DSCP to create a similar HOV lane for SCE would have gone miles in convincing people in the WG that L4S might really not be as swell as its proponents argue, IMHO it won the day more with its attractive promise of low latency for all instead of what it delivers.
>
> The original SCE design was without any mention of DSCP.  Since that pretty much stahled us in the battle with L4S there actually *IS* an SCE design that uses one or more DSCP.  This technically allows SCE to proceed forward on its own without any blessing from IETF so that we may gain some additional experience and data.  This *IS NOT* the ideal design, but I feel by taking this road we might gain ground.
>
> > > Do I have that right? Now, my question was, simply, in MPLS or X-G are
> > > they out of bits, and
> > > that's why they want to use up this one in L4S?
> >
> >       I do not think that MPLS folks are a driver in all of this, no? No idea what X-G is.
>
> We do not know fully all of there motivations for being so very insistant on doing it this way, only this way, and with no changes of any kind.  But they do seem very stuck on there design and unflexiable in any and all suggestions of change.
>
> > > --
> > > "For a successful technology, reality must take precedence over public
> > > relations, for Mother Nature cannot be fooled" - Richard Feynman
> > >
> > > dave@taht.net <Dave T?ht> CTO, TekLibre, LLC Tel: 1-831-435-0729
>
> Regards,
> --
> Rod Grimes                                                 rgrimes@freebsd.org



-- 
"For a successful technology, reality must take precedence over public
relations, for Mother Nature cannot be fooled" - Richard Feynman

dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

Re: [Ecn-sane] ect(1) queue selector question

[Rodney W. Grimes]

> I have basically hoped to find some sysadmin out there with battle
> experience with a real dctcp deployment (such as the one at facebook)
> that could share his or her
> insights as to this debate.

My "experience" comes from working with HPE who are very experienced
at building data centers running RDMA using DCTCP.  I do not know
who the end customer is of the products though.  This is now mostly
100Gb/s and above Aruba switch gear.

> 
> Most of the public testing with BBRv2 has instead been with a brick
> wall setting to CE_threshold, the published data on it was a 260us,
> which was about as low as raw iron linux can get. I tend to favor the
> brick wall approach over anything more complex for the AQM component
> of a SCE architecture, and to modify the transports to suit this
> assumption.

The current information I have from HPE on this is that infact they
do use a brick wall or cliff setting, ie the slope is infinity,
as basically with all the offload going on in NIC cards you simply
can not react in time to much queue depth at all.  I told them to
turn the port speed down, to which they pretty much ran me out....
I call it poor mans pacing :-)

> 
> On Tue, Mar 9, 2021 at 7:19 AM Rodney W. Grimes <4bone@gndrsh.dnsmgr.net> wrote:
> >
> > > I am of course much happier interacting here than over on tsvwg.
> > >
> > > I would like very much
> > >
> > > A large scale bit twiddling test from the larger providers of fq_codel
> > > and cake based hw and middleboxes.
> > >
> > > extensive testing on lte and wifi transports. even emulated.
> > > the sce patches polished up and submitted for review to netdev as what
> > > we call there, an RFC
> > > the ability to test both SCE and L4S on openwrt (backport thereof)
> > >
> > > I know there's been a lot of work on other qdiscs than cake, but
> > > haven't paid much attention. netdev review would be nice.
> > >
> > > A simple internet-wide test of say bittorrent, or using an adserver
> > > method like what apnic uses.
> > >
> > > Most of all I'd really like someone to take a stab at RFC3168 support
> > > for BBR. And by that, I don't mean a reduction by half, but to
> > > institute an RTT probe phase. A fixed rate reduction per RTT is simply
> > > not
> > > going to work IMHO, period, on lte and wifi. I'm sure dave miller
> > > would accept such a patch, and this would also lead towards better ecn
> > > support across the internet in general... at least from the
> > > perspective of the one provider I have an in with, dropbox.
> > >
> > > SCE support for BBRv2 would be nice also.
> > >
> > > And I know, a pony, all sparkly and stuff. I find it very difficult to
> > > summon the cojones to do a drop of work in this area, and I keep
> > > cheering you on - especially on the bug fixing and wonderful scripts
> > > and data you've provided so far, and I do wish I could find some way
> > > to help that didn't cause so much ptsd in me.
> > >
> > > I wish I merely knew more about how dctp was configured in the data
> > > center. So far as *I* know its on dedicated switches mostly. I would
> > > vastly prefer a dscp codepoint for this, also.
> >
> > If you want I can take a discussion on how DCTCP is configured in
> > data center switches in a side channel.  Its basically using a
> > drop function that has a starting queue depth, and slope that
> > as you go higher on the slope your propability of marking
> > increases.  The 40 gbit experiments we did with HPE modified
> > that part of the switch asic code to do SCE marks starting
> > at 1% mark probability for 1% Queue depth, up to 100% marking
> > at 100% Queue depth.  Sadly that was a slight mistake on
> > my part, and I thought we would get a chance to iterate
> > and retune these, I just pulled those values out as a WOG
> > and gave them to HPE to let them get on with setting it up.
> >
> > For most practical purposes an SCE mark and a DCTCP CE
> > mark convey very similiar, if not the same information,
> > as does a CE mark in L4S.
> >
> > Usually tunning of these queue depth and slope values involes
> > data collection and several iterations of experiments.
> >
> > TOR vs SPINE switches are usually configured with different values.
> >
> > I believe there is even some work by someone that does data
> > collection across the whole datacenter and tries to adjust
> > these automagically and dynamically.
> >
> > >
> > > On Mon, Mar 8, 2021 at 4:36 PM Pete Heist <pete@heistp.net> wrote:
> > > >
> > > > Sorry for reviving an old thread as I haven't been on this list in a
> > > > while:
> > > >
> > > > > > SCE proposes to use ect(1) as an indicator of some congestion and
> > > > does
> > > > > > not explictly
> > > > > > require a dscp codepoint in a FQ'd implementation.
> > > > >       Pretty much. I do think that a demonstration using an
> > > > > additional DSCP to create a similar HOV lane for SCE would have gone
> > > > > miles in convincing people in the WG that L4S might really not be as
> > > > > swell as its proponents argue, IMHO it won the day more with its
> > > > > attractive promise of low latency for all instead of what it
> > > > delivers.
> > > >
> > > > On that, I don't think any of us knows how things will end up or how
> > > > long it will take to get there...
> > > >
> > > > I do agree that the interim meeting leading up to the codepoint
> > > > decision could have gone better. Everything went great until it came to
> > > > how to deploy SCE in a small number of queues. We had dismissed the
> > > > idea of using DSCP, because we thought it would be panned for its poor
> > > > traversal over the Internet. That may still have been the case, but it
> > > > also may have worked if sold right. We thought that AF alone might be
> > > > enough to get past that part, but it wasn't.
> > > >
> > > > We already implemented a two-queue design that uses DSCP, but either
> > > > there wasn't much interest, or we didn't sell it enough. Plus, for
> > > > those who demand a two queue solution that requires no flow awareness
> > > > at all, DSCP alone may not get you there, because you still need some
> > > > reasonably fair way to schedule the two queues. So that might have been
> > > > the next line of criticism. Scheduling in proportion to the number of
> > > > flows each queue contains is one effective way to do that, but that
> > > > requires at least some concept of a flow. Perhaps there's another way
> > > > that doesn't introduce too much RTT unfairness, but I'm not sure.
> > > >
> > > > In our defense, there was already a lot of support built up for L4S,
> > > > and stepping in front of that was like stepping in front of a freight
> > > > train no matter what we did. I think we've made a decent argument in
> > > > the most recent version of the SCE draft that ECN is a "network
> > > > feature" which carries higher risks than drop-based signaling, and
> > > > warrants the requirement for unresponsive flow mitigation, for
> > > > starters. That of course requires some level of flow awareness, which
> > > > then makes various queueing designs possible. And, there may still be
> > > > deployment possibilities with DSCP, as Rodney mentioned.
> > > >
> > > > Anyway, there's progress being made on SCE, with some new ideas and
> > > > improvements to testing tools coming along.
> > > >
> > > > Pete
> > > >
> > > >
> > > > Ecn-sane@lists.bufferbloat.net
> > > dave@taht.net <Dave T?ht> CTO, TekLibre, LLC Tel: 1-831-435-0729
> > --
> > Rod Grimes                                                 rgrimes@freebsd.org
> 
> 
> 
> -- 
> "For a successful technology, reality must take precedence over public
> relations, for Mother Nature cannot be fooled" - Richard Feynman
> 
> dave@taht.net <Dave T?ht> CTO, TekLibre, LLC Tel: 1-831-435-0729
> 
> 

-- 
Rod Grimes                                                 rgrimes@freebsd.org

Re: [Ecn-sane] ect(1) queue selector question

[Dave Taht]

I have basically hoped to find some sysadmin out there with battle
experience with a real dctcp deployment (such as the one at facebook)
that could share his or her
insights as to this debate.

Most of the public testing with BBRv2 has instead been with a brick
wall setting to CE_threshold, the published data on it was a 260us,
which was about as low as raw iron linux can get. I tend to favor the
brick wall approach over anything more complex for the AQM component
of a SCE architecture, and to modify the transports to suit this
assumption.

On Tue, Mar 9, 2021 at 7:19 AM Rodney W. Grimes <4bone@gndrsh.dnsmgr.net> wrote:
>
> > I am of course much happier interacting here than over on tsvwg.
> >
> > I would like very much
> >
> > A large scale bit twiddling test from the larger providers of fq_codel
> > and cake based hw and middleboxes.
> >
> > extensive testing on lte and wifi transports. even emulated.
> > the sce patches polished up and submitted for review to netdev as what
> > we call there, an RFC
> > the ability to test both SCE and L4S on openwrt (backport thereof)
> >
> > I know there's been a lot of work on other qdiscs than cake, but
> > haven't paid much attention. netdev review would be nice.
> >
> > A simple internet-wide test of say bittorrent, or using an adserver
> > method like what apnic uses.
> >
> > Most of all I'd really like someone to take a stab at RFC3168 support
> > for BBR. And by that, I don't mean a reduction by half, but to
> > institute an RTT probe phase. A fixed rate reduction per RTT is simply
> > not
> > going to work IMHO, period, on lte and wifi. I'm sure dave miller
> > would accept such a patch, and this would also lead towards better ecn
> > support across the internet in general... at least from the
> > perspective of the one provider I have an in with, dropbox.
> >
> > SCE support for BBRv2 would be nice also.
> >
> > And I know, a pony, all sparkly and stuff. I find it very difficult to
> > summon the cojones to do a drop of work in this area, and I keep
> > cheering you on - especially on the bug fixing and wonderful scripts
> > and data you've provided so far, and I do wish I could find some way
> > to help that didn't cause so much ptsd in me.
> >
> > I wish I merely knew more about how dctp was configured in the data
> > center. So far as *I* know its on dedicated switches mostly. I would
> > vastly prefer a dscp codepoint for this, also.
>
> If you want I can take a discussion on how DCTCP is configured in
> data center switches in a side channel.  Its basically using a
> drop function that has a starting queue depth, and slope that
> as you go higher on the slope your propability of marking
> increases.  The 40 gbit experiments we did with HPE modified
> that part of the switch asic code to do SCE marks starting
> at 1% mark probability for 1% Queue depth, up to 100% marking
> at 100% Queue depth.  Sadly that was a slight mistake on
> my part, and I thought we would get a chance to iterate
> and retune these, I just pulled those values out as a WOG
> and gave them to HPE to let them get on with setting it up.
>
> For most practical purposes an SCE mark and a DCTCP CE
> mark convey very similiar, if not the same information,
> as does a CE mark in L4S.
>
> Usually tunning of these queue depth and slope values involes
> data collection and several iterations of experiments.
>
> TOR vs SPINE switches are usually configured with different values.
>
> I believe there is even some work by someone that does data
> collection across the whole datacenter and tries to adjust
> these automagically and dynamically.
>
> >
> > On Mon, Mar 8, 2021 at 4:36 PM Pete Heist <pete@heistp.net> wrote:
> > >
> > > Sorry for reviving an old thread as I haven't been on this list in a
> > > while:
> > >
> > > > > SCE proposes to use ect(1) as an indicator of some congestion and
> > > does
> > > > > not explictly
> > > > > require a dscp codepoint in a FQ'd implementation.
> > > >       Pretty much. I do think that a demonstration using an
> > > > additional DSCP to create a similar HOV lane for SCE would have gone
> > > > miles in convincing people in the WG that L4S might really not be as
> > > > swell as its proponents argue, IMHO it won the day more with its
> > > > attractive promise of low latency for all instead of what it
> > > delivers.
> > >
> > > On that, I don't think any of us knows how things will end up or how
> > > long it will take to get there...
> > >
> > > I do agree that the interim meeting leading up to the codepoint
> > > decision could have gone better. Everything went great until it came to
> > > how to deploy SCE in a small number of queues. We had dismissed the
> > > idea of using DSCP, because we thought it would be panned for its poor
> > > traversal over the Internet. That may still have been the case, but it
> > > also may have worked if sold right. We thought that AF alone might be
> > > enough to get past that part, but it wasn't.
> > >
> > > We already implemented a two-queue design that uses DSCP, but either
> > > there wasn't much interest, or we didn't sell it enough. Plus, for
> > > those who demand a two queue solution that requires no flow awareness
> > > at all, DSCP alone may not get you there, because you still need some
> > > reasonably fair way to schedule the two queues. So that might have been
> > > the next line of criticism. Scheduling in proportion to the number of
> > > flows each queue contains is one effective way to do that, but that
> > > requires at least some concept of a flow. Perhaps there's another way
> > > that doesn't introduce too much RTT unfairness, but I'm not sure.
> > >
> > > In our defense, there was already a lot of support built up for L4S,
> > > and stepping in front of that was like stepping in front of a freight
> > > train no matter what we did. I think we've made a decent argument in
> > > the most recent version of the SCE draft that ECN is a "network
> > > feature" which carries higher risks than drop-based signaling, and
> > > warrants the requirement for unresponsive flow mitigation, for
> > > starters. That of course requires some level of flow awareness, which
> > > then makes various queueing designs possible. And, there may still be
> > > deployment possibilities with DSCP, as Rodney mentioned.
> > >
> > > Anyway, there's progress being made on SCE, with some new ideas and
> > > improvements to testing tools coming along.
> > >
> > > Pete
> > >
> > >
> > > Ecn-sane@lists.bufferbloat.net
> > dave@taht.net <Dave T?ht> CTO, TekLibre, LLC Tel: 1-831-435-0729
> --
> Rod Grimes                                                 rgrimes@freebsd.org



-- 
"For a successful technology, reality must take precedence over public
relations, for Mother Nature cannot be fooled" - Richard Feynman

dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

Re: [Ecn-sane] ect(1) queue selector question

[Rodney W. Grimes]

> I am of course much happier interacting here than over on tsvwg.
> 
> I would like very much
> 
> A large scale bit twiddling test from the larger providers of fq_codel
> and cake based hw and middleboxes.
> 
> extensive testing on lte and wifi transports. even emulated.
> the sce patches polished up and submitted for review to netdev as what
> we call there, an RFC
> the ability to test both SCE and L4S on openwrt (backport thereof)
> 
> I know there's been a lot of work on other qdiscs than cake, but
> haven't paid much attention. netdev review would be nice.
> 
> A simple internet-wide test of say bittorrent, or using an adserver
> method like what apnic uses.
> 
> Most of all I'd really like someone to take a stab at RFC3168 support
> for BBR. And by that, I don't mean a reduction by half, but to
> institute an RTT probe phase. A fixed rate reduction per RTT is simply
> not
> going to work IMHO, period, on lte and wifi. I'm sure dave miller
> would accept such a patch, and this would also lead towards better ecn
> support across the internet in general... at least from the
> perspective of the one provider I have an in with, dropbox.
> 
> SCE support for BBRv2 would be nice also.
> 
> And I know, a pony, all sparkly and stuff. I find it very difficult to
> summon the cojones to do a drop of work in this area, and I keep
> cheering you on - especially on the bug fixing and wonderful scripts
> and data you've provided so far, and I do wish I could find some way
> to help that didn't cause so much ptsd in me.
> 
> I wish I merely knew more about how dctp was configured in the data
> center. So far as *I* know its on dedicated switches mostly. I would
> vastly prefer a dscp codepoint for this, also.

If you want I can take a discussion on how DCTCP is configured in
data center switches in a side channel.  Its basically using a
drop function that has a starting queue depth, and slope that
as you go higher on the slope your propability of marking
increases.  The 40 gbit experiments we did with HPE modified
that part of the switch asic code to do SCE marks starting
at 1% mark probability for 1% Queue depth, up to 100% marking
at 100% Queue depth.  Sadly that was a slight mistake on
my part, and I thought we would get a chance to iterate
and retune these, I just pulled those values out as a WOG
and gave them to HPE to let them get on with setting it up.

For most practical purposes an SCE mark and a DCTCP CE
mark convey very similiar, if not the same information,
as does a CE mark in L4S.

Usually tunning of these queue depth and slope values involes
data collection and several iterations of experiments.

TOR vs SPINE switches are usually configured with different values.

I believe there is even some work by someone that does data
collection across the whole datacenter and tries to adjust
these automagically and dynamically.

> 
> On Mon, Mar 8, 2021 at 4:36 PM Pete Heist <pete@heistp.net> wrote:
> >
> > Sorry for reviving an old thread as I haven't been on this list in a
> > while:
> >
> > > > SCE proposes to use ect(1) as an indicator of some congestion and
> > does
> > > > not explictly
> > > > require a dscp codepoint in a FQ'd implementation.
> > >       Pretty much. I do think that a demonstration using an
> > > additional DSCP to create a similar HOV lane for SCE would have gone
> > > miles in convincing people in the WG that L4S might really not be as
> > > swell as its proponents argue, IMHO it won the day more with its
> > > attractive promise of low latency for all instead of what it
> > delivers.
> >
> > On that, I don't think any of us knows how things will end up or how
> > long it will take to get there...
> >
> > I do agree that the interim meeting leading up to the codepoint
> > decision could have gone better. Everything went great until it came to
> > how to deploy SCE in a small number of queues. We had dismissed the
> > idea of using DSCP, because we thought it would be panned for its poor
> > traversal over the Internet. That may still have been the case, but it
> > also may have worked if sold right. We thought that AF alone might be
> > enough to get past that part, but it wasn't.
> >
> > We already implemented a two-queue design that uses DSCP, but either
> > there wasn't much interest, or we didn't sell it enough. Plus, for
> > those who demand a two queue solution that requires no flow awareness
> > at all, DSCP alone may not get you there, because you still need some
> > reasonably fair way to schedule the two queues. So that might have been
> > the next line of criticism. Scheduling in proportion to the number of
> > flows each queue contains is one effective way to do that, but that
> > requires at least some concept of a flow. Perhaps there's another way
> > that doesn't introduce too much RTT unfairness, but I'm not sure.
> >
> > In our defense, there was already a lot of support built up for L4S,
> > and stepping in front of that was like stepping in front of a freight
> > train no matter what we did. I think we've made a decent argument in
> > the most recent version of the SCE draft that ECN is a "network
> > feature" which carries higher risks than drop-based signaling, and
> > warrants the requirement for unresponsive flow mitigation, for
> > starters. That of course requires some level of flow awareness, which
> > then makes various queueing designs possible. And, there may still be
> > deployment possibilities with DSCP, as Rodney mentioned.
> >
> > Anyway, there's progress being made on SCE, with some new ideas and
> > improvements to testing tools coming along.
> >
> > Pete
> >
> >
> > Ecn-sane@lists.bufferbloat.net
> dave@taht.net <Dave T?ht> CTO, TekLibre, LLC Tel: 1-831-435-0729
-- 
Rod Grimes                                                 rgrimes@freebsd.org

Re: [Ecn-sane] ect(1) queue selector question

[Pete Heist]

Fwiw I captured a "Dave's wish list" file so we have it around. We do
come back to these lists sometimes.

There's not much here currently in our project plan except for more
testing on bursty mac layers, in the lab at first, as part of work that
should make it much easier to run more types of tests, and concurrently
so they get done faster. If HFCC has a deployment problem with bursty
traffic that can't be solved, it seems like something that needs to be
worked out asap. We made quite a utilization improvement by using Codel
to do SCE signaling, but what are the tradeoffs and how does that work
in different conditions? Worth trying to answer with testing.

On Mon, 2021-03-08 at 17:18 -0800, Dave Taht wrote:
> I am of course much happier interacting here than over on tsvwg.
> 
> I would like very much
> 
> A large scale bit twiddling test from the larger providers of
> fq_codel
> and cake based hw and middleboxes.
> 
> extensive testing on lte and wifi transports. even emulated.
> the sce patches polished up and submitted for review to netdev as
> what
> we call there, an RFC
> the ability to test both SCE and L4S on openwrt (backport thereof)
> 
> I know there's been a lot of work on other qdiscs than cake, but
> haven't paid much attention. netdev review would be nice.
> 
> A simple internet-wide test of say bittorrent, or using an adserver
> method like what apnic uses.
> 
> Most of all I'd really like someone to take a stab at RFC3168 support
> for BBR. And by that, I don't mean a reduction by half, but to
> institute an RTT probe phase. A fixed rate reduction per RTT is
> simply
> not
> going to work IMHO, period, on lte and wifi. I'm sure dave miller
> would accept such a patch, and this would also lead towards better
> ecn
> support across the internet in general... at least from the
> perspective of the one provider I have an in with, dropbox.
> 
> SCE support for BBRv2 would be nice also.
> 
> And I know, a pony, all sparkly and stuff. I find it very difficult
> to
> summon the cojones to do a drop of work in this area, and I keep
> cheering you on - especially on the bug fixing and wonderful scripts
> and data you've provided so far, and I do wish I could find some way
> to help that didn't cause so much ptsd in me.
> 
> I wish I merely knew more about how dctp was configured in the data
> center. So far as *I* know its on dedicated switches mostly. I would
> vastly prefer a dscp codepoint for this, also.
> 
> On Mon, Mar 8, 2021 at 4:36 PM Pete Heist <pete@heistp.net> wrote:
> > 
> > Sorry for reviving an old thread as I haven't been on this list in
> > a
> > while:
> > 
> > > > SCE proposes to use ect(1) as an indicator of some congestion
> > > > and
> > does
> > > > not explictly
> > > > require a dscp codepoint in a FQ'd implementation.
> > >       Pretty much. I do think that a demonstration using an
> > > additional DSCP to create a similar HOV lane for SCE would have
> > > gone
> > > miles in convincing people in the WG that L4S might really not be
> > > as
> > > swell as its proponents argue, IMHO it won the day more with its
> > > attractive promise of low latency for all instead of what it
> > delivers.
> > 
> > On that, I don't think any of us knows how things will end up or
> > how
> > long it will take to get there...
> > 
> > I do agree that the interim meeting leading up to the codepoint
> > decision could have gone better. Everything went great until it
> > came to
> > how to deploy SCE in a small number of queues. We had dismissed the
> > idea of using DSCP, because we thought it would be panned for its
> > poor
> > traversal over the Internet. That may still have been the case, but
> > it
> > also may have worked if sold right. We thought that AF alone might
> > be
> > enough to get past that part, but it wasn't.
> > 
> > We already implemented a two-queue design that uses DSCP, but
> > either
> > there wasn't much interest, or we didn't sell it enough. Plus, for
> > those who demand a two queue solution that requires no flow
> > awareness
> > at all, DSCP alone may not get you there, because you still need
> > some
> > reasonably fair way to schedule the two queues. So that might have
> > been
> > the next line of criticism. Scheduling in proportion to the number
> > of
> > flows each queue contains is one effective way to do that, but that
> > requires at least some concept of a flow. Perhaps there's another
> > way
> > that doesn't introduce too much RTT unfairness, but I'm not sure.
> > 
> > In our defense, there was already a lot of support built up for
> > L4S,
> > and stepping in front of that was like stepping in front of a
> > freight
> > train no matter what we did. I think we've made a decent argument
> > in
> > the most recent version of the SCE draft that ECN is a "network
> > feature" which carries higher risks than drop-based signaling, and
> > warrants the requirement for unresponsive flow mitigation, for
> > starters. That of course requires some level of flow awareness,
> > which
> > then makes various queueing designs possible. And, there may still
> > be
> > deployment possibilities with DSCP, as Rodney mentioned.
> > 
> > Anyway, there's progress being made on SCE, with some new ideas and
> > improvements to testing tools coming along.
> > 
> > Pete
> > 
> > 
> > _______________________________________________
> > Ecn-sane mailing list
> > Ecn-sane@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/ecn-sane
> 
> 
> 

Re: [Ecn-sane] ect(1) queue selector question

[Dave Taht]

I am of course much happier interacting here than over on tsvwg.

I would like very much

A large scale bit twiddling test from the larger providers of fq_codel
and cake based hw and middleboxes.

extensive testing on lte and wifi transports. even emulated.
the sce patches polished up and submitted for review to netdev as what
we call there, an RFC
the ability to test both SCE and L4S on openwrt (backport thereof)

I know there's been a lot of work on other qdiscs than cake, but
haven't paid much attention. netdev review would be nice.

A simple internet-wide test of say bittorrent, or using an adserver
method like what apnic uses.

Most of all I'd really like someone to take a stab at RFC3168 support
for BBR. And by that, I don't mean a reduction by half, but to
institute an RTT probe phase. A fixed rate reduction per RTT is simply
not
going to work IMHO, period, on lte and wifi. I'm sure dave miller
would accept such a patch, and this would also lead towards better ecn
support across the internet in general... at least from the
perspective of the one provider I have an in with, dropbox.

SCE support for BBRv2 would be nice also.

And I know, a pony, all sparkly and stuff. I find it very difficult to
summon the cojones to do a drop of work in this area, and I keep
cheering you on - especially on the bug fixing and wonderful scripts
and data you've provided so far, and I do wish I could find some way
to help that didn't cause so much ptsd in me.

I wish I merely knew more about how dctp was configured in the data
center. So far as *I* know its on dedicated switches mostly. I would
vastly prefer a dscp codepoint for this, also.

On Mon, Mar 8, 2021 at 4:36 PM Pete Heist <pete@heistp.net> wrote:
>
> Sorry for reviving an old thread as I haven't been on this list in a
> while:
>
> > > SCE proposes to use ect(1) as an indicator of some congestion and
> does
> > > not explictly
> > > require a dscp codepoint in a FQ'd implementation.
> >       Pretty much. I do think that a demonstration using an
> > additional DSCP to create a similar HOV lane for SCE would have gone
> > miles in convincing people in the WG that L4S might really not be as
> > swell as its proponents argue, IMHO it won the day more with its
> > attractive promise of low latency for all instead of what it
> delivers.
>
> On that, I don't think any of us knows how things will end up or how
> long it will take to get there...
>
> I do agree that the interim meeting leading up to the codepoint
> decision could have gone better. Everything went great until it came to
> how to deploy SCE in a small number of queues. We had dismissed the
> idea of using DSCP, because we thought it would be panned for its poor
> traversal over the Internet. That may still have been the case, but it
> also may have worked if sold right. We thought that AF alone might be
> enough to get past that part, but it wasn't.
>
> We already implemented a two-queue design that uses DSCP, but either
> there wasn't much interest, or we didn't sell it enough. Plus, for
> those who demand a two queue solution that requires no flow awareness
> at all, DSCP alone may not get you there, because you still need some
> reasonably fair way to schedule the two queues. So that might have been
> the next line of criticism. Scheduling in proportion to the number of
> flows each queue contains is one effective way to do that, but that
> requires at least some concept of a flow. Perhaps there's another way
> that doesn't introduce too much RTT unfairness, but I'm not sure.
>
> In our defense, there was already a lot of support built up for L4S,
> and stepping in front of that was like stepping in front of a freight
> train no matter what we did. I think we've made a decent argument in
> the most recent version of the SCE draft that ECN is a "network
> feature" which carries higher risks than drop-based signaling, and
> warrants the requirement for unresponsive flow mitigation, for
> starters. That of course requires some level of flow awareness, which
> then makes various queueing designs possible. And, there may still be
> deployment possibilities with DSCP, as Rodney mentioned.
>
> Anyway, there's progress being made on SCE, with some new ideas and
> improvements to testing tools coming along.
>
> Pete
>
>
> _______________________________________________
> Ecn-sane mailing list
> Ecn-sane@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/ecn-sane



-- 
"For a successful technology, reality must take precedence over public
relations, for Mother Nature cannot be fooled" - Richard Feynman

dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

Re: [Ecn-sane] ect(1) queue selector question

[Pete Heist]

Sorry for reviving an old thread as I haven't been on this list in a
while:

> > SCE proposes to use ect(1) as an indicator of some congestion and
does
> > not explictly
> > require a dscp codepoint in a FQ'd implementation.
> 	Pretty much. I do think that a demonstration using an
> additional DSCP to create a similar HOV lane for SCE would have gone
> miles in convincing people in the WG that L4S might really not be as
> swell as its proponents argue, IMHO it won the day more with its
> attractive promise of low latency for all instead of what it
delivers.

On that, I don't think any of us knows how things will end up or how
long it will take to get there...

I do agree that the interim meeting leading up to the codepoint
decision could have gone better. Everything went great until it came to
how to deploy SCE in a small number of queues. We had dismissed the
idea of using DSCP, because we thought it would be panned for its poor
traversal over the Internet. That may still have been the case, but it
also may have worked if sold right. We thought that AF alone might be
enough to get past that part, but it wasn't.

We already implemented a two-queue design that uses DSCP, but either
there wasn't much interest, or we didn't sell it enough. Plus, for
those who demand a two queue solution that requires no flow awareness
at all, DSCP alone may not get you there, because you still need some
reasonably fair way to schedule the two queues. So that might have been
the next line of criticism. Scheduling in proportion to the number of
flows each queue contains is one effective way to do that, but that
requires at least some concept of a flow. Perhaps there's another way
that doesn't introduce too much RTT unfairness, but I'm not sure.

In our defense, there was already a lot of support built up for L4S,
and stepping in front of that was like stepping in front of a freight
train no matter what we did. I think we've made a decent argument in
the most recent version of the SCE draft that ECN is a "network
feature" which carries higher risks than drop-based signaling, and
warrants the requirement for unresponsive flow mitigation, for
starters. That of course requires some level of flow awareness, which
then makes various queueing designs possible. And, there may still be
deployment possibilities with DSCP, as Rodney mentioned.

Anyway, there's progress being made on SCE, with some new ideas and
improvements to testing tools coming along.

Pete