[Cake] draft-ietf-tsvwg-nqb-15.txt vs the cake AQM

[Cake] draft-ietf-tsvwg-nqb-15.txt vs the cake AQM

[Dave Taht]

I have been sitting on the cake related patches for this for years
now, and it is my hope to get support for NQB into the next linux
release, regardless of whether it gets through last call at this time,
unless the selected codepoint number changes. (?)

Cake (please see the man page here:
https://man7.org/linux/man-pages/man8/tc-cake.8.html ) supports
multiple diffserv models.

besteffort is exactly that, besteffort, and will not gain NQB support.

The diffserv3 interpretation is the default, and given that flow
queuing handles most of the NQB-like problems naturally, and  Voice
(CS7, CS6, EF, VA, TOS4) is all that is handled there today, I am
thinking of *not* elevating NQB into that class is the right thing.

NQB fits nicely into the diffserv4 model in the video class, so I will
put it there. since covid we tend to use the diffserv4 model a lot to
manage videoconferencing better.

As for the CS0-CS7 precedence model inc cake, we have declared that
obsolete in the code, and wherever NQB falls into it, great. And the
diffserv8, I don´t know.

Anyway, does that work for everyone?

Part II of this would be a discussion of the various wash modes, but
merely getting the right byte into the right lookup tables after all
this discussion, would be nice.

Re: [Cake] draft-ietf-tsvwg-nqb-15.txt vs the cake AQM

[Ruediger.Geib]

Dave,

thanks for asking - I'm not an NQB author, and my know-how on Linux QoS / Cake is fairly zero. Did you want to address Greg?

I myself am still struggling to understand how NQB operates. I understand the idea behind it, but questions on operation still remain.

NQB has been designed for AC_VI, not AC_VO. So aggregating it with other video related DSCPs may make sense. Greg's draft partially suggests other PHBs to forward NQB, I think. My main concern is that no flow should be able to starve off Best Effort by design. If the Linux Cake implementation does so, also if combined with WiFi scheduling, then I'm fine. If the result is, let's all mark all traffic by (e.g.) NQB as then we'll certainly seize more bandwidth than BE/default, we don't need NQB.

This is not to say, NQB does or will starve off BE/default. I'm however not sure, whether I understood operation of it completely and I think, draft text is insufficient or not precise. I saw and appreciate that precise flow definitions are part of the Linux/cake implementation. Draft NQB offers none at all.

Regards,

Ruediger 

-----Ursprüngliche Nachricht-----
Von: Dave Taht <dave.taht@gmail.com> 
Gesendet: Dienstag, 14. März 2023 15:02
An: Geib, Rüdiger <Ruediger.Geib@telekom.de>
Cc: Greg White <g.white@cablelabs.com>; tsvwg IETF list <tsvwg@ietf.org>; Cake List <cake@lists.bufferbloat.net>
Betreff: draft-ietf-tsvwg-nqb-15.txt vs the cake AQM

I have been sitting on the cake related patches for this for years now, and it is my hope to get support for NQB into the next linux release, regardless of whether it gets through last call at this time, unless the selected codepoint number changes. (?)

Cake (please see the man page here:
https://man7.org/linux/man-pages/man8/tc-cake.8.html ) supports multiple diffserv models.

besteffort is exactly that, besteffort, and will not gain NQB support.

The diffserv3 interpretation is the default, and given that flow queuing handles most of the NQB-like problems naturally, and  Voice (CS7, CS6, EF, VA, TOS4) is all that is handled there today, I am thinking of *not* elevating NQB into that class is the right thing.

NQB fits nicely into the diffserv4 model in the video class, so I will put it there. since covid we tend to use the diffserv4 model a lot to manage videoconferencing better.

As for the CS0-CS7 precedence model inc cake, we have declared that obsolete in the code, and wherever NQB falls into it, great. And the diffserv8, I don´t know.

Anyway, does that work for everyone?

Part II of this would be a discussion of the various wash modes, but merely getting the right byte into the right lookup tables after all this discussion, would be nice.

Re: [Cake] [tsvwg] draft-ietf-tsvwg-nqb-15.txt vs the cake AQM

[Sebastian Moeller]

Hi Ruerdiger,


> On Mar 14, 2023, at 16:09, <Ruediger.Geib@telekom.de> <Ruediger.Geib@telekom.de> wrote:
> 
> Dave,
> 
> thanks for asking - I'm not an NQB author, and my know-how on Linux QoS / Cake is fairly zero. Did you want to address Greg?
> 
> I myself am still struggling to understand how NQB operates. I understand the idea behind it, but questions on operation still remain.
> 
> NQB has been designed for AC_VI, not AC_VO.

	This is not how I remember it... it is designed to operate at slightly elevated conditional priority over AC_BE, it is just that WiFI does not offer that so Greg went for the next best thing AC_VI happily accepting the airtime unfairness this is going to introice. I think calling this designed for AC_VI is maked "designed" do too much work in that sentence.


> So aggregating it with other video related DSCPs may make sense. Greg's draft partially suggests other PHBs to forward NQB, I think. My main concern is that no flow should be able to starve off Best Effort by design.

	Let's please use a realistic definition of "starve" here that is less absolute than "suppress to minimal congestion window" which occasional seems to be used in this WG.

> If the Linux Cake implementation does so, also if combined with WiFi scheduling, then I'm fine.

	In cake "Video" is really just a name for one of the elevated priority classes (to give users a sense of what they could put in there). Now cake does not use priority queueing but if a priority tin exceeds its capacity share that class is scheduled (round-robin) I believe with best effort. 


> If the result is, let's all mark all traffic by (e.g.) NQB as then we'll certainly seize more bandwidth than BE/default, we don't need NQB.

	That is what is going to happen in cake: the Video tin gets 50% of capacity at priority (under contention, without other takes that flow gets up to 100%), if even a single flow is in there that single flow gets that 50% (if it actually uses it) even if competing with more flows in best effort...

> 
> This is not to say, NQB does or will starve off BE/default.

	No cake avoids starvation, but a flow abusing a priority tin can gain a considerably unfair throughput gain.

However none of this is actually needed, just leaving it in best effort and letting flow isolation and sparse boosting do its thing should be good enough for NQB use cases. The fact is cake really does not need NQB for the kind of service NQB is designed for. If you disagree Dave I am happy to see data showing this to be wrong.

> I'm however not sure, whether I understood operation of it completely and I think, draft text is insufficient or not precise. I saw and appreciate that precise flow definitions are part of the Linux/cake implementation. Draft NQB offers none at all.

	Indirectly it does, after all that is what queue protection operates on. I do wonder though how this is going to fare with tunneled traffic...

Regards
	Sebastian


> 
> Regards,
> 
> Ruediger 
> 
> -----Ursprüngliche Nachricht-----
> Von: Dave Taht <dave.taht@gmail.com> 
> Gesendet: Dienstag, 14. März 2023 15:02
> An: Geib, Rüdiger <Ruediger.Geib@telekom.de>
> Cc: Greg White <g.white@cablelabs.com>; tsvwg IETF list <tsvwg@ietf.org>; Cake List <cake@lists.bufferbloat.net>
> Betreff: draft-ietf-tsvwg-nqb-15.txt vs the cake AQM
> 
> I have been sitting on the cake related patches for this for years now, and it is my hope to get support for NQB into the next linux release, regardless of whether it gets through last call at this time, unless the selected codepoint number changes. (?)
> 
> Cake (please see the man page here:
> https://man7.org/linux/man-pages/man8/tc-cake.8.html ) supports multiple diffserv models.
> 
> besteffort is exactly that, besteffort, and will not gain NQB support.
> 
> The diffserv3 interpretation is the default, and given that flow queuing handles most of the NQB-like problems naturally, and  Voice (CS7, CS6, EF, VA, TOS4) is all that is handled there today, I am thinking of *not* elevating NQB into that class is the right thing.
> 
> NQB fits nicely into the diffserv4 model in the video class, so I will put it there. since covid we tend to use the diffserv4 model a lot to manage videoconferencing better.
> 
> As for the CS0-CS7 precedence model inc cake, we have declared that obsolete in the code, and wherever NQB falls into it, great. And the diffserv8, I don´t know.
> 
> Anyway, does that work for everyone?
> 
> Part II of this would be a discussion of the various wash modes, but merely getting the right byte into the right lookup tables after all this discussion, would be nice.

Re: [Cake] [tsvwg] draft-ietf-tsvwg-nqb-15.txt vs the cake AQM

[Greg White]

Sebastian,

Please don't....

-Greg

On 3/14/23, 10:51 AM, "tsvwg on behalf of Sebastian Moeller" <tsvwg-bounces@ietf.org <mailto:tsvwg-bounces@ietf.org> on behalf of moeller0@gmx.de <mailto:moeller0@gmx.de>> wrote:
Hi Ruerdiger,

> On Mar 14, 2023, at 16:09, <Ruediger.Geib@telekom.de <mailto:Ruediger.Geib@telekom.de>> <Ruediger.Geib@telekom.de <mailto:Ruediger.Geib@telekom.de>> wrote:
> 
> Dave,
> 
> thanks for asking - I'm not an NQB author, and my know-how on Linux QoS / Cake is fairly zero. Did you want to address Greg?
> 
> I myself am still struggling to understand how NQB operates. I understand the idea behind it, but questions on operation still remain.
> 
> NQB has been designed for AC_VI, not AC_VO.

This is not how I remember it... it is designed to operate at slightly elevated conditional priority over AC_BE, it is just that WiFI does not offer that so Greg went for the next best thing AC_VI happily accepting the airtime unfairness this is going to introice. I think calling this designed for AC_VI is maked "designed" do too much work in that sentence.

[GW] There is no "slightly elevated conditional priority" in the NQB draft. The NQB queue is to be given equal priority to Default. That is written in the draft. Please don't try to misconstrue it.

[GW] I'm really upset about your implication that I am "happily accepting" the situation with legacy Wi-Fi.  This is extremely disrespectful and should not be tolerated by the WG.  As should be clear to everyone who's been reading the discussion on this (or who has read the draft) this decision was a compromise, and in my view the best option out of the available imperfect options. I would appreciate it if you would treat me, and the other members of this WG with respect.

Re: [Cake] draft-ietf-tsvwg-nqb-15.txt vs the cake AQM

[Greg White]

Hi Dave, 

The NQB requirement is that it shares capacity with and is at the same priority as Default (CS0).  So, for all priority queue options in CAKE (aside from precedence, perhaps), I would recommend that you align with that requirement. So, I think I agree with what you wrote below for besteffort, diffserv3 and precedence, but for diffserv4 CAKE would be non-compliant if it put NQB into the video class. I don't understand diffserv8, since the man page doesn't appear to describe it.  But, the same logic should hold there too. 

In most cases, I think the flow isolation in CAKE already provides the benefit that the NQB PHB is aiming to achieve.  But, in the flowblind, srchost, dsthost, & hosts modes, it doesn't.  Here is where you could consider doing a full implementation of the NQB PHB (a separate queue in the Best Effort tin). If you choose to take that on, the recommendation is to implement a traffic protection mechanism.  This would be a really interesting test case to see whether you think the draft is sufficiently detailed for you to come up with a design for linux. 

You mentioned a Part II to discuss the various wash modes.  I only see two modes (wash/nowash) am I missing something?

-Greg


On 3/14/23, 8:02 AM, "Dave Taht" <dave.taht@gmail.com <mailto:dave.taht@gmail.com>> wrote:


I have been sitting on the cake related patches for this for years
now, and it is my hope to get support for NQB into the next linux
release, regardless of whether it gets through last call at this time,
unless the selected codepoint number changes. (?)


Cake (please see the man page here:
https://man7.org/linux/man-pages/man8/tc-cake.8.html <https://man7.org/linux/man-pages/man8/tc-cake.8.html> ) supports
multiple diffserv models.


besteffort is exactly that, besteffort, and will not gain NQB support.


The diffserv3 interpretation is the default, and given that flow
queuing handles most of the NQB-like problems naturally, and Voice
(CS7, CS6, EF, VA, TOS4) is all that is handled there today, I am
thinking of *not* elevating NQB into that class is the right thing.


NQB fits nicely into the diffserv4 model in the video class, so I will
put it there. since covid we tend to use the diffserv4 model a lot to
manage videoconferencing better.


As for the CS0-CS7 precedence model inc cake, we have declared that
obsolete in the code, and wherever NQB falls into it, great. And the
diffserv8, I don´t know.


Anyway, does that work for everyone?


Part II of this would be a discussion of the various wash modes, but
merely getting the right byte into the right lookup tables after all
this discussion, would be nice.

Re: [Cake] [tsvwg] draft-ietf-tsvwg-nqb-15.txt vs the cake AQM

[Sebastian Moeller]

Hi Greg,


> On Mar 14, 2023, at 17:25, Greg White <g.white@cablelabs.com> wrote:
> 
> Hi Dave, 
> 
> The NQB requirement is that it shares capacity with and is at the same priority as Default (CS0).  So, for all priority queue options in CAKE (aside from precedence, perhaps), I would recommend that you align with that requirement. So, I think I agree with what you wrote below for besteffort, diffserv3 and precedence, but for diffserv4 CAKE would be non-compliant if it put NQB into the video class. I don't understand diffserv8, since the man page doesn't appear to describe it.  But, the same logic should hold there too. 
> 
> In most cases, I think the flow isolation in CAKE already provides the benefit that the NQB PHB is aiming to achieve.  But, in the flowblind, srchost, dsthost, & hosts modes, it doesn't.
>  Here is where you could consider doing a full implementation of the NQB PHB (a separate queue in the Best Effort tin). If you choose to take that on, the recommendation is to implement a traffic protection mechanism.  This would be a really interesting test case to see whether you think the draft is sufficiently detailed for you to come up with a design for linux. 

	But this additional queue would violate the promises made by flowblind, srchost, dsthost, & hosts, so you would need to add this as additional key word documented to add an additional queue for NQB, but what to do then, do this e.g. individually for each of the host queues or just a single one.

If you ask me don't do it at all or as specific modifier keyword for the isolation modes, not that you asked ;)


> You mentioned a Part II to discuss the various wash modes.  I only see two modes (wash/nowash) am I missing something?

	Same here, and again, if you want a special mode that still propagates NQB please give this a new descriptive name retain_e2e_DSCPs or similar, or an explicit way to enumerate which DSCPs to wash.

Regards
	Sebastian


> 
> -Greg
> 
> 
> On 3/14/23, 8:02 AM, "Dave Taht" <dave.taht@gmail.com <mailto:dave.taht@gmail.com>> wrote:
> 
> 
> I have been sitting on the cake related patches for this for years
> now, and it is my hope to get support for NQB into the next linux
> release, regardless of whether it gets through last call at this time,
> unless the selected codepoint number changes. (?)
> 
> 
> Cake (please see the man page here:
> https://man7.org/linux/man-pages/man8/tc-cake.8.html <https://man7.org/linux/man-pages/man8/tc-cake.8.html> ) supports
> multiple diffserv models.
> 
> 
> besteffort is exactly that, besteffort, and will not gain NQB support.
> 
> 
> The diffserv3 interpretation is the default, and given that flow
> queuing handles most of the NQB-like problems naturally, and Voice
> (CS7, CS6, EF, VA, TOS4) is all that is handled there today, I am
> thinking of *not* elevating NQB into that class is the right thing.
> 
> 
> NQB fits nicely into the diffserv4 model in the video class, so I will
> put it there. since covid we tend to use the diffserv4 model a lot to
> manage videoconferencing better.
> 
> 
> As for the CS0-CS7 precedence model inc cake, we have declared that
> obsolete in the code, and wherever NQB falls into it, great. And the
> diffserv8, I don´t know.
> 
> 
> Anyway, does that work for everyone?
> 
> 
> Part II of this would be a discussion of the various wash modes, but
> merely getting the right byte into the right lookup tables after all
> this discussion, would be nice.
> 
> 
> 

Re: [Cake] [tsvwg] draft-ietf-tsvwg-nqb-15.txt vs the cake AQM

[Greg White]

I agree with you that support for a separate NQB queue in the flowblind, srchost, dsthost, & hosts modes (if supported) should be controllable and documented. It would seem simplest to me that this configuration would apply to all instantiated "queues" (like other parameters: nat, wash, and even the flow isolation configuration itself).  CAKE doesn’t appear to have a way to configure e.g. "for host X use 'srchost' and for host Y use 'hosts'".

-Greg


On 3/14/23, 10:59 AM, "Sebastian Moeller" <moeller0@gmx.de <mailto:moeller0@gmx.de>> wrote:


Hi Greg,




> On Mar 14, 2023, at 17:25, Greg White <g.white@cablelabs.com <mailto:g.white@cablelabs.com>> wrote:
> 
> Hi Dave, 
> 
> The NQB requirement is that it shares capacity with and is at the same priority as Default (CS0). So, for all priority queue options in CAKE (aside from precedence, perhaps), I would recommend that you align with that requirement. So, I think I agree with what you wrote below for besteffort, diffserv3 and precedence, but for diffserv4 CAKE would be non-compliant if it put NQB into the video class. I don't understand diffserv8, since the man page doesn't appear to describe it. But, the same logic should hold there too. 
> 
> In most cases, I think the flow isolation in CAKE already provides the benefit that the NQB PHB is aiming to achieve. But, in the flowblind, srchost, dsthost, & hosts modes, it doesn't.
> Here is where you could consider doing a full implementation of the NQB PHB (a separate queue in the Best Effort tin). If you choose to take that on, the recommendation is to implement a traffic protection mechanism. This would be a really interesting test case to see whether you think the draft is sufficiently detailed for you to come up with a design for linux. 


But this additional queue would violate the promises made by flowblind, srchost, dsthost, & hosts, so you would need to add this as additional key word documented to add an additional queue for NQB, but what to do then, do this e.g. individually for each of the host queues or just a single one.


If you ask me don't do it at all or as specific modifier keyword for the isolation modes, not that you asked ;)




> You mentioned a Part II to discuss the various wash modes. I only see two modes (wash/nowash) am I missing something?


Same here, and again, if you want a special mode that still propagates NQB please give this a new descriptive name retain_e2e_DSCPs or similar, or an explicit way to enumerate which DSCPs to wash.


Regards
Sebastian




> 
> -Greg
> 
> 
> On 3/14/23, 8:02 AM, "Dave Taht" <dave.taht@gmail.com <mailto:dave.taht@gmail.com> <mailto:dave.taht@gmail.com <mailto:dave.taht@gmail.com>>> wrote:
> 
> 
> I have been sitting on the cake related patches for this for years
> now, and it is my hope to get support for NQB into the next linux
> release, regardless of whether it gets through last call at this time,
> unless the selected codepoint number changes. (?)
> 
> 
> Cake (please see the man page here:
> https://man7.org/linux/man-pages/man8/tc-cake.8.html <https://man7.org/linux/man-pages/man8/tc-cake.8.html> <https://man7.org/linux/man-pages/man8/tc-cake.8.html> <https://man7.org/linux/man-pages/man8/tc-cake.8.html&gt;> ) supports
> multiple diffserv models.
> 
> 
> besteffort is exactly that, besteffort, and will not gain NQB support.
> 
> 
> The diffserv3 interpretation is the default, and given that flow
> queuing handles most of the NQB-like problems naturally, and Voice
> (CS7, CS6, EF, VA, TOS4) is all that is handled there today, I am
> thinking of *not* elevating NQB into that class is the right thing.
> 
> 
> NQB fits nicely into the diffserv4 model in the video class, so I will
> put it there. since covid we tend to use the diffserv4 model a lot to
> manage videoconferencing better.
> 
> 
> As for the CS0-CS7 precedence model inc cake, we have declared that
> obsolete in the code, and wherever NQB falls into it, great. And the
> diffserv8, I don´t know.
> 
> 
> Anyway, does that work for everyone?
> 
> 
> Part II of this would be a discussion of the various wash modes, but
> merely getting the right byte into the right lookup tables after all
> this discussion, would be nice.
> 
> 
> 

Re: [Cake] [tsvwg] draft-ietf-tsvwg-nqb-15.txt vs the cake AQM

[Sebastian Moeller]

Hi Dave,


> On Mar 14, 2023, at 15:01, Dave Taht <dave.taht@gmail.com> wrote:
> 
> I have been sitting on the cake related patches for this for years
> now, and it is my hope to get support for NQB into the next linux
> release, regardless of whether it gets through last call at this time,
> unless the selected codepoint number changes. (?)
> 
> Cake (please see the man page here:
> https://man7.org/linux/man-pages/man8/tc-cake.8.html ) supports
> multiple diffserv models.
> 
> besteffort is exactly that, besteffort, and will not gain NQB support.
> 
> The diffserv3 interpretation is the default, and given that flow
> queuing handles most of the NQB-like problems naturally, and  Voice
> (CS7, CS6, EF, VA, TOS4) is all that is handled there today, I am
> thinking of *not* elevating NQB into that class is the right thing.
> 
> NQB fits nicely into the diffserv4 model in the video class, so I will
> put it there. since covid we tend to use the diffserv4 model a lot to
> manage videoconferencing better.

	Are you sure? Video gets up to 50% of shaper capacity at elevated priority.. this is not doing the required enforcement to keep NQB rate low...



> As for the CS0-CS7 precedence model inc cake, we have declared that
> obsolete in the code, and wherever NQB falls into it, great. And the
> diffserv8, I don´t know.
> 
> Anyway, does that work for everyone?

	Not for me, as you know I prefer to treat NQB like CS0 for all classes.

> 
> Part II of this would be a discussion of the various wash modes, but
> merely getting the right byte into the right lookup tables after all
> this discussion, would be nice.

	Wash is only a single mode and should stay so, remapping all DSCPs cake used to steer packets to priority tins to CS0 to not leak/reveal any of the internal details upstream. Users wanting to expose NQB to the world, simply do not configure wash or use nowash...
	I am not sure whether you are thinking in that direction at al or whether I am just "paranoid" but washing most DSCPs but retain NQB would IMHO a disservice to cake's users. If you believe such a functionality is needed, it should be configurable which DSCPs to retain and most importantly it should not be called wash to avoid confusion with the current behavior, please.

Regards
	Sebastian

P.S.: The bigger issue is that NQB's design goal is a shallow buffered queue where the side effects of that shallow buffers is hoped to discipline the NQB users not to try to game the system. While I generally believe this not to work, this is not what cake is going to offer, NQB flows will really only be bound by the capacity share. And e.g. a single NQB flow n diffserv4 (with no other non-BestEffort traffic, will be able to hog >= 50% of the total link capacity. That seems problematic... (this is also true for other DSCPs mapping into these priority tiers, but these are rarely exercised by applications without active user intervention in the first place).