[Cake] diffserv3 vs diffserv4

[Cake] diffserv3 vs diffserv4

[Justin Kilpatrick]

"sqm-scripts used 3 tiers of priority pretty successfully as does free.fr. - de-prioritization seems a good idea, prioritization not so much."

This is the best comment on why diffserv3 is the default that I could find on bufferbloat.net. I'm interested in hearing about what data (anecdotes welcome) lead to this conclusion.

-- 
  Justin Kilpatrick
  justin@althea.net

Re: [Cake] diffserv3 vs diffserv4

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 1091 bytes --]



> On 24 Jul 2020, at 16:56, Justin Kilpatrick <justin@althea.net> wrote:
> 
> "sqm-scripts used 3 tiers of priority pretty successfully as does free.fr. - de-prioritization seems a good idea, prioritization not so much."
> 
> This is the best comment on why diffserv3 is the default that I could find on bufferbloat.net. I'm interested in hearing about what data (anecdotes welcome) lead to this conclusion.

As someone who is currently trying (but not that hard) to get a diffserv5 implementation in upstream as opposed to a local hack the aim being to have 2 lower than default classes I have some opinion on this.  My use case is straightforward, I want somewhere to put ‘Least Effort’ traffic (eg Bittorrent) as a scavenger class that loses out to my Bulk transfers (backups)   At the other end of things I do want to prioritise Voice (VOIP) above Video (netflix/facetime) above ‘Best Effort’.  LE, BK, BE, VI, VO

The move from diffserv4 to diffserv5 WAS about de-prioritization.

Cheers,

Kevin D-B

gpg: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Cake] diffserv3 vs diffserv4

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 459 bytes --]



> On 24 Jul 2020, at 18:42, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
> 
> 
> The move from diffserv4 to diffserv5 WAS about de-prioritization.

It was also about minimum bandwidth allocations:

LE: 1/64th
BK: 1/16th
BE: 1/1
VI: 1/2
VO: 1/4

So worst case, best effort should get 11/64ths in the extreme case of all other tins in use.

Cheers,

Kevin D-B

gpg: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Cake] diffserv3 vs diffserv4

[Sebastian Moeller]

Hi Kevin,


> On Jul 25, 2020, at 12:12, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
> 
> 
> 
>> On 24 Jul 2020, at 18:42, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
>> 
>> 
>> The move from diffserv4 to diffserv5 WAS about de-prioritization.
> 
> It was also about minimum bandwidth allocations:
> 
> LE: 1/64th

That is 6 binary orders of magnitude, on a slow link, LE is effectively starved and there will be no real forward progress. For real scavenger services this might well be a sane policy, but this requires the very selective with assigning flows to this tin ;)

> BK: 1/16th
> BE: 1/1
> VI: 1/2
> VO: 1/4

So I see 1/64 + 1/16 + 1/1 + 1/2 + 1/4 = 1.828125 which seems excessive for actually guaranteed minimums. I was under the naive? impression the minima should add up to <= 1, no?


> 
> So worst case, best effort should get 11/64ths in the extreme case of all other tins in use.

	This seems only true, if on overload the lowest prioritiers tiers get their allotment first, no?

I am confused... but I am also confused by cake's output:
"

                   Bulk  Best Effort        Voice
  thresh       3062Kbit       49Mbit    12250Kbit"

as far as I can tell, Bulk's 3062Kbit must be the minimum, while BE and Voice give their maxima... That, or I am missing something important...
(I wonder whether it would not be clearer to give both min and max for each tin, then again I probably missing all the deyails of the actual implementation...)

Best Regards
	Sebastian

> 
> Cheers,
> 
> Kevin D-B
> 
> gpg: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A
> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] diffserv3 vs diffserv4

[Jonathan Morton]

> On 25 Jul, 2020, at 8:18 pm, Sebastian Moeller <moeller0@gmx.de> wrote:
> 
> I am confused... but I am also confused by cake's output:
> 
>                   Bulk  Best Effort        Voice
>  thresh       3062Kbit       49Mbit    12250Kbit"
> 
> as far as I can tell, Bulk's 3062Kbit must be the minimum, while BE and Voice give their maxima... That, or I am missing something important...
> (I wonder whether it would not be clearer to give both min and max for each tin, then again I probably missing all the deyails of the actual implementation...)

Cake delivers from the highest-priority tin that both has data to send and is "behind" its local schedule, defined by the threshold rate.  If no tin with data to send is behind schedule, then some tin that does have data to send is chosen (so Cake as a whole is work-conserving, modulo its global shaper).  IIRC, it'll be the highest priority such tin.

The notion of which tin is highest priority is a little counter-intuitive.  One tin must be at the global shaper rate, and will be the lowest priority tin - and normally that is the "best effort" tin.  So the BK tin is actually at a higher priority, but only up to its very limited threshold rate.  To avoid starving the best effort tin under all possible combinations of traffic, it is necessary and sufficient to ensure that the sum of all higher-priority tins' threshold rates is less than the global rate.

In the case of Diffserv3, the BK and VO tins both have higher priority than BE and sum to 5/16ths of the global rate. So with all tins saturated, the BE traffic gets 11/16ths which is pretty respectable.  If the BE and VO traffic goes away, BK is then able to use all available bandwidth.

 - Jonathan Morton

Re: [Cake] diffserv3 vs diffserv4

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 2721 bytes --]


This idea (dividing the link rate capacity, since "bandwidth" is an incorrect term not to be promulgated), is absurd, but typical of "bellhead" thinking.
 
Per packet latency is the key control variable, even for TCP. That's because capacity/rate is not controllable by routers, but by routing in a general Internet situation.
 
Latency is controlled by queuing delay in a packet network, not bitrate. And in mixed traffic, which after all is why traffic is classified in the first place, by its characteristics and response to increased latency end-to-end, is the core "control" for the internetwork as a whole.
 
So, by promoting thinking about "bandwidth" a whole sequence of misformulations of network management is embedded into the thinking of those designing queue management algorithms.
 
And make no mistake, queue management is the ONLY knob other than sending different packets on different routes that one has for routers.
 
I don't know who proposed this fractional division, but it is clearly a bellhead-influenced thinker who thinks all protocols are CBR flows like in the old phone system.
 
But almost no flows in the internet are CBR flows! File transfers are not, streaming TV is not, web ttraffic is not, game traffic is not. Only non-statistically multiplexed real-time telephony and *some* video conferencing is CBR.
 
Yet this bizarre idea of dividing "bandwidth" among all categories of flows pops up. Probably from employees of phone companies or phone equipment suppliers. Or folks who went to Uni and were trained in "communications" by former phone engineers.
 
Latency, latency, latency. Queue delay, queue delay, queue delay. Not link speed! Change your brains.
 
It's hard fo fight this bellhead crowd (or the bellheadedness in your own thinking) but think about packets and queues instead.
 
My good friend Len Kleinrock didn't invent "Bandwidth Theory"! He invented Queueing Theory. For a reason.
 
On Saturday, July 25, 2020 6:12am, "Kevin Darbyshire-Bryant" <kevin@darbyshire-bryant.me.uk> said:



> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
> 
> 
> > On 24 Jul 2020, at 18:42, Kevin Darbyshire-Bryant
> <kevin@darbyshire-bryant.me.uk> wrote:
> >
> >
> > The move from diffserv4 to diffserv5 WAS about de-prioritization.
> 
> It was also about minimum bandwidth allocations:
> 
> LE: 1/64th
> BK: 1/16th
> BE: 1/1
> VI: 1/2
> VO: 1/4
> 
> So worst case, best effort should get 11/64ths in the extreme case of all other
> tins in use.
> 
> Cheers,
> 
> Kevin D-B
> 
> gpg: 012C ACB2 28C6 C53E 9775 9123 B3A2 389B 9DE2 334A
> 
> 

[-- Attachment #2: Type: text/html, Size: 5438 bytes --]

Re: [Cake] diffserv3 vs diffserv4

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 3063 bytes --]

I didn’t sign up for this abuse.  Bellhead eh?  Well f**k off!

I’ve had enough - bye.

> On 25 Jul 2020, at 18:48, David P. Reed <dpreed@deepplum.com> wrote:
> 
> This idea (dividing the link rate capacity, since "bandwidth" is an incorrect term not to be promulgated), is absurd, but typical of "bellhead" thinking.
> 
> Per packet latency is the key control variable, even for TCP. That's because capacity/rate is not controllable by routers, but by routing in a general Internet situation.
> 
> Latency is controlled by queuing delay in a packet network, not bitrate. And in mixed traffic, which after all is why traffic is classified in the first place, by its characteristics and response to increased latency end-to-end, is the core "control" for the internetwork as a whole.
> 
> So, by promoting thinking about "bandwidth" a whole sequence of misformulations of network management is embedded into the thinking of those designing queue management algorithms.
> 
> And make no mistake, queue management is the ONLY knob other than sending different packets on different routes that one has for routers.
> 
> I don't know who proposed this fractional division, but it is clearly a bellhead-influenced thinker who thinks all protocols are CBR flows like in the old phone system.
> 
> But almost no flows in the internet are CBR flows! File transfers are not, streaming TV is not, web ttraffic is not, game traffic is not. Only non-statistically multiplexed real-time telephony and *some* video conferencing is CBR.
> 
> Yet this bizarre idea of dividing "bandwidth" among all categories of flows pops up. Probably from employees of phone companies or phone equipment suppliers. Or folks who went to Uni and were trained in "communications" by former phone engineers.
> 
> Latency, latency, latency. Queue delay, queue delay, queue delay. Not link speed! Change your brains.
> 
> It's hard fo fight this bellhead crowd (or the bellheadedness in your own thinking) but think about packets and queues instead.
> 
> My good friend Len Kleinrock didn't invent "Bandwidth Theory"! He invented Queueing Theory. For a reason.
> 
> On Saturday, July 25, 2020 6:12am, "Kevin Darbyshire-Bryant" <kevin@darbyshire-bryant.me.uk> said:
> 
> > _______________________________________________
> > Cake mailing list
> > Cake@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
> >
> >
> > > On 24 Jul 2020, at 18:42, Kevin Darbyshire-Bryant
> > <kevin@darbyshire-bryant.me.uk> wrote:
> > >
> > >
> > > The move from diffserv4 to diffserv5 WAS about de-prioritization.
> >
> > It was also about minimum bandwidth allocations:
> >
> > LE: 1/64th
> > BK: 1/16th
> > BE: 1/1
> > VI: 1/2
> > VO: 1/4
> >
> > So worst case, best effort should get 11/64ths in the extreme case of all other
> > tins in use.
> >
> > Cheers,
> >
> > Kevin D-B
> >
> > gpg: 012C ACB2 28C6 C53E 9775 9123 B3A2 389B 9DE2 334A
> >
> >


Cheers,

Kevin D-B

gpg: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Cake] diffserv3 vs diffserv4

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 4724 bytes --]


I want to apologize for any implication that you, Mr. Darbyshire-Bryant, were a "bellhead". AFAIK, you were quoting a staement from the designers of diffserv4, who apparently still believe in bandwidth division as a metric.
 
But I understand it might be painful to hear my critique of the diffserv design process.
 
Just be aware that it's my problem, not yours. I don't mean to offend you. I do, however, feel like the folks who did "design" diffserv (and continue to promote it) completely miss the whole point of why the Internet is architected the way it is. And since they haven't managed to respond to a clue-by-4 yet, I'm tired of just pointing out that the idea doesn't actually achieve any benefits, because no one (literally no one) has evern done a consistent assignment of end-to-end meaning to the various diffserv labels after decades of failed testing.
 
Since this is a group discussion, and not just a response to you, my comment was aimed at the general group (which is not dedicated to bellhead thinking, thank goodness).
 
And to be clear, AQM (cake, being an example) is not about bandwidth allocation. It does focus on latency/queueing-delay, for the most part.
 
Hence my concern that diffserv's fundamental misunderstanding of the responsibility of router queue management might contaminate a very, very important project.
 
 
On Saturday, July 25, 2020 1:54pm, "Kevin Darbyshire-Bryant" <kevin@darbyshire-bryant.me.uk> said:



> I didn’t sign up for this abuse. Bellhead eh? Well f**k off!
> 
> I’ve had enough - bye.
> 
> > On 25 Jul 2020, at 18:48, David P. Reed <dpreed@deepplum.com> wrote:
> >
> > This idea (dividing the link rate capacity, since "bandwidth" is an incorrect
> term not to be promulgated), is absurd, but typical of "bellhead" thinking.
> >
> > Per packet latency is the key control variable, even for TCP. That's because
> capacity/rate is not controllable by routers, but by routing in a general Internet
> situation.
> >
> > Latency is controlled by queuing delay in a packet network, not bitrate. And
> in mixed traffic, which after all is why traffic is classified in the first place,
> by its characteristics and response to increased latency end-to-end, is the core
> "control" for the internetwork as a whole.
> >
> > So, by promoting thinking about "bandwidth" a whole sequence of
> misformulations of network management is embedded into the thinking of those
> designing queue management algorithms.
> >
> > And make no mistake, queue management is the ONLY knob other than sending
> different packets on different routes that one has for routers.
> >
> > I don't know who proposed this fractional division, but it is clearly a
> bellhead-influenced thinker who thinks all protocols are CBR flows like in the old
> phone system.
> >
> > But almost no flows in the internet are CBR flows! File transfers are not,
> streaming TV is not, web ttraffic is not, game traffic is not. Only
> non-statistically multiplexed real-time telephony and *some* video conferencing is
> CBR.
> >
> > Yet this bizarre idea of dividing "bandwidth" among all categories of flows
> pops up. Probably from employees of phone companies or phone equipment suppliers.
> Or folks who went to Uni and were trained in "communications" by former phone
> engineers.
> >
> > Latency, latency, latency. Queue delay, queue delay, queue delay. Not link
> speed! Change your brains.
> >
> > It's hard fo fight this bellhead crowd (or the bellheadedness in your own
> thinking) but think about packets and queues instead.
> >
> > My good friend Len Kleinrock didn't invent "Bandwidth Theory"! He invented
> Queueing Theory. For a reason.
> >
> > On Saturday, July 25, 2020 6:12am, "Kevin Darbyshire-Bryant"
> <kevin@darbyshire-bryant.me.uk> said:
> >
> > > _______________________________________________
> > > Cake mailing list
> > > Cake@lists.bufferbloat.net
> > > https://lists.bufferbloat.net/listinfo/cake
> > >
> > >
> > > > On 24 Jul 2020, at 18:42, Kevin Darbyshire-Bryant
> > > <kevin@darbyshire-bryant.me.uk> wrote:
> > > >
> > > >
> > > > The move from diffserv4 to diffserv5 WAS about de-prioritization.
> > >
> > > It was also about minimum bandwidth allocations:
> > >
> > > LE: 1/64th
> > > BK: 1/16th
> > > BE: 1/1
> > > VI: 1/2
> > > VO: 1/4
> > >
> > > So worst case, best effort should get 11/64ths in the extreme case of
> all other
> > > tins in use.
> > >
> > > Cheers,
> > >
> > > Kevin D-B
> > >
> > > gpg: 012C ACB2 28C6 C53E 9775 9123 B3A2 389B 9DE2 334A
> > >
> > >
> 
> 
> Cheers,
> 
> Kevin D-B
> 
> gpg: 012C ACB2 28C6 C53E 9775 9123 B3A2 389B 9DE2 334A
> 
> 

[-- Attachment #2: Type: text/html, Size: 7236 bytes --]

Re: [Cake] diffserv3 vs diffserv4

[Sebastian Moeller]

Hi David,

I believe that folks here, certainly Kevin, have accepted the domain specificity of diffserv and we are mainly discussion, how many tiers of differential latency tolerance we desire ;). It goes without saying that this is all within the "home domain", and the goal is how little/few priority games we need to play for decent performance under load. Sure, end2end signaling would be desirable, but as you point out does not exist in diffserve today and is also not very likely to appear anytime soon, but in one's home it is still relatively easy to identify a few special cases, like bit-torrent (don't get in the way of "real" traffic) or VoIP (quite latency and especially jitter averse, but also typically of modest rate) that could/should be taken care of. 
	As far as I can tell that is all the cake/sqm's diffserve modes try to accomplish. DSCPs are simply used, as there exists machinery for routers/end-hosts OS to selectively set/re-set them and all IP packets will carry them. 
	Regarding the Bell-haededness, sure I might qualify for that moniker/abuse*, but the relevant factor, IMHO, is not so much the exact rate cut-offs of the different priority tiers, but the simple realization that low latency via prioritization requires relatively low rates, otherwise "priority" traffic will self congest, so these thresholds serve to establish a "cost" for using each priority tier. 

Best Regards
	Sebastian

*) By virtue of intellectual laziness, it is simply often easier for me to think in rate shares than the alternatives. But hey, I do this as a hobby, so I cut myself a lot of slack here ;) But I take no offense in being labeled that.


> On Jul 25, 2020, at 21:35, David P. Reed <dpreed@deepplum.com> wrote:
> 
> I want to apologize for any implication that you, Mr. Darbyshire-Bryant, were a "bellhead". AFAIK, you were quoting a staement from the designers of diffserv4, who apparently still believe in bandwidth division as a metric.
>  
> But I understand it might be painful to hear my critique of the diffserv design process.
>  
> Just be aware that it's my problem, not yours. I don't mean to offend you. I do, however, feel like the folks who did "design" diffserv (and continue to promote it) completely miss the whole point of why the Internet is architected the way it is. And since they haven't managed to respond to a clue-by-4 yet, I'm tired of just pointing out that the idea doesn't actually achieve any benefits, because no one (literally no one) has evern done a consistent assignment of end-to-end meaning to the various diffserv labels after decades of failed testing.
>  
> Since this is a group discussion, and not just a response to you, my comment was aimed at the general group (which is not dedicated to bellhead thinking, thank goodness).
>  
> And to be clear, AQM (cake, being an example) is not about bandwidth allocation. It does focus on latency/queueing-delay, for the most part.
>  
> Hence my concern that diffserv's fundamental misunderstanding of the responsibility of router queue management might contaminate a very, very important project.
>  
>  
> On Saturday, July 25, 2020 1:54pm, "Kevin Darbyshire-Bryant" <kevin@darbyshire-bryant.me.uk> said:
> 
> > I didn’t sign up for this abuse. Bellhead eh? Well f**k off!
> > 
> > I’ve had enough - bye.
> > 
> > > On 25 Jul 2020, at 18:48, David P. Reed <dpreed@deepplum.com> wrote:
> > >
> > > This idea (dividing the link rate capacity, since "bandwidth" is an incorrect
> > term not to be promulgated), is absurd, but typical of "bellhead" thinking.
> > >
> > > Per packet latency is the key control variable, even for TCP. That's because
> > capacity/rate is not controllable by routers, but by routing in a general Internet
> > situation.
> > >
> > > Latency is controlled by queuing delay in a packet network, not bitrate. And
> > in mixed traffic, which after all is why traffic is classified in the first place,
> > by its characteristics and response to increased latency end-to-end, is the core
> > "control" for the internetwork as a whole.
> > >
> > > So, by promoting thinking about "bandwidth" a whole sequence of
> > misformulations of network management is embedded into the thinking of those
> > designing queue management algorithms.
> > >
> > > And make no mistake, queue management is the ONLY knob other than sending
> > different packets on different routes that one has for routers.
> > >
> > > I don't know who proposed this fractional division, but it is clearly a
> > bellhead-influenced thinker who thinks all protocols are CBR flows like in the old
> > phone system.
> > >
> > > But almost no flows in the internet are CBR flows! File transfers are not,
> > streaming TV is not, web ttraffic is not, game traffic is not. Only
> > non-statistically multiplexed real-time telephony and *some* video conferencing is
> > CBR.
> > >
> > > Yet this bizarre idea of dividing "bandwidth" among all categories of flows
> > pops up. Probably from employees of phone companies or phone equipment suppliers.
> > Or folks who went to Uni and were trained in "communications" by former phone
> > engineers.
> > >
> > > Latency, latency, latency. Queue delay, queue delay, queue delay. Not link
> > speed! Change your brains.
> > >
> > > It's hard fo fight this bellhead crowd (or the bellheadedness in your own
> > thinking) but think about packets and queues instead.
> > >
> > > My good friend Len Kleinrock didn't invent "Bandwidth Theory"! He invented
> > Queueing Theory. For a reason.
> > >
> > > On Saturday, July 25, 2020 6:12am, "Kevin Darbyshire-Bryant"
> > <kevin@darbyshire-bryant.me.uk> said:
> > >
> > > > _______________________________________________
> > > > Cake mailing list
> > > > Cake@lists.bufferbloat.net
> > > > https://lists.bufferbloat.net/listinfo/cake
> > > >
> > > >
> > > > > On 24 Jul 2020, at 18:42, Kevin Darbyshire-Bryant
> > > > <kevin@darbyshire-bryant.me.uk> wrote:
> > > > >
> > > > >
> > > > > The move from diffserv4 to diffserv5 WAS about de-prioritization.
> > > >
> > > > It was also about minimum bandwidth allocations:
> > > >
> > > > LE: 1/64th
> > > > BK: 1/16th
> > > > BE: 1/1
> > > > VI: 1/2
> > > > VO: 1/4
> > > >
> > > > So worst case, best effort should get 11/64ths in the extreme case of
> > all other
> > > > tins in use.
> > > >
> > > > Cheers,
> > > >
> > > > Kevin D-B
> > > >
> > > > gpg: 012C ACB2 28C6 C53E 9775 9123 B3A2 389B 9DE2 334A
> > > >
> > > >
> > 
> > 
> > Cheers,
> > 
> > Kevin D-B
> > 
> > gpg: 012C ACB2 28C6 C53E 9775 9123 B3A2 389B 9DE2 334A
> > 
> >
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] diffserv3 vs diffserv4

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 10297 bytes --]

Apologies.  I think this is a cultural divide and misunderstanding, certainly on my part.  Bellhead is uncomfortably close to bellend and in the context of ‘absurd’ I’m afraid I rather took your bellhead to be rather closer to my interpretation of bellend.  In essence I thought you were saying I was a bit of a cock, which I probably am but for reasons other than the ones mentioned.

Possibly getting back to subject.  CAKE is probably the fairest thing I know and worships the god of (low) latency.  When I first heard of cake many years ago, the promise of ‘full link’ and ‘low latency’ I couldn’t believe it.  I’ve subsequently learned it’s all about queues, effective queue management and picking the right packet at the right time to fill the right tx slot.  It’s all about latency.  It’s all about shooting/marking the right packets to signal correctly and keep latency of each flow under control.  I understand the phrase “It’s the latency, stupid”.

Cake is so fair across flows that cake offers deliberate unfairness features, things that bias that fairness, some obvious, some not.  The obvious one is using packet categorisation to choose priority levels.  Instead of the default ‘every packet is equal’ of besteffort, a choice of categorisation are available from ‘diffserv3’ (a 3 tier system) to ‘diffserv8’ (an 8 tier system) all designed to introduce an unfairness, ‘least important’ to ‘most important’.  The categories or tins in cake parlance also have bandwidth thresholds, representing minimum capacities for that tin in the presence of traffic in competing tins.

A less obvious but deliberate unfairness mechanism in cake is ‘host fairness’.  This counts the number of flows to/from each host and divides the bandwidth amongst the flows such that each host gets an even share. e.g. 1 host with 9 flows vs 1 host with 1 flow will end up with the 9 flow host getting 50% of bandwidth across those 9 flows whilst the 1 flow host will get 50% of bandwidth across 1 flow.  This prevents gaming of bandwidth allocation simply by starting more flows.

Going all the way back to the start of this thread which spoke about ‘more important to de-prioritise’ my domestic use case/problem is to support ‘bittorrent’ but at a lower importance than bulk (my backups) at a lower importance than best effort (browsing) at a lower importance than latency/jitter sensitive video at a lower importance than voip.  That’s 5 categories (LE, BK, BE, VI, VO), but it could easily be 4 (LE, BK, BE, VO) with ‘video’ lumped in with Best Effort as is done with diffserv3.

Cake’s tin bandwidth thresholds say ‘you’re allowed to have at least this much’ and in my diffserv5 implementation it’s 1/64th of configured bandwidth simply ‘cos it can’t really be zero.  In the absence of other traffic, Least Effort under CAKE will happily consume ALL the bandwidth, great, nothing more important..bittorrent you go right ahead.  But as soon as something more important comes along, well, that takes (within limits) priority.  I think this is called ’soft admission’ but not totally sure.

I apologies if I have incorrectly used bandwidth/capacity/rate/whatever but hopefully everyone will understand what I’m trying to say.

Kevin

> On 25 Jul 2020, at 21:04, Sebastian Moeller <moeller0@gmx.de> wrote:
> 
> Hi David,
> 
> I believe that folks here, certainly Kevin, have accepted the domain specificity of diffserv and we are mainly discussion, how many tiers of differential latency tolerance we desire ;). It goes without saying that this is all within the "home domain", and the goal is how little/few priority games we need to play for decent performance under load. Sure, end2end signaling would be desirable, but as you point out does not exist in diffserve today and is also not very likely to appear anytime soon, but in one's home it is still relatively easy to identify a few special cases, like bit-torrent (don't get in the way of "real" traffic) or VoIP (quite latency and especially jitter averse, but also typically of modest rate) that could/should be taken care of.
> 	As far as I can tell that is all the cake/sqm's diffserve modes try to accomplish. DSCPs are simply used, as there exists machinery for routers/end-hosts OS to selectively set/re-set them and all IP packets will carry them.
> 	Regarding the Bell-haededness, sure I might qualify for that moniker/abuse*, but the relevant factor, IMHO, is not so much the exact rate cut-offs of the different priority tiers, but the simple realization that low latency via prioritization requires relatively low rates, otherwise "priority" traffic will self congest, so these thresholds serve to establish a "cost" for using each priority tier.
> 
> Best Regards
> 	Sebastian
> 
> *) By virtue of intellectual laziness, it is simply often easier for me to think in rate shares than the alternatives. But hey, I do this as a hobby, so I cut myself a lot of slack here ;) But I take no offense in being labeled that.
> 
> 
>> On Jul 25, 2020, at 21:35, David P. Reed <dpreed@deepplum.com> wrote:
>> 
>> I want to apologize for any implication that you, Mr. Darbyshire-Bryant, were a "bellhead". AFAIK, you were quoting a staement from the designers of diffserv4, who apparently still believe in bandwidth division as a metric.
>> 
>> But I understand it might be painful to hear my critique of the diffserv design process.
>> 
>> Just be aware that it's my problem, not yours. I don't mean to offend you. I do, however, feel like the folks who did "design" diffserv (and continue to promote it) completely miss the whole point of why the Internet is architected the way it is. And since they haven't managed to respond to a clue-by-4 yet, I'm tired of just pointing out that the idea doesn't actually achieve any benefits, because no one (literally no one) has evern done a consistent assignment of end-to-end meaning to the various diffserv labels after decades of failed testing.
>> 
>> Since this is a group discussion, and not just a response to you, my comment was aimed at the general group (which is not dedicated to bellhead thinking, thank goodness).
>> 
>> And to be clear, AQM (cake, being an example) is not about bandwidth allocation. It does focus on latency/queueing-delay, for the most part.
>> 
>> Hence my concern that diffserv's fundamental misunderstanding of the responsibility of router queue management might contaminate a very, very important project.
>> 
>> 
>> On Saturday, July 25, 2020 1:54pm, "Kevin Darbyshire-Bryant" <kevin@darbyshire-bryant.me.uk> said:
>> 
>>> I didn’t sign up for this abuse. Bellhead eh? Well f**k off!
>>> 
>>> I’ve had enough - bye.
>>> 
>>>> On 25 Jul 2020, at 18:48, David P. Reed <dpreed@deepplum.com> wrote:
>>>> 
>>>> This idea (dividing the link rate capacity, since "bandwidth" is an incorrect
>>> term not to be promulgated), is absurd, but typical of "bellhead" thinking.
>>>> 
>>>> Per packet latency is the key control variable, even for TCP. That's because
>>> capacity/rate is not controllable by routers, but by routing in a general Internet
>>> situation.
>>>> 
>>>> Latency is controlled by queuing delay in a packet network, not bitrate. And
>>> in mixed traffic, which after all is why traffic is classified in the first place,
>>> by its characteristics and response to increased latency end-to-end, is the core
>>> "control" for the internetwork as a whole.
>>>> 
>>>> So, by promoting thinking about "bandwidth" a whole sequence of
>>> misformulations of network management is embedded into the thinking of those
>>> designing queue management algorithms.
>>>> 
>>>> And make no mistake, queue management is the ONLY knob other than sending
>>> different packets on different routes that one has for routers.
>>>> 
>>>> I don't know who proposed this fractional division, but it is clearly a
>>> bellhead-influenced thinker who thinks all protocols are CBR flows like in the old
>>> phone system.
>>>> 
>>>> But almost no flows in the internet are CBR flows! File transfers are not,
>>> streaming TV is not, web ttraffic is not, game traffic is not. Only
>>> non-statistically multiplexed real-time telephony and *some* video conferencing is
>>> CBR.
>>>> 
>>>> Yet this bizarre idea of dividing "bandwidth" among all categories of flows
>>> pops up. Probably from employees of phone companies or phone equipment suppliers.
>>> Or folks who went to Uni and were trained in "communications" by former phone
>>> engineers.
>>>> 
>>>> Latency, latency, latency. Queue delay, queue delay, queue delay. Not link
>>> speed! Change your brains.
>>>> 
>>>> It's hard fo fight this bellhead crowd (or the bellheadedness in your own
>>> thinking) but think about packets and queues instead.
>>>> 
>>>> My good friend Len Kleinrock didn't invent "Bandwidth Theory"! He invented
>>> Queueing Theory. For a reason.
>>>> 
>>>> On Saturday, July 25, 2020 6:12am, "Kevin Darbyshire-Bryant"
>>> <kevin@darbyshire-bryant.me.uk> said:
>>>> 
>>>>> _______________________________________________
>>>>> Cake mailing list
>>>>> Cake@lists.bufferbloat.net
>>>>> https://lists.bufferbloat.net/listinfo/cake
>>>>> 
>>>>> 
>>>>>> On 24 Jul 2020, at 18:42, Kevin Darbyshire-Bryant
>>>>> <kevin@darbyshire-bryant.me.uk> wrote:
>>>>>> 
>>>>>> 
>>>>>> The move from diffserv4 to diffserv5 WAS about de-prioritization.
>>>>> 
>>>>> It was also about minimum bandwidth allocations:
>>>>> 
>>>>> LE: 1/64th
>>>>> BK: 1/16th
>>>>> BE: 1/1
>>>>> VI: 1/2
>>>>> VO: 1/4
>>>>> 
>>>>> So worst case, best effort should get 11/64ths in the extreme case of
>>> all other
>>>>> tins in use.
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Kevin D-B
>>>>> 
>>>>> gpg: 012C ACB2 28C6 C53E 9775 9123 B3A2 389B 9DE2 334A
>>>>> 
>>>>> 
>>> 
>>> 
>>> Cheers,
>>> 
>>> Kevin D-B
>>> 
>>> gpg: 012C ACB2 28C6 C53E 9775 9123 B3A2 389B 9DE2 334A
>>> 
>>> 
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
> 


Cheers,

Kevin D-B

gpg: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Cake] diffserv3 vs diffserv4

[Jonathan Morton]

> On 25 Jul, 2020, at 10:35 pm, David P. Reed <dpreed@deepplum.com> wrote:
> 
> And to be clear, AQM (cake, being an example) is not about bandwidth allocation. It does focus on latency/queueing-delay, for the most part.

Cake is not *just* an AQM, though I understand your point.  It is a qdisc with many interwoven functions.

Cake's Diffserv support is neither a pure priority scheme nor a pure bandwidth allocation.  By using a hybrid of the two for bandwidth allocation, I was hoping to avoid the main pitfalls that the simple Bell-headed approaches routinely encounter.  Each tin also has its own AQM parameters, which feed into the distinction between high-throughput and low-latency classes of traffic.

There are doubtless other approaches that could be tried, of course.  And there might be endless debate over exactly how many traffic classes are actually needed; I don't think five is the right number, and the symmetry argument is not persuasive.  But can we at least agree that Cake's attempt is a step in the right direction?

 - Jonathan Morton

Re: [Cake] diffserv3 vs diffserv4

[Jonathan Morton]

> On 24 Jul, 2020, at 6:56 pm, Justin Kilpatrick <justin@althea.net> wrote:
> 
> "sqm-scripts used 3 tiers of priority pretty successfully as does free.fr. - de-prioritization seems a good idea, prioritization not so much."
> 
> This is the best comment on why diffserv3 is the default that I could find on bufferbloat.net. I'm interested in hearing about what data (anecdotes welcome) lead to this conclusion.

In Cake, Diffserv4 maps conceptually (but not in detail) to the four priority buckets in Wifi - BK, BE, VI, VO.  In Diffserv3 the VI bucket is dropped, because Cake's flow isolation within BE is already good enough to give decent video streaming performance.  The BK and VO buckets are still useful to deal with certain specific problems; BK is the place to put "swarm" protocols which intend to be scavengers but which flow-isolation tends to prioritise, and VO is for latency-sensitive protocols which the user wants to specifically protect from random traffic fluctuations.

Thinking more broadly, I believe Diffserv would have been far more successful if it had replaced Precedence/TOS with a simple two-bit, four-way set of PHBs:

00: High Throughput - equivalent to traditional Best Effort service.

01: High Reliability - "Every Packet's Sacred".

10: Low Cost - a scavenging service for altruistic applications.

11: Low Latency - for the many protocols that are sensitive to delays more than throughput.

It may also have been reasonable to include a couple of extra bits for uses internal to an AS, on the understanding that the basic two bits would be preserved end-to-end as an indication of application intent.

Of the above four classes, Diffserv3 provides three - omitting only the High Reliability class.  But that is a class most useful within a datacentre, where it is actually practical to implement a lossless backplane with backpressure signals instead of loss.

What we *actually* have is a six-bit field with ill-defined semantics, that is neither preserved nor respected end-to-end, is consequently ignored by most applications, and consumes all the space in the former TOS byte that is not specifically set aside for ECN (a field which could profitably have been larger).  It is a serious problem.

Implementations of PHBs still tend to think in terms of bandwidth reservation (a Bell-head paradigm) and/or strict priority (like the Precedence system which was lifted directly from telegraphy practice).  Both approaches are inefficient, and go along with the misconception that if we can merely categorise traffic on the fly into a large enough number of pigeonholes, some magical method of dealing with the pigeonholes will make itself obvious.  However, both the easy, universal method of categorisation and the magical delivery strategy have failed to materialise.  It rather suggests that they're doing it wrong.

So that is why Diffserv3 is the default in Cake.  It offers explicit "low cost" and "low latency" service for suitably marked traffic, and for everything else the Best Effort service uses flow and host isolation strategies to maintain good behaviour.  It usually works well.

 - Jonathan Morton

Re: [Cake] diffserv3 vs diffserv4

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 4998 bytes --]


+1000%
 
I believe the problem with diffserv arose at conception, because it violated the core idea of IETF's operation:
 
"rough consensus and working code"
 
It was clear very, very early( to everyone but those working on it!) that no working approximate implementation ever existed, nor could it!
 
Had someone proposed a single better-efforts category, whose implementation would be Autonomous System by Autonomous System defined by a scheme roughly equivalent to "Paris Metro Pricing", it would have afforded experience at scale. (In Paris Metro Pricing, there are two knds of cars on each train, First Class and Second Class. If you pay for first class, you get to go into the first class cars. Cars change from second to first class iff the seats in first class are tending to be full. Trains run more often when there are lines waiting for second class cars. The analogy with router decisions is should be clear, except since trains can't run more often, congestion is signaled by drop or marking, which means that second class packets would be dropped or marked unless there were no first class packets.)
 
But instead the designers ignored implementation entirely, and invented "wish-based" classes.
 
This also violated an end-to-end argument - you should only put "in the network" functions that can be completely *implemented* "within the network".
And the TOS/QOS idea isn't meaningful to routers.
 
On Friday, July 24, 2020 11:13pm, "Jonathan Morton" <chromatix99@gmail.com> said:



> > On 24 Jul, 2020, at 6:56 pm, Justin Kilpatrick <justin@althea.net>
> wrote:
> >
> > "sqm-scripts used 3 tiers of priority pretty successfully as does free.fr. -
> de-prioritization seems a good idea, prioritization not so much."
> >
> > This is the best comment on why diffserv3 is the default that I could find on
> bufferbloat.net. I'm interested in hearing about what data (anecdotes welcome)
> lead to this conclusion.
> 
> In Cake, Diffserv4 maps conceptually (but not in detail) to the four priority
> buckets in Wifi - BK, BE, VI, VO. In Diffserv3 the VI bucket is dropped, because
> Cake's flow isolation within BE is already good enough to give decent video
> streaming performance. The BK and VO buckets are still useful to deal with
> certain specific problems; BK is the place to put "swarm" protocols which intend
> to be scavengers but which flow-isolation tends to prioritise, and VO is for
> latency-sensitive protocols which the user wants to specifically protect from
> random traffic fluctuations.
> 
> Thinking more broadly, I believe Diffserv would have been far more successful if
> it had replaced Precedence/TOS with a simple two-bit, four-way set of PHBs:
> 
> 00: High Throughput - equivalent to traditional Best Effort service.
> 
> 01: High Reliability - "Every Packet's Sacred".
> 
> 10: Low Cost - a scavenging service for altruistic applications.
> 
> 11: Low Latency - for the many protocols that are sensitive to delays more than
> throughput.
> 
> It may also have been reasonable to include a couple of extra bits for uses
> internal to an AS, on the understanding that the basic two bits would be preserved
> end-to-end as an indication of application intent.
> 
> Of the above four classes, Diffserv3 provides three - omitting only the High
> Reliability class. But that is a class most useful within a datacentre, where it
> is actually practical to implement a lossless backplane with backpressure signals
> instead of loss.
> 
> What we *actually* have is a six-bit field with ill-defined semantics, that is
> neither preserved nor respected end-to-end, is consequently ignored by most
> applications, and consumes all the space in the former TOS byte that is not
> specifically set aside for ECN (a field which could profitably have been larger). 
> It is a serious problem.
> 
> Implementations of PHBs still tend to think in terms of bandwidth reservation (a
> Bell-head paradigm) and/or strict priority (like the Precedence system which was
> lifted directly from telegraphy practice). Both approaches are inefficient, and
> go along with the misconception that if we can merely categorise traffic on the
> fly into a large enough number of pigeonholes, some magical method of dealing with
> the pigeonholes will make itself obvious. However, both the easy, universal
> method of categorisation and the magical delivery strategy have failed to
> materialise. It rather suggests that they're doing it wrong.
> 
> So that is why Diffserv3 is the default in Cake. It offers explicit "low cost"
> and "low latency" service for suitably marked traffic, and for everything else the
> Best Effort service uses flow and host isolation strategies to maintain good
> behaviour. It usually works well.
> 
> - Jonathan Morton
> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
> 

[-- Attachment #2: Type: text/html, Size: 7271 bytes --]