[Cerowrt-devel] SQM: tracking some diffserv related internet drafts better

[Cerowrt-devel] SQM: tracking some diffserv related internet drafts better

[Dave Taht]

This appears to be close to finalization, or finalized:

http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10

And this is complementary:

http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03

While wading through all this is tedious, and much of the advice contradictory,
there are a few things that could be done more right in the sqm system
that I'd like to discuss. (feel free to pour a cup of coffee and read
the drafts)

-1) They still think the old style tos imm bit is obsolete. Sigh. Am I
the last person that uses ssh or plays games?

0) Key to this draft is expecting that the AF code points on a single
5-tuple not be re-ordered, which means dumping AF41 into a priority
queue and AF42 into the BE queue is incorrect.

1) SQM only prioritizes a few diffserv codepoints (just the ones for
which I had tools doing classification, like ssh). Doing so with tc
rules is very inefficient presently. I had basically planned on
rolling a new tc and/or iptables filter to "do the right thing" to map
into all 64 codepoints via a simple lookup table (as what is in the
wifi code already), rather than use the existing mechanism... and
hesitated
as nobody had nailed down the definitions of each one.

That said, I have not measured recently the impact of the extra tc
filters and iptables rules required.

1a) Certainly only doing AF42 in sqm is pretty wrong (that was left
over from my test patches against mosh - mosh ran with AF42 for a
while until they crashed a couple routers with it)

The relevant lines are here:

https://github.com/dtaht/ceropackages-3.10/blob/master/net/sqm-scripts/files/usr/lib/sqm/functions.sh#L411

1b) The cake code presently does it pretty wrong, which is eminately fixable.

1c) And given that the standards are settling, it might be time to
start baking them into a new tc or iptables filter. This would be a
small, interesting project for someone who wants to get their feet wet
writing this sort of thing, and examples abound of how to do it.

2) A lot of these diffserv specs - notably all the AFxx codepoints -
are all about variable drop probability. (Not that this concept has
been proven to work in the real world) We don't do variable drop
probability... and I haven't the slightest clue as to how to do it in
fq_codel. But keeping variable diffserv codepoints in order on the
same 5 tuple seems to be the way things are going. Still I have
trouble folding these ideas into the 3 basic queue system fq_codel
uses, it looks to me as most of the AF codepoints end up in the
current best effort queue, as the priority queue is limited to 30% of
the bandwidth by default.


3) Squashing inbound dscp should still be the default option...

4) My patch set to the wifi code for diffserv support disables the VO
queue almost entirely in favor of punting things to the VI queue
(which can aggregate), but I'm not sure if I handled AFxx
appropriately.

5) So far as I know, no browser implements any of this stuff yet. So
far as I know nobody actually deployed a router that tries to do smart
things with this stuff yet.

6) I really wish there were more codepoints for background traffic than cs1.

-- 
Dave Täht

thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks

Re: [Cerowrt-devel] tracking some diffserv related internet drafts better

[Black, David]

Dave (T),

> This appears to be close to finalization, or finalized:
> 
> http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
> 
> And this is complementary:
> 
> http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03

The dart-dscp-rtp draft is now done and in the RFC Editor's Queue.

The tsvwg-rtcweb-qos draft is still a work in progress, and its
aggressive use of almost every PHB and DSCP is a concern.  Here's
another related draft that is likely to be adopted by the tsvwg WG soon:

https://datatracker.ietf.org/doc/draft-geib-tsvwg-diffserv-intercon/


> -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
> the last person that uses ssh or plays games?

See RFC 2474, published in 1998 ...

> 0) Key to this draft is expecting that the AF code points on a single
> 5-tuple not be re-ordered, which means dumping AF41 into a priority
> queue and AF42 into the BE queue is incorrect.

Yes.  That's supposed to be the case for any AF PHB group that's
supported., e.g., AF3x [AF31, AF32 and AF33].

https://datatracker.ietf.org/doc/draft-geib-tsvwg-diffserv-intercon/

> 1c) And given that the standards are settling, it might be time to
> start baking them into a new tc or iptables filter. This would be a
> small, interesting project for someone who wants to get their feet wet
> writing this sort of thing, and examples abound of how to do it.

I'm happy to help w/insight, advice etc. - please ask.  In turn,
experience w/what's more vs. less likely to work where and why would
be quite useful, e.g., for the tsvwg-rtcweb-qos draft.

> 2) A lot of these diffserv specs - notably all the AFxx codepoints -
> are all about variable drop probability. (Not that this concept has
> been proven to work in the real world) We don't do variable drop
> probability... and I haven't the slightest clue as to how to do it in
> fq_codel. But keeping variable diffserv codepoints in order on the
> same 5 tuple seems to be the way things are going.

That's UDP-only at the moment if that helps.  Variable drop probability
doesn't do anything useful for TCP, and there's no good experience
with other congestion controlled transports - see the dart-dscp-rtp
draft.

> 3) Squashing inbound dscp should still be the default option...

Yes, very common out there.  See the tsvwg-diffserv-intercon draft
which is trying to specify a few small holes to punch into that
"squash to zero" default ingress DSCP behavior.  Yes, I'm an
optimist ...

> 6) I really wish there were more codepoints for background traffic than cs1.

It's worse than that - CS1 may or may not be a background codepoint,
see the dart-dscp-rtp draft.

Thanks,
--David

> -----Original Message-----
> From: Dave Taht [mailto:dave.taht@gmail.com]
> Sent: Thursday, November 13, 2014 12:27 PM
> To: cerowrt-devel@lists.bufferbloat.net; Black, David
> Subject: SQM: tracking some diffserv related internet drafts better
> 
> This appears to be close to finalization, or finalized:
> 
> http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
> 
> And this is complementary:
> 
> http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03
> 
> While wading through all this is tedious, and much of the advice
> contradictory,
> there are a few things that could be done more right in the sqm system
> that I'd like to discuss. (feel free to pour a cup of coffee and read
> the drafts)
> 
> -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
> the last person that uses ssh or plays games?
> 
> 0) Key to this draft is expecting that the AF code points on a single
> 5-tuple not be re-ordered, which means dumping AF41 into a priority
> queue and AF42 into the BE queue is incorrect.
> 
> 1) SQM only prioritizes a few diffserv codepoints (just the ones for
> which I had tools doing classification, like ssh). Doing so with tc
> rules is very inefficient presently. I had basically planned on
> rolling a new tc and/or iptables filter to "do the right thing" to map
> into all 64 codepoints via a simple lookup table (as what is in the
> wifi code already), rather than use the existing mechanism... and
> hesitated
> as nobody had nailed down the definitions of each one.
> 
> That said, I have not measured recently the impact of the extra tc
> filters and iptables rules required.
> 
> 1a) Certainly only doing AF42 in sqm is pretty wrong (that was left
> over from my test patches against mosh - mosh ran with AF42 for a
> while until they crashed a couple routers with it)
> 
> The relevant lines are here:
> 
> https://github.com/dtaht/ceropackages-3.10/blob/master/net/sqm-
> scripts/files/usr/lib/sqm/functions.sh#L411
> 
> 1b) The cake code presently does it pretty wrong, which is eminately fixable.
> 
> 1c) And given that the standards are settling, it might be time to
> start baking them into a new tc or iptables filter. This would be a
> small, interesting project for someone who wants to get their feet wet
> writing this sort of thing, and examples abound of how to do it.
> 
> 2) A lot of these diffserv specs - notably all the AFxx codepoints -
> are all about variable drop probability. (Not that this concept has
> been proven to work in the real world) We don't do variable drop
> probability... and I haven't the slightest clue as to how to do it in
> fq_codel. But keeping variable diffserv codepoints in order on the
> same 5 tuple seems to be the way things are going. Still I have
> trouble folding these ideas into the 3 basic queue system fq_codel
> uses, it looks to me as most of the AF codepoints end up in the
> current best effort queue, as the priority queue is limited to 30% of
> the bandwidth by default.
> 
> 
> 3) Squashing inbound dscp should still be the default option...
> 
> 4) My patch set to the wifi code for diffserv support disables the VO
> queue almost entirely in favor of punting things to the VI queue
> (which can aggregate), but I'm not sure if I handled AFxx
> appropriately.
> 
> 5) So far as I know, no browser implements any of this stuff yet. So
> far as I know nobody actually deployed a router that tries to do smart
> things with this stuff yet.
> 
> 6) I really wish there were more codepoints for background traffic than cs1.
> 
> --
> Dave Täht
> 
> thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks

Re: [Cerowrt-devel] SQM: tracking some diffserv related internet drafts better

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 4493 bytes --]


The IETF used to require rough consensus and *working code*.  The latter seems to be out of fashion - especially with a zillion code points for which no working code has been produced, and worse yet, no real world testing has demonstrated any value whatsoever.
 
It's also true that almost no actual agreements exist at peering points about what to do at those points.  That's why diffserv appears to be a lot of energy wasted on something that has little to do with inter-networking.
 
"intserv" was a plausible idea, because within a vertically integrated system, one can enforce regularity and consistency.
 
So my view, for what it is worth, is that there are a zillion better things to put effort into than incorporating diffserv into CeroWRT!
 
Cui bono?
 


On Thursday, November 13, 2014 12:26pm, "Dave Taht" <dave.taht@gmail.com> said:



> This appears to be close to finalization, or finalized:
> 
> http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
> 
> And this is complementary:
> 
> http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03
> 
> While wading through all this is tedious, and much of the advice contradictory,
> there are a few things that could be done more right in the sqm system
> that I'd like to discuss. (feel free to pour a cup of coffee and read
> the drafts)
> 
> -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
> the last person that uses ssh or plays games?
> 
> 0) Key to this draft is expecting that the AF code points on a single
> 5-tuple not be re-ordered, which means dumping AF41 into a priority
> queue and AF42 into the BE queue is incorrect.
> 
> 1) SQM only prioritizes a few diffserv codepoints (just the ones for
> which I had tools doing classification, like ssh). Doing so with tc
> rules is very inefficient presently. I had basically planned on
> rolling a new tc and/or iptables filter to "do the right thing" to map
> into all 64 codepoints via a simple lookup table (as what is in the
> wifi code already), rather than use the existing mechanism... and
> hesitated
> as nobody had nailed down the definitions of each one.
> 
> That said, I have not measured recently the impact of the extra tc
> filters and iptables rules required.
> 
> 1a) Certainly only doing AF42 in sqm is pretty wrong (that was left
> over from my test patches against mosh - mosh ran with AF42 for a
> while until they crashed a couple routers with it)
> 
> The relevant lines are here:
> 
> https://github.com/dtaht/ceropackages-3.10/blob/master/net/sqm-scripts/files/usr/lib/sqm/functions.sh#L411
> 
> 1b) The cake code presently does it pretty wrong, which is eminately fixable.
> 
> 1c) And given that the standards are settling, it might be time to
> start baking them into a new tc or iptables filter. This would be a
> small, interesting project for someone who wants to get their feet wet
> writing this sort of thing, and examples abound of how to do it.
> 
> 2) A lot of these diffserv specs - notably all the AFxx codepoints -
> are all about variable drop probability. (Not that this concept has
> been proven to work in the real world) We don't do variable drop
> probability... and I haven't the slightest clue as to how to do it in
> fq_codel. But keeping variable diffserv codepoints in order on the
> same 5 tuple seems to be the way things are going. Still I have
> trouble folding these ideas into the 3 basic queue system fq_codel
> uses, it looks to me as most of the AF codepoints end up in the
> current best effort queue, as the priority queue is limited to 30% of
> the bandwidth by default.
> 
> 
> 3) Squashing inbound dscp should still be the default option...
> 
> 4) My patch set to the wifi code for diffserv support disables the VO
> queue almost entirely in favor of punting things to the VI queue
> (which can aggregate), but I'm not sure if I handled AFxx
> appropriately.
> 
> 5) So far as I know, no browser implements any of this stuff yet. So
> far as I know nobody actually deployed a router that tries to do smart
> things with this stuff yet.
> 
> 6) I really wish there were more codepoints for background traffic than cs1.
> 
> --
> Dave Täht
> 
> thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

[-- Attachment #2: Type: text/html, Size: 6392 bytes --]

Re: [Cerowrt-devel] SQM: tracking some diffserv related internet drafts better

[Sebastian Moeller]

Hi Dave,

I probably do not understand the topic fully, but...

On Nov 13, 2014, at 18:26 , Dave Taht <dave.taht@gmail.com> wrote:

> This appears to be close to finalization, or finalized:
> 
> http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
> 
> And this is complementary:
> 
> http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03

	Oha, that’s 15 priority levels (out of ~64 possible?) right there for a browser to mark packets with depending on media type. Now, not all need to map to real queues but that seems a lot, so that I would expect in real life a bunch of those will map to the same queues. 
	If I understand correctly we already have a problems getting  decent AQM implemented at core switching/routing equipment, how realistic is it to expect that these devices implement differential packet drop probabilities per diffserv markings? I f the answer is not realistic the last three DS bits become functionally equal… . Also CS1 for audio/video? I thought this to be the scavenger class and hence not suitable for anything but bulk background traffic if there is even the slightest contention on the path… (on second thought this will allow to turn a CS1 internet radio in a decent congestion monitor, if the audio skips you know the network is starting to develop issues…).

> 
> While wading through all this is tedious, and much of the advice contradictory,
> there are a few things that could be done more right in the sqm system
> that I'd like to discuss. (feel free to pour a cup of coffee and read
> the drafts)
> 
> -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
> the last person that uses ssh or plays games?

	Are we free in cerowrt/SQM to just ignore this and just keep imm (CS2?) above the best effort queue?

> 
> 0) Key to this draft is expecting that the AF code points on a single
> 5-tuple not be re-ordered, which means dumping AF41 into a priority
> queue and AF42 into the BE queue is incorrect.

	So what about sticking to the class selectors only in SQM? If I understand correctly we can match on the CS bits only and ignore the other bits; I think each AFNx map to the same CS(M) class… Looking at section "4.2.2.3  Using the Class Selector PHB Requirements for IP Precedence Compatibility” of http://tools.ietf.org/html/rfc2474#page-11 seems to confirm that interpretation…


> 
> 1) SQM only prioritizes a few diffserv codepoints (just the ones for
> which I had tools doing classification, like ssh). Doing so with tc
> rules is very inefficient presently. I had basically planned on
> rolling a new tc and/or iptables filter to "do the right thing" to map
> into all 64 codepoints via a simple lookup table (as what is in the
> wifi code already), rather than use the existing mechanism... and
> hesitated
> as nobody had nailed down the definitions of each one.

	Well, "tc filter” hurts us badly as I figured out implementing filters to look into PPP encapsulated packets to get to the TOS bits… But in theory all tests for the individual code points can be turned into a hawh operation in tc filter so that we only pay the price for each encapsulation type (IPv4 IPv6, IPv$ in PPP, IPv6 in PPP, to poke through the PPP layer costs a few additional ANDed match tests, but I really really hope that tc filter is smart enough to stop filter processing on the first mismatch…) On my TODO list for SQM is to use tc filter’s hash functionality to process all code points in one operation per packet. This should also allow/require mapping each of the 64 diffserve markings to our queues so that any “ietf-recommendation-of-the-day” can be a easily implemented by changing our mapping table...

> 
> That said, I have not measured recently the impact of the extra tc
> filters and iptables rules required.

	As far as I can tell tc filter is costly, in a “non-scientific” test with netperf-wrapper’s RRUL test I saw the ICMP-CDF “robust-range” (the delay span in which the CDF went from ~5% to 95%) incase from 10ms to 30ms. No idea about the iptables rules (well, the internet seems to argue iptables being much cheaper than tc filter).

> 
> 1a) Certainly only doing AF42 in sqm is pretty wrong (that was left
> over from my test patches against mosh - mosh ran with AF42 for a
> while until they crashed a couple routers with it)

	Why? We could just switch to stash all CS4 packets into this queue and be compliant again to the recommendation to treat packets in each AFN set equally?

> 
> The relevant lines are here:
> 
> https://github.com/dtaht/ceropackages-3.10/blob/master/net/sqm-scripts/files/usr/lib/sqm/functions.sh#L411
> 
> 1b) The cake code presently does it pretty wrong, which is eminately fixable.
> 
> 1c) And given that the standards are settling, it might be time to
> start baking them into a new tc or iptables filter. This would be a
> small, interesting project for someone who wants to get their feet wet
> writing this sort of thing, and examples abound of how to do it.

	So what I plan on doing until the end of the year is getting the hashed tc filter set up for SQM than implementing/testing different mappings will be a piece of cake just change 64 values and you are done...

> 
> 2) A lot of these diffserv specs - notably all the AFxx codepoints -
> are all about variable drop probability. (Not that this concept has
> been proven to work in the real world) We don't do variable drop
> probability... and I haven't the slightest clue as to how to do it in
> fq_codel. But keeping variable diffserv codepoints in order on the
> same 5 tuple seems to be the way things are going. Still I have
> trouble folding these ideas into the 3 basic queue system fq_codel
> uses, it looks to me as most of the AF codepoints end up in the
> current best effort queue, as the priority queue is limited to 30% of
> the bandwidth by default.

	Is this really relevant for the wider internet at all? As you argue below (and as is argued in the drafts cited above) each network can do what ever it likes with code points so the relevant question, as I see it is not what could we do with the code points if we had all 6 bits for us end to end, but rather how many and which bits actually survive a trip over the open internet ;)

> 
> 
> 3) Squashing inbound dscp should still be the default option…

My interpretation of http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10 section 3.2’s “ When DiffServ is used, the edge or boundary nodes of a network are responsible for ensuring that all traffic entering that network conforms to that network's policies for DSCP and PHB usage, and such nodes may change DSCP markings on traffic to achieve that result.” is anything goes including remapping to all zeros aka squashing. http://tools.ietf.org/html/rfc2474 talks about a MUST to put CS 6 and 7 into a higher priority class than CS0, but I really doubt that any ISP will allow me to label all my traffic CS7 an will treat it accordingly, so remapping to zero is okay if not by standard then by cause of reality ;). 




> 
> 4) My patch set to the wifi code for diffserv support disables the VO
> queue almost entirely in favor of punting things to the VI queue
> (which can aggregate), but I'm not sure if I handled AFxx
> appropriately.
> 
> 5) So far as I know, no browser implements any of this stuff yet. So
> far as I know nobody actually deployed a router that tries to do smart
> things with this stuff yet.

	I would love to know whether the proposed markings actually survive a trip through the open internet at all. I would like to argue that until that actually happens this a nicely academic discussion (cerowrt does a fine job already with its nice fq_codel hierarchy, and if all the new fancy stuff will be wiped directly by my ISP I am not sure that implementing the proposal in sam is going to change anything, especially nothing that can be measured. As they say, “measurement data or it did not happen” ;) )

> 
> 6) I really wish there were more codepoints for background traffic than cs1.

	But isn’t that what AF1x is all about?. I agree the range of the 6 DS bits is not used to its fullest extend: rather than bits treat is as a number and do: current priority = (DS - 32) so we have a range from -32 to 31 (or so) and simply require that higher values are not treated with less priority than smaller numbers. (Heck maybe special case CS0 to also mean zero for backward/reality compatibility ;) )
	But most likely I just have misunderstood the whole issue….

Best Regards
	Sebastian



> 
> -- 
> Dave Täht
> 
> thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] SQM: tracking some diffserv related internet drafts better

[Sebastian Moeller]

Hi David,

On Nov 14, 2014, at 03:41 , dpreed@reed.com wrote:

> The IETF used to require rough consensus and *working code*.  The latter seems to be out of fashion - especially with a zillion code points for which no working code has been produced, and worse yet, no real world testing has demonstrated any value whatsoever.
>  
> It's also true that almost no actual agreements exist at peering points about what to do at those points.  That's why diffserv appears to be a lot of energy wasted on something that has little to do with inter-networking.
>  
> "intserv" was a plausible idea, because within a vertically integrated system, one can enforce regularity and consistency.
>  
> So my view, for what it is worth, is that there are a zillion better things to put effort into than incorporating diffserv into CeroWRT!

	Well for cerowrt’s egress queue it is nice to be able to select the priority by modifying the DS bits of the packet as some applications can do this directly; the alternative (bear with me here I am a layman) would be deep packet inspection at the router with the problem that even without that the wndrs are having a hard time routing current link speeds even without needing to perform DPI plus I have no idea how to DPI an encrypted connection. So I think that the use of the DS bits inside cerowrt is justified for cero’s core job as a home router. Whether implementing the full DS set is needed of whether the 15 DS bits per browser will help anything is a different question, but sincere we want to handle the bits already it should not be too much work. (For the current SQM system we need to switch DS to queue mapping from independent tc filter invocations to a hashed filter as we run to many filters already and then the only additional work will be mapping 64 codes to 3 queues instead of just 8 or so ;) )

>  
> Cui bono?
>  
> 
> 
> On Thursday, November 13, 2014 12:26pm, "Dave Taht" <dave.taht@gmail.com> said:
> 
> > This appears to be close to finalization, or finalized:
> > 
> > http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
> > 
> > And this is complementary:
> > 
> > http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03
> > 
> > While wading through all this is tedious, and much of the advice contradictory,
> > there are a few things that could be done more right in the sqm system
> > that I'd like to discuss. (feel free to pour a cup of coffee and read
> > the drafts)
> > 
> > -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
> > the last person that uses ssh or plays games?
> > 
> > 0) Key to this draft is expecting that the AF code points on a single
> > 5-tuple not be re-ordered, which means dumping AF41 into a priority
> > queue and AF42 into the BE queue is incorrect.
> > 
> > 1) SQM only prioritizes a few diffserv codepoints (just the ones for
> > which I had tools doing classification, like ssh). Doing so with tc
> > rules is very inefficient presently. I had basically planned on
> > rolling a new tc and/or iptables filter to "do the right thing" to map
> > into all 64 codepoints via a simple lookup table (as what is in the
> > wifi code already), rather than use the existing mechanism... and
> > hesitated
> > as nobody had nailed down the definitions of each one.
> > 
> > That said, I have not measured recently the impact of the extra tc
> > filters and iptables rules required.
> > 
> > 1a) Certainly only doing AF42 in sqm is pretty wrong (that was left
> > over from my test patches against mosh - mosh ran with AF42 for a
> > while until they crashed a couple routers with it)
> > 
> > The relevant lines are here:
> > 
> > https://github.com/dtaht/ceropackages-3.10/blob/master/net/sqm-scripts/files/usr/lib/sqm/functions.sh#L411
> > 
> > 1b) The cake code presently does it pretty wrong, which is eminately fixable.
> > 
> > 1c) And given that the standards are settling, it might be time to
> > start baking them into a new tc or iptables filter. This would be a
> > small, interesting project for someone who wants to get their feet wet
> > writing this sort of thing, and examples abound of how to do it.
> > 
> > 2) A lot of these diffserv specs - notably all the AFxx codepoints -
> > are all about variable drop probability. (Not that this concept has
> > been proven to work in the real world) We don't do variable drop
> > probability... and I haven't the slightest clue as to how to do it in
> > fq_codel. But keeping variable diffserv codepoints in order on the
> > same 5 tuple seems to be the way things are going. Still I have
> > trouble folding these ideas into the 3 basic queue system fq_codel
> > uses, it looks to me as most of the AF codepoints end up in the
> > current best effort queue, as the priority queue is limited to 30% of
> > the bandwidth by default.
> > 
> > 
> > 3) Squashing inbound dscp should still be the default option...
> > 
> > 4) My patch set to the wifi code for diffserv support disables the VO
> > queue almost entirely in favor of punting things to the VI queue
> > (which can aggregate), but I'm not sure if I handled AFxx
> > appropriately.
> > 
> > 5) So far as I know, no browser implements any of this stuff yet. So
> > far as I know nobody actually deployed a router that tries to do smart
> > things with this stuff yet.
> > 
> > 6) I really wish there were more codepoints for background traffic than cs1.
> > 
> > --
> > Dave Täht
> > 
> > thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
> >
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] SQM: tracking some diffserv related internet drafts better

[Black, David]

Replying to Sebastian's shorter message first ;-).

> 	Well for cerowrt's egress queue it is nice to be able to select the
> priority by modifying the DS bits of the packet as some applications can do
> this directly; the alternative (bear with me here I am a layman) would be deep
> packet inspection at the router with the problem that even without that the
> wndrs are having a hard time routing current link speeds even without needing
> to perform DPI plus I have no idea how to DPI an encrypted connection. So I
> think that the use of the DS bits inside cerowrt is justified for cero's core
> job as a home router.

+1 - while that sort of DPI is done today (although I suspect that specialized
middleboxes are more common than router integration), the encryption backlash
to use and abuse of DPI (and other technologies) is only just beginning ...

... and please keep cc:'ing me, as I'm not on the cerowrt-devel list ...

Thanks,
--David

> -----Original Message-----
> From: Sebastian Moeller [mailto:moeller0@gmx.de]
> Sent: Friday, November 14, 2014 9:15 AM
> To: dpreed@reed.com
> Cc: Dave Täht; Black, David; cerowrt-devel@lists.bufferbloat.net
> Subject: Re: [Cerowrt-devel] SQM: tracking some diffserv related internet
> drafts better
> 
> Hi David,
> 
> On Nov 14, 2014, at 03:41 , dpreed@reed.com wrote:
> 
> > The IETF used to require rough consensus and *working code*.  The latter
> seems to be out of fashion - especially with a zillion code points for which
> no working code has been produced, and worse yet, no real world testing has
> demonstrated any value whatsoever.
> >
> > It's also true that almost no actual agreements exist at peering points
> about what to do at those points.  That's why diffserv appears to be a lot of
> energy wasted on something that has little to do with inter-networking.
> >
> > "intserv" was a plausible idea, because within a vertically integrated
> system, one can enforce regularity and consistency.
> >
> > So my view, for what it is worth, is that there are a zillion better things
> to put effort into than incorporating diffserv into CeroWRT!
> 
> 	Well for cerowrt's egress queue it is nice to be able to select the
> priority by modifying the DS bits of the packet as some applications can do
> this directly; the alternative (bear with me here I am a layman) would be deep
> packet inspection at the router with the problem that even without that the
> wndrs are having a hard time routing current link speeds even without needing
> to perform DPI plus I have no idea how to DPI an encrypted connection. So I
> think that the use of the DS bits inside cerowrt is justified for cero's core
> job as a home router. Whether implementing the full DS set is needed of
> whether the 15 DS bits per browser will help anything is a different question,
> but sincere we want to handle the bits already it should not be too much work.
> (For the current SQM system we need to switch DS to queue mapping from
> independent tc filter invocations to a hashed filter as we run to many filters
> already and then the only additional work will be mapping 64 codes to 3 queues
> instead of just 8 or so ;) )
> 
> >
> > Cui bono?
> >
> >
> >
> > On Thursday, November 13, 2014 12:26pm, "Dave Taht" <dave.taht@gmail.com>
> said:
> >
> > > This appears to be close to finalization, or finalized:
> > >
> > > http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
> > >
> > > And this is complementary:
> > >
> > > http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03
> > >
> > > While wading through all this is tedious, and much of the advice
> contradictory,
> > > there are a few things that could be done more right in the sqm system
> > > that I'd like to discuss. (feel free to pour a cup of coffee and read
> > > the drafts)
> > >
> > > -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
> > > the last person that uses ssh or plays games?
> > >
> > > 0) Key to this draft is expecting that the AF code points on a single
> > > 5-tuple not be re-ordered, which means dumping AF41 into a priority
> > > queue and AF42 into the BE queue is incorrect.
> > >
> > > 1) SQM only prioritizes a few diffserv codepoints (just the ones for
> > > which I had tools doing classification, like ssh). Doing so with tc
> > > rules is very inefficient presently. I had basically planned on
> > > rolling a new tc and/or iptables filter to "do the right thing" to map
> > > into all 64 codepoints via a simple lookup table (as what is in the
> > > wifi code already), rather than use the existing mechanism... and
> > > hesitated
> > > as nobody had nailed down the definitions of each one.
> > >
> > > That said, I have not measured recently the impact of the extra tc
> > > filters and iptables rules required.
> > >
> > > 1a) Certainly only doing AF42 in sqm is pretty wrong (that was left
> > > over from my test patches against mosh - mosh ran with AF42 for a
> > > while until they crashed a couple routers with it)
> > >
> > > The relevant lines are here:
> > >
> > > https://github.com/dtaht/ceropackages-3.10/blob/master/net/sqm-
> scripts/files/usr/lib/sqm/functions.sh#L411
> > >
> > > 1b) The cake code presently does it pretty wrong, which is eminately
> fixable.
> > >
> > > 1c) And given that the standards are settling, it might be time to
> > > start baking them into a new tc or iptables filter. This would be a
> > > small, interesting project for someone who wants to get their feet wet
> > > writing this sort of thing, and examples abound of how to do it.
> > >
> > > 2) A lot of these diffserv specs - notably all the AFxx codepoints -
> > > are all about variable drop probability. (Not that this concept has
> > > been proven to work in the real world) We don't do variable drop
> > > probability... and I haven't the slightest clue as to how to do it in
> > > fq_codel. But keeping variable diffserv codepoints in order on the
> > > same 5 tuple seems to be the way things are going. Still I have
> > > trouble folding these ideas into the 3 basic queue system fq_codel
> > > uses, it looks to me as most of the AF codepoints end up in the
> > > current best effort queue, as the priority queue is limited to 30% of
> > > the bandwidth by default.
> > >
> > >
> > > 3) Squashing inbound dscp should still be the default option...
> > >
> > > 4) My patch set to the wifi code for diffserv support disables the VO
> > > queue almost entirely in favor of punting things to the VI queue
> > > (which can aggregate), but I'm not sure if I handled AFxx
> > > appropriately.
> > >
> > > 5) So far as I know, no browser implements any of this stuff yet. So
> > > far as I know nobody actually deployed a router that tries to do smart
> > > things with this stuff yet.
> > >
> > > 6) I really wish there were more codepoints for background traffic than
> cs1.
> > >
> > > --
> > > Dave Täht
> > >
> > > thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks
> > > _______________________________________________
> > > Cerowrt-devel mailing list
> > > Cerowrt-devel@lists.bufferbloat.net
> > > https://lists.bufferbloat.net/listinfo/cerowrt-devel
> > >
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] SQM: tracking some diffserv related internet drafts better

[Black, David]

Sebastian,

Summarizing, as opposed to extending the inline responses ...

> > And this is complementary:
> >
> > http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03
> 
> 	Oha, that's 15 priority levels (out of ~64 possible?) right there for a
> browser to mark packets with depending on media type. Now, not all need to map
> to real queues but that seems a lot, so that I would expect in real life a
> bunch of those will map to the same queues.

Yes, that's completely realistic, and reasonable, IMHO.

> > -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
> > the last person that uses ssh or plays games?

Actually, in the form of CS2, it is very definitely not obsolete ...

> 	Are we free in cerowrt/SQM to just ignore this and just keep imm (CS2?)
> above the best effort queue?

Yes - that rtcweb-qos draft is only applicable to the DSCPs that the browser
will use with WebRTC application.  It is not supposed to tell the networks
to not support other DSCPs - in addition to CS2, CS6 and CS7 are important
as mentioned further below.

Beyond that, we can do better - that rtcweb-qos draft has a number of
significant open issues and could well change quite a bit.  Could you and
Dave T. put your heads together and send me a paragraph on what that draft
should say about CS2 and why?

A specific example is that the DSCP table in Section 5 of that draft
does not use CS2 - if you could suggest where CS2 should appear in that
table, I will (wearing my tsvwg WG co-chair "hat") ensure that the draft
authors do something about that input.

FWIW, independent of grousing about the overall importance of "running
code" to IETF, I do my best to pay attention, as specs that can't or
won't be implemented just waste everyone's time.

> > 0) Key to this draft is expecting that the AF code points on a single
> > 5-tuple not be re-ordered, which means dumping AF41 into a priority
> > queue and AF42 into the BE queue is incorrect.

Yes, that's definitely incorrect ... and this situation is not new -
see RFC 2597.

> 	So what about sticking to the class selectors only in SQM? If I
> understand correctly we can match on the CS bits only and ignore the other
> bits; I think each AFNx map to the same CS(M) class.

IMHO, that's reasonable, and DiffServ was designed to accommodate existing
implementations that do that, so I don't see a problem with a new
implementation decision to do something like that in pursuit of simplicity, etc.

One request - please describe this as "mapping the CCCxxx codepoints to one
queue, for all values of xxx" rather than "ignoring xxx" even if the results
are the same, and even if the code winds up masking off those bits instead
of using a full mapping table, as "mapping" language goes over better
than "masking" ...

> > 1a) Certainly only doing AF42 in sqm is pretty wrong (that was left
> > over from my test patches against mosh - mosh ran with AF42 for a
> > while until they crashed a couple routers with it)
> 
> 	Why? We could just switch to stash all CS4 packets into this queue and
> be compliant again to the recommendation to treat packets in each AFN set
> equally?

I see no problems with that approach.

> > 3) Squashing inbound dscp should still be the default option.
> 
> My interpretation of http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
> section 3.2's " When DiffServ is used, the edge or boundary nodes of a network
> are responsible for ensuring that all traffic entering that network conforms
> to that network's policies for DSCP and PHB usage, and such nodes may change
> DSCP markings on traffic to achieve that result." is anything goes including
> remapping to all zeros aka squashing.

That's correct, and there's a lot of "remapping to all zeros aka squashing"
deployed out there - it's definitely ok wrt these drafts and the DiffServ RFCs.

> http://tools.ietf.org/html/rfc2474 talks
> about a MUST to put CS 6 and 7 into a higher priority class than CS0, but I
> really doubt that any ISP will allow me to label all my traffic CS7 an will
> treat it accordingly, so remapping to zero is okay if not by standard then by
> cause of reality ;).

That RFC 2474 statement is primarily about the internal operation of a diffserv
domain, e.g., one probably wants OSPF traffic prioritized over web traffic sot
that routing converges faster after a disruption.  I would be very surprised to
see ISPs accept customer traffic at CS6 or CS7 w/o remarking at the edge unless
that traffic is immediately shoved into a tunnel that hides those codepoint
values from the ISPs network.

When networks peer, CS6 is appropriate for use across peering interfaces/
intereconnects but the traffic that uses CS6 is likely to terminate at or
close to the edge of the receiving network - see Section 3.2 of draft-geib:

http://tools.ietf.org/html/draft-geib-tsvwg-diffserv-intercon-08#section-3

I welcome any suggestions for improving that text.

Thanks,
--David

> -----Original Message-----
> From: Sebastian Moeller [mailto:moeller0@gmx.de]
> Sent: Friday, November 14, 2014 9:02 AM
> To: Dave Täht
> Cc: cerowrt-devel@lists.bufferbloat.net; Black, David
> Subject: Re: [Cerowrt-devel] SQM: tracking some diffserv related internet
> drafts better
> 
> Hi Dave,
> 
> I probably do not understand the topic fully, but...
> 
> On Nov 13, 2014, at 18:26 , Dave Taht <dave.taht@gmail.com> wrote:
> 
> > This appears to be close to finalization, or finalized:
> >
> > http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
> >
> > And this is complementary:
> >
> > http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03
> 
> 	Oha, that's 15 priority levels (out of ~64 possible?) right there for a
> browser to mark packets with depending on media type. Now, not all need to map
> to real queues but that seems a lot, so that I would expect in real life a
> bunch of those will map to the same queues.
> 	If I understand correctly we already have a problems getting  decent AQM
> implemented at core switching/routing equipment, how realistic is it to expect
> that these devices implement differential packet drop probabilities per
> diffserv markings? I f the answer is not realistic the last three DS bits
> become functionally equal. . Also CS1 for audio/video? I thought this to be
> the scavenger class and hence not suitable for anything but bulk background
> traffic if there is even the slightest contention on the path. (on second
> thought this will allow to turn a CS1 internet radio in a decent congestion
> monitor, if the audio skips you know the network is starting to develop
> issues.).
> 
> >
> > While wading through all this is tedious, and much of the advice
> contradictory,
> > there are a few things that could be done more right in the sqm system
> > that I'd like to discuss. (feel free to pour a cup of coffee and read
> > the drafts)
> >
> > -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
> > the last person that uses ssh or plays games?
> 
> 	Are we free in cerowrt/SQM to just ignore this and just keep imm (CS2?)
> above the best effort queue?
> 
> >
> > 0) Key to this draft is expecting that the AF code points on a single
> > 5-tuple not be re-ordered, which means dumping AF41 into a priority
> > queue and AF42 into the BE queue is incorrect.
> 
> 	So what about sticking to the class selectors only in SQM? If I
> understand correctly we can match on the CS bits only and ignore the other
> bits; I think each AFNx map to the same CS(M) class. Looking at section
> "4.2.2.3  Using the Class Selector PHB Requirements for IP Precedence
> Compatibility" of http://tools.ietf.org/html/rfc2474#page-11 seems to confirm
> that interpretation.
> 
> 
> >
> > 1) SQM only prioritizes a few diffserv codepoints (just the ones for
> > which I had tools doing classification, like ssh). Doing so with tc
> > rules is very inefficient presently. I had basically planned on
> > rolling a new tc and/or iptables filter to "do the right thing" to map
> > into all 64 codepoints via a simple lookup table (as what is in the
> > wifi code already), rather than use the existing mechanism... and
> > hesitated
> > as nobody had nailed down the definitions of each one.
> 
> 	Well, "tc filter" hurts us badly as I figured out implementing filters
> to look into PPP encapsulated packets to get to the TOS bits. But in theory
> all tests for the individual code points can be turned into a hawh operation
> in tc filter so that we only pay the price for each encapsulation type (IPv4
> IPv6, IPv$ in PPP, IPv6 in PPP, to poke through the PPP layer costs a few
> additional ANDed match tests, but I really really hope that tc filter is smart
> enough to stop filter processing on the first mismatch.) On my TODO list for
> SQM is to use tc filter's hash functionality to process all code points in one
> operation per packet. This should also allow/require mapping each of the 64
> diffserve markings to our queues so that any "ietf-recommendation-of-the-day"
> can be a easily implemented by changing our mapping table...
> 
> >
> > That said, I have not measured recently the impact of the extra tc
> > filters and iptables rules required.
> 
> 	As far as I can tell tc filter is costly, in a "non-scientific" test
> with netperf-wrapper's RRUL test I saw the ICMP-CDF "robust-range" (the delay
> span in which the CDF went from ~5% to 95%) incase from 10ms to 30ms. No idea
> about the iptables rules (well, the internet seems to argue iptables being
> much cheaper than tc filter).
> 
> >
> > 1a) Certainly only doing AF42 in sqm is pretty wrong (that was left
> > over from my test patches against mosh - mosh ran with AF42 for a
> > while until they crashed a couple routers with it)
> 
> 	Why? We could just switch to stash all CS4 packets into this queue and
> be compliant again to the recommendation to treat packets in each AFN set
> equally?
> 
> >
> > The relevant lines are here:
> >
> > https://github.com/dtaht/ceropackages-3.10/blob/master/net/sqm-
> scripts/files/usr/lib/sqm/functions.sh#L411
> >
> > 1b) The cake code presently does it pretty wrong, which is eminately
> fixable.
> >
> > 1c) And given that the standards are settling, it might be time to
> > start baking them into a new tc or iptables filter. This would be a
> > small, interesting project for someone who wants to get their feet wet
> > writing this sort of thing, and examples abound of how to do it.
> 
> 	So what I plan on doing until the end of the year is getting the hashed
> tc filter set up for SQM than implementing/testing different mappings will be
> a piece of cake just change 64 values and you are done...
> 
> >
> > 2) A lot of these diffserv specs - notably all the AFxx codepoints -
> > are all about variable drop probability. (Not that this concept has
> > been proven to work in the real world) We don't do variable drop
> > probability... and I haven't the slightest clue as to how to do it in
> > fq_codel. But keeping variable diffserv codepoints in order on the
> > same 5 tuple seems to be the way things are going. Still I have
> > trouble folding these ideas into the 3 basic queue system fq_codel
> > uses, it looks to me as most of the AF codepoints end up in the
> > current best effort queue, as the priority queue is limited to 30% of
> > the bandwidth by default.
> 
> 	Is this really relevant for the wider internet at all? As you argue
> below (and as is argued in the drafts cited above) each network can do what
> ever it likes with code points so the relevant question, as I see it is not
> what could we do with the code points if we had all 6 bits for us end to end,
> but rather how many and which bits actually survive a trip over the open
> internet ;)
> 
> >
> >
> > 3) Squashing inbound dscp should still be the default option.
> 
> My interpretation of http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
> section 3.2's " When DiffServ is used, the edge or boundary nodes of a network
> are responsible for ensuring that all traffic entering that network conforms
> to that network's policies for DSCP and PHB usage, and such nodes may change
> DSCP markings on traffic to achieve that result." is anything goes including
> remapping to all zeros aka squashing. http://tools.ietf.org/html/rfc2474 talks
> about a MUST to put CS 6 and 7 into a higher priority class than CS0, but I
> really doubt that any ISP will allow me to label all my traffic CS7 an will
> treat it accordingly, so remapping to zero is okay if not by standard then by
> cause of reality ;).
> 
> 
> 
> 
> >
> > 4) My patch set to the wifi code for diffserv support disables the VO
> > queue almost entirely in favor of punting things to the VI queue
> > (which can aggregate), but I'm not sure if I handled AFxx
> > appropriately.
> >
> > 5) So far as I know, no browser implements any of this stuff yet. So
> > far as I know nobody actually deployed a router that tries to do smart
> > things with this stuff yet.
> 
> 	I would love to know whether the proposed markings actually survive a
> trip through the open internet at all. I would like to argue that until that
> actually happens this a nicely academic discussion (cerowrt does a fine job
> already with its nice fq_codel hierarchy, and if all the new fancy stuff will
> be wiped directly by my ISP I am not sure that implementing the proposal in
> sam is going to change anything, especially nothing that can be measured. As
> they say, "measurement data or it did not happen" ;) )
> 
> >
> > 6) I really wish there were more codepoints for background traffic than cs1.
> 
> 	But isn't that what AF1x is all about?. I agree the range of the 6 DS
> bits is not used to its fullest extend: rather than bits treat is as a number
> and do: current priority = (DS - 32) so we have a range from -32 to 31 (or so)
> and simply require that higher values are not treated with less priority than
> smaller numbers. (Heck maybe special case CS0 to also mean zero for
> backward/reality compatibility ;) )
> 	But most likely I just have misunderstood the whole issue..
> 
> Best Regards
> 	Sebastian
> 
> 
> 
> >
> > --
> > Dave Täht
> >
> > thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] tracking some diffserv related internet drafts better

[Sebastian Moeller]

Hi Dave(s),

On Nov 13, 2014, at 19:45 , Black, David <david.black@emc.com> wrote:

> Dave (T),
> 
>> This appears to be close to finalization, or finalized:
>> 
>> http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
>> 
>> And this is complementary:
>> 
>> http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03
> 
> The dart-dscp-rtp draft is now done and in the RFC Editor's Queue.
> 
> The tsvwg-rtcweb-qos draft is still a work in progress, and its
> aggressive use of almost every PHB and DSCP is a concern.  Here's
> another related draft that is likely to be adopted by the tsvwg WG soon:
> 
> https://datatracker.ietf.org/doc/draft-geib-tsvwg-diffserv-intercon/

	I could be completely off my rocker, but that scheme would work perfectly if the edge and the core would use 3 bits each max then the core can just move the edge bits 0-2 to the so far empty bits 3-5 on ingress and back to 0-2 on egress and each can do their own thing. On the other hand, if the core does not actually use the edge markings then it could also jus ts quash them nd the edge would not be any wiser.

Best Regards
	Sebastian


> 
> 
>> -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
>> the last person that uses ssh or plays games?
> 
> See RFC 2474, published in 1998 ...
> 
>> 0) Key to this draft is expecting that the AF code points on a single
>> 5-tuple not be re-ordered, which means dumping AF41 into a priority
>> queue and AF42 into the BE queue is incorrect.
> 
> Yes.  That's supposed to be the case for any AF PHB group that's
> supported., e.g., AF3x [AF31, AF32 and AF33].
> 
> https://datatracker.ietf.org/doc/draft-geib-tsvwg-diffserv-intercon/
> 
>> 1c) And given that the standards are settling, it might be time to
>> start baking them into a new tc or iptables filter. This would be a
>> small, interesting project for someone who wants to get their feet wet
>> writing this sort of thing, and examples abound of how to do it.
> 
> I'm happy to help w/insight, advice etc. - please ask.  In turn,
> experience w/what's more vs. less likely to work where and why would
> be quite useful, e.g., for the tsvwg-rtcweb-qos draft.
> 
>> 2) A lot of these diffserv specs - notably all the AFxx codepoints -
>> are all about variable drop probability. (Not that this concept has
>> been proven to work in the real world) We don't do variable drop
>> probability... and I haven't the slightest clue as to how to do it in
>> fq_codel. But keeping variable diffserv codepoints in order on the
>> same 5 tuple seems to be the way things are going.
> 
> That's UDP-only at the moment if that helps.  Variable drop probability
> doesn't do anything useful for TCP, and there's no good experience
> with other congestion controlled transports - see the dart-dscp-rtp
> draft.
> 
>> 3) Squashing inbound dscp should still be the default option...
> 
> Yes, very common out there.  See the tsvwg-diffserv-intercon draft
> which is trying to specify a few small holes to punch into that
> "squash to zero" default ingress DSCP behavior.  Yes, I'm an
> optimist ...
> 
>> 6) I really wish there were more codepoints for background traffic than cs1.
> 
> It's worse than that - CS1 may or may not be a background codepoint,
> see the dart-dscp-rtp draft.
> 
> Thanks,
> --David
> 
>> -----Original Message-----
>> From: Dave Taht [mailto:dave.taht@gmail.com]
>> Sent: Thursday, November 13, 2014 12:27 PM
>> To: cerowrt-devel@lists.bufferbloat.net; Black, David
>> Subject: SQM: tracking some diffserv related internet drafts better
>> 
>> This appears to be close to finalization, or finalized:
>> 
>> http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
>> 
>> And this is complementary:
>> 
>> http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03
>> 
>> While wading through all this is tedious, and much of the advice
>> contradictory,
>> there are a few things that could be done more right in the sqm system
>> that I'd like to discuss. (feel free to pour a cup of coffee and read
>> the drafts)
>> 
>> -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
>> the last person that uses ssh or plays games?
>> 
>> 0) Key to this draft is expecting that the AF code points on a single
>> 5-tuple not be re-ordered, which means dumping AF41 into a priority
>> queue and AF42 into the BE queue is incorrect.
>> 
>> 1) SQM only prioritizes a few diffserv codepoints (just the ones for
>> which I had tools doing classification, like ssh). Doing so with tc
>> rules is very inefficient presently. I had basically planned on
>> rolling a new tc and/or iptables filter to "do the right thing" to map
>> into all 64 codepoints via a simple lookup table (as what is in the
>> wifi code already), rather than use the existing mechanism... and
>> hesitated
>> as nobody had nailed down the definitions of each one.
>> 
>> That said, I have not measured recently the impact of the extra tc
>> filters and iptables rules required.
>> 
>> 1a) Certainly only doing AF42 in sqm is pretty wrong (that was left
>> over from my test patches against mosh - mosh ran with AF42 for a
>> while until they crashed a couple routers with it)
>> 
>> The relevant lines are here:
>> 
>> https://github.com/dtaht/ceropackages-3.10/blob/master/net/sqm-
>> scripts/files/usr/lib/sqm/functions.sh#L411
>> 
>> 1b) The cake code presently does it pretty wrong, which is eminately fixable.
>> 
>> 1c) And given that the standards are settling, it might be time to
>> start baking them into a new tc or iptables filter. This would be a
>> small, interesting project for someone who wants to get their feet wet
>> writing this sort of thing, and examples abound of how to do it.
>> 
>> 2) A lot of these diffserv specs - notably all the AFxx codepoints -
>> are all about variable drop probability. (Not that this concept has
>> been proven to work in the real world) We don't do variable drop
>> probability... and I haven't the slightest clue as to how to do it in
>> fq_codel. But keeping variable diffserv codepoints in order on the
>> same 5 tuple seems to be the way things are going. Still I have
>> trouble folding these ideas into the 3 basic queue system fq_codel
>> uses, it looks to me as most of the AF codepoints end up in the
>> current best effort queue, as the priority queue is limited to 30% of
>> the bandwidth by default.
>> 
>> 
>> 3) Squashing inbound dscp should still be the default option...
>> 
>> 4) My patch set to the wifi code for diffserv support disables the VO
>> queue almost entirely in favor of punting things to the VI queue
>> (which can aggregate), but I'm not sure if I handled AFxx
>> appropriately.
>> 
>> 5) So far as I know, no browser implements any of this stuff yet. So
>> far as I know nobody actually deployed a router that tries to do smart
>> things with this stuff yet.
>> 
>> 6) I really wish there were more codepoints for background traffic than cs1.
>> 
>> --
>> Dave Täht
>> 
>> thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] tracking some diffserv related internet drafts better

[Black, David]

> 	I could be completely off my rocker, but that scheme would work
> perfectly if the edge and the core would use 3 bits each max then the core can
> just move the edge bits 0-2 to the so far empty bits 3-5 on ingres

"so far empty" ... not exactly ... AFxx and EF use those bits:

http://www.iana.org/assignments/dscp-registry/dscp-registry.xhtml#dscp-registry-1

> and back
> to 0-2 on egress and each can do their own thing. On the other hand, if the
> core does not actually use the edge markings then it could also jus ts quash
> them nd the edge would not be any wiser.

The core should be mapping edge markings to core markings and then to some
sort of edge marking at the far edge.  I'd prefer to view terms like "squash"
and "shift" as implementation-specific decisions about how to do those mappings.

DiffServ allows implementers and network operators to do what they want with
DSCPs, with the exception of the CSx DSCPs (details are in RFC 2474, some of
which are already in this discussion).

Thanks,
--David


> -----Original Message-----
> From: Sebastian Moeller [mailto:moeller0@gmx.de]
> Sent: Friday, November 14, 2014 3:36 PM
> To: Black, David
> Cc: Dave Täht; cerowrt-devel@lists.bufferbloat.net; draft-ietf-tsvwg-rtcweb-
> qos@tools.ietf.org; draft-ietf-dart-dscp-rtp@tools.ietf.org
> Subject: Re: [Cerowrt-devel] tracking some diffserv related internet drafts
> better
> 
> Hi Dave(s),
> 
> On Nov 13, 2014, at 19:45 , Black, David <david.black@emc.com> wrote:
> 
> > Dave (T),
> >
> >> This appears to be close to finalization, or finalized:
> >>
> >> http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
> >>
> >> And this is complementary:
> >>
> >> http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03
> >
> > The dart-dscp-rtp draft is now done and in the RFC Editor's Queue.
> >
> > The tsvwg-rtcweb-qos draft is still a work in progress, and its
> > aggressive use of almost every PHB and DSCP is a concern.  Here's
> > another related draft that is likely to be adopted by the tsvwg WG soon:
> >
> > https://datatracker.ietf.org/doc/draft-geib-tsvwg-diffserv-intercon/
> 
> 	I could be completely off my rocker, but that scheme would work
> perfectly if the edge and the core would use 3 bits each max then the core can
> just move the edge bits 0-2 to the so far empty bits 3-5 on ingress and back
> to 0-2 on egress and each can do their own thing. On the other hand, if the
> core does not actually use the edge markings then it could also jus ts quash
> them nd the edge would not be any wiser.
> 
> Best Regards
> 	Sebastian
> 
> 
> >
> >
> >> -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
> >> the last person that uses ssh or plays games?
> >
> > See RFC 2474, published in 1998 ...
> >
> >> 0) Key to this draft is expecting that the AF code points on a single
> >> 5-tuple not be re-ordered, which means dumping AF41 into a priority
> >> queue and AF42 into the BE queue is incorrect.
> >
> > Yes.  That's supposed to be the case for any AF PHB group that's
> > supported., e.g., AF3x [AF31, AF32 and AF33].
> >
> > https://datatracker.ietf.org/doc/draft-geib-tsvwg-diffserv-intercon/
> >
> >> 1c) And given that the standards are settling, it might be time to
> >> start baking them into a new tc or iptables filter. This would be a
> >> small, interesting project for someone who wants to get their feet wet
> >> writing this sort of thing, and examples abound of how to do it.
> >
> > I'm happy to help w/insight, advice etc. - please ask.  In turn,
> > experience w/what's more vs. less likely to work where and why would
> > be quite useful, e.g., for the tsvwg-rtcweb-qos draft.
> >
> >> 2) A lot of these diffserv specs - notably all the AFxx codepoints -
> >> are all about variable drop probability. (Not that this concept has
> >> been proven to work in the real world) We don't do variable drop
> >> probability... and I haven't the slightest clue as to how to do it in
> >> fq_codel. But keeping variable diffserv codepoints in order on the
> >> same 5 tuple seems to be the way things are going.
> >
> > That's UDP-only at the moment if that helps.  Variable drop probability
> > doesn't do anything useful for TCP, and there's no good experience
> > with other congestion controlled transports - see the dart-dscp-rtp
> > draft.
> >
> >> 3) Squashing inbound dscp should still be the default option...
> >
> > Yes, very common out there.  See the tsvwg-diffserv-intercon draft
> > which is trying to specify a few small holes to punch into that
> > "squash to zero" default ingress DSCP behavior.  Yes, I'm an
> > optimist ...
> >
> >> 6) I really wish there were more codepoints for background traffic than
> cs1.
> >
> > It's worse than that - CS1 may or may not be a background codepoint,
> > see the dart-dscp-rtp draft.
> >
> > Thanks,
> > --David
> >
> >> -----Original Message-----
> >> From: Dave Taht [mailto:dave.taht@gmail.com]
> >> Sent: Thursday, November 13, 2014 12:27 PM
> >> To: cerowrt-devel@lists.bufferbloat.net; Black, David
> >> Subject: SQM: tracking some diffserv related internet drafts better
> >>
> >> This appears to be close to finalization, or finalized:
> >>
> >> http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
> >>
> >> And this is complementary:
> >>
> >> http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03
> >>
> >> While wading through all this is tedious, and much of the advice
> >> contradictory,
> >> there are a few things that could be done more right in the sqm system
> >> that I'd like to discuss. (feel free to pour a cup of coffee and read
> >> the drafts)
> >>
> >> -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
> >> the last person that uses ssh or plays games?
> >>
> >> 0) Key to this draft is expecting that the AF code points on a single
> >> 5-tuple not be re-ordered, which means dumping AF41 into a priority
> >> queue and AF42 into the BE queue is incorrect.
> >>
> >> 1) SQM only prioritizes a few diffserv codepoints (just the ones for
> >> which I had tools doing classification, like ssh). Doing so with tc
> >> rules is very inefficient presently. I had basically planned on
> >> rolling a new tc and/or iptables filter to "do the right thing" to map
> >> into all 64 codepoints via a simple lookup table (as what is in the
> >> wifi code already), rather than use the existing mechanism... and
> >> hesitated
> >> as nobody had nailed down the definitions of each one.
> >>
> >> That said, I have not measured recently the impact of the extra tc
> >> filters and iptables rules required.
> >>
> >> 1a) Certainly only doing AF42 in sqm is pretty wrong (that was left
> >> over from my test patches against mosh - mosh ran with AF42 for a
> >> while until they crashed a couple routers with it)
> >>
> >> The relevant lines are here:
> >>
> >> https://github.com/dtaht/ceropackages-3.10/blob/master/net/sqm-
> >> scripts/files/usr/lib/sqm/functions.sh#L411
> >>
> >> 1b) The cake code presently does it pretty wrong, which is eminately
> fixable.
> >>
> >> 1c) And given that the standards are settling, it might be time to
> >> start baking them into a new tc or iptables filter. This would be a
> >> small, interesting project for someone who wants to get their feet wet
> >> writing this sort of thing, and examples abound of how to do it.
> >>
> >> 2) A lot of these diffserv specs - notably all the AFxx codepoints -
> >> are all about variable drop probability. (Not that this concept has
> >> been proven to work in the real world) We don't do variable drop
> >> probability... and I haven't the slightest clue as to how to do it in
> >> fq_codel. But keeping variable diffserv codepoints in order on the
> >> same 5 tuple seems to be the way things are going. Still I have
> >> trouble folding these ideas into the 3 basic queue system fq_codel
> >> uses, it looks to me as most of the AF codepoints end up in the
> >> current best effort queue, as the priority queue is limited to 30% of
> >> the bandwidth by default.
> >>
> >>
> >> 3) Squashing inbound dscp should still be the default option...
> >>
> >> 4) My patch set to the wifi code for diffserv support disables the VO
> >> queue almost entirely in favor of punting things to the VI queue
> >> (which can aggregate), but I'm not sure if I handled AFxx
> >> appropriately.
> >>
> >> 5) So far as I know, no browser implements any of this stuff yet. So
> >> far as I know nobody actually deployed a router that tries to do smart
> >> things with this stuff yet.
> >>
> >> 6) I really wish there were more codepoints for background traffic than
> cs1.
> >>
> >> --
> >> Dave Täht
> >>
> >> thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] tracking some diffserv related internet drafts better

[Sebastian Moeller]

Hi David,

On Nov 14, 2014, at 21:49 , Black, David <david.black@emc.com> wrote:

>> 	I could be completely off my rocker, but that scheme would work
>> perfectly if the edge and the core would use 3 bits each max then the core can
>> just move the edge bits 0-2 to the so far empty bits 3-5 on ingres
> 
> "so far empty" ... not exactly ... AFxx and EF use those bits:
> 
> http://www.iana.org/assignments/dscp-registry/dscp-registry.xhtml#dscp-registry-1

	Well, I guess I forgot the smiley face there. But looking at information I just learned from the intercom RFC you referenced: if the core really only has room for 3bits of DS marking can the edge actually expect to use more than three bits of DS marking in a meaningful way? Thanks for the link that helps me bait to understand the beauty of DSCP, including that none intended to use of 64 concurrent code points ;) (only 32 of which 10 seem not assigned yet). in that light the 15 proposed code points for browsers spann half the standard pool (actually rather 68%) using the diffserv “dynamic range” quite well. I think I start to realize that there is basically one chance to change browser marking behaviour and it is much easier to map down to fewer priority queues than it is to introduce new “standard” marking later (heck it will probably tricky enough to get the proposed scheme implemented in all popular dowsers in a timely fashion).

> 
>> and back
>> to 0-2 on egress and each can do their own thing. On the other hand, if the
>> core does not actually use the edge markings then it could also jus ts quash
>> them nd the edge would not be any wiser.
> 
> The core should be mapping edge markings to core markings and then to some
> sort of edge marking at the far edge.  

	Sure, but with only 3 bits worth of core marking space, the egress marking will also just contain 8 combinations (unless a side channel to transfer the original marking or DPI is used in addition to the existing core markings). But this is really not my area of expertise as I have amply shown...

> I'd prefer to view terms like "squash"
> and "shift" as implementation-specific decisions about how to do those mappings.

	I happily agree, mapping is a more general term, so no more squashing just "map to zero” ;)

> 
> DiffServ allows implementers and network operators to do what they want with
> DSCPs, with the exception of the CSx DSCPs (details are in RFC 2474, some of
> which are already in this discussion).

	But the “do what they want part” is what makes using DSCPs across network boundaries so taxing/frustrating as even my EF marked packets might end up in my ISPs scavenger class… (First allowed "map to zero” followed by a DPI based new marking, I think that would be RFC conform)...

Best Regards
	Sebastian

> 
> Thanks,
> --David
> 
> 
>> -----Original Message-----
>> From: Sebastian Moeller [mailto:moeller0@gmx.de]
>> Sent: Friday, November 14, 2014 3:36 PM
>> To: Black, David
>> Cc: Dave Täht; cerowrt-devel@lists.bufferbloat.net; draft-ietf-tsvwg-rtcweb-
>> qos@tools.ietf.org; draft-ietf-dart-dscp-rtp@tools.ietf.org
>> Subject: Re: [Cerowrt-devel] tracking some diffserv related internet drafts
>> better
>> 
>> Hi Dave(s),
>> 
>> On Nov 13, 2014, at 19:45 , Black, David <david.black@emc.com> wrote:
>> 
>>> Dave (T),
>>> 
>>>> This appears to be close to finalization, or finalized:
>>>> 
>>>> http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
>>>> 
>>>> And this is complementary:
>>>> 
>>>> http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03
>>> 
>>> The dart-dscp-rtp draft is now done and in the RFC Editor's Queue.
>>> 
>>> The tsvwg-rtcweb-qos draft is still a work in progress, and its
>>> aggressive use of almost every PHB and DSCP is a concern.  Here's
>>> another related draft that is likely to be adopted by the tsvwg WG soon:
>>> 
>>> https://datatracker.ietf.org/doc/draft-geib-tsvwg-diffserv-intercon/
>> 
>> 	I could be completely off my rocker, but that scheme would work
>> perfectly if the edge and the core would use 3 bits each max then the core can
>> just move the edge bits 0-2 to the so far empty bits 3-5 on ingress and back
>> to 0-2 on egress and each can do their own thing. On the other hand, if the
>> core does not actually use the edge markings then it could also jus ts quash
>> them nd the edge would not be any wiser.
>> 
>> Best Regards
>> 	Sebastian
>> 
>> 
>>> 
>>> 
>>>> -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
>>>> the last person that uses ssh or plays games?
>>> 
>>> See RFC 2474, published in 1998 ...
>>> 
>>>> 0) Key to this draft is expecting that the AF code points on a single
>>>> 5-tuple not be re-ordered, which means dumping AF41 into a priority
>>>> queue and AF42 into the BE queue is incorrect.
>>> 
>>> Yes.  That's supposed to be the case for any AF PHB group that's
>>> supported., e.g., AF3x [AF31, AF32 and AF33].
>>> 
>>> https://datatracker.ietf.org/doc/draft-geib-tsvwg-diffserv-intercon/
>>> 
>>>> 1c) And given that the standards are settling, it might be time to
>>>> start baking them into a new tc or iptables filter. This would be a
>>>> small, interesting project for someone who wants to get their feet wet
>>>> writing this sort of thing, and examples abound of how to do it.
>>> 
>>> I'm happy to help w/insight, advice etc. - please ask.  In turn,
>>> experience w/what's more vs. less likely to work where and why would
>>> be quite useful, e.g., for the tsvwg-rtcweb-qos draft.
>>> 
>>>> 2) A lot of these diffserv specs - notably all the AFxx codepoints -
>>>> are all about variable drop probability. (Not that this concept has
>>>> been proven to work in the real world) We don't do variable drop
>>>> probability... and I haven't the slightest clue as to how to do it in
>>>> fq_codel. But keeping variable diffserv codepoints in order on the
>>>> same 5 tuple seems to be the way things are going.
>>> 
>>> That's UDP-only at the moment if that helps.  Variable drop probability
>>> doesn't do anything useful for TCP, and there's no good experience
>>> with other congestion controlled transports - see the dart-dscp-rtp
>>> draft.
>>> 
>>>> 3) Squashing inbound dscp should still be the default option...
>>> 
>>> Yes, very common out there.  See the tsvwg-diffserv-intercon draft
>>> which is trying to specify a few small holes to punch into that
>>> "squash to zero" default ingress DSCP behavior.  Yes, I'm an
>>> optimist ...
>>> 
>>>> 6) I really wish there were more codepoints for background traffic than
>> cs1.
>>> 
>>> It's worse than that - CS1 may or may not be a background codepoint,
>>> see the dart-dscp-rtp draft.
>>> 
>>> Thanks,
>>> --David
>>> 
>>>> -----Original Message-----
>>>> From: Dave Taht [mailto:dave.taht@gmail.com]
>>>> Sent: Thursday, November 13, 2014 12:27 PM
>>>> To: cerowrt-devel@lists.bufferbloat.net; Black, David
>>>> Subject: SQM: tracking some diffserv related internet drafts better
>>>> 
>>>> This appears to be close to finalization, or finalized:
>>>> 
>>>> http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
>>>> 
>>>> And this is complementary:
>>>> 
>>>> http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03
>>>> 
>>>> While wading through all this is tedious, and much of the advice
>>>> contradictory,
>>>> there are a few things that could be done more right in the sqm system
>>>> that I'd like to discuss. (feel free to pour a cup of coffee and read
>>>> the drafts)
>>>> 
>>>> -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
>>>> the last person that uses ssh or plays games?
>>>> 
>>>> 0) Key to this draft is expecting that the AF code points on a single
>>>> 5-tuple not be re-ordered, which means dumping AF41 into a priority
>>>> queue and AF42 into the BE queue is incorrect.
>>>> 
>>>> 1) SQM only prioritizes a few diffserv codepoints (just the ones for
>>>> which I had tools doing classification, like ssh). Doing so with tc
>>>> rules is very inefficient presently. I had basically planned on
>>>> rolling a new tc and/or iptables filter to "do the right thing" to map
>>>> into all 64 codepoints via a simple lookup table (as what is in the
>>>> wifi code already), rather than use the existing mechanism... and
>>>> hesitated
>>>> as nobody had nailed down the definitions of each one.
>>>> 
>>>> That said, I have not measured recently the impact of the extra tc
>>>> filters and iptables rules required.
>>>> 
>>>> 1a) Certainly only doing AF42 in sqm is pretty wrong (that was left
>>>> over from my test patches against mosh - mosh ran with AF42 for a
>>>> while until they crashed a couple routers with it)
>>>> 
>>>> The relevant lines are here:
>>>> 
>>>> https://github.com/dtaht/ceropackages-3.10/blob/master/net/sqm-
>>>> scripts/files/usr/lib/sqm/functions.sh#L411
>>>> 
>>>> 1b) The cake code presently does it pretty wrong, which is eminately
>> fixable.
>>>> 
>>>> 1c) And given that the standards are settling, it might be time to
>>>> start baking them into a new tc or iptables filter. This would be a
>>>> small, interesting project for someone who wants to get their feet wet
>>>> writing this sort of thing, and examples abound of how to do it.
>>>> 
>>>> 2) A lot of these diffserv specs - notably all the AFxx codepoints -
>>>> are all about variable drop probability. (Not that this concept has
>>>> been proven to work in the real world) We don't do variable drop
>>>> probability... and I haven't the slightest clue as to how to do it in
>>>> fq_codel. But keeping variable diffserv codepoints in order on the
>>>> same 5 tuple seems to be the way things are going. Still I have
>>>> trouble folding these ideas into the 3 basic queue system fq_codel
>>>> uses, it looks to me as most of the AF codepoints end up in the
>>>> current best effort queue, as the priority queue is limited to 30% of
>>>> the bandwidth by default.
>>>> 
>>>> 
>>>> 3) Squashing inbound dscp should still be the default option...
>>>> 
>>>> 4) My patch set to the wifi code for diffserv support disables the VO
>>>> queue almost entirely in favor of punting things to the VI queue
>>>> (which can aggregate), but I'm not sure if I handled AFxx
>>>> appropriately.
>>>> 
>>>> 5) So far as I know, no browser implements any of this stuff yet. So
>>>> far as I know nobody actually deployed a router that tries to do smart
>>>> things with this stuff yet.
>>>> 
>>>> 6) I really wish there were more codepoints for background traffic than
>> cs1.
>>>> 
>>>> --
>>>> Dave Täht
>>>> 
>>>> thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks
>>> _______________________________________________
>>> Cerowrt-devel mailing list
>>> Cerowrt-devel@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

Re: [Cerowrt-devel] tracking some diffserv related internet drafts better

[Black, David]

Hi Sebastian,

> 	Well, I guess I forgot the smiley face there. But looking at information
> I just learned from the intercom RFC you referenced: if the core really only
> has room for 3bits of DS marking can the edge actually expect to use more than
> three bits of DS marking in a meaningful way?

That is one of the open issues on the tsvwg-rtcweb-qos draft ;-).

The situation in network cores varies:

- MPLS-based: About 2 bits is all that's really widely possible.  The MPLS TC
	field that supports this is 3 bits, but has other uses.

- IP-based carrier: Harder to say, but 3 bits is a good guesstimate.  There
	was a lot of 3-bit IP Precedence support prior to RFC 2474 - that
	corresponds to the 8 class selector codepoints, and every
	additional DSCP used has network config and management cost.

- IP-based Enterprise: Ditto on pre-RFC-2474 IP Precedence, but there seems
	to be more flexibility here.  Even so, if I take all the traffic classes
	in RFC 4594, plus add an admission-controlled telephony/voice class
	(RFC 5865) I still wind up somewhere around 4 bits, and most networks
	will implement a subset of those classes, not all of them.

> I think I start to realize that there is basically one chance to change browser
> marking behaviour and it is much easier to map down to fewer priority queues
> than it is to introduce new "standard" marking later (heck it will probably
> tricky enough to get the proposed scheme implemented in all popular dowsers in
> a timely fashion).

Yes - WebRTC is expected to provide significant encouragement for browser adoption.

> 	But the "do what they want part" is what makes using DSCPs across
> network boundaries so taxing/frustrating as even my EF marked packets might
> end up in my ISPs scavenger class. (First allowed "map to zero" followed by a
> DPI based new marking, I think that would be RFC conform)...

Indeed it would be, and draft-geib-tsvwg-diffserv-intercon is intended as a
first step towards encouraging something better at network boundaries.

Thanks,
--David

> -----Original Message-----
> From: Sebastian Moeller [mailto:moeller0@gmx.de]
> Sent: Friday, November 14, 2014 4:40 PM
> To: Black, David
> Cc: Dave Täht; cerowrt-devel@lists.bufferbloat.net; draft-ietf-tsvwg-rtcweb-
> qos@tools.ietf.org; draft-ietf-dart-dscp-rtp@tools.ietf.org
> Subject: Re: [Cerowrt-devel] tracking some diffserv related internet drafts
> better
> 
> Hi David,
> 
> On Nov 14, 2014, at 21:49 , Black, David <david.black@emc.com> wrote:
> 
> >> 	I could be completely off my rocker, but that scheme would work
> >> perfectly if the edge and the core would use 3 bits each max then the core
> can
> >> just move the edge bits 0-2 to the so far empty bits 3-5 on ingres
> >
> > "so far empty" ... not exactly ... AFxx and EF use those bits:
> >
> > http://www.iana.org/assignments/dscp-registry/dscp-registry.xhtml#dscp-
> registry-1
> 
> 	Well, I guess I forgot the smiley face there. But looking at information
> I just learned from the intercom RFC you referenced: if the core really only
> has room for 3bits of DS marking can the edge actually expect to use more than
> three bits of DS marking in a meaningful way? Thanks for the link that helps
> me bait to understand the beauty of DSCP, including that none intended to use
> of 64 concurrent code points ;) (only 32 of which 10 seem not assigned yet).
> in that light the 15 proposed code points for browsers spann half the standard
> pool (actually rather 68%) using the diffserv "dynamic range" quite well. I
> think I start to realize that there is basically one chance to change browser
> marking behaviour and it is much easier to map down to fewer priority queues
> than it is to introduce new "standard" marking later (heck it will probably
> tricky enough to get the proposed scheme implemented in all popular dowsers in
> a timely fashion).
> 
> >
> >> and back
> >> to 0-2 on egress and each can do their own thing. On the other hand, if the
> >> core does not actually use the edge markings then it could also jus ts
> quash
> >> them nd the edge would not be any wiser.
> >
> > The core should be mapping edge markings to core markings and then to some
> > sort of edge marking at the far edge.
> 
> 	Sure, but with only 3 bits worth of core marking space, the egress
> marking will also just contain 8 combinations (unless a side channel to
> transfer the original marking or DPI is used in addition to the existing core
> markings). But this is really not my area of expertise as I have amply
> shown...
> 
> > I'd prefer to view terms like "squash"
> > and "shift" as implementation-specific decisions about how to do those
> mappings.
> 
> 	I happily agree, mapping is a more general term, so no more squashing
> just "map to zero" ;)
> 
> >
> > DiffServ allows implementers and network operators to do what they want with
> > DSCPs, with the exception of the CSx DSCPs (details are in RFC 2474, some of
> > which are already in this discussion).
> 
> 	But the "do what they want part" is what makes using DSCPs across
> network boundaries so taxing/frustrating as even my EF marked packets might
> end up in my ISPs scavenger class. (First allowed "map to zero" followed by a
> DPI based new marking, I think that would be RFC conform)...
> 
> Best Regards
> 	Sebastian
> 
> >
> > Thanks,
> > --David
> >
> >
> >> -----Original Message-----
> >> From: Sebastian Moeller [mailto:moeller0@gmx.de]
> >> Sent: Friday, November 14, 2014 3:36 PM
> >> To: Black, David
> >> Cc: Dave Täht; cerowrt-devel@lists.bufferbloat.net; draft-ietf-tsvwg-
> rtcweb-
> >> qos@tools.ietf.org; draft-ietf-dart-dscp-rtp@tools.ietf.org
> >> Subject: Re: [Cerowrt-devel] tracking some diffserv related internet drafts
> >> better
> >>
> >> Hi Dave(s),
> >>
> >> On Nov 13, 2014, at 19:45 , Black, David <david.black@emc.com> wrote:
> >>
> >>> Dave (T),
> >>>
> >>>> This appears to be close to finalization, or finalized:
> >>>>
> >>>> http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
> >>>>
> >>>> And this is complementary:
> >>>>
> >>>> http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03
> >>>
> >>> The dart-dscp-rtp draft is now done and in the RFC Editor's Queue.
> >>>
> >>> The tsvwg-rtcweb-qos draft is still a work in progress, and its
> >>> aggressive use of almost every PHB and DSCP is a concern.  Here's
> >>> another related draft that is likely to be adopted by the tsvwg WG soon:
> >>>
> >>> https://datatracker.ietf.org/doc/draft-geib-tsvwg-diffserv-intercon/
> >>
> >> 	I could be completely off my rocker, but that scheme would work
> >> perfectly if the edge and the core would use 3 bits each max then the core
> can
> >> just move the edge bits 0-2 to the so far empty bits 3-5 on ingress and
> back
> >> to 0-2 on egress and each can do their own thing. On the other hand, if the
> >> core does not actually use the edge markings then it could also jus ts
> quash
> >> them nd the edge would not be any wiser.
> >>
> >> Best Regards
> >> 	Sebastian
> >>
> >>
> >>>
> >>>
> >>>> -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
> >>>> the last person that uses ssh or plays games?
> >>>
> >>> See RFC 2474, published in 1998 ...
> >>>
> >>>> 0) Key to this draft is expecting that the AF code points on a single
> >>>> 5-tuple not be re-ordered, which means dumping AF41 into a priority
> >>>> queue and AF42 into the BE queue is incorrect.
> >>>
> >>> Yes.  That's supposed to be the case for any AF PHB group that's
> >>> supported., e.g., AF3x [AF31, AF32 and AF33].
> >>>
> >>> https://datatracker.ietf.org/doc/draft-geib-tsvwg-diffserv-intercon/
> >>>
> >>>> 1c) And given that the standards are settling, it might be time to
> >>>> start baking them into a new tc or iptables filter. This would be a
> >>>> small, interesting project for someone who wants to get their feet wet
> >>>> writing this sort of thing, and examples abound of how to do it.
> >>>
> >>> I'm happy to help w/insight, advice etc. - please ask.  In turn,
> >>> experience w/what's more vs. less likely to work where and why would
> >>> be quite useful, e.g., for the tsvwg-rtcweb-qos draft.
> >>>
> >>>> 2) A lot of these diffserv specs - notably all the AFxx codepoints -
> >>>> are all about variable drop probability. (Not that this concept has
> >>>> been proven to work in the real world) We don't do variable drop
> >>>> probability... and I haven't the slightest clue as to how to do it in
> >>>> fq_codel. But keeping variable diffserv codepoints in order on the
> >>>> same 5 tuple seems to be the way things are going.
> >>>
> >>> That's UDP-only at the moment if that helps.  Variable drop probability
> >>> doesn't do anything useful for TCP, and there's no good experience
> >>> with other congestion controlled transports - see the dart-dscp-rtp
> >>> draft.
> >>>
> >>>> 3) Squashing inbound dscp should still be the default option...
> >>>
> >>> Yes, very common out there.  See the tsvwg-diffserv-intercon draft
> >>> which is trying to specify a few small holes to punch into that
> >>> "squash to zero" default ingress DSCP behavior.  Yes, I'm an
> >>> optimist ...
> >>>
> >>>> 6) I really wish there were more codepoints for background traffic than
> >> cs1.
> >>>
> >>> It's worse than that - CS1 may or may not be a background codepoint,
> >>> see the dart-dscp-rtp draft.
> >>>
> >>> Thanks,
> >>> --David
> >>>
> >>>> -----Original Message-----
> >>>> From: Dave Taht [mailto:dave.taht@gmail.com]
> >>>> Sent: Thursday, November 13, 2014 12:27 PM
> >>>> To: cerowrt-devel@lists.bufferbloat.net; Black, David
> >>>> Subject: SQM: tracking some diffserv related internet drafts better
> >>>>
> >>>> This appears to be close to finalization, or finalized:
> >>>>
> >>>> http://tools.ietf.org/html/draft-ietf-dart-dscp-rtp-10
> >>>>
> >>>> And this is complementary:
> >>>>
> >>>> http://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-03
> >>>>
> >>>> While wading through all this is tedious, and much of the advice
> >>>> contradictory,
> >>>> there are a few things that could be done more right in the sqm system
> >>>> that I'd like to discuss. (feel free to pour a cup of coffee and read
> >>>> the drafts)
> >>>>
> >>>> -1) They still think the old style tos imm bit is obsolete. Sigh. Am I
> >>>> the last person that uses ssh or plays games?
> >>>>
> >>>> 0) Key to this draft is expecting that the AF code points on a single
> >>>> 5-tuple not be re-ordered, which means dumping AF41 into a priority
> >>>> queue and AF42 into the BE queue is incorrect.
> >>>>
> >>>> 1) SQM only prioritizes a few diffserv codepoints (just the ones for
> >>>> which I had tools doing classification, like ssh). Doing so with tc
> >>>> rules is very inefficient presently. I had basically planned on
> >>>> rolling a new tc and/or iptables filter to "do the right thing" to map
> >>>> into all 64 codepoints via a simple lookup table (as what is in the
> >>>> wifi code already), rather than use the existing mechanism... and
> >>>> hesitated
> >>>> as nobody had nailed down the definitions of each one.
> >>>>
> >>>> That said, I have not measured recently the impact of the extra tc
> >>>> filters and iptables rules required.
> >>>>
> >>>> 1a) Certainly only doing AF42 in sqm is pretty wrong (that was left
> >>>> over from my test patches against mosh - mosh ran with AF42 for a
> >>>> while until they crashed a couple routers with it)
> >>>>
> >>>> The relevant lines are here:
> >>>>
> >>>> https://github.com/dtaht/ceropackages-3.10/blob/master/net/sqm-
> >>>> scripts/files/usr/lib/sqm/functions.sh#L411
> >>>>
> >>>> 1b) The cake code presently does it pretty wrong, which is eminately
> >> fixable.
> >>>>
> >>>> 1c) And given that the standards are settling, it might be time to
> >>>> start baking them into a new tc or iptables filter. This would be a
> >>>> small, interesting project for someone who wants to get their feet wet
> >>>> writing this sort of thing, and examples abound of how to do it.
> >>>>
> >>>> 2) A lot of these diffserv specs - notably all the AFxx codepoints -
> >>>> are all about variable drop probability. (Not that this concept has
> >>>> been proven to work in the real world) We don't do variable drop
> >>>> probability... and I haven't the slightest clue as to how to do it in
> >>>> fq_codel. But keeping variable diffserv codepoints in order on the
> >>>> same 5 tuple seems to be the way things are going. Still I have
> >>>> trouble folding these ideas into the 3 basic queue system fq_codel
> >>>> uses, it looks to me as most of the AF codepoints end up in the
> >>>> current best effort queue, as the priority queue is limited to 30% of
> >>>> the bandwidth by default.
> >>>>
> >>>>
> >>>> 3) Squashing inbound dscp should still be the default option...
> >>>>
> >>>> 4) My patch set to the wifi code for diffserv support disables the VO
> >>>> queue almost entirely in favor of punting things to the VI queue
> >>>> (which can aggregate), but I'm not sure if I handled AFxx
> >>>> appropriately.
> >>>>
> >>>> 5) So far as I know, no browser implements any of this stuff yet. So
> >>>> far as I know nobody actually deployed a router that tries to do smart
> >>>> things with this stuff yet.
> >>>>
> >>>> 6) I really wish there were more codepoints for background traffic than
> >> cs1.
> >>>>
> >>>> --
> >>>> Dave Täht
> >>>>
> >>>> thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks
> >>> _______________________________________________
> >>> Cerowrt-devel mailing list
> >>> Cerowrt-devel@lists.bufferbloat.net
> >>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> >