[Cake] DSCP ramblings

[Cake] DSCP ramblings

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 3656 bytes --]

During these strange times of lockdown I’ve been trying to keep myself occupied/entertained/sane(???) by ‘fiddling with stuff’ and improving my coding.  This started with an idea of learning Python which was great until the on-line bit of it ran out and someone posted an idea on the Openwrt forum about graphing Cake stats.

That had nothing to do with Python and involved (new to me) technologies such as ‘collectd’, ‘JSON’, a bit of javascript and my usual level of cobbling something together in ‘ash’…. So that course was well spent :-)

Anyway, data was collected and graphs produced in a very small household.  What’s immediately apparent from those graphs and cake in ‘diffserv4’ mode is that very, very few applications are using DSCP at all.  Most things are to port 443.

I was also a little surprised to see that my DNS over foo proxies such as stubby & https-dns-proxy don’t use DSCP coding.  It surprised me even more to see RFC recommendations that DNS be treated as ‘Best Effort’.  Now in the days of udp only and no dnssec (with fallback to tcp) this may be good enough, but I wonder if this is realistic these days?

So putting aside the discussion of what codepoint should be used, I then wondered how hard it would be to actually set a dscp in these applications.  And this is where I had another surprise.  For example https-dns-proxy uses libcurl.  libcurl has no standard ‘in-library’ method for setting a socket’s dscp.  I cobbled a workaround in the application https://github.com/aarond10/https_dns_proxy/pull/83 - it works.

Next I attacked stubby, which uses getdns.  getdns doesn’t even have a callback or parameters passing so you can set a dscp on the socket from a client application, pure ‘hack the library’ stuff.

To be blunt and on a small sample of 2 libraries/applications, it seems that DSCP is completely ignored.  Applications signalling ’this is/isnt latency sensitive/bulk’ isn’t going to happen if it isn’t easy to do.

Apple should be marking facetime calls as being ‘video conference’ or whatever.  BBC iplayer Radio apps should be marking ‘audio streaming’. But every f*ing thing is CS0 port 443.  And I’m wondering how much of this is because library support is simply missing.  Maybe gaming apps are better? (I don’t game)

Right, I’m off for a lie down.  Sorry for the rant.


Hack for getdns/stubby

diff --git a/src/stub.c b/src/stub.c
index 2547d10f..7e47aba5 100644
--- a/src/stub.c
+++ b/src/stub.c
@@ -52,6 +52,7 @@
 #include "platform.h"
 #include "general.h"
 #include "pubkey-pinning.h"
+#include <netinet/ip.h>

 /* WSA TODO:
  * STUB_TCP_RETRY added to deal with edge triggered event loops (versus
@@ -381,6 +382,9 @@ tcp_connect(getdns_upstream *upstream, getdns_transport_list_t transport)
 # else
        static const int  enable = 1;
 # endif
+#endif
+#if defined(IP_TOS)
+       int dscp = IPTOS_CLASS_CS4;
 #endif
        int fd = -1;

@@ -390,6 +394,12 @@ tcp_connect(getdns_upstream *upstream, getdns_transport_list_t transport)
                   __FUNC__, (void*)upstream);
        if ((fd = socket(upstream->addr.ss_family, SOCK_STREAM, IPPROTO_TCP)) == -1)
                return -1;
+#if defined(IP_TOS)
+       if (upstream->addr.ss_family == AF_INET6)
+               (void)setsockopt(fd, IPPROTO_IPV6, IP_TOS, &dscp, sizeof(dscp));
+       else if (upstream->addr.ss_family == AF_INET)
+               (void)setsockopt(fd, IPPROTO_IP, IP_TOS, &dscp, sizeof(dscp));
+#endif


Cheers,

Kevin D-B

gpg: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Cake] DSCP ramblings

[Dave Taht]

On Wed, Apr 22, 2020 at 8:58 AM Kevin Darbyshire-Bryant
<kevin@darbyshire-bryant.me.uk> wrote:
>
> During these strange times of lockdown I’ve been trying to keep myself occupied/entertained/sane(???) by ‘fiddling with stuff’ and improving my coding.  This started with an idea of learning Python which was great until the on-line bit of it ran out and someone posted an idea on the Openwrt forum about graphing Cake stats.
>
> That had nothing to do with Python and involved (new to me) technologies such as ‘collectd’, ‘JSON’, a bit of javascript and my usual level of cobbling something together in ‘ash’…. So that course was well spent :-)
>
> Anyway, data was collected and graphs produced in a very small household.  What’s immediately apparent from those graphs and cake in ‘diffserv4’ mode is that very, very few applications are using DSCP at all.  Most things are to port 443.
>
> I was also a little surprised to see that my DNS over foo proxies such as stubby & https-dns-proxy don’t use DSCP coding.  It surprised me even more to see RFC recommendations that DNS be treated as ‘Best Effort’.  Now in the days of udp only and no dnssec (with fallback to tcp) this may be good enough, but I wonder if this is realistic these days?
>
> So putting aside the discussion of what codepoint should be used, I then wondered how hard it would be to actually set a dscp in these applications.  And this is where I had another surprise.  For example https-dns-proxy uses libcurl.  libcurl has no standard ‘in-library’ method for setting a socket’s dscp.  I cobbled a workaround in the application https://github.com/aarond10/https_dns_proxy/pull/83 - it works.
>
> Next I attacked stubby, which uses getdns.  getdns doesn’t even have a callback or parameters passing so you can set a dscp on the socket from a client application, pure ‘hack the library’ stuff.
>
> To be blunt and on a small sample of 2 libraries/applications, it seems that DSCP is completely ignored.  Applications signalling ’this is/isnt latency sensitive/bulk’ isn’t going to happen if it isn’t easy to do.
>
> Apple should be marking facetime calls as being ‘video conference’ or whatever.  BBC iplayer Radio apps should be marking ‘audio streaming’. But every f*ing thing is CS0 port 443.  And I’m wondering how much of this is because library support is simply missing.  Maybe gaming apps are better? (I don’t game)
>
> Right, I’m off for a lie down.  Sorry for the rant.

Welcome to my explorations... in 2011. Diffserv is rather underused, isn't it?

I took a survey of every (500+) gaming console at a convention. nearly
zero diffserv usage and it was all over the map, and I think, mostly,
from osx.

windows requires admin privs to set the tos bits at all
webrtc has an api to set the bits, but it doesn't work on windows.

ssh will set the imm bit for interactive, I forget what it sets for bulk
bgp sets cs6. so does babel. Arguably both usages are wrong.
some windows stuff sets cs1 for things like ping
I got the mosh folk to use AF42 as a (worldwide) test, for nearly a
year. they had one user with a problem and they turned it off. It was
funny, keith thought I was making an expert recommendation rather than
a test and just copy pasted my code into the tree and shipped it.

linux implements a strict priority queue in pfifo_fast. You can dos it
if you hit it by setting the bits.
irtt and netperf let you set the bits. iperf also.

I produced a patch for rsync in particular (since I use it heavily)

sqm at least used to mark dns and ntp as some elivated prio, but I
forget which and for all I know the cake qos system doesn't implement
those filters.

A few multi-queue ethernet devices actually do interpret the bits.
Undocumented as to which one..

and lets not get started on ecn.

>
>
> Hack for getdns/stubby
>
> diff --git a/src/stub.c b/src/stub.c
> index 2547d10f..7e47aba5 100644
> --- a/src/stub.c
> +++ b/src/stub.c
> @@ -52,6 +52,7 @@
>  #include "platform.h"
>  #include "general.h"
>  #include "pubkey-pinning.h"
> +#include <netinet/ip.h>
>
>  /* WSA TODO:
>   * STUB_TCP_RETRY added to deal with edge triggered event loops (versus
> @@ -381,6 +382,9 @@ tcp_connect(getdns_upstream *upstream, getdns_transport_list_t transport)
>  # else
>         static const int  enable = 1;
>  # endif
> +#endif
> +#if defined(IP_TOS)
> +       int dscp = IPTOS_CLASS_CS4;
>  #endif
>         int fd = -1;
>
> @@ -390,6 +394,12 @@ tcp_connect(getdns_upstream *upstream, getdns_transport_list_t transport)
>                    __FUNC__, (void*)upstream);
>         if ((fd = socket(upstream->addr.ss_family, SOCK_STREAM, IPPROTO_TCP)) == -1)
>                 return -1;
> +#if defined(IP_TOS)
> +       if (upstream->addr.ss_family == AF_INET6)
> +               (void)setsockopt(fd, IPPROTO_IPV6, IP_TOS, &dscp, sizeof(dscp));
> +       else if (upstream->addr.ss_family == AF_INET)
> +               (void)setsockopt(fd, IPPROTO_IP, IP_TOS, &dscp, sizeof(dscp));
> +#endif
>
>
> Cheers,
>
> Kevin D-B
>
> gpg: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Cake] DSCP ramblings

[Dave Taht]

and because of your I'm off building collectd because those graphs
look so good. :)

https://forum.openwrt.org/t/sqm-reporting/59960/24

I have long just used snmpd, and collectd looks interesting. I fear
it's too heavyweight, particularly shelling out to a script....

On Wed, Apr 22, 2020 at 9:15 AM Dave Taht <dave.taht@gmail.com> wrote:
>
> On Wed, Apr 22, 2020 at 8:58 AM Kevin Darbyshire-Bryant
> <kevin@darbyshire-bryant.me.uk> wrote:
> >
> > During these strange times of lockdown I’ve been trying to keep myself occupied/entertained/sane(???) by ‘fiddling with stuff’ and improving my coding.  This started with an idea of learning Python which was great until the on-line bit of it ran out and someone posted an idea on the Openwrt forum about graphing Cake stats.
> >
> > That had nothing to do with Python and involved (new to me) technologies such as ‘collectd’, ‘JSON’, a bit of javascript and my usual level of cobbling something together in ‘ash’…. So that course was well spent :-)
> >
> > Anyway, data was collected and graphs produced in a very small household.  What’s immediately apparent from those graphs and cake in ‘diffserv4’ mode is that very, very few applications are using DSCP at all.  Most things are to port 443.
> >
> > I was also a little surprised to see that my DNS over foo proxies such as stubby & https-dns-proxy don’t use DSCP coding.  It surprised me even more to see RFC recommendations that DNS be treated as ‘Best Effort’.  Now in the days of udp only and no dnssec (with fallback to tcp) this may be good enough, but I wonder if this is realistic these days?
> >
> > So putting aside the discussion of what codepoint should be used, I then wondered how hard it would be to actually set a dscp in these applications.  And this is where I had another surprise.  For example https-dns-proxy uses libcurl.  libcurl has no standard ‘in-library’ method for setting a socket’s dscp.  I cobbled a workaround in the application https://github.com/aarond10/https_dns_proxy/pull/83 - it works.
> >
> > Next I attacked stubby, which uses getdns.  getdns doesn’t even have a callback or parameters passing so you can set a dscp on the socket from a client application, pure ‘hack the library’ stuff.
> >
> > To be blunt and on a small sample of 2 libraries/applications, it seems that DSCP is completely ignored.  Applications signalling ’this is/isnt latency sensitive/bulk’ isn’t going to happen if it isn’t easy to do.
> >
> > Apple should be marking facetime calls as being ‘video conference’ or whatever.  BBC iplayer Radio apps should be marking ‘audio streaming’. But every f*ing thing is CS0 port 443.  And I’m wondering how much of this is because library support is simply missing.  Maybe gaming apps are better? (I don’t game)
> >
> > Right, I’m off for a lie down.  Sorry for the rant.
>
> Welcome to my explorations... in 2011. Diffserv is rather underused, isn't it?
>
> I took a survey of every (500+) gaming console at a convention. nearly
> zero diffserv usage and it was all over the map, and I think, mostly,
> from osx.
>
> windows requires admin privs to set the tos bits at all
> webrtc has an api to set the bits, but it doesn't work on windows.
>
> ssh will set the imm bit for interactive, I forget what it sets for bulk
> bgp sets cs6. so does babel. Arguably both usages are wrong.
> some windows stuff sets cs1 for things like ping
> I got the mosh folk to use AF42 as a (worldwide) test, for nearly a
> year. they had one user with a problem and they turned it off. It was
> funny, keith thought I was making an expert recommendation rather than
> a test and just copy pasted my code into the tree and shipped it.
>
> linux implements a strict priority queue in pfifo_fast. You can dos it
> if you hit it by setting the bits.
> irtt and netperf let you set the bits. iperf also.
>
> I produced a patch for rsync in particular (since I use it heavily)
>
> sqm at least used to mark dns and ntp as some elivated prio, but I
> forget which and for all I know the cake qos system doesn't implement
> those filters.
>
> A few multi-queue ethernet devices actually do interpret the bits.
> Undocumented as to which one..
>
> and lets not get started on ecn.
>
> >
> >
> > Hack for getdns/stubby
> >
> > diff --git a/src/stub.c b/src/stub.c
> > index 2547d10f..7e47aba5 100644
> > --- a/src/stub.c
> > +++ b/src/stub.c
> > @@ -52,6 +52,7 @@
> >  #include "platform.h"
> >  #include "general.h"
> >  #include "pubkey-pinning.h"
> > +#include <netinet/ip.h>
> >
> >  /* WSA TODO:
> >   * STUB_TCP_RETRY added to deal with edge triggered event loops (versus
> > @@ -381,6 +382,9 @@ tcp_connect(getdns_upstream *upstream, getdns_transport_list_t transport)
> >  # else
> >         static const int  enable = 1;
> >  # endif
> > +#endif
> > +#if defined(IP_TOS)
> > +       int dscp = IPTOS_CLASS_CS4;
> >  #endif
> >         int fd = -1;
> >
> > @@ -390,6 +394,12 @@ tcp_connect(getdns_upstream *upstream, getdns_transport_list_t transport)
> >                    __FUNC__, (void*)upstream);
> >         if ((fd = socket(upstream->addr.ss_family, SOCK_STREAM, IPPROTO_TCP)) == -1)
> >                 return -1;
> > +#if defined(IP_TOS)
> > +       if (upstream->addr.ss_family == AF_INET6)
> > +               (void)setsockopt(fd, IPPROTO_IPV6, IP_TOS, &dscp, sizeof(dscp));
> > +       else if (upstream->addr.ss_family == AF_INET)
> > +               (void)setsockopt(fd, IPPROTO_IP, IP_TOS, &dscp, sizeof(dscp));
> > +#endif
> >
> >
> > Cheers,
> >
> > Kevin D-B
> >
> > gpg: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A
> >
> > _______________________________________________
> > Cake mailing list
> > Cake@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
>
>
>
> --
> Make Music, Not War
>
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-435-0729



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Cake] DSCP ramblings

[Stephen Hemminger]

On Wed, 22 Apr 2020 09:20:29 -0700
Dave Taht <dave.taht@gmail.com> wrote:

> and because of your I'm off building collectd because those graphs
> look so good. :)
> 
> https://forum.openwrt.org/t/sqm-reporting/59960/24
> 
> I have long just used snmpd, and collectd looks interesting. I fear
> it's too heavyweight, particularly shelling out to a script....
> 
> On Wed, Apr 22, 2020 at 9:15 AM Dave Taht <dave.taht@gmail.com> wrote:
> >
> > On Wed, Apr 22, 2020 at 8:58 AM Kevin Darbyshire-Bryant
> > <kevin@darbyshire-bryant.me.uk> wrote:  
> > >
> > > During these strange times of lockdown I’ve been trying to keep myself occupied/entertained/sane(???) by ‘fiddling with stuff’ and improving my coding.  This started with an idea of learning Python which was great until the on-line bit of it ran out and someone posted an idea on the Openwrt forum about graphing Cake stats.
> > >
> > > That had nothing to do with Python and involved (new to me) technologies such as ‘collectd’, ‘JSON’, a bit of javascript and my usual level of cobbling something together in ‘ash’…. So that course was well spent :-)
> > >
> > > Anyway, data was collected and graphs produced in a very small household.  What’s immediately apparent from those graphs and cake in ‘diffserv4’ mode is that very, very few applications are using DSCP at all.  Most things are to port 443.
> > >
> > > I was also a little surprised to see that my DNS over foo proxies such as stubby & https-dns-proxy don’t use DSCP coding.  It surprised me even more to see RFC recommendations that DNS be treated as ‘Best Effort’.  Now in the days of udp only and no dnssec (with fallback to tcp) this may be good enough, but I wonder if this is realistic these days?
> > >
> > > So putting aside the discussion of what codepoint should be used, I then wondered how hard it would be to actually set a dscp in these applications.  And this is where I had another surprise.  For example https-dns-proxy uses libcurl.  libcurl has no standard ‘in-library’ method for setting a socket’s dscp.  I cobbled a workaround in the application https://github.com/aarond10/https_dns_proxy/pull/83 - it works.
> > >
> > > Next I attacked stubby, which uses getdns.  getdns doesn’t even have a callback or parameters passing so you can set a dscp on the socket from a client application, pure ‘hack the library’ stuff.
> > >
> > > To be blunt and on a small sample of 2 libraries/applications, it seems that DSCP is completely ignored.  Applications signalling ’this is/isnt latency sensitive/bulk’ isn’t going to happen if it isn’t easy to do.
> > >
> > > Apple should be marking facetime calls as being ‘video conference’ or whatever.  BBC iplayer Radio apps should be marking ‘audio streaming’. But every f*ing thing is CS0 port 443.  And I’m wondering how much of this is because library support is simply missing.  Maybe gaming apps are better? (I don’t game)
> > >
> > > Right, I’m off for a lie down.  Sorry for the rant.  
> >
> > Welcome to my explorations... in 2011. Diffserv is rather underused, isn't it?
> >
> > I took a survey of every (500+) gaming console at a convention. nearly
> > zero diffserv usage and it was all over the map, and I think, mostly,
> > from osx.
> >
> > windows requires admin privs to set the tos bits at all
> > webrtc has an api to set the bits, but it doesn't work on windows.
> >
> > ssh will set the imm bit for interactive, I forget what it sets for bulk
> > bgp sets cs6. so does babel. Arguably both usages are wrong.
> > some windows stuff sets cs1 for things like ping
> > I got the mosh folk to use AF42 as a (worldwide) test, for nearly a
> > year. they had one user with a problem and they turned it off. It was
> > funny, keith thought I was making an expert recommendation rather than
> > a test and just copy pasted my code into the tree and shipped it.
> >
> > linux implements a strict priority queue in pfifo_fast. You can dos it
> > if you hit it by setting the bits.
> > irtt and netperf let you set the bits. iperf also.
> >
> > I produced a patch for rsync in particular (since I use it heavily)
> >
> > sqm at least used to mark dns and ntp as some elivated prio, but I
> > forget which and for all I know the cake qos system doesn't implement
> > those filters.
> >
> > A few multi-queue ethernet devices actually do interpret the bits.
> > Undocumented as to which one..
> >
> > and lets not get started on ecn.
> >  
> > >
> > >
> > > Hack for getdns/stubby
> > >
> > > diff --git a/src/stub.c b/src/stub.c
> > > index 2547d10f..7e47aba5 100644
> > > --- a/src/stub.c
> > > +++ b/src/stub.c
> > > @@ -52,6 +52,7 @@
> > >  #include "platform.h"
> > >  #include "general.h"
> > >  #include "pubkey-pinning.h"
> > > +#include <netinet/ip.h>
> > >
> > >  /* WSA TODO:
> > >   * STUB_TCP_RETRY added to deal with edge triggered event loops (versus
> > > @@ -381,6 +382,9 @@ tcp_connect(getdns_upstream *upstream, getdns_transport_list_t transport)
> > >  # else
> > >         static const int  enable = 1;
> > >  # endif
> > > +#endif
> > > +#if defined(IP_TOS)
> > > +       int dscp = IPTOS_CLASS_CS4;
> > >  #endif
> > >         int fd = -1;
> > >
> > > @@ -390,6 +394,12 @@ tcp_connect(getdns_upstream *upstream, getdns_transport_list_t transport)
> > >                    __FUNC__, (void*)upstream);
> > >         if ((fd = socket(upstream->addr.ss_family, SOCK_STREAM, IPPROTO_TCP)) == -1)
> > >                 return -1;
> > > +#if defined(IP_TOS)
> > > +       if (upstream->addr.ss_family == AF_INET6)
> > > +               (void)setsockopt(fd, IPPROTO_IPV6, IP_TOS, &dscp, sizeof(dscp));
> > > +       else if (upstream->addr.ss_family == AF_INET)
> > > +               (void)setsockopt(fd, IPPROTO_IP, IP_TOS, &dscp, sizeof(dscp));
> > > +#endif
> > >
> > >
> > > Cheers,
> > >
> > > Kevin D-B
> > >
> > > gpg: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A
> > >

In my experience, except for a small number of cases (RDMA etc) Diffserv is a
complete waste of time. There is no global ordering, there is no guarantee against
starvation and any sane ISP strips the bits off or ignores them.

Diffserv is even an issue at scale in the cloud. What does DSCP mean exactly on
outer headers, who gets to decide for which service. And what about inner headers
and propogating inner to outer. Its a mess.

Re: [Cake] DSCP ramblings

[Dave Taht]

On Wed, Apr 22, 2020 at 9:44 AM Stephen Hemminger
<stephen@networkplumber.org> wrote:
>
> On Wed, 22 Apr 2020 09:20:29 -0700
> Dave Taht <dave.taht@gmail.com> wrote:
>
> > and because of your I'm off building collectd because those graphs
> > look so good. :)
> >
> > https://forum.openwrt.org/t/sqm-reporting/59960/24
> >
> > I have long just used snmpd, and collectd looks interesting. I fear
> > it's too heavyweight, particularly shelling out to a script....
> >
> > On Wed, Apr 22, 2020 at 9:15 AM Dave Taht <dave.taht@gmail.com> wrote:
> > >
> > > On Wed, Apr 22, 2020 at 8:58 AM Kevin Darbyshire-Bryant
> > > <kevin@darbyshire-bryant.me.uk> wrote:
> > > >
> > > > During these strange times of lockdown I’ve been trying to keep myself occupied/entertained/sane(???) by ‘fiddling with stuff’ and improving my coding.  This started with an idea of learning Python which was great until the on-line bit of it ran out and someone posted an idea on the Openwrt forum about graphing Cake stats.
> > > >
> > > > That had nothing to do with Python and involved (new to me) technologies such as ‘collectd’, ‘JSON’, a bit of javascript and my usual level of cobbling something together in ‘ash’…. So that course was well spent :-)
> > > >
> > > > Anyway, data was collected and graphs produced in a very small household.  What’s immediately apparent from those graphs and cake in ‘diffserv4’ mode is that very, very few applications are using DSCP at all.  Most things are to port 443.
> > > >
> > > > I was also a little surprised to see that my DNS over foo proxies such as stubby & https-dns-proxy don’t use DSCP coding.  It surprised me even more to see RFC recommendations that DNS be treated as ‘Best Effort’.  Now in the days of udp only and no dnssec (with fallback to tcp) this may be good enough, but I wonder if this is realistic these days?
> > > >
> > > > So putting aside the discussion of what codepoint should be used, I then wondered how hard it would be to actually set a dscp in these applications.  And this is where I had another surprise.  For example https-dns-proxy uses libcurl.  libcurl has no standard ‘in-library’ method for setting a socket’s dscp.  I cobbled a workaround in the application https://github.com/aarond10/https_dns_proxy/pull/83 - it works.
> > > >
> > > > Next I attacked stubby, which uses getdns.  getdns doesn’t even have a callback or parameters passing so you can set a dscp on the socket from a client application, pure ‘hack the library’ stuff.
> > > >
> > > > To be blunt and on a small sample of 2 libraries/applications, it seems that DSCP is completely ignored.  Applications signalling ’this is/isnt latency sensitive/bulk’ isn’t going to happen if it isn’t easy to do.
> > > >
> > > > Apple should be marking facetime calls as being ‘video conference’ or whatever.  BBC iplayer Radio apps should be marking ‘audio streaming’. But every f*ing thing is CS0 port 443.  And I’m wondering how much of this is because library support is simply missing.  Maybe gaming apps are better? (I don’t game)
> > > >
> > > > Right, I’m off for a lie down.  Sorry for the rant.
> > >
> > > Welcome to my explorations... in 2011. Diffserv is rather underused, isn't it?
> > >
> > > I took a survey of every (500+) gaming console at a convention. nearly
> > > zero diffserv usage and it was all over the map, and I think, mostly,
> > > from osx.
> > >
> > > windows requires admin privs to set the tos bits at all
> > > webrtc has an api to set the bits, but it doesn't work on windows.
> > >
> > > ssh will set the imm bit for interactive, I forget what it sets for bulk
> > > bgp sets cs6. so does babel. Arguably both usages are wrong.
> > > some windows stuff sets cs1 for things like ping
> > > I got the mosh folk to use AF42 as a (worldwide) test, for nearly a
> > > year. they had one user with a problem and they turned it off. It was
> > > funny, keith thought I was making an expert recommendation rather than
> > > a test and just copy pasted my code into the tree and shipped it.
> > >
> > > linux implements a strict priority queue in pfifo_fast. You can dos it
> > > if you hit it by setting the bits.
> > > irtt and netperf let you set the bits. iperf also.
> > >
> > > I produced a patch for rsync in particular (since I use it heavily)
> > >
> > > sqm at least used to mark dns and ntp as some elivated prio, but I
> > > forget which and for all I know the cake qos system doesn't implement
> > > those filters.
> > >
> > > A few multi-queue ethernet devices actually do interpret the bits.
> > > Undocumented as to which one..
> > >
> > > and lets not get started on ecn.
> > >
> > > >
> > > >
> > > > Hack for getdns/stubby
> > > >
> > > > diff --git a/src/stub.c b/src/stub.c
> > > > index 2547d10f..7e47aba5 100644
> > > > --- a/src/stub.c
> > > > +++ b/src/stub.c
> > > > @@ -52,6 +52,7 @@
> > > >  #include "platform.h"
> > > >  #include "general.h"
> > > >  #include "pubkey-pinning.h"
> > > > +#include <netinet/ip.h>
> > > >
> > > >  /* WSA TODO:
> > > >   * STUB_TCP_RETRY added to deal with edge triggered event loops (versus
> > > > @@ -381,6 +382,9 @@ tcp_connect(getdns_upstream *upstream, getdns_transport_list_t transport)
> > > >  # else
> > > >         static const int  enable = 1;
> > > >  # endif
> > > > +#endif
> > > > +#if defined(IP_TOS)
> > > > +       int dscp = IPTOS_CLASS_CS4;
> > > >  #endif
> > > >         int fd = -1;
> > > >
> > > > @@ -390,6 +394,12 @@ tcp_connect(getdns_upstream *upstream, getdns_transport_list_t transport)
> > > >                    __FUNC__, (void*)upstream);
> > > >         if ((fd = socket(upstream->addr.ss_family, SOCK_STREAM, IPPROTO_TCP)) == -1)
> > > >                 return -1;
> > > > +#if defined(IP_TOS)
> > > > +       if (upstream->addr.ss_family == AF_INET6)
> > > > +               (void)setsockopt(fd, IPPROTO_IPV6, IP_TOS, &dscp, sizeof(dscp));
> > > > +       else if (upstream->addr.ss_family == AF_INET)
> > > > +               (void)setsockopt(fd, IPPROTO_IP, IP_TOS, &dscp, sizeof(dscp));
> > > > +#endif
> > > >
> > > >
> > > > Cheers,
> > > >
> > > > Kevin D-B
> > > >
> > > > gpg: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A
> > > >
>
> In my experience, except for a small number of cases (RDMA etc) Diffserv is a
> complete waste of time. There is no global ordering, there is no guarantee against
> starvation and any sane ISP strips the bits off or ignores them.

I am under the impression that comcast at least, uses diffserv
internally, and also has all kinds of fun
mapping from MPLS back and forth. they also annoyingly, on some
CMTSes, remark most traffic to CS1,
which is why I was so insistent on getting the wash option into cake.

diffserv is totally OK on your internal network, but most of the time,
you really, really don't care. really time
sensitive traffic gets tossed onto a priority vlan....

given the track record of diffserv, the idea of handing ECT(1) to the
same folk as an identifier does not seem like
a good idea, either.

the first thing cake got when it hit the real world was proper tc
support so you could identify and prioritize packets
without twiddling the diffserv field. That said, I do rather support
the ability to mark you own traffic within your domain,
and in particular, I like the idea of a least effort or background
marking for traffic you don't care about much.

transmission does support qos markings... for tcp traffic.ledbat, no.
Bunch of one line patches
to a lot of tools and then where do we get?

> Diffserv is even an issue at scale in the cloud. What does DSCP mean exactly on
> outer headers, who gets to decide for which service. And what about inner headers
> and propogating inner to outer. Its a mess.

yep. we tried, by following the proposed standards for webrtc and so
on, to come up with
a sane set of interpretations. If you think diffserv was a messy idea... see:

https://datatracker.ietf.org/doc/draft-white-tsvwg-nqb/

and for that matter the recent thread on a new hop by hop options
header from china telecom on netdev titled

"net: ipv6: support Application-aware IPv6 Network (APN6)"

inmates, asylum, running amuck.



>


--
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Cake] DSCP ramblings

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 1000 bytes --]



> On 22 Apr 2020, at 17:20, Dave Taht <dave.taht@gmail.com> wrote:
> 
> and because of your I'm off building collectd because those graphs
> look so good. :)

Oh dear, sorry about that :-)  The collection bit https://github.com/ldir-EDB0/packages/commit/932bb4b022bdbf3ab0fa1e43842f7c94da7f046a
The display bit https://github.com/ldir-EDB0/luci/commit/a0a95da1703079887a85c4d9b6929e74d2c77a29

Don’t break it, or if you do, send fixes ;-)

The idea of using collectd_exec and hence a sh script was the quickest way of spinning something up.  It is inherently going to be heavier than a proper C based plugin/collector…and beyond my skill/patience limits (You should have seen how many combinations of ‘*' & ‘&’ were involved in getting https-dns-proxy/lubcurl "static curl_socket_t opensocket_callback(void *clientp, curlsocktype purpose, struct curl_sockaddr *addr)  (void)setsockopt(sock, IPPROTO_IPV6, IP_TOS, (int *)clientp, sizeof(int));” to work :-) )

Kevin




[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Cake] DSCP ramblings

[Dave Taht]

On Wed, Apr 22, 2020 at 10:17 AM Kevin Darbyshire-Bryant
<kevin@darbyshire-bryant.me.uk> wrote:
>
>
>
> > On 22 Apr 2020, at 17:20, Dave Taht <dave.taht@gmail.com> wrote:
> >
> > and because of your I'm off building collectd because those graphs
> > look so good. :)
>
> Oh dear, sorry about that :-)  The collection bit https://github.com/ldir-EDB0/packages/commit/932bb4b022bdbf3ab0fa1e43842f7c94da7f046a
> The display bit https://github.com/ldir-EDB0/luci/commit/a0a95da1703079887a85c4d9b6929e74d2c77a29
>
> Don’t break it, or if you do, send fixes ;-)

I just - amazingly - for being years out of practice - before my first
cup of coffee - patched openwrt for reducing the codel target, updated
babeld to 1.9.2, and the kernel to 5.4... reflashed a ubnt mesh ap...
and it worked, first time. I am afraid to push my luck, further.

>
> The idea of using collectd_exec and hence a sh script was the quickest way of spinning something up.  It is inherently going to be heavier than a proper C based plugin/collector…and beyond my skill/patience limits

I get it. :)

My intent however is to somehow have the collector send stuff back to
another collector elsewhere and not
store anything locally. haven't figured out how to do that. (The
uap-lite mesh only has 8MB flash)


>(You should have seen how many combinations of ‘*' & ‘&’ were involved in getting https-dns-proxy/lubcurl "static curl_socket_t opensocket_callback(void *clientp, curlsocktype purpose, struct curl_sockaddr *addr)  (void)setsockopt(sock, IPPROTO_IPV6, IP_TOS, (int *)clientp, sizeof(int));” to work :-) )

the cdecl tool is your friend here.

> Kevin
>
>
>


-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Cake] DSCP ramblings

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 2439 bytes --]



> On 22 Apr 2020, at 17:44, Stephen Hemminger <stephen@networkplumber.org> wrote:
> 
> In my experience, except for a small number of cases (RDMA etc) Diffserv is a
> complete waste of time. There is no global ordering, there is no guarantee against
> starvation and any sane ISP strips the bits off or ignores them.
> 
> Diffserv is even an issue at scale in the cloud. What does DSCP mean exactly on
> outer headers, who gets to decide for which service. And what about inner headers
> and propogating inner to outer. Its a mess.

That’s…depressing :-/ And suggests there are at least 6 bits spare in the ’TOS’ byte, perhaps that should go forward at the next IETF meeting ;-)

In my own naive home network fiefdom I would like to have some level of control as to who/what gets the lion’s/fairshare of my ISP wan link.  I don’t actually care whether that DSCP decision makes it/returns on the wan link…I have my own combination of tc act_ctinfo (in upstream linux) and ‘iptables connmark —setdscp’ (not in upstream linux yet) to do my DSCP mark saving/restoration, so what I set sticks and affects CAKE’s tin allocation & shaping decisions.  I want my wife’s facetime call to her sister to work perfectly, I want my bittorrenting to sit there in the background unnoticed, I want my network backup job/s to run in good time but not at the expense of interactive web browsing, ssh’ing etc.  I want my BBC iplayer radio to sit there streaming away at the best quality, no interruptions.  It’s all possible.

As it currently stands I’ve a series of guestimate iptables rules mainly based on source or destination ip address (occasionally a bit of port number) but that’s pretty coarse.  Why can’t the application do it?

LE - Least Effort - 0/16 can be starved - bittorrent
GE - Good Effort - 1/16 not starved - background downloads, windows update
BE - Best Effort - 16/16 Normal activity, default
SP - Streaming Priority - 8/16 high bitrate streaming, Video/Audio streaming,  Video portion of video conferencing
IIP - Interactive/Important Priority - 4/16 low bitrate streaming, SIP/VOIP, interactive SSH, DNS, Audio portion of video conferencing

I’m still stunned/shocked by the lack of obvious DSCP support in libcurl.  If DSCP setting isn’t easy to do, no one will do it.  Perhaps if it is built, people will come?

I’m probably stupid.  And now depressed!

Kevin

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]