[Bloat] this explains speedtest stuff

[Bloat] this explains speedtest stuff

[Dave Taht]

dslreports.com is only on the third page of the search results.

https://www.google.com/search?q=internet+speed+test

Re: [Bloat] this explains speedtest stuff

[Jonathan Foulkes]

Confirmed, and I go there all time as well, you’d think it would be the first thing Google would show us.

At least Fast.com is on the first page, but they don’t pro-actively show latency tests, especially on the upload.

BTW- I’ve suggested they support a URL request format where we can pre-set options that engage and show results for the bloat tests. This way we can share that pre-formated link and the users who click on it immediately see a bloat metric.
Maybe if a few more suggest this as well, it will climb in priority. 

- Jonathan

> On Apr 23, 2020, at 3:38 PM, Dave Taht <dave.taht@gmail.com> wrote:
> 
> dslreports.com is only on the third page of the search results.
> 
> https://www.google.com/search?q=internet+speed+test
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] this explains speedtest stuff

[Sergey Fedorov]

[-- Attachment #1: Type: text/plain, Size: 1597 bytes --]

Hi folks,

No need to put more pressure - I've seen Jonathan's suggestion and it makes
a lot of sense to add the option to deep-link to an expanded version with
all detailed parameters shown.
This will be added some time this quarter (Q2), but not within a few next
weeks.

SERGEY FEDOROV

Director of Engineering

sfedorov@netflix.com

121 Albright Way | Los Gatos, CA 95032



On Thu, Apr 23, 2020 at 1:00 PM Jonathan Foulkes <jf@jonathanfoulkes.com>
wrote:

> Confirmed, and I go there all time as well, you’d think it would be the
> first thing Google would show us.
>
> At least Fast.com is on the first page, but they don’t pro-actively show
> latency tests, especially on the upload.
>
> BTW- I’ve suggested they support a URL request format where we can pre-set
> options that engage and show results for the bloat tests. This way we can
> share that pre-formated link and the users who click on it immediately see
> a bloat metric.
> Maybe if a few more suggest this as well, it will climb in priority.
>
> - Jonathan
>
> > On Apr 23, 2020, at 3:38 PM, Dave Taht <dave.taht@gmail.com> wrote:
> >
> > dslreports.com is only on the third page of the search results.
> >
> > https://www.google.com/search?q=internet+speed+test
> > _______________________________________________
> > Bloat mailing list
> > Bloat@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

[-- Attachment #2: Type: text/html, Size: 3807 bytes --]

Re: [Bloat] this explains speedtest stuff

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 2646 bytes --]

On Thu, Apr 23, 2020 at 1:15 PM Sergey Fedorov <sfedorov@netflix.com> wrote:

> Hi folks,
>
> No need to put more pressure - I've seen Jonathan's suggestion and it
> makes a lot of sense to add the option to deep-link to an expanded version
> with all detailed parameters shown.
> This will be added some time this quarter (Q2), but not within a few next
> weeks.
>

that's wonderful, thanks!

One thing I've been seeing is way too many articles basically talking about
traffic increases (or not), and how many are videoconferencing from home...
but no metrics of import to videoconferencing folk. I've been engaged in a
conversation about increasing a certain videoconferencing platform's
default jitterbuffer to... wait for it... *1 sec* - based on how badly LTE
was performing for some people

so if you control your own videoconferencing tools, collecting more metrics
there would be usef

One very interesting metric a netflix streamer could be collecting is
differences in tcp RTT (assuming you slowed traffic down in europe,
especially, to a lower quality?), hour by hour, day by day.

another one is packet loss... retransmits...



> SERGEY FEDOROV
>
> Director of Engineering
>
> sfedorov@netflix.com
>
> 121 Albright Way | Los Gatos, CA 95032
>
>
>
> On Thu, Apr 23, 2020 at 1:00 PM Jonathan Foulkes <jf@jonathanfoulkes.com>
> wrote:
>
>> Confirmed, and I go there all time as well, you’d think it would be the
>> first thing Google would show us.
>>
>> At least Fast.com is on the first page, but they don’t pro-actively show
>> latency tests, especially on the upload.
>>
>> BTW- I’ve suggested they support a URL request format where we can
>> pre-set options that engage and show results for the bloat tests. This way
>> we can share that pre-formated link and the users who click on it
>> immediately see a bloat metric.
>> Maybe if a few more suggest this as well, it will climb in priority.
>>
>> - Jonathan
>>
>> > On Apr 23, 2020, at 3:38 PM, Dave Taht <dave.taht@gmail.com> wrote:
>> >
>> > dslreports.com is only on the third page of the search results.
>> >
>> > https://www.google.com/search?q=internet+speed+test
>> > _______________________________________________
>> > Bloat mailing list
>> > Bloat@lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/bloat
>>
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>>
>

-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

[-- Attachment #2: Type: text/html, Size: 5517 bytes --]

Re: [Bloat] this explains speedtest stuff

[Sergey Fedorov]

[-- Attachment #1: Type: text/plain, Size: 4065 bytes --]

>
> One thing I've been seeing is way too many articles basically talking
> about traffic increases (or not), and how many are videoconferencing from
> home... but no metrics of import to videoconferencing folk. I've been
> engaged in a conversation about increasing a certain videoconferencing
> platform's default jitterbuffer to... wait for it... *1 sec* - based on how
> badly LTE was performing for some people
> so if you control your own videoconferencing tools, collecting more
> metrics there would be usef
> One very interesting metric a netflix streamer could be collecting is
> differences in tcp RTT (assuming you slowed traffic down in europe,
> especially, to a lower quality?), hour by hour, day by day.

All good points and I generally agree with your observation on lack of good
resources/recommendations to improve latency-sensitive network
interactions, especially as those are becoming critical for the users.
I hope we'll be able to share some of the
findings/observations/recommendations based on our experience (white paper,
blog etc), but no hard promises at this point.

SERGEY FEDOROV

Director of Engineering

sfedorov@netflix.com

121 Albright Way | Los Gatos, CA 95032



On Thu, Apr 23, 2020 at 1:27 PM Dave Taht <dave.taht@gmail.com> wrote:

>
>
> On Thu, Apr 23, 2020 at 1:15 PM Sergey Fedorov <sfedorov@netflix.com>
> wrote:
>
>> Hi folks,
>>
>> No need to put more pressure - I've seen Jonathan's suggestion and it
>> makes a lot of sense to add the option to deep-link to an expanded version
>> with all detailed parameters shown.
>> This will be added some time this quarter (Q2), but not within a few next
>> weeks.
>>
>
> that's wonderful, thanks!
>
> One thing I've been seeing is way too many articles basically talking
> about traffic increases (or not), and how many are videoconferencing from
> home... but no metrics of import to videoconferencing folk. I've been
> engaged in a conversation about increasing a certain videoconferencing
> platform's default jitterbuffer to... wait for it... *1 sec* - based on how
> badly LTE was performing for some people
>
> so if you control your own videoconferencing tools, collecting more
> metrics there would be usef
>
> One very interesting metric a netflix streamer could be collecting is
> differences in tcp RTT (assuming you slowed traffic down in europe,
> especially, to a lower quality?), hour by hour, day by day.
>
> another one is packet loss... retransmits...
>
>
>
>> SERGEY FEDOROV
>>
>> Director of Engineering
>>
>> sfedorov@netflix.com
>>
>> 121 Albright Way | Los Gatos, CA 95032
>>
>>
>>
>> On Thu, Apr 23, 2020 at 1:00 PM Jonathan Foulkes <jf@jonathanfoulkes.com>
>> wrote:
>>
>>> Confirmed, and I go there all time as well, you’d think it would be the
>>> first thing Google would show us.
>>>
>>> At least Fast.com is on the first page, but they don’t pro-actively show
>>> latency tests, especially on the upload.
>>>
>>> BTW- I’ve suggested they support a URL request format where we can
>>> pre-set options that engage and show results for the bloat tests. This way
>>> we can share that pre-formated link and the users who click on it
>>> immediately see a bloat metric.
>>> Maybe if a few more suggest this as well, it will climb in priority.
>>>
>>> - Jonathan
>>>
>>> > On Apr 23, 2020, at 3:38 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>> >
>>> > dslreports.com is only on the third page of the search results.
>>> >
>>> > https://www.google.com/search?q=internet+speed+test
>>> > _______________________________________________
>>> > Bloat mailing list
>>> > Bloat@lists.bufferbloat.net
>>> > https://lists.bufferbloat.net/listinfo/bloat
>>>
>>> _______________________________________________
>>> Bloat mailing list
>>> Bloat@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/bloat
>>>
>>
>
> --
> Make Music, Not War
>
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-435-0729
>

[-- Attachment #2: Type: text/html, Size: 8501 bytes --]

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

--On Thursday, April 23, 2020 1:38 PM -0700 Dave Taht <dave.taht@gmail.com> 
wrote:

> dslreports.com is only on the third page of the search results.

What does it mean that my bloat indicator is a grey dot?

<http://www.dslreports.com/speedtest/62741609>

Re: [Bloat] this explains speedtest stuff

[Jonathan Morton]

> On 24 Apr, 2020, at 3:44 am, Kenneth Porter <shiva@sewingwitch.com> wrote:
> 
>> dslreports.com is only on the third page of the search results.
> 
> What does it mean that my bloat indicator is a grey dot?
> 
> <http://www.dslreports.com/speedtest/62741609>

It looks like there was a websockets error during the test, so try it again and it might work.

 - Jonathan Morton

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

--On Friday, April 24, 2020 4:47 AM +0300 Jonathan Morton 
<chromatix99@gmail.com> wrote:

> It looks like there was a websockets error during the test, so try it
> again and it might work.

This time I got an A score.

<http://www.dslreports.com/speedtest/62743044>

I'm using OpenWrt 18.06.4 on a Zyxel NBG6716 with the piece_of_cake.qos 
queue setup script.

Re: [Bloat] this explains speedtest stuff

[Dave Taht]

On Thu, Apr 23, 2020 at 6:17 PM Kenneth Porter <shiva@sewingwitch.com> wrote:
>
> --On Friday, April 24, 2020 4:47 AM +0300 Jonathan Morton
> <chromatix99@gmail.com> wrote:
>
> > It looks like there was a websockets error during the test, so try it
> > again and it might work.
>
> This time I got an A score.
>
> <http://www.dslreports.com/speedtest/62743044>
>
> I'm using OpenWrt 18.06.4 on a Zyxel NBG6716 with the piece_of_cake.qos
> queue setup script.

That is still oscillating more than I would expect. Do you have docsis
framing enabled? if you are doing nat, nat?

I used layer_cake with this overriding the defaults on my cable modems
in /etc/config/sqm

        option iqdisc_opts 'docsis wash besteffort nat ingress '
        option eqdisc_opts 'docsis ack-filter nat'




>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Bloat] this explains speedtest stuff

[Matt Taggart]

On 4/23/20 6:20 PM, Dave Taht wrote:
> On Thu, Apr 23, 2020 at 6:17 PM Kenneth Porter <shiva@sewingwitch.com> wrote:
>>
>> --On Friday, April 24, 2020 4:47 AM +0300 Jonathan Morton
>> <chromatix99@gmail.com> wrote:
>>
>>> It looks like there was a websockets error during the test, so try it
>>> again and it might work.
>>
>> This time I got an A score.
>>
>> <http://www.dslreports.com/speedtest/62743044>
>>
>> I'm using OpenWrt 18.06.4 on a Zyxel NBG6716 with the piece_of_cake.qos
>> queue setup script.
> 
> That is still oscillating more than I would expect. Do you have docsis
> framing enabled? if you are doing nat, nat?

ISP is Comcast in this case. In the past I have always gotten exactly 
the rates in the service plan, I think because the neighborhood was 
properly built-out and the modem itself was doing the limiting. But...

Today I was running betterspeedtest.sh on a Comcast link with 125/12 
service and I was getting results like 98/12, 75/12. I think the 
quarantine is really resulting in lot more download use, that might also 
explain the variability in the download (note the upload is pretty 
consistent).

-- 
Matt Taggart
matt@lackof.org

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

On 4/23/2020 6:20 PM, Dave Taht wrote:
> I used layer_cake with this overriding the defaults on my cable modems
> in /etc/config/sqm
>
>          option iqdisc_opts 'docsis wash besteffort nat ingress '
>          option eqdisc_opts 'docsis ack-filter nat'

I think I see where to do this in the GUI. (I can't remember if I 
installed a mini-emacs and I can never remember how to drive vi, so raw 
file editing might be harder than a GUI field.)

Where can I find documentation on what those options do?

Re: [Bloat] this explains speedtest stuff

[Sebastian Moeller]

For occasional small edits directly on the router I tend to use nano (`opkg update ; opkg install nano`), not super powerful, but it displays its keyboard shortcuts at the bottom of the screen...

Good luck
	Sebastian


> On Apr 24, 2020, at 16:40, Kenneth Porter <shiva@sewingwitch.com> wrote:
> 
> On 4/23/2020 6:20 PM, Dave Taht wrote:
>> I used layer_cake with this overriding the defaults on my cable modems
>> in /etc/config/sqm
>> 
>>         option iqdisc_opts 'docsis wash besteffort nat ingress '
>>         option eqdisc_opts 'docsis ack-filter nat'
> 
> I think I see where to do this in the GUI. (I can't remember if I installed a mini-emacs and I can never remember how to drive vi, so raw file editing might be harder than a GUI field.)
> 
> Where can I find documentation on what those options do?
> 
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] this explains speedtest stuff

[Toke Høiland-Jørgensen]

Kenneth Porter <shiva@sewingwitch.com> writes:

> On 4/23/2020 6:20 PM, Dave Taht wrote:
>> I used layer_cake with this overriding the defaults on my cable modems
>> in /etc/config/sqm
>>
>>          option iqdisc_opts 'docsis wash besteffort nat ingress '
>>          option eqdisc_opts 'docsis ack-filter nat'
>
> I think I see where to do this in the GUI. (I can't remember if I 
> installed a mini-emacs and I can never remember how to drive vi, so raw 
> file editing might be harder than a GUI field.)
>
> Where can I find documentation on what those options do?

`man tc-cake`

-Toke

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

On 4/24/2020 7:56 AM, Toke Høiland-Jørgensen via Bloat wrote:
> `man tc-cake`

CentOS 7 lacks that man page but I found it online. Thanks!

http://man7.org/linux/man-pages/man8/tc-cake.8.html

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

--On Thursday, April 23, 2020 7:20 PM -0700 Dave Taht <dave.taht@gmail.com> 
wrote:

> I used layer_cake with this overriding the defaults on my cable modems
> in /etc/config/sqm
>
>         option iqdisc_opts 'docsis wash besteffort nat ingress '
>         option eqdisc_opts 'docsis ack-filter nat'

It turned out I didn't actually have the SQM enabled before. I hadn't 
spotted the checkbox to turn it on so I'd just loaded the bandwidth 
parameter but not enabled it.

Now with your settings I get this result:

<http://www.dslreports.com/speedtest/62769414>

My next project will be to enable cake on my CentOS 7 box that just got a 
new 45 Mbps symmetric fiber connection from AT&T ("Business in a Box"). We 
upgraded from 1.5Mbps/128kbps ADSL. Any hints on what settings to use? 
Right now everyone's giddy with the massive speedup. But I'm sure that'll 
change as people start using more bandwidth. (About a dozen users.) The big 
win will be for us moving large solid model files and binary software 
distributions between partners using either SMB over VPN or secure FTP. I 
expect transferring a few GB files could saturate the link and drop some 
SIP calls.

Re: [Bloat] this explains speedtest stuff

[Jonathan Morton]

> On 24 Apr, 2020, at 7:22 pm, Kenneth Porter <shiva@sewingwitch.com> wrote:
> 
> My next project will be to enable cake on my CentOS 7 box that just got a new 45 Mbps symmetric fiber connection from AT&T ("Business in a Box"). We upgraded from 1.5Mbps/128kbps ADSL. Any hints on what settings to use?

Fibre probably uses Ethernet-style framing, or at least it does at the provisioning shaper.  So the following settings should probably work well:

# outbound
tc qdisc replace dev $WAN root cake bandwidth 44.5Mbit besteffort dual-srchost nonat ethernet ack-filter

# inbound
tc qdisc replace dev $IFB4WAN root cake bandwidth 42Mbit besteffort dual-dsthost nonat ethernet ingress

With, of course, the usual redirecting of $WAN ingress to $IFB4WAN.  The dual-src/dsthost settings should share things nicely between different users, including the server, even if one uses a lot more flows than another.

 - Jonathan Morton

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

Alas, CentOS 7 lacks cake. It does have fq_codel so I used the 
simple.qos script from sqm-scripts, with uplink 50000 and downlink 45000:

http://www.dslreports.com/speedtest/62797600

Re: [Bloat] this explains speedtest stuff

[Jonathan Morton]

> On 25 Apr, 2020, at 4:16 am, Kenneth Porter <shiva@sewingwitch.com> wrote:
> 
> Alas, CentOS 7 lacks cake. It does have fq_codel so I used the simple.qos script from sqm-scripts, with uplink 50000 and downlink 45000:
> 
> http://www.dslreports.com/speedtest/62797600

Those bandwidth settings are definitely too high; you don't have complete control of the queue here, and that's visible particularly with the steady increase in the upload latency during the test.  Try 44500 up, 42000 down, equivalent to my suggestions for Cake.

 - Jonathan Morton

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

before:

http://www.dslreports.com/speedtest/62767361

after:

http://www.dslreports.com/speedtest/62803997

Using simple.qos with:

UPLINK=45000
DOWNLINK=42500

(The link is supposed to be 50 Mbps symmetric and speed test does show it 
bursting that high sometimes.)

Re: [Bloat] this explains speedtest stuff

[Jonathan Morton]

> On 25 Apr, 2020, at 4:49 pm, Kenneth Porter <shiva@sewingwitch.com> wrote:
> 
> before:
> 
> http://www.dslreports.com/speedtest/62767361
> 
> after:
> 
> http://www.dslreports.com/speedtest/62803997
> 
> Using simple.qos with:
> 
> UPLINK=45000
> DOWNLINK=42500
> 
> (The link is supposed to be 50 Mbps symmetric and speed test does show it bursting that high sometimes.)

Looks like a definite improvement.  The Quality grade of C may indicate that you haven't enabled ECN on your client; without it, Codel has to drop packets to do congestion signalling.

 - Jonathan Morton

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

--On Saturday, April 25, 2020 6:00 PM +0300 Jonathan Morton 
<chromatix99@gmail.com> wrote:

> Looks like a definite improvement.  The Quality grade of C may indicate
> that you haven't enabled ECN on your client; without it, Codel has to
> drop packets to do congestion signalling.

I see "ecn" in the qdisc commands. Here's the log from running the script 
today:

<https://pastebin.com/E9BQDQC4>

I'm using sqm-scripts 1.3.0 from <https://github.com/tohojo/sqm-scripts>. I 
just checked and see he has a 1.4 tag. I'll have to go look at the 
changelog and consider upgrading.

Re: [Bloat] this explains speedtest stuff

[Jonathan Morton]

> On 25 Apr, 2020, at 5:14 pm, Kenneth Porter <shiva@sewingwitch.com> wrote:
> 
> I see "ecn" in the qdisc commands.

No, not the qdisc (where ECN is enabled by default), but on the client.

Linux:
# sysctl net.ipv4.tcp_ecn=1

Windows:
> netsh interface tcp set global ecncapability=enabled

OSX:
$ sudo sysctl -w net.inet.tcp.ecn_initiate_out=1  
$ sudo sysctl -w net.inet.tcp.ecn_negotiate_in=1

In Linux and OSX, to make the setting persist across reboots, edit /etc/sysctl.conf.

 - Jonathan Morton

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

On 4/25/2020 7:19 AM, Jonathan Morton wrote:
> No, not the qdisc (where ECN is enabled by default), but on the client.

I enabled it on both the router (Linux) and the client where I'm 
conducting the speed test (Win10):

http://www.dslreports.com/speedtest/62818220

Re: [Bloat] this explains speedtest stuff

[Dave Taht]

On Sat, Apr 25, 2020 at 8:31 AM Kenneth Porter <shiva@sewingwitch.com> wrote:
>
> On 4/25/2020 7:19 AM, Jonathan Morton wrote:
> > No, not the qdisc (where ECN is enabled by default), but on the client.
>
> I enabled it on both the router (Linux) and the client where I'm
> conducting the speed test (Win10):
>
> http://www.dslreports.com/speedtest/62818220

doesn't appear to have worked. retransmits are still high.

a packet capture during the test would be useful to see if the syn +
ecn request was successfully sent and echoed.

>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Bloat] this explains speedtest stuff

[Dave Taht]

Trying to improve the dslreports misguided "Quality" metric by enabling ecn is
essentially "engineering to the test" rather than providing much
actual benefit. It would be rather useful
at the moment to do that test given the 50/50 mbit symmetric nature of
this one, and also collect QoE metrics
over time. I'd rather like kenneth to try it!

The use case here includes vpn, smb filesharing, and rdp. Enabling ecn
universally for all the endpoints involved
is adding in a few lines of configuration for all those endpoints, and
rdp itself is a hybrid tcp/udp protocol.

https://en.wikipedia.org/wiki/Remote_Desktop_Protocol about the
congestion control characteristics of the udp
portion in all the competing implementations is kind of unknown but
seem likely to be poor.

It would be a useful test for an intrepid windows administrator to
actually enable ecn fully and see what breaks in their
vpn, smb, and rdp implementations, and across their remote workforce,
and observe any difference in QoE. I have long
tried to get one drunk enough to deploy ecn across their windows
infrastructure without any success.

However it is perhaps useful to point to a few detailed statistics the
dslreports test also gets incorrect. Going to the "good" result:
http://www.dslreports.com/speedtest/62803997

The summary stats on the "Server view" of this page appear to be
incorrect. In particular they claim severe flow unfairness by RTT,
where the logs further down on the page, do not.

retransmits are pretty high, nearly 5%, but that too could be an
error. certainly repeating the test with ecn enabled should
eliminate retransmits caused by loss on this part of the path!

IMHO: Packet loss, in modest amounts, really has no impact on
"quality" for most TCP flows. If you lose a few packets in a flow that
takes 2 seconds to complete, only tail loss is going to hurt you, (if
it happens), otherwise, the maximum rtt observed here was a physical
64ms +- 12ms, which means that (so long as it isn't tail loss) that
the total time to complete a 2 second tcp
transaction would be inflated 130ms or so, and most of the time... 0.

When you look at the overall benefit of applying advanced AQM
techniques to this link verses the original test
( http://www.dslreports.com/speedtest/62767361) , there is a savings
of 480ms of peak latency in the uplink direction and 150ms in the
down. On a path measured in the 4ms range, that's a really enormous
savings, and I'd hope that his
users could see an observable difference - or rather, merely not
notice anyone else was using the link, all the time.

The characteristics of smb traffic are very different from web
traffic. file downloads and uploads are common, file locking
and directory operations are very short, and there's all kinds of
short administrative traffic for kerberos tickets and the like.

since (I think?) the proposed vpn traffic is client -> server (?) not,
clients -> router -> server, each individual using this link
over the vpn will tend to get their own queue, and a big upload or
download through their openvpn will tend to "do them in",
because openvpn has lousy queue management internally... and (when
last I looked), windows itself had not a lot of backpressure when
dealing with that device.

as for rdp traffic, well... I do kind of wish rdp worked over webrtc
instead and rather than using vpns we could actually expose good
applications to ipv6 and the internet at large, but I do like to sleep
at night, also. I wish (for example) I could trust smb3 to be deployed
everywhere and we could go back to good ole drag and drop filesharing,
with support for remote locking,
and none of this port 433 only nonsense.


On Sat, Apr 25, 2020 at 7:19 AM Jonathan Morton <chromatix99@gmail.com> wrote:
>
> > On 25 Apr, 2020, at 5:14 pm, Kenneth Porter <shiva@sewingwitch.com> wrote:
> >
> > I see "ecn" in the qdisc commands.
>
> No, not the qdisc (where ECN is enabled by default), but on the client.
>
> Linux:
> # sysctl net.ipv4.tcp_ecn=1
>
> Windows:
> > netsh interface tcp set global ecncapability=enabled
>
> OSX:
> $ sudo sysctl -w net.inet.tcp.ecn_initiate_out=1
> $ sudo sysctl -w net.inet.tcp.ecn_negotiate_in=1
>
> In Linux and OSX, to make the setting persist across reboots, edit /etc/sysctl.conf.
>
>  - Jonathan Morton
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Bloat] this explains speedtest stuff

[Dave Taht]

and the last flaw of this test series is that ken took the dslreports
"fiber" setting for the dslreports test as "The right thing". the
"fiber" test is structured to stress test an asymmetric 1gbit/100mbit
connection, not a shaped fiber connection running at 50mbit symmetric.
The number of uploads is 4, downloads, 32.... it's totally ok to pick
a given fiber/cable/whatever test, but it does help to apply the same
characteristics to more of the tests you do, if you are trying to
compare technologies.

that said, I'm rather happy to see cake survive the onslaught of 32
simultaneous download flows at a range of rtts from 4ms to 64ms, with
flow fairness.

On Sat, Apr 25, 2020 at 8:32 AM Dave Taht <dave.taht@gmail.com> wrote:
>
> Trying to improve the dslreports misguided "Quality" metric by enabling ecn is
> essentially "engineering to the test" rather than providing much
> actual benefit. It would be rather useful
> at the moment to do that test given the 50/50 mbit symmetric nature of
> this one, and also collect QoE metrics
> over time. I'd rather like kenneth to try it!
>
> The use case here includes vpn, smb filesharing, and rdp. Enabling ecn
> universally for all the endpoints involved
> is adding in a few lines of configuration for all those endpoints, and
> rdp itself is a hybrid tcp/udp protocol.
>
> https://en.wikipedia.org/wiki/Remote_Desktop_Protocol about the
> congestion control characteristics of the udp
> portion in all the competing implementations is kind of unknown but
> seem likely to be poor.
>
> It would be a useful test for an intrepid windows administrator to
> actually enable ecn fully and see what breaks in their
> vpn, smb, and rdp implementations, and across their remote workforce,
> and observe any difference in QoE. I have long
> tried to get one drunk enough to deploy ecn across their windows
> infrastructure without any success.
>
> However it is perhaps useful to point to a few detailed statistics the
> dslreports test also gets incorrect. Going to the "good" result:
> http://www.dslreports.com/speedtest/62803997
>
> The summary stats on the "Server view" of this page appear to be
> incorrect. In particular they claim severe flow unfairness by RTT,
> where the logs further down on the page, do not.
>
> retransmits are pretty high, nearly 5%, but that too could be an
> error. certainly repeating the test with ecn enabled should
> eliminate retransmits caused by loss on this part of the path!
>
> IMHO: Packet loss, in modest amounts, really has no impact on
> "quality" for most TCP flows. If you lose a few packets in a flow that
> takes 2 seconds to complete, only tail loss is going to hurt you, (if
> it happens), otherwise, the maximum rtt observed here was a physical
> 64ms +- 12ms, which means that (so long as it isn't tail loss) that
> the total time to complete a 2 second tcp
> transaction would be inflated 130ms or so, and most of the time... 0.
>
> When you look at the overall benefit of applying advanced AQM
> techniques to this link verses the original test
> ( http://www.dslreports.com/speedtest/62767361) , there is a savings
> of 480ms of peak latency in the uplink direction and 150ms in the
> down. On a path measured in the 4ms range, that's a really enormous
> savings, and I'd hope that his
> users could see an observable difference - or rather, merely not
> notice anyone else was using the link, all the time.
>
> The characteristics of smb traffic are very different from web
> traffic. file downloads and uploads are common, file locking
> and directory operations are very short, and there's all kinds of
> short administrative traffic for kerberos tickets and the like.
>
> since (I think?) the proposed vpn traffic is client -> server (?) not,
> clients -> router -> server, each individual using this link
> over the vpn will tend to get their own queue, and a big upload or
> download through their openvpn will tend to "do them in",
> because openvpn has lousy queue management internally... and (when
> last I looked), windows itself had not a lot of backpressure when
> dealing with that device.
>
> as for rdp traffic, well... I do kind of wish rdp worked over webrtc
> instead and rather than using vpns we could actually expose good
> applications to ipv6 and the internet at large, but I do like to sleep
> at night, also. I wish (for example) I could trust smb3 to be deployed
> everywhere and we could go back to good ole drag and drop filesharing,
> with support for remote locking,
> and none of this port 433 only nonsense.
>
>
> On Sat, Apr 25, 2020 at 7:19 AM Jonathan Morton <chromatix99@gmail.com> wrote:
> >
> > > On 25 Apr, 2020, at 5:14 pm, Kenneth Porter <shiva@sewingwitch.com> wrote:
> > >
> > > I see "ecn" in the qdisc commands.
> >
> > No, not the qdisc (where ECN is enabled by default), but on the client.
> >
> > Linux:
> > # sysctl net.ipv4.tcp_ecn=1
> >
> > Windows:
> > > netsh interface tcp set global ecncapability=enabled
> >
> > OSX:
> > $ sudo sysctl -w net.inet.tcp.ecn_initiate_out=1
> > $ sudo sysctl -w net.inet.tcp.ecn_negotiate_in=1
> >
> > In Linux and OSX, to make the setting persist across reboots, edit /etc/sysctl.conf.
> >
> >  - Jonathan Morton
> > _______________________________________________
> > Bloat mailing list
> > Bloat@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat
>
>
>
> --
> Make Music, Not War
>
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-435-0729



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

On 4/25/2020 8:43 AM, Dave Taht wrote:
> and the last flaw of this test series is that ken took the dslreports
> "fiber" setting for the dslreports test as "The right thing". the
> "fiber" test is structured to stress test an asymmetric 1gbit/100mbit
> connection, not a shaped fiber connection running at 50mbit symmetric.
> The number of uploads is 4, downloads, 32.... it's totally ok to pick
> a given fiber/cable/whatever test, but it does help to apply the same
> characteristics to more of the tests you do, if you are trying to
> compare technologies.

I'd be happy to test with a different set of speed test settings. Would 
Corporate/Edu be a better choice?

In going through this exercise, I can imagine someone saying "where'd 
all the bandwidth I paid for go?" as we trade bandwidth for reduced 
latency. How do you sell that to end users? ISPs have trained people to 
think only in terms of raw speed (obviously that's best for an ISP's 
bottom line as they can upsell you the higher-priced package) and most 
people (outside of gamers) don't really grasp latency with the 
intuitiveness of speed. Maybe we need some Youtube videos showing end 
user experiences of the benefit of reduced latency.

Re: [Bloat] this explains speedtest stuff

[Dave Taht]

On Sat, Apr 25, 2020 at 8:59 AM Kenneth Porter <shiva@sewingwitch.com> wrote:
>
> On 4/25/2020 8:43 AM, Dave Taht wrote:
> > and the last flaw of this test series is that ken took the dslreports
> > "fiber" setting for the dslreports test as "The right thing". the
> > "fiber" test is structured to stress test an asymmetric 1gbit/100mbit
> > connection, not a shaped fiber connection running at 50mbit symmetric.
> > The number of uploads is 4, downloads, 32.... it's totally ok to pick
> > a given fiber/cable/whatever test, but it does help to apply the same
> > characteristics to more of the tests you do, if you are trying to
> > compare technologies.
>
> I'd be happy to test with a different set of speed test settings. Would
> Corporate/Edu be a better choice?

Sticking with the fiber test is ok, though I wish it tried more upload flows.

There is a "hires bufferbloat" option in the dslreports settings that
I find useful.

>
> In going through this exercise, I can imagine someone saying "where'd
> all the bandwidth I paid for go?" as we trade bandwidth for reduced
> latency. How do you sell that to end users? ISPs have trained people to
> think only in terms of raw speed (obviously that's best for an ISP's
> bottom line as they can upsell you the higher-priced package) and most
> people (outside of gamers) don't really grasp latency with the
> intuitiveness of speed. Maybe we need some Youtube videos showing end
> user experiences of the benefit of reduced latency.

You didn't lose anything really on the upload. the wild swings of
throughput with such an overbuffered link balance
out with the relative smoothness inherent in fq_codel, and eliminating
the side effects of other users of the link
is really wonderful (partially why I wanted to see cake in this environment)

on the download, yea, you lost a bit there, but also got rid of wild
swings of throughput, and things like dns
will always just "fly" which IMHO is the biggest bottleneck for most
web pages. Caching your corporate dns locally
if at all possible might help a bit, but kind of difficult to do
right. Worse, most openvpn implementations I've seen
tend to route *all traffic* through it, rather than intelligently
using local resources for stuff outside the vpn.

most users ALSO don't run speedtest, either. They just want things to
work, and some tiny percentage difference
in perceived slowness of download speed they won't notice. jitter and
voip/video glitches far more apparent to most users.
>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Bloat] this explains speedtest stuff

[Sebastian Moeller]

[-- Attachment #1: Type: text/plain, Size: 1803 bytes --]

Hi Ken,

I tried to summarize a few recommendations under https://forum.openwrt.org/t/sqm-qos-recommended-settings-for-the-dslreports-speedtest-bufferbloat-testing/2803 maybe that is of some help...

Good luck
        Sebastian

On 25 April 2020 17:59:47 CEST, Kenneth Porter <shiva@sewingwitch.com> wrote:
>On 4/25/2020 8:43 AM, Dave Taht wrote:
>> and the last flaw of this test series is that ken took the dslreports
>> "fiber" setting for the dslreports test as "The right thing". the
>> "fiber" test is structured to stress test an asymmetric 1gbit/100mbit
>> connection, not a shaped fiber connection running at 50mbit
>symmetric.
>> The number of uploads is 4, downloads, 32.... it's totally ok to pick
>> a given fiber/cable/whatever test, but it does help to apply the same
>> characteristics to more of the tests you do, if you are trying to
>> compare technologies.
>
>I'd be happy to test with a different set of speed test settings. Would
>
>Corporate/Edu be a better choice?
>
>In going through this exercise, I can imagine someone saying "where'd 
>all the bandwidth I paid for go?" as we trade bandwidth for reduced 
>latency. How do you sell that to end users? ISPs have trained people to
>
>think only in terms of raw speed (obviously that's best for an ISP's 
>bottom line as they can upsell you the higher-priced package) and most 
>people (outside of gamers) don't really grasp latency with the 
>intuitiveness of speed. Maybe we need some Youtube videos showing end 
>user experiences of the benefit of reduced latency.
>
>
>_______________________________________________
>Bloat mailing list
>Bloat@lists.bufferbloat.net
>https://lists.bufferbloat.net/listinfo/bloat

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 2326 bytes --]

Re: [Bloat] this explains speedtest stuff

[Toke Høiland-Jørgensen]

Kenneth Porter <shiva@sewingwitch.com> writes:

> Maybe we need some Youtube videos showing end user experiences of the
> benefit of reduced latency.

The (now defunct) RITE project did this one, which I thought was rather
good: https://youtu.be/F1a-eMF9xdY

-Toke

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

On 4/25/2020 8:32 AM, Dave Taht wrote:
> It would be a useful test for an intrepid windows administrator to
> actually enable ecn fully and see what breaks in their
> vpn, smb, and rdp implementations, and across their remote workforce,
> and observe any difference in QoE. I have long
> tried to get one drunk enough to deploy ecn across their windows
> infrastructure without any success.

I can easily do it for myself, now that I know where that setting hides. 
Is there reason to make others do it or is it sufficient that I be the 
only guinea pig here?


> since (I think?) the proposed vpn traffic is client -> server (?) not,
> clients -> router -> server, each individual using this link
> over the vpn will tend to get their own queue, and a big upload or
> download through their openvpn will tend to "do them in",
> because openvpn has lousy queue management internally... and (when
> last I looked), windows itself had not a lot of backpressure when
> dealing with that device.

I'm using openvpn to avoid exposing raw RDP servers to the Internet. I 
still don't trust MS enough to allow a Windows box directly connected to 
the Internet. So it's Win10(@home) -> openvpn(client on Win10 @home) 
->openvpn(Linux router @office)->Win10(@office). I think another user 
may use RDP's file transfer capabilities, but I personally use a 
separate ssh (scp) connection for that. I have no problem with command 
lines.

Re: [Bloat] this explains speedtest stuff

[Dave Taht]

On Sat, Apr 25, 2020 at 8:53 AM Kenneth Porter <shiva@sewingwitch.com> wrote:
>
> On 4/25/2020 8:32 AM, Dave Taht wrote:
> > It would be a useful test for an intrepid windows administrator to
> > actually enable ecn fully and see what breaks in their
> > vpn, smb, and rdp implementations, and across their remote workforce,
> > and observe any difference in QoE. I have long
> > tried to get one drunk enough to deploy ecn across their windows
> > infrastructure without any success.
>
> I can easily do it for myself, now that I know where that setting hides.
> Is there reason to make others do it or is it sufficient that I be the
> only guinea pig here?

You would be the first person in the history of the bufferbloat
effort, to attempt enabling ecn more fully on windows, and
observing the results on real traffic and real traffic types.

be prepared to take a lot of packet captures.

/me passes the vodka

>
>
> > since (I think?) the proposed vpn traffic is client -> server (?) not,
> > clients -> router -> server, each individual using this link
> > over the vpn will tend to get their own queue, and a big upload or
> > download through their openvpn will tend to "do them in",
> > because openvpn has lousy queue management internally... and (when
> > last I looked), windows itself had not a lot of backpressure when
> > dealing with that device.
>
> I'm using openvpn to avoid exposing raw RDP servers to the Internet. I
> still don't trust MS enough to allow a Windows box directly connected to
> the Internet. So it's Win10(@home) -> openvpn(client on Win10 @home)
> ->openvpn(Linux router @office)->Win10(@office). I think another user
> may use RDP's file transfer capabilities, but I personally use a
> separate ssh (scp) connection for that. I have no problem with command
> lines.
>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

On 4/25/2020 8:55 AM, Dave Taht wrote:
> You would be the first person in the history of the bufferbloat
> effort, to attempt enabling ecn more fully on windows, and
> observing the results on real traffic and real traffic types.
>
> be prepared to take a lot of packet captures.
>
> /me passes the vodka

I'm more a gin and tonic guy. I'll keep some ready. :D

Time to update Wireshark.

I realized I have two machines I can mess with at home (my game machine 
and my development machine) and a family member with some tech savvy can 
do this with his devel machine that he uses for a lot of video meetings 
lately. All Win10Pro. This would be behind our OpenWrt/Comcast router 
with cake.

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

On 4/25/2020 7:19 AM, Jonathan Morton wrote:
> In Linux and OSX, to make the setting persist across reboots, edit /etc/sysctl.conf.

For the lurkers, CentOS (and presumably other Red Hat distros) now has 
an /etc/sysctl.d for vendor-specific settings. The prevents modifying 
package-supplied config files so that package upgrades are easier. Much 
of Linux seems to be moving towards this model of a config subdirectory 
to place overrides.

I created 51-bufferbloat.conf in that directory to add the ecn setting 
and to select the default_qdisc.

Re: [Bloat] this explains speedtest stuff

[Y]

Hi,all

ECN on
http://www.dslreports.com/speedtest/62823326
ECN off
http://www.dslreports.com/speedtest/62823112

Thanks.

Yutaka

On Sat, 25 Apr 2020 17:00:19 +0300
Jonathan Morton <chromatix99@gmail.com> wrote:

> > On 25 Apr, 2020, at 4:49 pm, Kenneth Porter <shiva@sewingwitch.com> wrote:
> > 
> > before:
> > 
> > http://www.dslreports.com/speedtest/62767361
> > 
> > after:
> > 
> > http://www.dslreports.com/speedtest/62803997
> > 
> > Using simple.qos with:
> > 
> > UPLINK=45000
> > DOWNLINK=42500
> > 
> > (The link is supposed to be 50 Mbps symmetric and speed test does show it bursting that high sometimes.)
> 
> Looks like a definite improvement.  The Quality grade of C may indicate that you haven't enabled ECN on your client; without it, Codel has to drop packets to do congestion signalling.
> 
>  - Jonathan Morton
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat


-- 
Y <intruder_tkyf@yahoo.fr>

Re: [Bloat] this explains speedtest stuff

[Jonathan Morton]

> On 25 Apr, 2020, at 8:24 pm, Y via Bloat <bloat@lists.bufferbloat.net> wrote:
> 
> ECN on
> http://www.dslreports.com/speedtest/62823326
> ECN off
> http://www.dslreports.com/speedtest/62823112

Yup, that's what I mean.

> doesn't appear to have worked. retransmits are still high.

Ken, it might be that your version of fq_codel doesn't actually have ECN support on by default.  So try adding the "ecn" keyword to the qdisc.

 - Jonathan Morton

Re: [Bloat] this explains speedtest stuff

[Dave Taht]

in /etc/config/sqm, with fq_codel, you need to uncomment or enter the
following lines

        option ingress_ecn 'ECN'
        option egress_ecn 'ECN'

I still am not much of an ecn fan. I use it primarily for debugging
codel. :/ When I see marking, I know its working,
and if I see loss where I shouldn't, I know something else in the
stack is broken somewhere.

Example - I ran into this while trying to get the ath10k performing
sanely: https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/92
- dropping packets elsewhere in the stack. The "fix" got bidir
throughput up by 10x, and it's still 4x lower than it could be for...
unknown reasons.

I worry a lot about checksummng problems and incorrect encapsulation.
Hit that multiple times myself, and most recently a ddpk person hit
it. "Look ecn's enabled!" but the actual receiver was dropping it
instead because the checksum was wrong. I also hit it with 6 months of
data from all over the world on osx. The second and successive runs of
the rrul test disabled ecn neg attempts entirely for those ip
addresses. I didn't figure it out until
I sat down to write the paper and was agast at my packet captures.

osx is completely useless for evaluating ecn behaviors given the
heuristics in place on it today.

Would love to have one valid windows + openvpn + fq_codel result.

On Sat, Apr 25, 2020 at 10:37 AM Jonathan Morton <chromatix99@gmail.com> wrote:
>
> > On 25 Apr, 2020, at 8:24 pm, Y via Bloat <bloat@lists.bufferbloat.net> wrote:
> >
> > ECN on
> > http://www.dslreports.com/speedtest/62823326
> > ECN off
> > http://www.dslreports.com/speedtest/62823112
>
> Yup, that's what I mean.
>
> > doesn't appear to have worked. retransmits are still high.
>
> Ken, it might be that your version of fq_codel doesn't actually have ECN support on by default.  So try adding the "ecn" keyword to the qdisc.
>
>  - Jonathan Morton
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

--On Saturday, April 25, 2020 12:19 PM -0700 Dave Taht 
<dave.taht@gmail.com> wrote:

> in /etc/config/sqm, with fq_codel, you need to uncomment or enter the
> following lines
>
>         option ingress_ecn 'ECN'
>         option egress_ecn 'ECN'

Does this only affect flows that terminate on the box, or also those for 
which this is a (NAT) router in the middle? The tests I'm running are from 
an internal Win10 box, not from the router.

Re: [Bloat] this explains speedtest stuff

[Dave Taht]

On Sat, Apr 25, 2020 at 1:49 PM Kenneth Porter <shiva@sewingwitch.com> wrote:
>
> --On Saturday, April 25, 2020 12:19 PM -0700 Dave Taht
> <dave.taht@gmail.com> wrote:
>
> > in /etc/config/sqm, with fq_codel, you need to uncomment or enter the
> > following lines
> >
> >         option ingress_ecn 'ECN'
> >         option egress_ecn 'ECN'
>
> Does this only affect flows that terminate on the box, or also those for
> which this is a (NAT) router in the middle?

all flows passing through fq_codel.

even encapsulated ones if the encap type is well known or the
client app follows
https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-encap-guidelines-13



> The tests I'm running are from
> an internal Win10 box, not from the router.

a packet cap of a syn exchange would be goo
>
>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Bloat] this explains speedtest stuff

[Dave Taht]

On Sat, Apr 25, 2020 at 6:49 AM Kenneth Porter <shiva@sewingwitch.com> wrote:
>
> before:
>
> http://www.dslreports.com/speedtest/62767361
>
> after:
>
> http://www.dslreports.com/speedtest/62803997
>
> Using simple.qos with:
>
> UPLINK=45000
> DOWNLINK=42500
>
> (The link is supposed to be 50 Mbps symmetric and speed test does show it
> bursting that high sometimes.)

Oh, I misread your report. I thought this was cake, not fq_codel. Care
to try that?

>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

On 4/25/2020 9:00 AM, Dave Taht wrote:
> Oh, I misread your report. I thought this was cake, not fq_codel. Care
> to try that?

I'd love to if I knew how to add cake to CentOS 7. I've installed kernel 
modules for unsupported Ethernet interfaces before, so perhaps cake is 
available that way from a 3rd party repo? Or I could adapt a 3rd party 
driver RPM's source to add cake, instead. I really don't want to do a 
full custom kernel, though. That's hard to maintain over time as there's 
a new kernel in the updates every month or two. Some hints on how to add 
a qdisc to a kernel would be welcome.

Re: [Bloat] this explains speedtest stuff

[Toke Høiland-Jørgensen]

Kenneth Porter <shiva@sewingwitch.com> writes:

> On 4/25/2020 9:00 AM, Dave Taht wrote:
>> Oh, I misread your report. I thought this was cake, not fq_codel. Care
>> to try that?
>
> I'd love to if I knew how to add cake to CentOS 7. I've installed kernel 
> modules for unsupported Ethernet interfaces before, so perhaps cake is 
> available that way from a 3rd party repo? Or I could adapt a 3rd party 
> driver RPM's source to add cake, instead. I really don't want to do a 
> full custom kernel, though. That's hard to maintain over time as there's 
> a new kernel in the updates every month or two. Some hints on how to add 
> a qdisc to a kernel would be welcome.

There's an out-of-tree version of cake here that should theoretically
build as a module if you have the right kernel-headers installed

https://github.com/dtaht/sch_cake:

However, RHEL kernels (and thus CentOS) lie about their kernel versions,
so it may be that all the compatibility stuff for old kernels we have in
that repo is going to break. Feel free to give it a shot, though.

Or, y'know, just upgrade to CentOS 8 ;)

-Toke

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

Here's what the fiber connection looks like with no SQM applied:

<http://www.dslreports.com/speedtest/62767361>

Re: [Bloat] this explains speedtest stuff

[Dave Taht]

It looks to me you should reduce your download settings a bit for the
docsis service.

On Fri, Apr 24, 2020 at 9:26 AM Kenneth Porter <shiva@sewingwitch.com> wrote:
>
> Here's what the fiber connection looks like with no SQM applied:
>
> <http://www.dslreports.com/speedtest/62767361>

That's miserable. 480ms latency on fiber?? You can do so much better.
But why centos? sure the sqm-scripts work with that but you should be
able to shape 45Mbits with even a wndr3800. openwrt works great on x86
hw, also. :)

do you get dedicated ipv6 with that AT&T service?

What will be the vpn type? ipsec, terminating on the router, works
well with fq_codel because the hash is propagated to the tunnel,
wireguard and openvpn currently do not.

I have been tempted, with smb3 supporting crypto e2e, and ipv6, to
ditch the vpn entirely. But only tempted!

> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Bloat] this explains speedtest stuff

[Dave Taht]

On Fri, Apr 24, 2020 at 9:32 AM Dave Taht <dave.taht@gmail.com> wrote:
>
> It looks to me you should reduce your download settings a bit for the
> docsis service.
>
> On Fri, Apr 24, 2020 at 9:26 AM Kenneth Porter <shiva@sewingwitch.com> wrote:
> >
> > Here's what the fiber connection looks like with no SQM applied:
> >
> > <http://www.dslreports.com/speedtest/62767361>
>
> That's miserable. 480ms latency on fiber?? You can do so much better.
> But why centos? sure the sqm-scripts work with that but you should be
> able to shape 45Mbits with even a wndr3800. openwrt works great on x86
> hw, also. :)
>
> do you get dedicated ipv6 with that AT&T service?
>
> What will be the vpn type? ipsec, terminating on the router, works
> well with fq_codel because the hash is propagated to the tunnel,
> wireguard and openvpn currently do not.

And cake  does not propagate the hash because it recalculates it. So
if your use case is ipsec, terminating on the router, and it's
primarily vpn traffic, I would use fq_codel, rather than cake. Any
other use case, cake. :) These days I slam another cake instance in
front of my wireguard tunnels, set to a slightly lower bandwidth than
the next hop (20% less). I'm always willing to live with a bandwidth
hit in order to generally get better latency on my vpn'd traffic.
YMMV.
> I have been tempted, with smb3 supporting crypto e2e, and ipv6, to
> ditch the vpn entirely. But only tempted!
>
> > _______________________________________________
> > Bloat mailing list
> > Bloat@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat
>
>
>
> --
> Make Music, Not War
>
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-435-0729



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Bloat] this explains speedtest stuff

[Kenneth Porter]

--On Friday, April 24, 2020 10:32 AM -0700 Dave Taht <dave.taht@gmail.com> 
wrote:

> That's miserable. 480ms latency on fiber?? You can do so much better.
> But why centos? sure the sqm-scripts work with that but you should be
> able to shape 45Mbits with even a wndr3800. openwrt works great on x86
> hw, also. :)

The same box is providing a bunch of other public-facing services, so I 
need some moderately heavy iron. (Still a cheap server, though.) If it were 
JUST a NAT router, I'd consider a cheap OpenWrt-capable router like the one 
I'm using at home.

Note that this test was without any shaping parameters. I think CentOS 
(like Fedora) defaults to fq_codel, though.

> do you get dedicated ipv6 with that AT&T service?

Yep, they give us a /56, which seems to be the default for "sites" unless 
you ask for something bigger. So I'm assigning a /64 to the link between 
our box and their gateway, and another to our LAN. That leaves 254 more for 
whatever. I need to assign a AAAA to the public side and test. Haven't 
gotten to that, yet.

We also get some VOIP lines that their gateway deals with. So no SIP yet 
within the LAN. We do use the "WiFi calling" feature on our mobiles, 
though. Cellular coverage at our location is terrible.

> What will be the vpn type? ipsec, terminating on the router, works
> well with fq_codel because the hash is propagated to the tunnel,
> wireguard and openvpn currently do not.

I'm using OpenVPN with proto udp and dev tun. Our main use is to run Remote 
Desktop from home to our office and lab PCs. If I need to move files, I 
usually use scp. Outbound, we use Cisco's VPN to connect to customers to 
check binaries into their Subversion repo over HTTPS.

For customers and vendors, we have secure FTP drops. Mostly used for CAD 
drawings.