[Make-wifi-fast] dslreports is no longer free

[Make-wifi-fast] dslreports is no longer free

[Dave Taht]

https://www.reddit.com/r/HomeNetworking/comments/gbd6g0/dsl_reports_speed_test_no_longer_free/

They ran out of bandwidth.

Message to users here:

http://www.dslreports.com/speedtest


-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Make-wifi-fast] [Cake] dslreports is no longer free

[Sebastian Moeller]

Hi Dave,

well, it was a free service and it lasted a long time. I want to raise a toast to Justin and convey my sincere thanks for years of investing into the "good" of the internet. 

Now, the question is which test is going to be the rightful successor? 

Short of running netperf/irtt/iper2/iperf3 on a hosted server, I see lots of potential but none of the tests are really there yet (grievances in now particular order):

OOKLA: speedtest.net.
	Pros: ubiquitious, allows selection of single flow versus multi-flow test, allows server selection
	Cons: only IPv4, only static unloaded RTT measurement, no control over measurement duration
	BUFFERBLOAT verdict: incomplete, maybe usable as load generator


NETFLIX: fast.com.
	Pros: allows selection of upload testing, supposedly decent back-end, duration configurable
		allows unloaded, loaded download and loaded upload RTT measurements (but reports sinlge numbers for loaded and unloaded RTT, that are not the max)
	Cons: RTT report as two numbers one for the loaded and one for unloaded RTT, time-course of RTTs missing
	BUFFERBLOAT verdict: incomplete, but oh, so close...


NPERF: nperf.com
	Pros: allows server selection, RTT measurement and report as time course, also reports average rates and static RTT/jitter for Up- and Download
	Cons: RTT measurement for unloaded only, reported RTT static only , no control over measurement duration
	BUFFERBLOAT verdict: incomplete,


THINKBROADBAND: www.thinkbroadband.com/speedtest
	Pros: IPv6, reports coarse RTT time courses for all three measurement phases
	Cons: only static unloaded RTT report in final results, time courses only visible immediately after testing, no control over measurement duration
	BUFFERBLOAT verdict: a bit coarse, might work for users within a reasonable distance to the UK for acute de-bloating sessions (history reporting is bad though)


honorable mentioning:
	BREITBANDMESSUNG: breitbandmessung.de
	Pros: query of contracted internet access speed before measurement, with a scheduler that will only start a test when the backend has sufficient capacity to saturate the user-supplied contracted rates, IPv6 (happy-eyeballs)
	Cons: only static unloaded RTT measurement, no control over measurement duration
	BUFFERBLOAT verdict: unsuitable, exceot as load generator, but the bandwidth reservation feature is quite nice.

Best Regards
	Sebastian


> On May 1, 2020, at 18:44, Dave Taht <dave.taht@gmail.com> wrote:
> 
> https://www.reddit.com/r/HomeNetworking/comments/gbd6g0/dsl_reports_speed_test_no_longer_free/
> 
> They ran out of bandwidth.
> 
> Message to users here:
> 
> http://www.dslreports.com/speedtest
> 
> 
> -- 
> Make Music, Not War
> 
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-435-0729
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Bloat] [Cake] dslreports is no longer free

[Sergey Fedorov]

[-- Attachment #1: Type: text/plain, Size: 4738 bytes --]

Great review, Sebastian!


> NETFLIX: fast.com.
>         Pros: allows selection of upload testing, supposedly decent
> back-end, duration configurable
>                 allows unloaded, loaded download and loaded upload RTT
> measurements (but reports sinlge numbers for loaded and unloaded RTT, that
> are not the max)
>         Cons: RTT report as two numbers one for the loaded and one for
> unloaded RTT, time-course of RTTs missing
>         BUFFERBLOAT verdict: incomplete, but oh, so close...

Just a note that I have a plan to separate the loaded latency into
upload/download. It's not great UX now they way it's implemented.
The timeline view is a bit more nuanced, in the spirit of the simplistic
UX, but I've been thinking on a good way to show that for super users as
well.
Two latency numbers - that's more user friendly, we want the general user
to understand the meaning. And latency under load is much easier than
bufferbloat.

As a side note, if our backend is decent, I'm curious what are the backends
for the speed tests that exist that are great :)


SERGEY FEDOROV

Director of Engineering

sfedorov@netflix.com

121 Albright Way | Los Gatos, CA 95032



On Fri, May 1, 2020 at 12:48 PM Sebastian Moeller <moeller0@gmx.de> wrote:

> Hi Dave,
>
> well, it was a free service and it lasted a long time. I want to raise a
> toast to Justin and convey my sincere thanks for years of investing into
> the "good" of the internet.
>
> Now, the question is which test is going to be the rightful successor?
>
> Short of running netperf/irtt/iper2/iperf3 on a hosted server, I see lots
> of potential but none of the tests are really there yet (grievances in now
> particular order):
>
> OOKLA: speedtest.net.
>         Pros: ubiquitious, allows selection of single flow versus
> multi-flow test, allows server selection
>         Cons: only IPv4, only static unloaded RTT measurement, no control
> over measurement duration
>         BUFFERBLOAT verdict: incomplete, maybe usable as load generator
>
>
> NETFLIX: fast.com.
>         Pros: allows selection of upload testing, supposedly decent
> back-end, duration configurable
>                 allows unloaded, loaded download and loaded upload RTT
> measurements (but reports sinlge numbers for loaded and unloaded RTT, that
> are not the max)
>         Cons: RTT report as two numbers one for the loaded and one for
> unloaded RTT, time-course of RTTs missing
>         BUFFERBLOAT verdict: incomplete, but oh, so close...
>
>
> NPERF: nperf.com
>         Pros: allows server selection, RTT measurement and report as time
> course, also reports average rates and static RTT/jitter for Up- and
> Download
>         Cons: RTT measurement for unloaded only, reported RTT static only
> , no control over measurement duration
>         BUFFERBLOAT verdict: incomplete,
>
>
> THINKBROADBAND: www.thinkbroadband.com/speedtest
>         Pros: IPv6, reports coarse RTT time courses for all three
> measurement phases
>         Cons: only static unloaded RTT report in final results, time
> courses only visible immediately after testing, no control over measurement
> duration
>         BUFFERBLOAT verdict: a bit coarse, might work for users within a
> reasonable distance to the UK for acute de-bloating sessions (history
> reporting is bad though)
>
>
> honorable mentioning:
>         BREITBANDMESSUNG: breitbandmessung.de
>         Pros: query of contracted internet access speed before
> measurement, with a scheduler that will only start a test when the backend
> has sufficient capacity to saturate the user-supplied contracted rates,
> IPv6 (happy-eyeballs)
>         Cons: only static unloaded RTT measurement, no control over
> measurement duration
>         BUFFERBLOAT verdict: unsuitable, exceot as load generator, but the
> bandwidth reservation feature is quite nice.
>
> Best Regards
>         Sebastian
>
>
> > On May 1, 2020, at 18:44, Dave Taht <dave.taht@gmail.com> wrote:
> >
> >
> https://www.reddit.com/r/HomeNetworking/comments/gbd6g0/dsl_reports_speed_test_no_longer_free/
> >
> > They ran out of bandwidth.
> >
> > Message to users here:
> >
> > http://www.dslreports.com/speedtest
> >
> >
> > --
> > Make Music, Not War
> >
> > Dave Täht
> > CTO, TekLibre, LLC
> > http://www.teklibre.com
> > Tel: 1-831-435-0729
> > _______________________________________________
> > Cake mailing list
> > Cake@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

[-- Attachment #2: Type: text/html, Size: 7843 bytes --]

Re: [Make-wifi-fast] [Bloat] [Cake] dslreports is no longer free

[Sebastian Moeller]

Hi Sergey,



> On May 1, 2020, at 22:09, Sergey Fedorov <sfedorov@netflix.com> wrote:
> 
> Great review, Sebastian!
>  
> NETFLIX: fast.com.
>         Pros: allows selection of upload testing, supposedly decent back-end, duration configurable
>                 allows unloaded, loaded download and loaded upload RTT measurements (but reports sinlge numbers for loaded and unloaded RTT, that are not the max)
>         Cons: RTT report as two numbers one for the loaded and one for unloaded RTT, time-course of RTTs missing
>         BUFFERBLOAT verdict: incomplete, but oh, so close...
> Just a note that I have a plan to separate the loaded latency into upload/download. It's not great UX now they way it's implemented.

	Great! I really appreciate the way fast.com evolves carefully to not confuse the intended users and to stay true to its core mission while it still gaining additional features that are not directly part of Netflix business case to operate that test in the first place. Don't get me wrong, I absolutely love that I can easily understand why you should be interested in getting reliable robust speedtests from all existing or potential customers to your back-end; and unlike an ISP's internal speedtest, you are not likely to sugar coat things ;) as your goal and the end-user's goal are fully aligned.

> The timeline view is a bit more nuanced, in the spirit of the simplistic UX, but I've been thinking on a good way to show that for super users as well.

	Great again! I see the beauty of keeping things simple while maybe hiding optional information behind an additional "click".

> Two latency numbers - that's more user friendly, we want the general user to understand the meaning.

	+1; for normal users that is already bliss. For de-bloating a link however a bit more time resolution generally makes things a bit easier to reason about ;)

> And latency under load is much easier than bufferbloat.

	+1; as far as I can tell that term sort of was a decent description of the observed phenomenon that then got a life of its own; in retrospect it was not the most self explanatory term. I like to talk about the latency-under-load-increase when helping people to debloat their links, but that also is a tad on the long side.

> 
> As a side note, if our backend is decent, I'm curious what are the backends for the speed tests that exist that are great :)

	Ah, I might have tried too hard at understatement, this was the only back-end worth mentioning in the "pros" section...
(well, I also like how breitbandmessung.de deals with their purposefully limited backend (all located in a single" data center in Germany located in an AS that is not directly owned by any ISP, it's the german regulators official speedtest for germany against which we can effectively measure and get an early exit from contracts if the ISPs can not deliver the contracted rates (with a bit of slack)))

Best Regards
	Sebastian

>  
> SERGEY FEDOROV
> Director of Engineering
> sfedorov@netflix.com
> 121 Albright Way  |  Los Gatos, CA 95032
> 
> 
> 
> On Fri, May 1, 2020 at 12:48 PM Sebastian Moeller <moeller0@gmx.de> wrote:
> Hi Dave,
> 
> well, it was a free service and it lasted a long time. I want to raise a toast to Justin and convey my sincere thanks for years of investing into the "good" of the internet. 
> 
> Now, the question is which test is going to be the rightful successor? 
> 
> Short of running netperf/irtt/iper2/iperf3 on a hosted server, I see lots of potential but none of the tests are really there yet (grievances in now particular order):
> 
> OOKLA: speedtest.net.
>         Pros: ubiquitious, allows selection of single flow versus multi-flow test, allows server selection
>         Cons: only IPv4, only static unloaded RTT measurement, no control over measurement duration
>         BUFFERBLOAT verdict: incomplete, maybe usable as load generator
> 
> 
> NETFLIX: fast.com.
>         Pros: allows selection of upload testing, supposedly decent back-end, duration configurable
>                 allows unloaded, loaded download and loaded upload RTT measurements (but reports sinlge numbers for loaded and unloaded RTT, that are not the max)
>         Cons: RTT report as two numbers one for the loaded and one for unloaded RTT, time-course of RTTs missing
>         BUFFERBLOAT verdict: incomplete, but oh, so close...
> 
> 
> NPERF: nperf.com
>         Pros: allows server selection, RTT measurement and report as time course, also reports average rates and static RTT/jitter for Up- and Download
>         Cons: RTT measurement for unloaded only, reported RTT static only , no control over measurement duration
>         BUFFERBLOAT verdict: incomplete,
> 
> 
> THINKBROADBAND: www.thinkbroadband.com/speedtest
>         Pros: IPv6, reports coarse RTT time courses for all three measurement phases
>         Cons: only static unloaded RTT report in final results, time courses only visible immediately after testing, no control over measurement duration
>         BUFFERBLOAT verdict: a bit coarse, might work for users within a reasonable distance to the UK for acute de-bloating sessions (history reporting is bad though)
> 
> 
> honorable mentioning:
>         BREITBANDMESSUNG: breitbandmessung.de
>         Pros: query of contracted internet access speed before measurement, with a scheduler that will only start a test when the backend has sufficient capacity to saturate the user-supplied contracted rates, IPv6 (happy-eyeballs)
>         Cons: only static unloaded RTT measurement, no control over measurement duration
>         BUFFERBLOAT verdict: unsuitable, exceot as load generator, but the bandwidth reservation feature is quite nice.
> 
> Best Regards
>         Sebastian
> 
> 
> > On May 1, 2020, at 18:44, Dave Taht <dave.taht@gmail.com> wrote:
> > 
> > https://www.reddit.com/r/HomeNetworking/comments/gbd6g0/dsl_reports_speed_test_no_longer_free/
> > 
> > They ran out of bandwidth.
> > 
> > Message to users here:
> > 
> > http://www.dslreports.com/speedtest
> > 
> > 
> > -- 
> > Make Music, Not War
> > 
> > Dave Täht
> > CTO, TekLibre, LLC
> > http://www.teklibre.com
> > Tel: 1-831-435-0729
> > _______________________________________________
> > Cake mailing list
> > Cake@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Cake] dslreports is no longer free

[Sergey Fedorov]

[-- Attachment #1: Type: text/plain, Size: 8302 bytes --]

Thanks for the kind words, Sebastian!

 +1; for normal users that is already bliss. For de-bloating a link however
> a bit more time resolution generally makes things a bit easier to reason
> about ;)

Apologies, I misunderstood your original statement. I interpreted it as a
vote to keep a single bufferbloat metric (vs loaded/unloaded latency).
Agreed on time resolution and its value. No question it's useful for
diagnostics. Open question is to what extent browser-based tools should be
used for detailed troubleshooting (due to sandboxing limitations), and when
is the time for the big guns (like flent) to enter the scene.

 I like to talk about the latency-under-load-increase when helping people
> to debloat their links, but that also is a tad on the long side.

Fully agree on length, don't like the verboseness as well. Still looking
for a term that is shorter and yet generic enough that I can explain to my
mom.

Ah, I might have tried too hard at understatement, this was the only
> back-end worth mentioning in the "pros" section...

Got it. The breitbandmessung case is indeed interesting.

SERGEY FEDOROV

Director of Engineering

sfedorov@netflix.com

121 Albright Way | Los Gatos, CA 95032



On Fri, May 1, 2020 at 2:11 PM Sebastian Moeller <moeller0@gmx.de> wrote:

> Hi Sergey,
>
>
>
> > On May 1, 2020, at 22:09, Sergey Fedorov <sfedorov@netflix.com> wrote:
> >
> > Great review, Sebastian!
> >
> > NETFLIX: fast.com.
> >         Pros: allows selection of upload testing, supposedly decent
> back-end, duration configurable
> >                 allows unloaded, loaded download and loaded upload RTT
> measurements (but reports sinlge numbers for loaded and unloaded RTT, that
> are not the max)
> >         Cons: RTT report as two numbers one for the loaded and one for
> unloaded RTT, time-course of RTTs missing
> >         BUFFERBLOAT verdict: incomplete, but oh, so close...
> > Just a note that I have a plan to separate the loaded latency into
> upload/download. It's not great UX now they way it's implemented.
>
>         Great! I really appreciate the way fast.com evolves carefully to
> not confuse the intended users and to stay true to its core mission while
> it still gaining additional features that are not directly part of Netflix
> business case to operate that test in the first place. Don't get me wrong,
> I absolutely love that I can easily understand why you should be interested
> in getting reliable robust speedtests from all existing or potential
> customers to your back-end; and unlike an ISP's internal speedtest, you are
> not likely to sugar coat things ;) as your goal and the end-user's goal are
> fully aligned.
>
> > The timeline view is a bit more nuanced, in the spirit of the simplistic
> UX, but I've been thinking on a good way to show that for super users as
> well.
>
>         Great again! I see the beauty of keeping things simple while maybe
> hiding optional information behind an additional "click".
>
> > Two latency numbers - that's more user friendly, we want the general
> user to understand the meaning.
>
>         +1; for normal users that is already bliss. For de-bloating a link
> however a bit more time resolution generally makes things a bit easier to
> reason about ;)
>
> > And latency under load is much easier than bufferbloat.
>
>         +1; as far as I can tell that term sort of was a decent
> description of the observed phenomenon that then got a life of its own; in
> retrospect it was not the most self explanatory term. I like to talk about
> the latency-under-load-increase when helping people to debloat their links,
> but that also is a tad on the long side.
>
> >
> > As a side note, if our backend is decent, I'm curious what are the
> backends for the speed tests that exist that are great :)
>
>         Ah, I might have tried too hard at understatement, this was the
> only back-end worth mentioning in the "pros" section...
> (well, I also like how breitbandmessung.de deals with their purposefully
> limited backend (all located in a single" data center in Germany located in
> an AS that is not directly owned by any ISP, it's the german regulators
> official speedtest for germany against which we can effectively measure and
> get an early exit from contracts if the ISPs can not deliver the contracted
> rates (with a bit of slack)))
>
> Best Regards
>         Sebastian
>
> >
> > SERGEY FEDOROV
> > Director of Engineering
> > sfedorov@netflix.com
> > 121 Albright Way  |  Los Gatos, CA 95032
> >
> >
> >
> > On Fri, May 1, 2020 at 12:48 PM Sebastian Moeller <moeller0@gmx.de>
> wrote:
> > Hi Dave,
> >
> > well, it was a free service and it lasted a long time. I want to raise a
> toast to Justin and convey my sincere thanks for years of investing into
> the "good" of the internet.
> >
> > Now, the question is which test is going to be the rightful successor?
> >
> > Short of running netperf/irtt/iper2/iperf3 on a hosted server, I see
> lots of potential but none of the tests are really there yet (grievances in
> now particular order):
> >
> > OOKLA: speedtest.net.
> >         Pros: ubiquitious, allows selection of single flow versus
> multi-flow test, allows server selection
> >         Cons: only IPv4, only static unloaded RTT measurement, no
> control over measurement duration
> >         BUFFERBLOAT verdict: incomplete, maybe usable as load generator
> >
> >
> > NETFLIX: fast.com.
> >         Pros: allows selection of upload testing, supposedly decent
> back-end, duration configurable
> >                 allows unloaded, loaded download and loaded upload RTT
> measurements (but reports sinlge numbers for loaded and unloaded RTT, that
> are not the max)
> >         Cons: RTT report as two numbers one for the loaded and one for
> unloaded RTT, time-course of RTTs missing
> >         BUFFERBLOAT verdict: incomplete, but oh, so close...
> >
> >
> > NPERF: nperf.com
> >         Pros: allows server selection, RTT measurement and report as
> time course, also reports average rates and static RTT/jitter for Up- and
> Download
> >         Cons: RTT measurement for unloaded only, reported RTT static
> only , no control over measurement duration
> >         BUFFERBLOAT verdict: incomplete,
> >
> >
> > THINKBROADBAND: www.thinkbroadband.com/speedtest
> >         Pros: IPv6, reports coarse RTT time courses for all three
> measurement phases
> >         Cons: only static unloaded RTT report in final results, time
> courses only visible immediately after testing, no control over measurement
> duration
> >         BUFFERBLOAT verdict: a bit coarse, might work for users within a
> reasonable distance to the UK for acute de-bloating sessions (history
> reporting is bad though)
> >
> >
> > honorable mentioning:
> >         BREITBANDMESSUNG: breitbandmessung.de
> >         Pros: query of contracted internet access speed before
> measurement, with a scheduler that will only start a test when the backend
> has sufficient capacity to saturate the user-supplied contracted rates,
> IPv6 (happy-eyeballs)
> >         Cons: only static unloaded RTT measurement, no control over
> measurement duration
> >         BUFFERBLOAT verdict: unsuitable, exceot as load generator, but
> the bandwidth reservation feature is quite nice.
> >
> > Best Regards
> >         Sebastian
> >
> >
> > > On May 1, 2020, at 18:44, Dave Taht <dave.taht@gmail.com> wrote:
> > >
> > >
> https://www.reddit.com/r/HomeNetworking/comments/gbd6g0/dsl_reports_speed_test_no_longer_free/
> > >
> > > They ran out of bandwidth.
> > >
> > > Message to users here:
> > >
> > > http://www.dslreports.com/speedtest
> > >
> > >
> > > --
> > > Make Music, Not War
> > >
> > > Dave Täht
> > > CTO, TekLibre, LLC
> > > http://www.teklibre.com
> > > Tel: 1-831-435-0729
> > > _______________________________________________
> > > Cake mailing list
> > > Cake@lists.bufferbloat.net
> > > https://lists.bufferbloat.net/listinfo/cake
> >
> > _______________________________________________
> > Bloat mailing list
> > Bloat@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat
>
>

[-- Attachment #2: Type: text/html, Size: 12423 bytes --]

Re: [Make-wifi-fast] [Bloat] [Cake] dslreports is no longer free

[Michael Richardson]

[-- Attachment #1: Type: text/plain, Size: 1440 bytes --]


{Do I need all the lists?}

Sergey Fedorov via Bloat <bloat@lists.bufferbloat.net> wrote:
    > Just a note that I have a plan to separate the loaded latency into
    > upload/download. It's not great UX now they way it's implemented.
    > The timeline view is a bit more nuanced, in the spirit of the simplistic
    > UX, but I've been thinking on a good way to show that for super users as
    > well.
    > Two latency numbers - that's more user friendly, we want the general user
    > to understand the meaning. And latency under load is much easier than
    > bufferbloat.

    > As a side note, if our backend is decent, I'm curious what are the backends
    > for the speed tests that exist that are great :)

Does it find/use my nearest Netflix cache?

As others asked, it would be great if we could put the settings into a URL,
and having the "latency under upload" is probably the most important number
that people trying to videoconference need to know.

(it's also the thing that they can mostly directly/cheaply fix)

    > SERGEY FEDOROV
    > Director of Engineering
    > sfedorov@netflix.com
    > 121 Albright Way | Los Gatos, CA 95032

Very happy that you are looped in here.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     mcr@sandelman.ca  http://www.sandelman.ca/        |   ruby on rails    [


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [Bloat] [Cake] dslreports is no longer free

[Sergey Fedorov]

[-- Attachment #1: Type: text/plain, Size: 1917 bytes --]

Hi Michael,

This blog post <https://netflixtechblog.com/building-fast-com-4857fe0f8adb>
describes how
the test steers to the server(s).
Noted on the other thread, I hope to add the url param option reasonably
soon.

SERGEY FEDOROV

Director of Engineering

sfedorov@netflix.com

121 Albright Way | Los Gatos, CA 95032



On Fri, May 1, 2020 at 3:07 PM Michael Richardson <mcr@sandelman.ca> wrote:

>
> {Do I need all the lists?}
>
> Sergey Fedorov via Bloat <bloat@lists.bufferbloat.net> wrote:
>     > Just a note that I have a plan to separate the loaded latency into
>     > upload/download. It's not great UX now they way it's implemented.
>     > The timeline view is a bit more nuanced, in the spirit of the
> simplistic
>     > UX, but I've been thinking on a good way to show that for super
> users as
>     > well.
>     > Two latency numbers - that's more user friendly, we want the general
> user
>     > to understand the meaning. And latency under load is much easier than
>     > bufferbloat.
>
>     > As a side note, if our backend is decent, I'm curious what are the
> backends
>     > for the speed tests that exist that are great :)
>
> Does it find/use my nearest Netflix cache?
>
> As others asked, it would be great if we could put the settings into a URL,
> and having the "latency under upload" is probably the most important number
> that people trying to videoconference need to know.
>
> (it's also the thing that they can mostly directly/cheaply fix)
>
>     > SERGEY FEDOROV
>     > Director of Engineering
>     > sfedorov@netflix.com
>     > 121 Albright Way | Los Gatos, CA 95032
>
> Very happy that you are looped in here.
>
> --
> ]               Never tell me the odds!                 | ipv6 mesh
> networks [
> ]   Michael Richardson, Sandelman Software Works        |    IoT
> architect   [
> ]     mcr@sandelman.ca  http://www.sandelman.ca/        |   ruby on
> rails    [
>
>

[-- Attachment #2: Type: text/html, Size: 4039 bytes --]

Re: [Make-wifi-fast] [Bloat] [Cake] dslreports is no longer free

[Michael Richardson]

[-- Attachment #1: Type: text/plain, Size: 681 bytes --]


Given QUIC uses UDP and does congestion control essentially within the
browser, it seems that maybe one could built latency under load measuring
into the QUIC infrastructure in the browser.

Maybe we don't have to create JS tools like fast.com to get good and
regular measurements of bufferbloat.  Maybe it could be a part of
browsers.   Maybe web site designers could ask for the current
"latency-under-load" value from the browser DOM.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     mcr@sandelman.ca  http://www.sandelman.ca/        |   ruby on rails    [


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [Make-wifi-fast] [Bloat] [Cake] dslreports is no longer free

[Jannie Hanekom]

Michael Richardson <mcr@sandelman.ca>:
> Does it find/use my nearest Netflix cache?

Thankfully, it appears so.  The DSLReports bloat test was interesting, but
the jitter on the ~240ms base latency from South Africa (and other parts of
the world) was significant enough that the figures returned were often
unreliable and largely unusable - at least in my experience.

Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms and
mentions servers located in local cities.  I finally have a test I can share
with local non-technical people!

(Agreed, upload test would be nice, but this is a huge step forward from
what I had access to before.)

Jannie Hanekom

Re: [Make-wifi-fast] [Cake] [Bloat] dslreports is no longer free

[Benjamin Cronce]

[-- Attachment #1: Type: text/plain, Size: 1246 bytes --]

> Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms

For download, I show 6ms unloaded and 6-7 loaded. But for upload the loaded
shows as 7-8 and I see it blip upwards of 12ms. But I am no longer using
any traffic shaping. Any anti-bufferbloat is from my ISP. A graph of the
bloat would be nice.

On Sat, May 2, 2020 at 9:51 AM Jannie Hanekom <jannie@hanekom.net> wrote:

> Michael Richardson <mcr@sandelman.ca>:
> > Does it find/use my nearest Netflix cache?
>
> Thankfully, it appears so.  The DSLReports bloat test was interesting, but
> the jitter on the ~240ms base latency from South Africa (and other parts of
> the world) was significant enough that the figures returned were often
> unreliable and largely unusable - at least in my experience.
>
> Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms and
> mentions servers located in local cities.  I finally have a test I can
> share
> with local non-technical people!
>
> (Agreed, upload test would be nice, but this is a huge step forward from
> what I had access to before.)
>
> Jannie Hanekom
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>

[-- Attachment #2: Type: text/html, Size: 1875 bytes --]

Re: [Make-wifi-fast] [Cake] [Bloat] dslreports is no longer free

[Dave Taht]

On Sat, May 2, 2020 at 9:37 AM Benjamin Cronce <bcronce@gmail.com> wrote:
>
> > Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms

I guess one of my questions is that with a switch to BBR netflix is
going to do pretty well. If fast.com is using bbr, well... that
excludes much of the current side of the internet.

> For download, I show 6ms unloaded and 6-7 loaded. But for upload the loaded shows as 7-8 and I see it blip upwards of 12ms. But I am no longer using any traffic shaping. Any anti-bufferbloat is from my ISP. A graph of the bloat would be nice.

The tests do need to last a fairly long time.

> On Sat, May 2, 2020 at 9:51 AM Jannie Hanekom <jannie@hanekom.net> wrote:
>>
>> Michael Richardson <mcr@sandelman.ca>:
>> > Does it find/use my nearest Netflix cache?
>>
>> Thankfully, it appears so.  The DSLReports bloat test was interesting, but
>> the jitter on the ~240ms base latency from South Africa (and other parts of
>> the world) was significant enough that the figures returned were often
>> unreliable and largely unusable - at least in my experience.
>>
>> Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms and
>> mentions servers located in local cities.  I finally have a test I can share
>> with local non-technical people!
>>
>> (Agreed, upload test would be nice, but this is a huge step forward from
>> what I had access to before.)
>>
>> Jannie Hanekom
>>
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Make-wifi-fast] [Cake] [Bloat] dslreports is no longer free

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 4600 bytes --]


I am still a bit worried about properly defining "latency under load" for a NAT routed situation. If the test is based on ICMP Ping packets *from the server*,  it will NOT be measuring the full path latency, and if the potential congestion is in the uplink path from the access provider's residential box to the access provider's router/switch, it will NOT measure congestion caused by bufferbloat reliably on either side, since the bufferbloat will be outside the ICMP Ping path.
 
I realize that a browser based speed test has to be basically run from the "server" end, because browsers are not that good at time measurement on a packet basis. However, there are ways to solve this and avoid the ICMP Ping issue, with a cooperative server.
 
I once built a test that fixed this issue reasonably well. It carefully created a TCP based RTT measurement channel (over HTTP) that made the echo have to traverse the whole end-to-end path, which is the best and only way to accurately define lag under load from the user's perspective. The client end of an unloaded TCP connection can depend on TCP (properly prepared by getting it past slowstart) to generate a single packet response.
 
This "TCP ping" is thus compatible with getting the end-to-end measurement on the server end of a true RTT.
 
It's like tcp-traceroute tool, in that it tricks anyone in the middle boxes into thinking this is a real, serious packet, not an optional low priority packet.
 
The same issue comes up with non-browser-based techniques for measuring true lag-under-load.
 
Now as we move HTTP to QUIC, this actually gets easier to do.
 
One other opportunity I haven't explored, but which is pregnant with potential is the use of WebRTC, which runs over UDP internally. Since JavaScript has direct access to create WebRTC connections (multiple ones), this makes detailed testing in the browser quite reasonable.
 
And the time measurements can resolve well below 100 microseconds, if the JS is based on modern JIT compilation (Chrome, Firefox, Edge all compile to machine code speed if the code is restricted and in a loop). Then again, there is Web Assembly if you want to write C code that runs in the brower fast. WebAssembly is a low level language that compiles to machine code in the browser execution, and still has access to all the browser networking facilities.
 
On Saturday, May 2, 2020 12:52pm, "Dave Taht" <dave.taht@gmail.com> said:



> On Sat, May 2, 2020 at 9:37 AM Benjamin Cronce <bcronce@gmail.com> wrote:
> >
> > > Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms
> 
> I guess one of my questions is that with a switch to BBR netflix is
> going to do pretty well. If fast.com is using bbr, well... that
> excludes much of the current side of the internet.
> 
> > For download, I show 6ms unloaded and 6-7 loaded. But for upload the loaded
> shows as 7-8 and I see it blip upwards of 12ms. But I am no longer using any
> traffic shaping. Any anti-bufferbloat is from my ISP. A graph of the bloat would
> be nice.
> 
> The tests do need to last a fairly long time.
> 
> > On Sat, May 2, 2020 at 9:51 AM Jannie Hanekom <jannie@hanekom.net>
> wrote:
> >>
> >> Michael Richardson <mcr@sandelman.ca>:
> >> > Does it find/use my nearest Netflix cache?
> >>
> >> Thankfully, it appears so. The DSLReports bloat test was interesting,
> but
> >> the jitter on the ~240ms base latency from South Africa (and other parts
> of
> >> the world) was significant enough that the figures returned were often
> >> unreliable and largely unusable - at least in my experience.
> >>
> >> Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms
> and
> >> mentions servers located in local cities. I finally have a test I can
> share
> >> with local non-technical people!
> >>
> >> (Agreed, upload test would be nice, but this is a huge step forward from
> >> what I had access to before.)
> >>
> >> Jannie Hanekom
> >>
> >> _______________________________________________
> >> Cake mailing list
> >> Cake@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/cake
> >
> > _______________________________________________
> > Cake mailing list
> > Cake@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
> 
> 
> 
> --
> Make Music, Not War
> 
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-435-0729
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
> 

[-- Attachment #2: Type: text/html, Size: 7316 bytes --]

Re: [Cake] [Make-wifi-fast] [Bloat] dslreports is no longer free

[Sergey Fedorov]

[-- Attachment #1: Type: text/plain, Size: 6670 bytes --]

Dave, thanks for sharing interesting thoughts and context.

> I am still a bit worried about properly defining "latency under load" for
> a NAT routed situation. If the test is based on ICMP Ping packets *from the
> server*,  it will NOT be measuring the full path latency, and if the
> potential congestion is in the uplink path from the access provider's
> residential box to the access provider's router/switch, it will NOT measure
> congestion caused by bufferbloat reliably on either side, since the
> bufferbloat will be outside the ICMP Ping path.
>
> I realize that a browser based speed test has to be basically run from the
> "server" end, because browsers are not that good at time measurement on a
> packet basis. However, there are ways to solve this and avoid the ICMP Ping
> issue, with a cooperative server.

This erroneously assumes that fast.com measures latency from the server
side. It does not. The measurements are done from the client, over http,
with a parallel connection(s) to the same or similar set of servers, by
sending empty requests over a previously established connection (you can
see that in the browser web inspector).
It should be noted that the value is not precisely the "RTT on a
TCP/UDP flow that is loaded with traffic", but "user delay given the
presence of heavy parallel flows". With that, some of the challenges you
mentioned do not apply.
In line with another point I've shared earlier - the goal is to measure and
explain the user experience, not to be a diagnostic tool showing internal
transport metrics.

SERGEY FEDOROV

Director of Engineering

sfedorov@netflix.com

121 Albright Way | Los Gatos, CA 95032



On Sat, May 2, 2020 at 10:38 AM David P. Reed <dpreed@deepplum.com> wrote:

> I am still a bit worried about properly defining "latency under load" for
> a NAT routed situation. If the test is based on ICMP Ping packets *from the
> server*,  it will NOT be measuring the full path latency, and if the
> potential congestion is in the uplink path from the access provider's
> residential box to the access provider's router/switch, it will NOT measure
> congestion caused by bufferbloat reliably on either side, since the
> bufferbloat will be outside the ICMP Ping path.
>
>
>
> I realize that a browser based speed test has to be basically run from the
> "server" end, because browsers are not that good at time measurement on a
> packet basis. However, there are ways to solve this and avoid the ICMP Ping
> issue, with a cooperative server.
>
>
>
> I once built a test that fixed this issue reasonably well. It carefully
> created a TCP based RTT measurement channel (over HTTP) that made the echo
> have to traverse the whole end-to-end path, which is the best and only way
> to accurately define lag under load from the user's perspective. The client
> end of an unloaded TCP connection can depend on TCP (properly prepared by
> getting it past slowstart) to generate a single packet response.
>
>
>
> This "TCP ping" is thus compatible with getting the end-to-end measurement
> on the server end of a true RTT.
>
>
>
> It's like tcp-traceroute tool, in that it tricks anyone in the middle
> boxes into thinking this is a real, serious packet, not an optional low
> priority packet.
>
>
>
> The same issue comes up with non-browser-based techniques for measuring
> true lag-under-load.
>
>
>
> Now as we move HTTP to QUIC, this actually gets easier to do.
>
>
>
> One other opportunity I haven't explored, but which is pregnant with
> potential is the use of WebRTC, which runs over UDP internally. Since
> JavaScript has direct access to create WebRTC connections (multiple ones),
> this makes detailed testing in the browser quite reasonable.
>
>
>
> And the time measurements can resolve well below 100 microseconds, if the
> JS is based on modern JIT compilation (Chrome, Firefox, Edge all compile to
> machine code speed if the code is restricted and in a loop). Then again,
> there is Web Assembly if you want to write C code that runs in the brower
> fast. WebAssembly is a low level language that compiles to machine code in
> the browser execution, and still has access to all the browser networking
> facilities.
>
>
>
> On Saturday, May 2, 2020 12:52pm, "Dave Taht" <dave.taht@gmail.com> said:
>
> > On Sat, May 2, 2020 at 9:37 AM Benjamin Cronce <bcronce@gmail.com>
> wrote:
> > >
> > > > Fast.com reports my unloaded latency as 4ms, my loaded latency as
> ~7ms
> >
> > I guess one of my questions is that with a switch to BBR netflix is
> > going to do pretty well. If fast.com is using bbr, well... that
> > excludes much of the current side of the internet.
> >
> > > For download, I show 6ms unloaded and 6-7 loaded. But for upload the
> loaded
> > shows as 7-8 and I see it blip upwards of 12ms. But I am no longer using
> any
> > traffic shaping. Any anti-bufferbloat is from my ISP. A graph of the
> bloat would
> > be nice.
> >
> > The tests do need to last a fairly long time.
> >
> > > On Sat, May 2, 2020 at 9:51 AM Jannie Hanekom <jannie@hanekom.net>
> > wrote:
> > >>
> > >> Michael Richardson <mcr@sandelman.ca>:
> > >> > Does it find/use my nearest Netflix cache?
> > >>
> > >> Thankfully, it appears so. The DSLReports bloat test was interesting,
> > but
> > >> the jitter on the ~240ms base latency from South Africa (and other
> parts
> > of
> > >> the world) was significant enough that the figures returned were often
> > >> unreliable and largely unusable - at least in my experience.
> > >>
> > >> Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms
> > and
> > >> mentions servers located in local cities. I finally have a test I can
> > share
> > >> with local non-technical people!
> > >>
> > >> (Agreed, upload test would be nice, but this is a huge step forward
> from
> > >> what I had access to before.)
> > >>
> > >> Jannie Hanekom
> > >>
> > >> _______________________________________________
> > >> Cake mailing list
> > >> Cake@lists.bufferbloat.net
> > >> https://lists.bufferbloat.net/listinfo/cake
> > >
> > > _______________________________________________
> > > Cake mailing list
> > > Cake@lists.bufferbloat.net
> > > https://lists.bufferbloat.net/listinfo/cake
> >
> >
> >
> > --
> > Make Music, Not War
> >
> > Dave Täht
> > CTO, TekLibre, LLC
> > http://www.teklibre.com
> > Tel: 1-831-435-0729
> > _______________________________________________
> > Cake mailing list
> > Cake@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
> >
>

[-- Attachment #2: Type: text/html, Size: 11006 bytes --]

Re: [Make-wifi-fast] [Cake] [Bloat] dslreports is no longer free

[Sebastian Moeller]

Hi David,

in principle I agree, a NATed IPv4 ICMP probe will be at best reflected at the NAT router (CPE)  (some commercial home gateways do not respond to ICMP echo requests in the name of security theatre). So it is pretty hard to measure the full end to end path in that configuration. I believe that IPv6 should make that easier/simpler in that NAT hopefully will be out of the path (but let's see what ingenuity ISPs will come up with).
Then again, traditionally the relevant bottlenecks often are a) the internet access link itself and there the CPE is in a reasonable position as a reflector on the other side of the bottleneck as seen from an internet server, b) the home network between CPE and end-host, often with variable rate wifi, here I agree reflecting echos at the CPE hides part of the issue.



> On May 2, 2020, at 19:38, David P. Reed <dpreed@deepplum.com> wrote:
> 
> I am still a bit worried about properly defining "latency under load" for a NAT routed situation. If the test is based on ICMP Ping packets *from the server*,  it will NOT be measuring the full path latency, and if the potential congestion is in the uplink path from the access provider's residential box to the access provider's router/switch, it will NOT measure congestion caused by bufferbloat reliably on either side, since the bufferbloat will be outside the ICMP Ping path.

	Puzzled, as i believe it is going to be the residential box that will respond here, or will it be the AFTRs for CG-NAT that reflect the ICMP echo requests?

>  
> I realize that a browser based speed test has to be basically run from the "server" end, because browsers are not that good at time measurement on a packet basis. However, there are ways to solve this and avoid the ICMP Ping issue, with a cooperative server.
>  
> I once built a test that fixed this issue reasonably well. It carefully created a TCP based RTT measurement channel (over HTTP) that made the echo have to traverse the whole end-to-end path, which is the best and only way to accurately define lag under load from the user's perspective. The client end of an unloaded TCP connection can depend on TCP (properly prepared by getting it past slowstart) to generate a single packet response.
>  
> This "TCP ping" is thus compatible with getting the end-to-end measurement on the server end of a true RTT.
>  
> It's like tcp-traceroute tool, in that it tricks anyone in the middle boxes into thinking this is a real, serious packet, not an optional low priority packet.
>  
> The same issue comes up with non-browser-based techniques for measuring true lag-under-load.
>  
> Now as we move HTTP to QUIC, this actually gets easier to do.
>  
> One other opportunity I haven't explored, but which is pregnant with potential is the use of WebRTC, which runs over UDP internally. Since JavaScript has direct access to create WebRTC connections (multiple ones), this makes detailed testing in the browser quite reasonable.
>  
> And the time measurements can resolve well below 100 microseconds, if the JS is based on modern JIT compilation (Chrome, Firefox, Edge all compile to machine code speed if the code is restricted and in a loop). Then again, there is Web Assembly if you want to write C code that runs in the brower fast. WebAssembly is a low level language that compiles to machine code in the browser execution, and still has access to all the browser networking facilities.

	Mmmh, according to https://github.com/w3c/hr-time/issues/56 due to spectre side-channel vulnerabilities many browsers seemed to have lowered the timer resolution, but even the ~1ms resolution should be fine for typical RTTs.

Best Regards
	Sebastian

P.S.: I assume that I simply do not see/understand the full scope of the issue at hand yet.


>  
> On Saturday, May 2, 2020 12:52pm, "Dave Taht" <dave.taht@gmail.com> said:
> 
> > On Sat, May 2, 2020 at 9:37 AM Benjamin Cronce <bcronce@gmail.com> wrote:
> > >
> > > > Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms
> > 
> > I guess one of my questions is that with a switch to BBR netflix is
> > going to do pretty well. If fast.com is using bbr, well... that
> > excludes much of the current side of the internet.
> > 
> > > For download, I show 6ms unloaded and 6-7 loaded. But for upload the loaded
> > shows as 7-8 and I see it blip upwards of 12ms. But I am no longer using any
> > traffic shaping. Any anti-bufferbloat is from my ISP. A graph of the bloat would
> > be nice.
> > 
> > The tests do need to last a fairly long time.
> > 
> > > On Sat, May 2, 2020 at 9:51 AM Jannie Hanekom <jannie@hanekom.net>
> > wrote:
> > >>
> > >> Michael Richardson <mcr@sandelman.ca>:
> > >> > Does it find/use my nearest Netflix cache?
> > >>
> > >> Thankfully, it appears so. The DSLReports bloat test was interesting,
> > but
> > >> the jitter on the ~240ms base latency from South Africa (and other parts
> > of
> > >> the world) was significant enough that the figures returned were often
> > >> unreliable and largely unusable - at least in my experience.
> > >>
> > >> Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms
> > and
> > >> mentions servers located in local cities. I finally have a test I can
> > share
> > >> with local non-technical people!
> > >>
> > >> (Agreed, upload test would be nice, but this is a huge step forward from
> > >> what I had access to before.)
> > >>
> > >> Jannie Hanekom
> > >>
> > >> _______________________________________________
> > >> Cake mailing list
> > >> Cake@lists.bufferbloat.net
> > >> https://lists.bufferbloat.net/listinfo/cake
> > >
> > > _______________________________________________
> > > Cake mailing list
> > > Cake@lists.bufferbloat.net
> > > https://lists.bufferbloat.net/listinfo/cake
> > 
> > 
> > 
> > --
> > Make Music, Not War
> > 
> > Dave Täht
> > CTO, TekLibre, LLC
> > http://www.teklibre.com
> > Tel: 1-831-435-0729
> > _______________________________________________
> > Cake mailing list
> > Cake@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
> > 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Make-wifi-fast] [Cake] [Bloat] dslreports is no longer free

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 7376 bytes --]


Sergey - I wasn't assuming anything about fast.com. The document you shared wasn't clear about the methodology's details here. Others sadly, have actually used ICMP pings in the way I described. I was making a generic comment of concern.
 
That said, it sounds like what you are doing is really helpful (esp. given that your measure is aimed at end user experiential qualities).
 
Good luck!
 
 
On Saturday, May 2, 2020 3:00pm, "Sergey Fedorov" <sfedorov@netflix.com> said:





Dave, thanks for sharing interesting thoughts and context. I am still a bit worried about properly defining "latency under load" for a NAT routed situation. If the test is based on ICMP Ping packets *from the server*,  it will NOT be measuring the full path latency, and if the potential congestion is in the uplink path from the access provider's residential box to the access provider's router/switch, it will NOT measure congestion caused by bufferbloat reliably on either side, since the bufferbloat will be outside the ICMP Ping path.
 
I realize that a browser based speed test has to be basically run from the "server" end, because browsers are not that good at time measurement on a packet basis. However, there are ways to solve this and avoid the ICMP Ping issue, with a cooperative server.
This erroneously assumes that [ fast.com ]( http://fast.com ) measures latency from the server side. It does not. The measurements are done from the client, over http, with a parallel connection(s) to the same or similar set of servers, by sending empty requests over a previously established connection (you can see that in the browser web inspector).
It should be noted that the value is not precisely the "RTT on a TCP/UDP flow that is loaded with traffic", but "user delay given the presence of heavy parallel flows". With that, some of the challenges you mentioned do not apply.
In line with another point I've shared earlier - the goal is to measure and explain the user experience, not to be a diagnostic tool showing internal transport metrics.






SERGEY FEDOROV
Director of Engineering
[ sfedorov@netflix.com ]( mailto:sfedorov@netflix.com )
121 Albright Way | Los Gatos, CA 95032


On Sat, May 2, 2020 at 10:38 AM David P. Reed <[ dpreed@deepplum.com ]( mailto:dpreed@deepplum.com )> wrote:
I am still a bit worried about properly defining "latency under load" for a NAT routed situation. If the test is based on ICMP Ping packets *from the server*,  it will NOT be measuring the full path latency, and if the potential congestion is in the uplink path from the access provider's residential box to the access provider's router/switch, it will NOT measure congestion caused by bufferbloat reliably on either side, since the bufferbloat will be outside the ICMP Ping path.
 
I realize that a browser based speed test has to be basically run from the "server" end, because browsers are not that good at time measurement on a packet basis. However, there are ways to solve this and avoid the ICMP Ping issue, with a cooperative server.
 
I once built a test that fixed this issue reasonably well. It carefully created a TCP based RTT measurement channel (over HTTP) that made the echo have to traverse the whole end-to-end path, which is the best and only way to accurately define lag under load from the user's perspective. The client end of an unloaded TCP connection can depend on TCP (properly prepared by getting it past slowstart) to generate a single packet response.
 
This "TCP ping" is thus compatible with getting the end-to-end measurement on the server end of a true RTT.
 
It's like tcp-traceroute tool, in that it tricks anyone in the middle boxes into thinking this is a real, serious packet, not an optional low priority packet.
 
The same issue comes up with non-browser-based techniques for measuring true lag-under-load.
 
Now as we move HTTP to QUIC, this actually gets easier to do.
 
One other opportunity I haven't explored, but which is pregnant with potential is the use of WebRTC, which runs over UDP internally. Since JavaScript has direct access to create WebRTC connections (multiple ones), this makes detailed testing in the browser quite reasonable.
 
And the time measurements can resolve well below 100 microseconds, if the JS is based on modern JIT compilation (Chrome, Firefox, Edge all compile to machine code speed if the code is restricted and in a loop). Then again, there is Web Assembly if you want to write C code that runs in the brower fast. WebAssembly is a low level language that compiles to machine code in the browser execution, and still has access to all the browser networking facilities.
 
On Saturday, May 2, 2020 12:52pm, "Dave Taht" <[ dave.taht@gmail.com ]( mailto:dave.taht@gmail.com )> said:



> On Sat, May 2, 2020 at 9:37 AM Benjamin Cronce <[ bcronce@gmail.com ]( mailto:bcronce@gmail.com )> wrote:
> >
> > > Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms
> 
> I guess one of my questions is that with a switch to BBR netflix is
> going to do pretty well. If [ fast.com ]( http://fast.com ) is using bbr, well... that
> excludes much of the current side of the internet.
> 
> > For download, I show 6ms unloaded and 6-7 loaded. But for upload the loaded
> shows as 7-8 and I see it blip upwards of 12ms. But I am no longer using any
> traffic shaping. Any anti-bufferbloat is from my ISP. A graph of the bloat would
> be nice.
> 
> The tests do need to last a fairly long time.
> 
> > On Sat, May 2, 2020 at 9:51 AM Jannie Hanekom <[ jannie@hanekom.net ]( mailto:jannie@hanekom.net )>
> wrote:
> >>
> >> Michael Richardson <[ mcr@sandelman.ca ]( mailto:mcr@sandelman.ca )>:
> >> > Does it find/use my nearest Netflix cache?
> >>
> >> Thankfully, it appears so. The DSLReports bloat test was interesting,
> but
> >> the jitter on the ~240ms base latency from South Africa (and other parts
> of
> >> the world) was significant enough that the figures returned were often
> >> unreliable and largely unusable - at least in my experience.
> >>
> >> Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms
> and
> >> mentions servers located in local cities. I finally have a test I can
> share
> >> with local non-technical people!
> >>
> >> (Agreed, upload test would be nice, but this is a huge step forward from
> >> what I had access to before.)
> >>
> >> Jannie Hanekom
> >>
> >> _______________________________________________
> >> Cake mailing list
> >> [ Cake@lists.bufferbloat.net ]( mailto:Cake@lists.bufferbloat.net )
> >> [ https://lists.bufferbloat.net/listinfo/cake ]( https://lists.bufferbloat.net/listinfo/cake )
> >
> > _______________________________________________
> > Cake mailing list
> > [ Cake@lists.bufferbloat.net ]( mailto:Cake@lists.bufferbloat.net )
> > [ https://lists.bufferbloat.net/listinfo/cake ]( https://lists.bufferbloat.net/listinfo/cake )
> 
> 
> 
> --
> Make Music, Not War
> 
> Dave Täht
> CTO, TekLibre, LLC
> [ http://www.teklibre.com ]( http://www.teklibre.com )
> Tel: 1-831-435-0729
> _______________________________________________
> Cake mailing list
> [ Cake@lists.bufferbloat.net ]( mailto:Cake@lists.bufferbloat.net )
> [ https://lists.bufferbloat.net/listinfo/cake ]( https://lists.bufferbloat.net/listinfo/cake )
>

[-- Attachment #2: Type: text/html, Size: 13135 bytes --]

[Make-wifi-fast] fast.com quality

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 7571 bytes --]


Sergey -
 
I am very happy to report that fast.com reports the following from my inexpensive Chromebook, over 802.11ac, my Linux-on-Celeron cake entry router setup, through RCN's "Gigabit service". It's a little surprising, only in how good it is.
 
460 Mbps down/17 Mbps up, 11 ms. unloaded, 18 ms. loaded.
 
I'm a little bit curious about the extra 7 ms. due to load. I'm wondering if it is in my WiFi path, or whether Cake is building a queue.
 
The 11 ms. to South Boston from my Needham home seems a bit high. I used to be about 7 msec. away from that switch. But I'm not complaiing.
On Saturday, May 2, 2020 3:00pm, "Sergey Fedorov" <sfedorov@netflix.com> said:





Dave, thanks for sharing interesting thoughts and context. I am still a bit worried about properly defining "latency under load" for a NAT routed situation. If the test is based on ICMP Ping packets *from the server*,  it will NOT be measuring the full path latency, and if the potential congestion is in the uplink path from the access provider's residential box to the access provider's router/switch, it will NOT measure congestion caused by bufferbloat reliably on either side, since the bufferbloat will be outside the ICMP Ping path.
 
I realize that a browser based speed test has to be basically run from the "server" end, because browsers are not that good at time measurement on a packet basis. However, there are ways to solve this and avoid the ICMP Ping issue, with a cooperative server.
This erroneously assumes that [ fast.com ]( http://fast.com ) measures latency from the server side. It does not. The measurements are done from the client, over http, with a parallel connection(s) to the same or similar set of servers, by sending empty requests over a previously established connection (you can see that in the browser web inspector).
It should be noted that the value is not precisely the "RTT on a TCP/UDP flow that is loaded with traffic", but "user delay given the presence of heavy parallel flows". With that, some of the challenges you mentioned do not apply.
In line with another point I've shared earlier - the goal is to measure and explain the user experience, not to be a diagnostic tool showing internal transport metrics.






SERGEY FEDOROV
Director of Engineering
[ sfedorov@netflix.com ]( mailto:sfedorov@netflix.com )
121 Albright Way | Los Gatos, CA 95032


On Sat, May 2, 2020 at 10:38 AM David P. Reed <[ dpreed@deepplum.com ]( mailto:dpreed@deepplum.com )> wrote:
I am still a bit worried about properly defining "latency under load" for a NAT routed situation. If the test is based on ICMP Ping packets *from the server*,  it will NOT be measuring the full path latency, and if the potential congestion is in the uplink path from the access provider's residential box to the access provider's router/switch, it will NOT measure congestion caused by bufferbloat reliably on either side, since the bufferbloat will be outside the ICMP Ping path.
 
I realize that a browser based speed test has to be basically run from the "server" end, because browsers are not that good at time measurement on a packet basis. However, there are ways to solve this and avoid the ICMP Ping issue, with a cooperative server.
 
I once built a test that fixed this issue reasonably well. It carefully created a TCP based RTT measurement channel (over HTTP) that made the echo have to traverse the whole end-to-end path, which is the best and only way to accurately define lag under load from the user's perspective. The client end of an unloaded TCP connection can depend on TCP (properly prepared by getting it past slowstart) to generate a single packet response.
 
This "TCP ping" is thus compatible with getting the end-to-end measurement on the server end of a true RTT.
 
It's like tcp-traceroute tool, in that it tricks anyone in the middle boxes into thinking this is a real, serious packet, not an optional low priority packet.
 
The same issue comes up with non-browser-based techniques for measuring true lag-under-load.
 
Now as we move HTTP to QUIC, this actually gets easier to do.
 
One other opportunity I haven't explored, but which is pregnant with potential is the use of WebRTC, which runs over UDP internally. Since JavaScript has direct access to create WebRTC connections (multiple ones), this makes detailed testing in the browser quite reasonable.
 
And the time measurements can resolve well below 100 microseconds, if the JS is based on modern JIT compilation (Chrome, Firefox, Edge all compile to machine code speed if the code is restricted and in a loop). Then again, there is Web Assembly if you want to write C code that runs in the brower fast. WebAssembly is a low level language that compiles to machine code in the browser execution, and still has access to all the browser networking facilities.
 
On Saturday, May 2, 2020 12:52pm, "Dave Taht" <[ dave.taht@gmail.com ]( mailto:dave.taht@gmail.com )> said:



> On Sat, May 2, 2020 at 9:37 AM Benjamin Cronce <[ bcronce@gmail.com ]( mailto:bcronce@gmail.com )> wrote:
> >
> > > Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms
> 
> I guess one of my questions is that with a switch to BBR netflix is
> going to do pretty well. If [ fast.com ]( http://fast.com ) is using bbr, well... that
> excludes much of the current side of the internet.
> 
> > For download, I show 6ms unloaded and 6-7 loaded. But for upload the loaded
> shows as 7-8 and I see it blip upwards of 12ms. But I am no longer using any
> traffic shaping. Any anti-bufferbloat is from my ISP. A graph of the bloat would
> be nice.
> 
> The tests do need to last a fairly long time.
> 
> > On Sat, May 2, 2020 at 9:51 AM Jannie Hanekom <[ jannie@hanekom.net ]( mailto:jannie@hanekom.net )>
> wrote:
> >>
> >> Michael Richardson <[ mcr@sandelman.ca ]( mailto:mcr@sandelman.ca )>:
> >> > Does it find/use my nearest Netflix cache?
> >>
> >> Thankfully, it appears so. The DSLReports bloat test was interesting,
> but
> >> the jitter on the ~240ms base latency from South Africa (and other parts
> of
> >> the world) was significant enough that the figures returned were often
> >> unreliable and largely unusable - at least in my experience.
> >>
> >> Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms
> and
> >> mentions servers located in local cities. I finally have a test I can
> share
> >> with local non-technical people!
> >>
> >> (Agreed, upload test would be nice, but this is a huge step forward from
> >> what I had access to before.)
> >>
> >> Jannie Hanekom
> >>
> >> _______________________________________________
> >> Cake mailing list
> >> [ Cake@lists.bufferbloat.net ]( mailto:Cake@lists.bufferbloat.net )
> >> [ https://lists.bufferbloat.net/listinfo/cake ]( https://lists.bufferbloat.net/listinfo/cake )
> >
> > _______________________________________________
> > Cake mailing list
> > [ Cake@lists.bufferbloat.net ]( mailto:Cake@lists.bufferbloat.net )
> > [ https://lists.bufferbloat.net/listinfo/cake ]( https://lists.bufferbloat.net/listinfo/cake )
> 
> 
> 
> --
> Make Music, Not War
> 
> Dave Täht
> CTO, TekLibre, LLC
> [ http://www.teklibre.com ]( http://www.teklibre.com )
> Tel: 1-831-435-0729
> _______________________________________________
> Cake mailing list
> [ Cake@lists.bufferbloat.net ]( mailto:Cake@lists.bufferbloat.net )
> [ https://lists.bufferbloat.net/listinfo/cake ]( https://lists.bufferbloat.net/listinfo/cake )
>

[-- Attachment #2: Type: text/html, Size: 13526 bytes --]

Re: [Make-wifi-fast] fast.com quality

[Dave Taht]

turn off cake, do it over wired. :) TAKE a packet cap of before and after.Thx.

On Sun, May 3, 2020 at 8:31 AM David P. Reed <dpreed@deepplum.com> wrote:
>
> Sergey -
>
>
>
> I am very happy to report that fast.com reports the following from my inexpensive Chromebook, over 802.11ac, my Linux-on-Celeron cake entry router setup, through RCN's "Gigabit service". It's a little surprising, only in how good it is.
>
>
>
> 460 Mbps down/17 Mbps up, 11 ms. unloaded, 18 ms. loaded.
>
>
>
> I'm a little bit curious about the extra 7 ms. due to load. I'm wondering if it is in my WiFi path, or whether Cake is building a queue.
>
>
>
> The 11 ms. to South Boston from my Needham home seems a bit high. I used to be about 7 msec. away from that switch. But I'm not complaiing.
>
> On Saturday, May 2, 2020 3:00pm, "Sergey Fedorov" <sfedorov@netflix.com> said:
>
> Dave, thanks for sharing interesting thoughts and context.
>>
>> I am still a bit worried about properly defining "latency under load" for a NAT routed situation. If the test is based on ICMP Ping packets *from the server*,  it will NOT be measuring the full path latency, and if the potential congestion is in the uplink path from the access provider's residential box to the access provider's router/switch, it will NOT measure congestion caused by bufferbloat reliably on either side, since the bufferbloat will be outside the ICMP Ping path.
>>
>> I realize that a browser based speed test has to be basically run from the "server" end, because browsers are not that good at time measurement on a packet basis. However, there are ways to solve this and avoid the ICMP Ping issue, with a cooperative server.
>
> This erroneously assumes that fast.com measures latency from the server side. It does not. The measurements are done from the client, over http, with a parallel connection(s) to the same or similar set of servers, by sending empty requests over a previously established connection (you can see that in the browser web inspector).
> It should be noted that the value is not precisely the "RTT on a TCP/UDP flow that is loaded with traffic", but "user delay given the presence of heavy parallel flows". With that, some of the challenges you mentioned do not apply.
> In line with another point I've shared earlier - the goal is to measure and explain the user experience, not to be a diagnostic tool showing internal transport metrics.
>
> SERGEY FEDOROV
>
> Director of Engineering
>
> sfedorov@netflix.com
>
> 121 Albright Way | Los Gatos, CA 95032
>
>
> On Sat, May 2, 2020 at 10:38 AM David P. Reed <dpreed@deepplum.com> wrote:
>>
>> I am still a bit worried about properly defining "latency under load" for a NAT routed situation. If the test is based on ICMP Ping packets *from the server*,  it will NOT be measuring the full path latency, and if the potential congestion is in the uplink path from the access provider's residential box to the access provider's router/switch, it will NOT measure congestion caused by bufferbloat reliably on either side, since the bufferbloat will be outside the ICMP Ping path.
>>
>>
>>
>> I realize that a browser based speed test has to be basically run from the "server" end, because browsers are not that good at time measurement on a packet basis. However, there are ways to solve this and avoid the ICMP Ping issue, with a cooperative server.
>>
>>
>>
>> I once built a test that fixed this issue reasonably well. It carefully created a TCP based RTT measurement channel (over HTTP) that made the echo have to traverse the whole end-to-end path, which is the best and only way to accurately define lag under load from the user's perspective. The client end of an unloaded TCP connection can depend on TCP (properly prepared by getting it past slowstart) to generate a single packet response.
>>
>>
>>
>> This "TCP ping" is thus compatible with getting the end-to-end measurement on the server end of a true RTT.
>>
>>
>>
>> It's like tcp-traceroute tool, in that it tricks anyone in the middle boxes into thinking this is a real, serious packet, not an optional low priority packet.
>>
>>
>>
>> The same issue comes up with non-browser-based techniques for measuring true lag-under-load.
>>
>>
>>
>> Now as we move HTTP to QUIC, this actually gets easier to do.
>>
>>
>>
>> One other opportunity I haven't explored, but which is pregnant with potential is the use of WebRTC, which runs over UDP internally. Since JavaScript has direct access to create WebRTC connections (multiple ones), this makes detailed testing in the browser quite reasonable.
>>
>>
>>
>> And the time measurements can resolve well below 100 microseconds, if the JS is based on modern JIT compilation (Chrome, Firefox, Edge all compile to machine code speed if the code is restricted and in a loop). Then again, there is Web Assembly if you want to write C code that runs in the brower fast. WebAssembly is a low level language that compiles to machine code in the browser execution, and still has access to all the browser networking facilities.
>>
>>
>>
>> On Saturday, May 2, 2020 12:52pm, "Dave Taht" <dave.taht@gmail.com> said:
>>
>> > On Sat, May 2, 2020 at 9:37 AM Benjamin Cronce <bcronce@gmail.com> wrote:
>> > >
>> > > > Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms
>> >
>> > I guess one of my questions is that with a switch to BBR netflix is
>> > going to do pretty well. If fast.com is using bbr, well... that
>> > excludes much of the current side of the internet.
>> >
>> > > For download, I show 6ms unloaded and 6-7 loaded. But for upload the loaded
>> > shows as 7-8 and I see it blip upwards of 12ms. But I am no longer using any
>> > traffic shaping. Any anti-bufferbloat is from my ISP. A graph of the bloat would
>> > be nice.
>> >
>> > The tests do need to last a fairly long time.
>> >
>> > > On Sat, May 2, 2020 at 9:51 AM Jannie Hanekom <jannie@hanekom.net>
>> > wrote:
>> > >>
>> > >> Michael Richardson <mcr@sandelman.ca>:
>> > >> > Does it find/use my nearest Netflix cache?
>> > >>
>> > >> Thankfully, it appears so. The DSLReports bloat test was interesting,
>> > but
>> > >> the jitter on the ~240ms base latency from South Africa (and other parts
>> > of
>> > >> the world) was significant enough that the figures returned were often
>> > >> unreliable and largely unusable - at least in my experience.
>> > >>
>> > >> Fast.com reports my unloaded latency as 4ms, my loaded latency as ~7ms
>> > and
>> > >> mentions servers located in local cities. I finally have a test I can
>> > share
>> > >> with local non-technical people!
>> > >>
>> > >> (Agreed, upload test would be nice, but this is a huge step forward from
>> > >> what I had access to before.)
>> > >>
>> > >> Jannie Hanekom
>> > >>
>> > >> _______________________________________________
>> > >> Cake mailing list
>> > >> Cake@lists.bufferbloat.net
>> > >> https://lists.bufferbloat.net/listinfo/cake
>> > >
>> > > _______________________________________________
>> > > Cake mailing list
>> > > Cake@lists.bufferbloat.net
>> > > https://lists.bufferbloat.net/listinfo/cake
>> >
>> >
>> >
>> > --
>> > Make Music, Not War
>> >
>> > Dave Täht
>> > CTO, TekLibre, LLC
>> > http://www.teklibre.com
>> > Tel: 1-831-435-0729
>> > _______________________________________________
>> > Cake mailing list
>> > Cake@lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/cake
>> >



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Make-wifi-fast] [Bloat] [Cake] dslreports is no longer free

[Matthew Ford]

What's the bufferbloat verdict on https://speed.cloudflare.com/ ?

Mat

> On 1 May 2020, at 20:48, Sebastian Moeller <moeller0@gmx.de> wrote:
> 
> Hi Dave,
> 
> well, it was a free service and it lasted a long time. I want to raise a toast to Justin and convey my sincere thanks for years of investing into the "good" of the internet. 
> 
> Now, the question is which test is going to be the rightful successor? 
> 
> Short of running netperf/irtt/iper2/iperf3 on a hosted server, I see lots of potential but none of the tests are really there yet (grievances in now particular order):
> 
> OOKLA: speedtest.net.
> 	Pros: ubiquitious, allows selection of single flow versus multi-flow test, allows server selection
> 	Cons: only IPv4, only static unloaded RTT measurement, no control over measurement duration
> 	BUFFERBLOAT verdict: incomplete, maybe usable as load generator
> 
> 
> NETFLIX: fast.com.
> 	Pros: allows selection of upload testing, supposedly decent back-end, duration configurable
> 		allows unloaded, loaded download and loaded upload RTT measurements (but reports sinlge numbers for loaded and unloaded RTT, that are not the max)
> 	Cons: RTT report as two numbers one for the loaded and one for unloaded RTT, time-course of RTTs missing
> 	BUFFERBLOAT verdict: incomplete, but oh, so close...
> 
> 
> NPERF: nperf.com
> 	Pros: allows server selection, RTT measurement and report as time course, also reports average rates and static RTT/jitter for Up- and Download
> 	Cons: RTT measurement for unloaded only, reported RTT static only , no control over measurement duration
> 	BUFFERBLOAT verdict: incomplete,
> 
> 
> THINKBROADBAND: www.thinkbroadband.com/speedtest
> 	Pros: IPv6, reports coarse RTT time courses for all three measurement phases
> 	Cons: only static unloaded RTT report in final results, time courses only visible immediately after testing, no control over measurement duration
> 	BUFFERBLOAT verdict: a bit coarse, might work for users within a reasonable distance to the UK for acute de-bloating sessions (history reporting is bad though)
> 
> 
> honorable mentioning:
> 	BREITBANDMESSUNG: breitbandmessung.de
> 	Pros: query of contracted internet access speed before measurement, with a scheduler that will only start a test when the backend has sufficient capacity to saturate the user-supplied contracted rates, IPv6 (happy-eyeballs)
> 	Cons: only static unloaded RTT measurement, no control over measurement duration
> 	BUFFERBLOAT verdict: unsuitable, exceot as load generator, but the bandwidth reservation feature is quite nice.
> 
> Best Regards
> 	Sebastian
> 
> 
>> On May 1, 2020, at 18:44, Dave Taht <dave.taht@gmail.com> wrote:
>> 
>> https://www.reddit.com/r/HomeNetworking/comments/gbd6g0/dsl_reports_speed_test_no_longer_free/
>> 
>> They ran out of bandwidth.
>> 
>> Message to users here:
>> 
>> http://www.dslreports.com/speedtest
>> 
>> 
>> -- 
>> Make Music, Not War
>> 
>> Dave Täht
>> CTO, TekLibre, LLC
>> http://www.teklibre.com
>> Tel: 1-831-435-0729
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Make-wifi-fast] [Bloat] [Cake] dslreports is no longer free

[Toke Høiland-Jørgensen]

Matthew Ford <ford@isoc.org> writes:

> What's the bufferbloat verdict on https://speed.cloudflare.com/ ?

Huh, didn't know about that. Seems they're measuring the latency before
the download test, though, so no bufferbloat numbers. If anyone knows
someone at Cloudflare we could try to bug to get this fixed, that would
be awesome!

Their FAQ links to https://www.speedcheck.org/ for "troubleshooting
tips". And of course that page doesn't seem to mention latency or
bufferbloat at all :(

-Toke

Re: [Make-wifi-fast] [Bloat] [Cake] dslreports is no longer free

[Sebastian Moeller]

Hi Mat,

> On May 27, 2020, at 11:08, Matthew Ford <ford@isoc.org> wrote:
> 
> What's the bufferbloat verdict on https://speed.cloudflare.com/ ?

	Not a verdict per se, but this has potential, but is not there yet.

Pros: Decent reporting of the Download rates including intermediate values
	Decent reporting for the idle latency (I like the box whisker plots, ans the details revealed on mouse-over, as well as the individual samples)

Cons: Upload seems missing
	Latency is only measured for a pre-download idle phase, that is important, but for bufferbloat testing we really need to see the latency-under-load numbers (separately for down- and upload).
	Test duration not configurable. A number of ISP techniques, like power-boost can give higher throughput for a limited amount of time, which often accidentally coincides with typical durations of speedtests*, so being able to confirm bufferbloat remedies at longer test run times is really helpful (nothing crazy, but if a test can run 30-60 seconds instead of just 10-20 seconds that already helps a lot).

Best Regards
	Sebastian



*) I believe this to be accidental, as the duration for "fair" power-boosting are naturally in the same few dozends of seconds range as typical speedtests take, nothing nefarious here.



> 
> Mat
> 
>> On 1 May 2020, at 20:48, Sebastian Moeller <moeller0@gmx.de> wrote:
>> 
>> Hi Dave,
>> 
>> well, it was a free service and it lasted a long time. I want to raise a toast to Justin and convey my sincere thanks for years of investing into the "good" of the internet. 
>> 
>> Now, the question is which test is going to be the rightful successor? 
>> 
>> Short of running netperf/irtt/iper2/iperf3 on a hosted server, I see lots of potential but none of the tests are really there yet (grievances in now particular order):
>> 
>> OOKLA: speedtest.net.
>> 	Pros: ubiquitious, allows selection of single flow versus multi-flow test, allows server selection
>> 	Cons: only IPv4, only static unloaded RTT measurement, no control over measurement duration
>> 	BUFFERBLOAT verdict: incomplete, maybe usable as load generator
>> 
>> 
>> NETFLIX: fast.com.
>> 	Pros: allows selection of upload testing, supposedly decent back-end, duration configurable
>> 		allows unloaded, loaded download and loaded upload RTT measurements (but reports sinlge numbers for loaded and unloaded RTT, that are not the max)
>> 	Cons: RTT report as two numbers one for the loaded and one for unloaded RTT, time-course of RTTs missing
>> 	BUFFERBLOAT verdict: incomplete, but oh, so close...
>> 
>> 
>> NPERF: nperf.com
>> 	Pros: allows server selection, RTT measurement and report as time course, also reports average rates and static RTT/jitter for Up- and Download
>> 	Cons: RTT measurement for unloaded only, reported RTT static only , no control over measurement duration
>> 	BUFFERBLOAT verdict: incomplete,
>> 
>> 
>> THINKBROADBAND: www.thinkbroadband.com/speedtest
>> 	Pros: IPv6, reports coarse RTT time courses for all three measurement phases
>> 	Cons: only static unloaded RTT report in final results, time courses only visible immediately after testing, no control over measurement duration
>> 	BUFFERBLOAT verdict: a bit coarse, might work for users within a reasonable distance to the UK for acute de-bloating sessions (history reporting is bad though)
>> 
>> 
>> honorable mentioning:
>> 	BREITBANDMESSUNG: breitbandmessung.de
>> 	Pros: query of contracted internet access speed before measurement, with a scheduler that will only start a test when the backend has sufficient capacity to saturate the user-supplied contracted rates, IPv6 (happy-eyeballs)
>> 	Cons: only static unloaded RTT measurement, no control over measurement duration
>> 	BUFFERBLOAT verdict: unsuitable, exceot as load generator, but the bandwidth reservation feature is quite nice.
>> 
>> Best Regards
>> 	Sebastian
>> 
>> 
>>> On May 1, 2020, at 18:44, Dave Taht <dave.taht@gmail.com> wrote:
>>> 
>>> https://www.reddit.com/r/HomeNetworking/comments/gbd6g0/dsl_reports_speed_test_no_longer_free/
>>> 
>>> They ran out of bandwidth.
>>> 
>>> Message to users here:
>>> 
>>> http://www.dslreports.com/speedtest
>>> 
>>> 
>>> -- 
>>> Make Music, Not War
>>> 
>>> Dave Täht
>>> CTO, TekLibre, LLC
>>> http://www.teklibre.com
>>> Tel: 1-831-435-0729
>>> _______________________________________________
>>> Cake mailing list
>>> Cake@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cake
>> 
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat

Re: [Cake] [Make-wifi-fast] [Bloat] dslreports is no longer free

[Sergey Fedorov]

[-- Attachment #1: Type: text/plain, Size: 10784 bytes --]

>
> Sergey - I wasn't assuming anything about fast.com. The document you
> shared wasn't clear about the methodology's details here. Others sadly,
> have actually used ICMP pings in the way I described. I was making a
> generic comment of concern.
>
> That said, it sounds like what you are doing is really helpful (esp. given
> that your measure is aimed at end user experiential qualities).

David - my apologies, I incorrectly interpreted your statement as being
said in context of fast.com measurements. The blog post linked indeed
doesn't provide the latency measurement details - was written before we
added the extra metrics. We'll see if we can publish an update.

1) a clear definition of lag under load that is from end-to-end in latency,
> and involves, ideally, independent traffic from multiple sources through
> the bottleneck.

 Curious if by multiple sources you mean multiple clients (devices) or
multiple connections sending data?


SERGEY FEDOROV

Director of Engineering

sfedorov@netflix.com

121 Albright Way | Los Gatos, CA 95032




On Sun, May 3, 2020 at 8:07 AM David P. Reed <dpreed@deepplum.com> wrote:

> Thanks Sebastian. I do agree that in many cases, reflecting the ICMP off
> the entry device that has the external IP address for the NAT gets most of
> the RTT measure, and if there's no queueing built up in the NAT device,
> that's a reasonable measure. But...
>
>
>
> However, if the router has "taken up the queueing delay" by rate limiting
> its uplink traffic to slightly less than the capacity (as with Cake and
> other TC shaping that isn't as good as cake), then there is a queue in the
> TC layer itself. This is what concerns me as a distortion in the
> measurement that can fool one into thinking the TC shaper is doing a good
> job, when in fact, lag under load may be quite high from inside the routed
> domain (the home).
>
>
>
> As you point out this unmeasured queueing delay can also be a problem with
> WiFi inside the home. But it isn't limited to that.
>
>
>
> A badly set up shaping/congestion management subsystem inside the NAT can
> look "very good" in its echo of ICMP packets, but be terrible in response
> time to trivial HTTP requests from inside, or equally terrible in twitch
> games and video conferencing.
>
>
>
> So, for example, for tuning settings with "Cake" it is useless.
>
>
>
> To be fair, usually the Access Provider has no control of what is done
> after the cable is terminated at the home, so as a way to decide if the
> provider is badly engineering its side, a ping from a server is a
> reasonable quality measure of the provider.
>
>
>
> But not a good measure of the user experience, and if the provider
> provides the NAT box, even if it has a good shaper in it, like Cake or
> fq_codel, it will just confuse the user and create the opportunity for a
> "finger pointing" argument where neither side understands what is going on.
>
>
>
> This is why we need
>
>
>
> 1) a clear definition of lag under load that is from end-to-end in
> latency, and involves, ideally, independent traffic from multiple sources
> through the bottleneck.
>
>
>
> 2) ideally, a better way to localize where the queues are building up and
> present that to users and access providers.  The flent graphs are not
> interpretable by most non-experts. What we need is a simple visualization
> of a sketch-map of the path (like traceroute might provide) with queueing
> delay measures  shown at key points that the user can understand.
>
> On Saturday, May 2, 2020 4:19pm, "Sebastian Moeller" <moeller0@gmx.de>
> said:
>
> > Hi David,
> >
> > in principle I agree, a NATed IPv4 ICMP probe will be at best reflected
> at the NAT
> > router (CPE) (some commercial home gateways do not respond to ICMP echo
> requests
> > in the name of security theatre). So it is pretty hard to measure the
> full end to
> > end path in that configuration. I believe that IPv6 should make that
> > easier/simpler in that NAT hopefully will be out of the path (but let's
> see what
> > ingenuity ISPs will come up with).
> > Then again, traditionally the relevant bottlenecks often are a) the
> internet
> > access link itself and there the CPE is in a reasonable position as a
> reflector on
> > the other side of the bottleneck as seen from an internet server, b) the
> home
> > network between CPE and end-host, often with variable rate wifi, here I
> agree
> > reflecting echos at the CPE hides part of the issue.
> >
> >
> >
> > > On May 2, 2020, at 19:38, David P. Reed <dpreed@deepplum.com> wrote:
> > >
> > > I am still a bit worried about properly defining "latency under load"
> for a
> > NAT routed situation. If the test is based on ICMP Ping packets *from
> the server*,
> > it will NOT be measuring the full path latency, and if the potential
> congestion
> > is in the uplink path from the access provider's residential box to the
> access
> > provider's router/switch, it will NOT measure congestion caused by
> bufferbloat
> > reliably on either side, since the bufferbloat will be outside the ICMP
> Ping
> > path.
> >
> > Puzzled, as i believe it is going to be the residential box that will
> respond
> > here, or will it be the AFTRs for CG-NAT that reflect the ICMP echo
> requests?
> >
> > >
> > > I realize that a browser based speed test has to be basically run from
> the
> > "server" end, because browsers are not that good at time measurement on
> a packet
> > basis. However, there are ways to solve this and avoid the ICMP Ping
> issue, with a
> > cooperative server.
> > >
> > > I once built a test that fixed this issue reasonably well. It carefully
> > created a TCP based RTT measurement channel (over HTTP) that made the
> echo have to
> > traverse the whole end-to-end path, which is the best and only way to
> accurately
> > define lag under load from the user's perspective. The client end of an
> unloaded
> > TCP connection can depend on TCP (properly prepared by getting it past
> slowstart)
> > to generate a single packet response.
> > >
> > > This "TCP ping" is thus compatible with getting the end-to-end
> measurement on
> > the server end of a true RTT.
> > >
> > > It's like tcp-traceroute tool, in that it tricks anyone in the middle
> boxes
> > into thinking this is a real, serious packet, not an optional low
> priority
> > packet.
> > >
> > > The same issue comes up with non-browser-based techniques for
> measuring true
> > lag-under-load.
> > >
> > > Now as we move HTTP to QUIC, this actually gets easier to do.
> > >
> > > One other opportunity I haven't explored, but which is pregnant with
> > potential is the use of WebRTC, which runs over UDP internally. Since
> JavaScript
> > has direct access to create WebRTC connections (multiple ones), this
> makes
> > detailed testing in the browser quite reasonable.
> > >
> > > And the time measurements can resolve well below 100 microseconds, if
> the JS
> > is based on modern JIT compilation (Chrome, Firefox, Edge all compile to
> machine
> > code speed if the code is restricted and in a loop). Then again, there
> is Web
> > Assembly if you want to write C code that runs in the brower fast.
> WebAssembly is
> > a low level language that compiles to machine code in the browser
> execution, and
> > still has access to all the browser networking facilities.
> >
> > Mmmh, according to https://github.com/w3c/hr-time/issues/56 due to
> spectre
> > side-channel vulnerabilities many browsers seemed to have lowered the
> timer
> > resolution, but even the ~1ms resolution should be fine for typical RTTs.
> >
> > Best Regards
> > Sebastian
> >
> > P.S.: I assume that I simply do not see/understand the full scope of the
> issue at
> > hand yet.
> >
> >
> > >
> > > On Saturday, May 2, 2020 12:52pm, "Dave Taht" <dave.taht@gmail.com>
> > said:
> > >
> > > > On Sat, May 2, 2020 at 9:37 AM Benjamin Cronce <bcronce@gmail.com>
> > wrote:
> > > > >
> > > > > > Fast.com reports my unloaded latency as 4ms, my loaded latency
> > as ~7ms
> > > >
> > > > I guess one of my questions is that with a switch to BBR netflix is
> > > > going to do pretty well. If fast.com is using bbr, well... that
> > > > excludes much of the current side of the internet.
> > > >
> > > > > For download, I show 6ms unloaded and 6-7 loaded. But for upload
> > the loaded
> > > > shows as 7-8 and I see it blip upwards of 12ms. But I am no longer
> using
> > any
> > > > traffic shaping. Any anti-bufferbloat is from my ISP. A graph of the
> > bloat would
> > > > be nice.
> > > >
> > > > The tests do need to last a fairly long time.
> > > >
> > > > > On Sat, May 2, 2020 at 9:51 AM Jannie Hanekom
> > <jannie@hanekom.net>
> > > > wrote:
> > > > >>
> > > > >> Michael Richardson <mcr@sandelman.ca>:
> > > > >> > Does it find/use my nearest Netflix cache?
> > > > >>
> > > > >> Thankfully, it appears so. The DSLReports bloat test was
> > interesting,
> > > > but
> > > > >> the jitter on the ~240ms base latency from South Africa (and
> > other parts
> > > > of
> > > > >> the world) was significant enough that the figures returned
> > were often
> > > > >> unreliable and largely unusable - at least in my experience.
> > > > >>
> > > > >> Fast.com reports my unloaded latency as 4ms, my loaded latency
> > as ~7ms
> > > > and
> > > > >> mentions servers located in local cities. I finally have a test
> > I can
> > > > share
> > > > >> with local non-technical people!
> > > > >>
> > > > >> (Agreed, upload test would be nice, but this is a huge step
> > forward from
> > > > >> what I had access to before.)
> > > > >>
> > > > >> Jannie Hanekom
> > > > >>
> > > > >> _______________________________________________
> > > > >> Cake mailing list
> > > > >> Cake@lists.bufferbloat.net
> > > > >> https://lists.bufferbloat.net/listinfo/cake
> > > > >
> > > > > _______________________________________________
> > > > > Cake mailing list
> > > > > Cake@lists.bufferbloat.net
> > > > > https://lists.bufferbloat.net/listinfo/cake
> > > >
> > > >
> > > >
> > > > --
> > > > Make Music, Not War
> > > >
> > > > Dave Täht
> > > > CTO, TekLibre, LLC
> > > > http://www.teklibre.com
> > > > Tel: 1-831-435-0729
> > > > _______________________________________________
> > > > Cake mailing list
> > > > Cake@lists.bufferbloat.net
> > > > https://lists.bufferbloat.net/listinfo/cake
> > > >
> > > _______________________________________________
> > > Cake mailing list
> > > Cake@lists.bufferbloat.net
> > > https://lists.bufferbloat.net/listinfo/cake
> >
> >
>
>
>

[-- Attachment #2: Type: text/html, Size: 16703 bytes --]