Re: [Bloat] We built a new bufferbloat test and keen for feedback

Re: [Bloat] We built a new bufferbloat test and keen for feedback

[Rich Brown]

It's great that you're tackling this problem. In addition to various speed test websites, it's worth comparing your results to the other tests we use for measuring bufferbloat. You probably know about these, but I figured I would list them for completeness:

1) Flent.org - the gold standard that provides repeatable, measurable results. See also the RRUL Chart Explanation for Cliff Notes about how to read the resulting plots. https://www.bufferbloat.net/projects/bloat/wiki/RRUL_Chart_Explanation/

2) betterspeedtest.sh - an easy-to-run script that fires off a ping test, then runs netperf to generate upload and download traffic. https://github.com/richb-hanover/OpenWrtScripts#betterspeedtestsh

Since these are not web-based, their results may not match yours, although it would be good to be able to explain differences.

Best regards,

Rich Brown

Re: [Bloat] We built a new bufferbloat test and keen for feedback

[Sam]

[-- Attachment #1: Type: text/plain, Size: 3297 bytes --]

On 11/4/20 3:30 PM, Sam Westwood wrote:
> Hi everyone,
> 
> My name is Sam and I'm the co-founder and COO of Waveform.com. At 
> Waveform we provide equipment to help improve cell phone service, and 
> being in the industry we've always been interested in all aspects of 
> network connectivity. Bufferbloat for us has always been interesting, 
> and while there are a few tests out there we never found one that was 
> fantastic. So we thought we'd try and build one!
> 
> My colleague Arshan has built the test, which we based upon the 
> Cloudflare Speedtest template that was discussed earlier in the summer 
> in a previous thread.
> 
> We measure bufferbloat under two conditions: when downlink is saturated 
> and when uplink is saturated. The test involves three stages: Unloaded, 
> Downlink Saturated, and Uplink Saturated. In the first stage we simply 
> measure latency to a file hosted on a CDN. This is usually around 5ms 
> and could vary a bit based on the user's location. We use the webTiming 
> API to find the time-to-first-byte, and consider that as the latency. In 
> the second stage we run a download, while simultaneously measuring 
> latency. In the third stage we do the same but for upload. Both download 
> and upload usually take around 5 seconds. We show the median, first 
> quartile and the third quartile on distribution charts corresponding to 
> each stage to provide a visual representation of the latency variations. 
> For download and upload we have used Cloudflare's speedtest backend.
> 
> You can find the test here: https://www.waveform.com/apps/dev-arshan 
> <https://www.waveform.com/apps/dev-arshan>
> 
> We built testing it on Chrome, but it works on Firefox and mobile too. 
> On mobile results may be a little different, as the APIs aren't 
> available and so instead we implemented a more manual method, which can 
> be a little noisier.
> 
> This is a really early alpha, and so we are keen to get any and all 
> feedback you have :-). Things that we would particularly like feedback on:
> 
>   * How does the bufferbloat measure compare to other tests you may have
>     run on the same connection (e.g. dslreports, fast.com <http://fast.com>)
>   * How the throughput results (download/upload/latency) look compared
>     to other tools
>   * Any feedback on the user interface of the test itself? We know that
>     before releasing more widely we will put more effort into explaining
>     bufferbloat than we have so far.
>   * Anything else you would like to give feedback on?
> 
> We have added a feature to share results via a URL, so please feel free 
> to share these if you have specific feedback.
> 
> Thanks!
> Sam and Arshan
> 
> *************************
> Sam Westwood
> Co-Founder & COO | RSRF & Waveform
> E sam@waveform.com <mailto:sam@waveform.com>
> D   (949) 207-3175
> T   1-800-761-3041 Ext. 100
> W www.rsrf.com <http://www.rsrf.com> & www.waveform.com 
> <http://www.waveform.com>
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
> 


Looks pretty identical to what fast.com gave me. I'm on 50/50 fiber and 
firefox 82.
https://www.waveform.com/apps/dev-arshan?test-id=58dfa326-23d4-44a3-9059-b6011b104ccb

--Sam

[-- Attachment #2: Screenshot_20201105_152751.png --]
[-- Type: image/png, Size: 32907 bytes --]

[-- Attachment #3: Screenshot_20201105_152830.png --]
[-- Type: image/png, Size: 41976 bytes --]

Re: [Bloat] We built a new bufferbloat test and keen for feedback

[Toke Høiland-Jørgensen]

Dave Collier-Brown <dave.collier-brown@indexexchange.com> writes:

> Tried it, and I really like the header and use of candle-charts!
>
> I got this:
>
> [cid:part1.81AC21AC.758FE66F@indexexchange.com]
>
> I'd like to be able to explain it to non-techie folks (my grandma, and also my IT team at work (;-)), so I wonder on their behalf...
>
>   *   Why is unloaded a large number, and loaded a small one?
>      *   milliseconds sound like delay, so 111.7 ms sounds slower than 0.0 ms
>   *   Is bloat and latency something bad? The zeroes are in green, does that mean they're good?
>   *   Is max "bad"? In that case I'd call it "worst" and min "best"
>   *   Is median the middle or the average? (no kidding, I've been asked that! I'd call it average)
>   *   Is 25% twenty-five percent of the packets? (I suspect it's a percentile)
>   *   What does this mean in terms of how many Skype calls I can have happening at my house? I have two kids, a wife and a grandmother, all of whom Skype a lot.
>
> Looking at the cool stuff in the banner, it looks like I can browse,
> do audio calls, video calls (just one, or many?) but not streaming
> (any or just 4k?) or gaming.  Emphasizing that would be instantly
> understandable by grandma and the kids.

Also, holy cow, what's going on with your connection? The unloaded
latency says 17/110/200 min/median/max RTT. Is that due to bad
measurements, or do you have a lot of cross traffic and a really bloated
link? :/

-Toke

Re: [Bloat] We built a new bufferbloat test and keen for feedback

[Sebastian Moeller]

Hi Sam,
 
first thanks, always good to see more dedicated tools to asses latency under load, especially tools that are easy to use and do not require the user to maintain her/his own dedicated endpoints!
More below in-line, prefixed [SM].
 
 

Gesendet: Mittwoch, 04. November 2020 um 22:30 Uhr
Von: "Sam Westwood" <sam@repeaterstore.com>
An: bloat@lists.bufferbloat.net
Betreff: [Bloat] We built a new bufferbloat test and keen for feedback

Hi everyone, 
My name is Sam and I'm the co-founder and COO of Waveform.com. At Waveform we provide equipment to help improve cell phone service, and being in the industry we've always been interested in all aspects of network connectivity. Bufferbloat for us has always been interesting, and while there are a few tests out there we never found one that was fantastic. So we thought we'd try and build one!
My colleague Arshan has built the test, which we based upon the Cloudflare Speedtest template that was discussed earlier in the summer in a previous thread.
 
We measure bufferbloat under two conditions: when downlink is saturated and when uplink is saturated.
 
        [SM] This is a decent starting point. In addition it might be helpful to at least optionally include a test with with bidirectional saturating load, in the past such tests typically were quite successful in detecting bufferbloat sources, that were less obvious in the uni-directional load tests. I am not sure however how well that can work with a browser based test?
 
 
The test involves three stages: Unloaded, Downlink Saturated, and Uplink Saturated. In the first stage we simply measure latency to a file hosted on a CDN. This is usually around 5ms and could vary a bit based on the user's location. We use the webTiming API to find the time-to-first-byte, and consider that as the latency. 

        [SM] Mmmh, I like that this is a relevant latency measure, it might make sense though to make sure users realize that this is not the eqivalent number to runing a ICMP eche request against the same endpoint?


In the second stage we run a download, while simultaneously measuring latency. In the third stage we do the same but for upload. Both download and upload usually take around 5 seconds. 

        [SM] On heavily bufferbloated links it often takes a considerable amount of time for the bottleneck buffers to drain after a uni-directional test, so it might make sense to separate the two direction test with an additional phase of idle latency measurements. If that latency is like the initial unloaded latency, all is well, but if latency slowly ramps down in that phase you have a smoking gun for bad bufferbloat.
         Also, there are link technologies and scheduler techniques that can prioritise relative short flows (e.g. Comcast's powerboost) to avoid just measuring the properties of these short duration special modes, it might make sense to optionally and considerably lengthen the duration of the test durations to say 30 seconds (empirically powerboost does not engage for a full 30second perid at full rate, but that might be arms race). Also to assess possible root causes for latency and rate issues, it is very helpful to show time resolved plots, that show the development of rate and latency over the duration of all phases of the test. For example, using longer running flent tests I could pinpoint the cyclic channel scanning of my laptop's wifi as a source of repeated bufferbloat with a period of ~10 seconds, by seeing evenly spaced latency spikes and rate dips every 10 seconds then went away when switching to wired ethernet...

We show the median, first quartile and the third quartile on distribution charts corresponding to each stage to provide a visual representation of the latency variations. For download and upload we have used Cloudflare's speedtest backend.

        [SM] This is great, it would be nice though to also add a graphical representation, be it a histogram or a cumulative density plot of latencies (split out for idle, download, upload and the idle period between down- and upload).


Best Regards
        Sebastian


 
You can find the test here: https://www.waveform.com/apps/dev-arshan
 
We built testing it on Chrome, but it works on Firefox and mobile too. On mobile results may be a little different, as the APIs aren't available and so instead we implemented a more manual method, which can be a little noisier.
 
This is a really early alpha, and so we are keen to get any and all feedback you have :-). Things that we would particularly like feedback on:
How does the bufferbloat measure compare to other tests you may have run on the same connection (e.g. dslreports, fast.com[http://fast.com])How the throughput results (download/upload/latency) look compared to other toolsAny feedback on the user interface of the test itself? We know that before releasing more widely we will put more effort into explaining bufferbloat than we have so far.  Anything else you would like to give feedback on?We have added a feature to share results via a URL, so please feel free to share these if you have specific feedback. 
Thanks!
Sam and Arshan
 
*************************
Sam Westwood
Co-Founder & COO | RSRF & Waveform
E   sam@waveform.com[mailto:sam@waveform.com]
D   (949) 207-3175  
T   1-800-761-3041 Ext. 100
W   www.rsrf.com[http://www.rsrf.com] & www.waveform.com[http://www.waveform.com]

 _______________________________________________ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat[https://lists.bufferbloat.net/listinfo/bloat]

Re: [Bloat] We built a new bufferbloat test and keen for feedback

[Y]

https://www.waveform.com/apps/dev-arshan?test-id=a9f994a0-171c-48d3-a40d-1550f13b2319
Great job. This is the result of my slow internees. I would like to know the criteria for the grade.







Le jeudi 5 novembre 2020 à 08:54:57 UTC+9, Toke Høiland-Jørgensen via Bloat <bloat@lists.bufferbloat.net> a écrit : 





Sam Westwood <sam@repeaterstore.com> writes:

> Hi everyone,
>
> My name is Sam and I'm the co-founder and COO of Waveform.com. At Waveform
> we provide equipment to help improve cell phone service, and being in the
> industry we've always been interested in all aspects of network
> connectivity. Bufferbloat for us has always been interesting, and while
> there are a few tests out there we never found one that was fantastic. So
> we thought we'd try and build one!
>
> My colleague Arshan has built the test, which we based upon the Cloudflare
> Speedtest template that was discussed earlier in the summer in a previous
> thread.
>
> We measure bufferbloat under two conditions: when downlink is saturated and
> when uplink is saturated. The test involves three stages: Unloaded,
> Downlink Saturated, and Uplink Saturated. In the first stage we simply
> measure latency to a file hosted on a CDN. This is usually around 5ms and
> could vary a bit based on the user's location. We use the webTiming API to
> find the time-to-first-byte, and consider that as the latency. In the
> second stage we run a download, while simultaneously measuring latency. In
> the third stage we do the same but for upload. Both download and upload
> usually take around 5 seconds. We show the median, first quartile and the
> third quartile on distribution charts corresponding to each stage to
> provide a visual representation of the latency variations. For download and
> upload we have used Cloudflare's speedtest backend.

This sounds great, thanks for doing this! It certainly sounds like
you're on the right track here. Some comments below...

> You can find the test here: https://www.waveform.com/apps/dev-arshan
>
> We built testing it on Chrome, but it works on Firefox and mobile too. On
> mobile results may be a little different, as the APIs aren't available and
> so instead we implemented a more manual method, which can be a little
> noisier.
>
> This is a really early alpha, and so we are keen to get any and all
> feedback you have :-). Things that we would particularly like feedback on:
>
>    - How does the bufferbloat measure compare to other tests you may have
>    run on the same connection (e.g. dslreports, fast.com)
>    - How the throughput results (download/upload/latency) look compared to
>    other tools

I'm fortunate enough to have a full Gbps fibre link, which makes it
really hard to saturate the connection from a browser test (or at all,
sometimes). Your test does a decent job and comes pretty close, at least
in Chromium (about 800 Mbps which is not too far off at the application
layer, considering I have a constant 100Mbps flow in the background
taking up some of the bandwidth). Firefox seems way off (one test said
500Mbps the other >1000).

This does mean that I can't say much that's useful about your
bufferbloat scores, unfortunately. The latency measurement puzzled me a
bit (your tool says 16.6ms, but I get half that when I ping the
cloudfront.net CDN, which I think is what you're measuring against?),
but it does seem to stay fairly constant.

How do you calculate the jitter score? It's not obvious how you get from
the percentiles to the jitter.

Link to the test in Chromium:
https://www.waveform.com/apps/dev-arshan?test-id=91a55adc-7513-4b55-b8a6-0fa698ce634e

>    - Any feedback on the user interface of the test itself? We know that
>    before releasing more widely we will put more effort into explaining
>    bufferbloat than we have so far.

Brain dump of thoughts on the UI:

I found it hard to tell whether it was doing anything while the test was
running. Most other tests have some kind of very obvious feedback
(moving graphs of bandwidth-over-time for cloudflare/dslreports, a
honking big number going up and down for fast.com), which I was missing
here. I would also have liked to a measure of bandwidth over time, it
seems a bit suspicious (from a "verify that this is doing something
reasonable" PoV) that it just spits out a number at the end without
telling me how long it ran, or how it got to that number.

It wasn't obvious at first either that the header changes from
"bufferbloat test" to "your bufferbloat grade" once the test is over I
think the stages + result would be better put somewhere else where it's
more obvious (the rest of the page grows downwards, so why isn't the
result at the "end"?).

Also, what are the shields below the grade supposed to mean? Do they
change depending on the result? On which criteria? And it's telling me I
have an A+ grade, so why is there a link to fix my bufferbloat issues?

Smaller nit, I found the up/down arrows in "up saturated" and "down
saturated" a bit hard to grasp at first, I think spelling out
upload/download would be better. Also not sure I like the "saturated"
term in the first place; do people know what that means in a networking
context? And can you be sure the network is actually *being* saturated?

Why is the "before you start" text below the page? Shouldn't it be at
the top? And maybe explain *why* this is important?

>    - Anything else you would like to give feedback on?

As far as the web page itself is concerned, holy cross-domain script
deluge, Batman! I'm one of those people who run uMatrix in my regular
Firefox session, and I disallow all cross-site script loads by default.
I had to allow 15(!) different cross-site requests, *and* whitelist a
few domains in my ad blocker as well to even get the test to run. Please
fix this! I get that you can't completely avoid cross-domain requests
due to the nature of the test, but why is a speedtest pulling in scripts
from 'shopify.com' and three different ad tracking networks?

I realise some of this may just be because you put this thing on your
corporate web site, and thus a lot of it is not directly related to the
speed test itself. But all the same, having to wade through chest-high
piles of javascript just to get to the test doesn't exactly scream
"trustworthy company that just wants to help me fight bufferbloat" :)

(For comparison, dslreports, fast.com and speed.cloudflare.com all pull
in scripts from exactly three domains, all of them directly related to
the test, and none of them fail with my default uBlock blocklist).

-Toke

_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] We built a new bufferbloat test and keen for feedback

[Dave Collier-Brown]

[-- Attachment #1: Type: text/plain, Size: 5361 bytes --]

Tried it, and I really like the header and use of candle-charts!

I got this:

[cid:part1.81AC21AC.758FE66F@indexexchange.com]

I'd like to be able to explain it to non-techie folks (my grandma, and also my IT team at work (;-)), so I wonder on their behalf...

  *   Why is unloaded a large number, and loaded a small one?
     *   milliseconds sound like delay, so 111.7 ms sounds slower than 0.0 ms
  *   Is bloat and latency something bad? The zeroes are in green, does that mean they're good?
  *   Is max "bad"? In that case I'd call it "worst" and min "best"
  *   Is median the middle or the average? (no kidding, I've been asked that! I'd call it average)
  *   Is 25% twenty-five percent of the packets? (I suspect it's a percentile)
  *   What does this mean in terms of how many Skype calls I can have happening at my house? I have two kids, a wife and a grandmother, all of whom Skype a lot.

Looking at the cool stuff in the banner, it looks like I can browse, do audio calls, video calls (just one, or many?) but not streaming (any or just 4k?) or gaming.  Emphasizing that would be instantly understandable by grandma and the kids.

--dave


On 2020-11-04 4:30 p.m., Sam Westwood wrote:
Hi everyone,

My name is Sam and I'm the co-founder and COO of Waveform.com. At Waveform we provide equipment to help improve cell phone service, and being in the industry we've always been interested in all aspects of network connectivity. Bufferbloat for us has always been interesting, and while there are a few tests out there we never found one that was fantastic. So we thought we'd try and build one!

My colleague Arshan has built the test, which we based upon the Cloudflare Speedtest template that was discussed earlier in the summer in a previous thread.

We measure bufferbloat under two conditions: when downlink is saturated and when uplink is saturated. The test involves three stages: Unloaded, Downlink Saturated, and Uplink Saturated. In the first stage we simply measure latency to a file hosted on a CDN. This is usually around 5ms and could vary a bit based on the user's location. We use the webTiming API to find the time-to-first-byte, and consider that as the latency. In the second stage we run a download, while simultaneously measuring latency. In the third stage we do the same but for upload. Both download and upload usually take around 5 seconds. We show the median, first quartile and the third quartile on distribution charts corresponding to each stage to provide a visual representation of the latency variations. For download and upload we have used Cloudflare's speedtest backend.

You can find the test here: https://www.waveform.com/apps/dev-arshan

We built testing it on Chrome, but it works on Firefox and mobile too. On mobile results may be a little different, as the APIs aren't available and so instead we implemented a more manual method, which can be a little noisier.

This is a really early alpha, and so we are keen to get any and all feedback you have :-). Things that we would particularly like feedback on:

  *   How does the bufferbloat measure compare to other tests you may have run on the same connection (e.g. dslreports, fast.com<http://fast.com>)
  *   How the throughput results (download/upload/latency) look compared to other tools
  *   Any feedback on the user interface of the test itself? We know that before releasing more widely we will put more effort into explaining bufferbloat than we have so far.
  *   Anything else you would like to give feedback on?

We have added a feature to share results via a URL, so please feel free to share these if you have specific feedback.

Thanks!
Sam and Arshan

*************************
Sam Westwood
Co-Founder & COO | RSRF & Waveform
E   sam@waveform.com<mailto:sam@waveform.com>
D   (949) 207-3175
T   1-800-761-3041 Ext. 100
W   www.rsrf.com<http://www.rsrf.com> & www.waveform.com<http://www.waveform.com>



_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net<mailto:Bloat@lists.bufferbloat.net>
https://lists.bufferbloat.net/listinfo/bloat


--
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
dave.collier-brown@indexexchange.com<mailto:dave.collier-brown@indexexchange.com> |              -- Mark Twain



CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, including any and all attachments, contains confidential information intended only for the person(s) to whom it is addressed. Any dissemination, distribution, copying or disclosure is strictly prohibited and is not a waiver of confidentiality. If you have received this telecommunication in error, please notify the sender immediately by return electronic mail and delete the message from your inbox and deleted items folders. This telecommunication does not constitute an express or implied agreement to conduct transactions by electronic means, nor does it constitute a contract offer, a contract amendment or an acceptance of a contract offer. Contract terms contained in this telecommunication are subject to legal review and the completion of formal documentation and are not binding until same is confirmed in writing and has been signed by an authorized signatory.

[-- Attachment #2.1: Type: text/html, Size: 7022 bytes --]

[-- Attachment #2.2: mbbhaappkdeojcia.png --]
[-- Type: image/png, Size: 121135 bytes --]

Re: [Bloat] We built a new bufferbloat test and keen for feedback

[Toke Høiland-Jørgensen]

Sam Westwood <sam@repeaterstore.com> writes:

> Hi everyone,
>
> My name is Sam and I'm the co-founder and COO of Waveform.com. At Waveform
> we provide equipment to help improve cell phone service, and being in the
> industry we've always been interested in all aspects of network
> connectivity. Bufferbloat for us has always been interesting, and while
> there are a few tests out there we never found one that was fantastic. So
> we thought we'd try and build one!
>
> My colleague Arshan has built the test, which we based upon the Cloudflare
> Speedtest template that was discussed earlier in the summer in a previous
> thread.
>
> We measure bufferbloat under two conditions: when downlink is saturated and
> when uplink is saturated. The test involves three stages: Unloaded,
> Downlink Saturated, and Uplink Saturated. In the first stage we simply
> measure latency to a file hosted on a CDN. This is usually around 5ms and
> could vary a bit based on the user's location. We use the webTiming API to
> find the time-to-first-byte, and consider that as the latency. In the
> second stage we run a download, while simultaneously measuring latency. In
> the third stage we do the same but for upload. Both download and upload
> usually take around 5 seconds. We show the median, first quartile and the
> third quartile on distribution charts corresponding to each stage to
> provide a visual representation of the latency variations. For download and
> upload we have used Cloudflare's speedtest backend.

This sounds great, thanks for doing this! It certainly sounds like
you're on the right track here. Some comments below...

> You can find the test here: https://www.waveform.com/apps/dev-arshan
>
> We built testing it on Chrome, but it works on Firefox and mobile too. On
> mobile results may be a little different, as the APIs aren't available and
> so instead we implemented a more manual method, which can be a little
> noisier.
>
> This is a really early alpha, and so we are keen to get any and all
> feedback you have :-). Things that we would particularly like feedback on:
>
>    - How does the bufferbloat measure compare to other tests you may have
>    run on the same connection (e.g. dslreports, fast.com)
>    - How the throughput results (download/upload/latency) look compared to
>    other tools

I'm fortunate enough to have a full Gbps fibre link, which makes it
really hard to saturate the connection from a browser test (or at all,
sometimes). Your test does a decent job and comes pretty close, at least
in Chromium (about 800 Mbps which is not too far off at the application
layer, considering I have a constant 100Mbps flow in the background
taking up some of the bandwidth). Firefox seems way off (one test said
500Mbps the other >1000).

This does mean that I can't say much that's useful about your
bufferbloat scores, unfortunately. The latency measurement puzzled me a
bit (your tool says 16.6ms, but I get half that when I ping the
cloudfront.net CDN, which I think is what you're measuring against?),
but it does seem to stay fairly constant.

How do you calculate the jitter score? It's not obvious how you get from
the percentiles to the jitter.

Link to the test in Chromium:
https://www.waveform.com/apps/dev-arshan?test-id=91a55adc-7513-4b55-b8a6-0fa698ce634e

>    - Any feedback on the user interface of the test itself? We know that
>    before releasing more widely we will put more effort into explaining
>    bufferbloat than we have so far.

Brain dump of thoughts on the UI:

I found it hard to tell whether it was doing anything while the test was
running. Most other tests have some kind of very obvious feedback
(moving graphs of bandwidth-over-time for cloudflare/dslreports, a
honking big number going up and down for fast.com), which I was missing
here. I would also have liked to a measure of bandwidth over time, it
seems a bit suspicious (from a "verify that this is doing something
reasonable" PoV) that it just spits out a number at the end without
telling me how long it ran, or how it got to that number.

It wasn't obvious at first either that the header changes from
"bufferbloat test" to "your bufferbloat grade" once the test is over I
think the stages + result would be better put somewhere else where it's
more obvious (the rest of the page grows downwards, so why isn't the
result at the "end"?).

Also, what are the shields below the grade supposed to mean? Do they
change depending on the result? On which criteria? And it's telling me I
have an A+ grade, so why is there a link to fix my bufferbloat issues?

Smaller nit, I found the up/down arrows in "up saturated" and "down
saturated" a bit hard to grasp at first, I think spelling out
upload/download would be better. Also not sure I like the "saturated"
term in the first place; do people know what that means in a networking
context? And can you be sure the network is actually *being* saturated?

Why is the "before you start" text below the page? Shouldn't it be at
the top? And maybe explain *why* this is important?

>    - Anything else you would like to give feedback on?

As far as the web page itself is concerned, holy cross-domain script
deluge, Batman! I'm one of those people who run uMatrix in my regular
Firefox session, and I disallow all cross-site script loads by default.
I had to allow 15(!) different cross-site requests, *and* whitelist a
few domains in my ad blocker as well to even get the test to run. Please
fix this! I get that you can't completely avoid cross-domain requests
due to the nature of the test, but why is a speedtest pulling in scripts
from 'shopify.com' and three different ad tracking networks?

I realise some of this may just be because you put this thing on your
corporate web site, and thus a lot of it is not directly related to the
speed test itself. But all the same, having to wade through chest-high
piles of javascript just to get to the test doesn't exactly scream
"trustworthy company that just wants to help me fight bufferbloat" :)

(For comparison, dslreports, fast.com and speed.cloudflare.com all pull
in scripts from exactly three domains, all of them directly related to
the test, and none of them fail with my default uBlock blocklist).

-Toke

Re: [Bloat] We built a new bufferbloat test and keen for feedback

[Michael Richardson]

[-- Attachment #1: Type: text/plain, Size: 1035 bytes --]


Sam Westwood <sam@repeaterstore.com> wrote:
    > Downlink Saturated, and Uplink Saturated. In the first stage we simply
    > measure latency to a file hosted on a CDN. This is usually around 5ms
    > and could vary a bit based on the user's location. We use the webTiming
    > API to find the time-to-first-byte, and consider that as the
    > latency. In the second stage we run a download, while simultaneously

Just to clarify:
  the latency measurement involves a TCP three-way handshake, with
  the fourth packet providing the end of the process.
  No TLS, I hope?

    > We built testing it on Chrome, but it works on Firefox and mobile
    > too. On mobile results may be a little different, as the APIs aren't
    > available and so instead we implemented a more manual method, which can
    > be a little noisier.

Would webrtc APIs have helped?


--
Michael Richardson <mcr+IETF@sandelman.ca>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide





[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[Bloat] We built a new bufferbloat test and keen for feedback

[Sam Westwood]

[-- Attachment #1: Type: text/plain, Size: 2643 bytes --]

Hi everyone,

My name is Sam and I'm the co-founder and COO of Waveform.com. At Waveform
we provide equipment to help improve cell phone service, and being in the
industry we've always been interested in all aspects of network
connectivity. Bufferbloat for us has always been interesting, and while
there are a few tests out there we never found one that was fantastic. So
we thought we'd try and build one!

My colleague Arshan has built the test, which we based upon the Cloudflare
Speedtest template that was discussed earlier in the summer in a previous
thread.

We measure bufferbloat under two conditions: when downlink is saturated and
when uplink is saturated. The test involves three stages: Unloaded,
Downlink Saturated, and Uplink Saturated. In the first stage we simply
measure latency to a file hosted on a CDN. This is usually around 5ms and
could vary a bit based on the user's location. We use the webTiming API to
find the time-to-first-byte, and consider that as the latency. In the
second stage we run a download, while simultaneously measuring latency. In
the third stage we do the same but for upload. Both download and upload
usually take around 5 seconds. We show the median, first quartile and the
third quartile on distribution charts corresponding to each stage to
provide a visual representation of the latency variations. For download and
upload we have used Cloudflare's speedtest backend.

You can find the test here: https://www.waveform.com/apps/dev-arshan

We built testing it on Chrome, but it works on Firefox and mobile too. On
mobile results may be a little different, as the APIs aren't available and
so instead we implemented a more manual method, which can be a little
noisier.

This is a really early alpha, and so we are keen to get any and all
feedback you have :-). Things that we would particularly like feedback on:

   - How does the bufferbloat measure compare to other tests you may have
   run on the same connection (e.g. dslreports, fast.com)
   - How the throughput results (download/upload/latency) look compared to
   other tools
   - Any feedback on the user interface of the test itself? We know that
   before releasing more widely we will put more effort into explaining
   bufferbloat than we have so far.
   - Anything else you would like to give feedback on?

We have added a feature to share results via a URL, so please feel free to
share these if you have specific feedback.

Thanks!
Sam and Arshan

*************************
Sam Westwood
Co-Founder & COO | RSRF & Waveform
E   sam@waveform.com
D   (949) 207-3175
T   1-800-761-3041 Ext. 100
W   www.rsrf.com & www.waveform.com

[-- Attachment #2: Type: text/html, Size: 3251 bytes --]