[Bloat] Updated Bufferbloat Test

[Toke Høiland-Jørgensen]

Sina Khanifar <sina at waveform.com> writes:

> Based on Toke’s feedback:
> https://lists.bufferbloat.net/pipermail/bloat/2020-November/015960.html
> https://lists.bufferbloat.net/pipermail/bloat/2020-November/015976.html

Thank you for the update, and especially this very detailed changelog!
I'm impressed! A few points on the specific items below:

>   * We changed the way the speed tests run to show an instantaneous
> speed as the test is being run.

Much better, I can actually see what's going on now :)
Maybe an 'abort' button somewhere would be useful? Once you've clicked
start the only way to abort is currently to close the browser tab...

>   * We moved the bufferbloat grade into the main results box.

Also very good!

>   * We tried really hard to get as close to saturating gigabit
> connections as possible. We redesigned completely the way we chunk
> files, added a “warming up” period, and spent quite a bit optimizing
> our code to minimize CPU usage, as we found that was often the
> limiting factor to our speed test results.

Yup, this seems to work better now! I can basically saturate my
connection now; Chromium seems to be a bit better than Firefox in this
respect, but I ended up getting very close on both:

Chromium:
https://www.waveform.com/tools/bufferbloat?test-id=b14731d3-46d7-49ba-8cc7-3641b495e6c7
Firefox:
https://www.waveform.com/tools/bufferbloat?test-id=877f496a-457a-4cc2-8f4c-91e23065c59e

(this is with a ~100Mbps base load on a Gbps connection, so at least the
Chromium result is pretty much link speed).

Interestingly, while my link is not bloated (the above results are
without running any shaping, just FQ-CoDel on the physical link), it did
manage to knock out the BFD exchange with my upstream BGP peers, causing
routes to flap. So it's definitely saturating something! :D

>   * We changed the shield grades altogether and went through a few
> different iterations of how to show the effect of bufferbloat on
> connectivity, and ended up with a “table view” to try to show the
> effect that bufferbloat specifically is having on the connection
> (compared to when the connection is unloaded).

I like this, with one caveat: When you have a good score, you end up
with a table that has all checkmarks in both the "normally" and "with
bufferbloat" columns, which is a bit confusing (makes one think "huh, I
can do low-latency gaming with bufferbloat?"). So I think changing the
column headings would be good; if I'm interpreting what you're trying to
convey, the second column should really say "your connection", right?
And maybe "normally" should be "Ideally"?

>   * We now link from the results table view to the FAQ where the
> conditions for each type of connection are explained.

This works well. I also like the FAQ in general (the water/oil in the
sink analogy is great!). What did you base the router recommendations
on? I haven't heard about that Asus gaming router before, does that ship
SQM? Also, the first time you mention the open source distributions,
OpenWrt is not a link (but it is the second time around).

>   * We also changed the way we measure latency and now use the faster
> of either Google’s CDN or Cloudflare at any given location.

Are you sure this is working? Mine seems to pick the Google fonts CDN.
The Javascript console outputs 'latency_source_selector.js:26 times
(2) [12.81000001472421, 12.80999998562038]', but in the network tab I
see two OPTIONS requests to fonts.gstatic.com, so I suspect those two
requests both go there? My ICMP ping time to Google is ~11ms, and it's
1.8ms to speed.cloudflare.com, so it seems a bit odd that it would pick
the other one... But maybe it's replying faster to HTTP?

> We’re also using the WebTiming APIs to get a more accurate latency
> number, though this does not work on some mobile browsers (e.g. iOS
> Safari) and as a result we show a higher latency on mobile devices.
> Since our test is less a test of absolute latency and more a test of
> relative latency with and without load, we felt this was workable.

That seems reasonable.

>   * Our jitter is now an average (was previously RMS).

I'll echo what the others have said about jitter.

>   * The “before you start” text was rewritten and moved above the start button.
>   * We now spell out upload and download instead of having arrows.
>   * We hugely reduced the number of cross-site scripts. I was a bit
> embarrassed by this if I’m honest - I spent a long time building web
> tools for the EFF, where we almost never allowed any cross-site
> scripts. * Our site is hosted on Shopify, and adding any features via
> their app store ends up adding a whole lot of gunk. But we uninstalled
> some apps, rewrote our template, and ended up removing a whole lot of
> the gunk. There’s still plenty of room for improvement, but it should
> be a lot better than before.

Thank you for this! It even works without allowing all the shopify
scripts, so that's all good :)

-Toke