Re: [Bloat] Updated Bufferbloat Test

[Marco Belmonte <marco@heavenlysanctuary.com>]

Wow, this is awesome and what a great contribution! Thanks so much!

On 2/24/2021 10:22 AM, Sina Khanifar wrote:
> Hi all,
>
> A couple of months ago my co-founder Sam posted an early beta of the
> Bufferbloat test that we’ve been working on, and Dave also linked to
> it a couple of weeks ago.
>
> Thank you all so much for your feedback - we almost entirely
> redesigned the tool and the UI based on the comments we received.
> We’re almost ready to launch the tool officially today at this URL,
> but wanted to show it to the list in case anyone finds any last bugs
> that we might have overlooked:
>
> https://www.waveform.com/tools/bufferbloat
>
> If you find a bug, please share the "Share Your Results" link with us
> along with what happened. We capture some debugging information on the
> backend, and having a share link allows us to diagnose any issues.
>
> This is really more of a passion project than anything else for us –
> we don’t anticipate we’ll try to commercialize it or anything like
> that. We're very thankful for all the work the folks on this list have
> done to identify and fix bufferbloat, and hope this is a useful
> contribution. I’ve personally been very frustrated by bufferbloat on a
> range of devices, and decided it might be helpful to build another
> bufferbloat test when the DSLReports test was down at some point last
> year.
>
> Our goals with this project were:
>    * To build a second solid bufferbloat test in case DSLReports goes down again.
>    * Build a test where bufferbloat is front and center as the primary
> purpose of the test, rather than just a feature.
>    * Try to explain bufferbloat and its effect on a user's connection
> as clearly as possible for a lay audience.
>
> A few notes:
>    * On the backend, we’re using Cloudflare’s CDN to perform the actual
> download and upload speed test. I know John Graham-Cunning has posted
> to this list in the past; if he or anyone from Cloudflare sees this,
> we’d love some help. Our Cloudflare Workers are being
> bandwidth-throttled due to having a non-enterprise grade account.
> We’ve worked around this in a kludgy way, but we’d love to get it
> resolved.
>    * We have lots of ideas for improvements, e.g. simultaneous
> upload/downloads, trying different file size chunks, time-series
> latency graphs, using WebRTC to test UDP traffic etc, but in the
> interest of getting things launched we're sticking with the current
> featureset.
>    * There are a lot of browser-specific workarounds that we had to
> implement, and latency itself is measured in different ways on
> Safari/Webkit vs Chromium/Firefox due to limitations of the
> PerformanceTiming APIs. You may notice that latency is different on
> different browsers, however the actual bufferbloat (relative increase
> in latency) should be pretty consistent.
>
> In terms of some of the changes we made based on the feedback we
> receive on this list:
>
> Based on Toke’s feedback:
> https://lists.bufferbloat.net/pipermail/bloat/2020-November/015960.html
> https://lists.bufferbloat.net/pipermail/bloat/2020-November/015976.html
>    * We changed the way the speed tests run to show an instantaneous
> speed as the test is being run.
>    * We moved the bufferbloat grade into the main results box.
>    * We tried really hard to get as close to saturating gigabit
> connections as possible. We redesigned completely the way we chunk
> files, added a “warming up” period, and spent quite a bit optimizing
> our code to minimize CPU usage, as we found that was often the
> limiting factor to our speed test results.
>    * We changed the shield grades altogether and went through a few
> different iterations of how to show the effect of bufferbloat on
> connectivity, and ended up with a “table view” to try to show the
> effect that bufferbloat specifically is having on the connection
> (compared to when the connection is unloaded).
>    * We now link from the results table view to the FAQ where the
> conditions for each type of connection are explained.
>    * We also changed the way we measure latency and now use the faster
> of either Google’s CDN or Cloudflare at any given location. We’re also
> using the WebTiming APIs to get a more accurate latency number, though
> this does not work on some mobile browsers (e.g. iOS Safari) and as a
> result we show a higher latency on mobile devices. Since our test is
> less a test of absolute latency and more a test of relative latency
> with and without load, we felt this was workable.
>    * Our jitter is now an average (was previously RMS).
>    * The “before you start” text was rewritten and moved above the start button.
>    * We now spell out upload and download instead of having arrows.
>    * We hugely reduced the number of cross-site scripts. I was a bit
> embarrassed by this if I’m honest - I spent a long time building web
> tools for the EFF, where we almost never allowed any cross-site
> scripts. * Our site is hosted on Shopify, and adding any features via
> their app store ends up adding a whole lot of gunk. But we uninstalled
> some apps, rewrote our template, and ended up removing a whole lot of
> the gunk. There’s still plenty of room for improvement, but it should
> be a lot better than before.
>
> Based on Dave Collier-Brown’s feedback:
> https://lists.bufferbloat.net/pipermail/bloat/2020-November/015966.html
>    * We replaced the “unloaded” and “loaded” language with “unloaded”
> and then “download active”  and “upload active.” In the grade box we
> indicate that, for example, “Your latency increased moderately under
> load.”
>    * We tried to generally make it easier for non-techie folks to
> understand by emphasizing the grade and adding the table showing how
> bufferbloat affects some commonly-used services.
>    * We didn’t really change the candle charts too much - they’re
> mostly just to give a basic visual - we focused more on the actual
> meat of the results above that.
>
> Based on Sebastian Moeller’s feedback:
> https://lists.bufferbloat.net/pipermail/bloat/2020-November/015963.html
>    * We considered doing a bidirectional saturating load, but decided
> to skip on implementing it for now. * It’s definitely something we’d
> like to experiment with more in the future.
>    * We added a “warming up” period as well as a “draining” period to
> help fill and empty the buffer. We haven’t added the option for an
> extended test, but have this on our list of backlog changes to make in
> the future.
>
> Based on Y’s feedback (link):
> https://lists.bufferbloat.net/pipermail/bloat/2020-November/015962.html
>    * We actually ended up removing the grades, but we explained our
> criteria for the new table in the FAQ.
>
> Based on Greg White's feedback (shared privately):
> * We added an FAQ answer explaining jitter and how we measure it.
>
> We’d love for you all to play with the new version of the tool and
> send over any feedback you might have. We’re going to be in a feature
> freeze before launch but we'd love to get any bugs sorted out. We'll
> likely put this project aside after we iron out a last round of bugs
> and launch, and turn back to working on projects that help us pay the
> bills, but we definitely hope to revisit and improve the tool over
> time.
>
> Best,
>
> Sina, Arshan, and Sam.
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat