From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 86B393B29D for ; Wed, 24 Feb 2021 13:22:19 -0500 (EST) Received: by mail-ej1-x633.google.com with SMTP id t11so4622732ejx.6 for ; Wed, 24 Feb 2021 10:22:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=waveform.com; s=google; h=mime-version:from:date:message-id:subject:to:cc :content-transfer-encoding; bh=QgJNZGJrnZmpQx2lP+IVL+07G4pjF2FufMDVJC1bJxk=; b=QgptY3EnFM6qIr/TmwnIO/Naa7CC8hdQ9r2zI/Xef9g0etniv8V2vTCsYDHCPQZkKE VsYmyxkfqQHqN6efoz3SU7r/1ZaKDh/vVg6YP9sov7aLLbGcIgfuHQCBicb6Jk3GWN3M iPI1TKXjZBaS+kRtRGNh221yy9SZWdymeKL5IuBymLTDQayKjVXPh71UhwZRTzNJ2YUH 68PSY/B6C/bXMksVmMlUDgjiYz+DWJYNEgxblv2er5PlkqUthcLzbnIAbUjqxfwfqmWU +1/PP+o5Z8AZUfZ1+GKLdm7/k1Jqoa4uoZ0LSpewRfQNK3v4H+fY5v28xrEnw+r2yeJ6 oFhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc :content-transfer-encoding; bh=QgJNZGJrnZmpQx2lP+IVL+07G4pjF2FufMDVJC1bJxk=; b=lo0fLo9V/4Gb8ZVVSzGhSgnQ512lNNpos7VLwrVrwoidr4+tJs+aGgItDdjAHZ1iyY dByDoBt7Xaw2C82vQHqQJ2N9NnURWWAijvrAZdd/yTYm9kzdObqhZADdTLLctKbPacjZ Ln0SYcUJGwfD4hzEB0GH4HQYfDz9S76sgQcumUUk2TM04cLKNaxYMNfZlNfxkANY3wJd kgZnKwy38CsxbCKtHoaI990YXfLaJGEaCdt+WUJosPX2zZ3J4qJNheVbL1V2uYz+ZD6i ZNNtL0o/YXMnwo7vODzwcSgcFZSBNAWwipGSBe2ezroNvnr0oPaAH/0UQWQx5ldCn3I0 hOFA== X-Gm-Message-State: AOAM533hVE5MJMXOm0o/FNdFy8L4AVrWQUtgCwxOUPBjue7GvIKUr6fQ Ew92AYaHytlzn+OV3bLglSDWkci8gsCP205OG6ONgOcYOcOerSrPIb5plkUyv8l2Z4XL4VopU9I rcZeVprG6lmYwV9tOVnVTI8OplIrI3r+aWAfpWfkdBUlG9aHHLsDPRYFSc6Crp6U3+yAwfEBBUy /yO7hfBfQ= X-Google-Smtp-Source: ABdhPJxiSdnr0BdALhT1Q7WDKTdKDjKhDUCSBYcgHNWzEC/r/Piop3Uz0g8LjbgddSrJdtXmGBzBoA== X-Received: by 2002:a17:907:3fa6:: with SMTP id hr38mr32557042ejc.24.1614190937943; Wed, 24 Feb 2021 10:22:17 -0800 (PST) Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com. [209.85.208.49]) by smtp.gmail.com with ESMTPSA id k4sm1789223ejv.73.2021.02.24.10.22.17 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 24 Feb 2021 10:22:17 -0800 (PST) Received: by mail-ed1-f49.google.com with SMTP id c6so3795073ede.0 for ; Wed, 24 Feb 2021 10:22:17 -0800 (PST) X-Received: by 2002:aa7:ca4b:: with SMTP id j11mr33368888edt.232.1614190937069; Wed, 24 Feb 2021 10:22:17 -0800 (PST) MIME-Version: 1.0 From: Sina Khanifar Date: Wed, 24 Feb 2021 10:22:05 -0800 X-Gmail-Original-Message-ID: Message-ID: To: bloat Cc: sam@waveform.com, arshankhanifar@gmail.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: [Bloat] Updated Bufferbloat Test X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2021 18:22:19 -0000 Hi all, A couple of months ago my co-founder Sam posted an early beta of the Bufferbloat test that we=E2=80=99ve been working on, and Dave also linked t= o it a couple of weeks ago. Thank you all so much for your feedback - we almost entirely redesigned the tool and the UI based on the comments we received. We=E2=80=99re almost ready to launch the tool officially today at this URL, but wanted to show it to the list in case anyone finds any last bugs that we might have overlooked: https://www.waveform.com/tools/bufferbloat If you find a bug, please share the "Share Your Results" link with us along with what happened. We capture some debugging information on the backend, and having a share link allows us to diagnose any issues. This is really more of a passion project than anything else for us =E2=80= =93 we don=E2=80=99t anticipate we=E2=80=99ll try to commercialize it or anythi= ng like that. We're very thankful for all the work the folks on this list have done to identify and fix bufferbloat, and hope this is a useful contribution. I=E2=80=99ve personally been very frustrated by bufferbloat o= n a range of devices, and decided it might be helpful to build another bufferbloat test when the DSLReports test was down at some point last year. Our goals with this project were: * To build a second solid bufferbloat test in case DSLReports goes down a= gain. * Build a test where bufferbloat is front and center as the primary purpose of the test, rather than just a feature. * Try to explain bufferbloat and its effect on a user's connection as clearly as possible for a lay audience. A few notes: * On the backend, we=E2=80=99re using Cloudflare=E2=80=99s CDN to perform= the actual download and upload speed test. I know John Graham-Cunning has posted to this list in the past; if he or anyone from Cloudflare sees this, we=E2=80=99d love some help. Our Cloudflare Workers are being bandwidth-throttled due to having a non-enterprise grade account. We=E2=80=99ve worked around this in a kludgy way, but we=E2=80=99d love to = get it resolved. * We have lots of ideas for improvements, e.g. simultaneous upload/downloads, trying different file size chunks, time-series latency graphs, using WebRTC to test UDP traffic etc, but in the interest of getting things launched we're sticking with the current featureset. * There are a lot of browser-specific workarounds that we had to implement, and latency itself is measured in different ways on Safari/Webkit vs Chromium/Firefox due to limitations of the PerformanceTiming APIs. You may notice that latency is different on different browsers, however the actual bufferbloat (relative increase in latency) should be pretty consistent. In terms of some of the changes we made based on the feedback we receive on this list: Based on Toke=E2=80=99s feedback: https://lists.bufferbloat.net/pipermail/bloat/2020-November/015960.html https://lists.bufferbloat.net/pipermail/bloat/2020-November/015976.html * We changed the way the speed tests run to show an instantaneous speed as the test is being run. * We moved the bufferbloat grade into the main results box. * We tried really hard to get as close to saturating gigabit connections as possible. We redesigned completely the way we chunk files, added a =E2=80=9Cwarming up=E2=80=9D period, and spent quite a bit o= ptimizing our code to minimize CPU usage, as we found that was often the limiting factor to our speed test results. * We changed the shield grades altogether and went through a few different iterations of how to show the effect of bufferbloat on connectivity, and ended up with a =E2=80=9Ctable view=E2=80=9D to try to sh= ow the effect that bufferbloat specifically is having on the connection (compared to when the connection is unloaded). * We now link from the results table view to the FAQ where the conditions for each type of connection are explained. * We also changed the way we measure latency and now use the faster of either Google=E2=80=99s CDN or Cloudflare at any given location. We=E2= =80=99re also using the WebTiming APIs to get a more accurate latency number, though this does not work on some mobile browsers (e.g. iOS Safari) and as a result we show a higher latency on mobile devices. Since our test is less a test of absolute latency and more a test of relative latency with and without load, we felt this was workable. * Our jitter is now an average (was previously RMS). * The =E2=80=9Cbefore you start=E2=80=9D text was rewritten and moved abo= ve the start button. * We now spell out upload and download instead of having arrows. * We hugely reduced the number of cross-site scripts. I was a bit embarrassed by this if I=E2=80=99m honest - I spent a long time building we= b tools for the EFF, where we almost never allowed any cross-site scripts. * Our site is hosted on Shopify, and adding any features via their app store ends up adding a whole lot of gunk. But we uninstalled some apps, rewrote our template, and ended up removing a whole lot of the gunk. There=E2=80=99s still plenty of room for improvement, but it shou= ld be a lot better than before. Based on Dave Collier-Brown=E2=80=99s feedback: https://lists.bufferbloat.net/pipermail/bloat/2020-November/015966.html * We replaced the =E2=80=9Cunloaded=E2=80=9D and =E2=80=9Cloaded=E2=80=9D= language with =E2=80=9Cunloaded=E2=80=9D and then =E2=80=9Cdownload active=E2=80=9D and =E2=80=9Cupload active.=E2= =80=9D In the grade box we indicate that, for example, =E2=80=9CYour latency increased moderately unde= r load.=E2=80=9D * We tried to generally make it easier for non-techie folks to understand by emphasizing the grade and adding the table showing how bufferbloat affects some commonly-used services. * We didn=E2=80=99t really change the candle charts too much - they=E2=80= =99re mostly just to give a basic visual - we focused more on the actual meat of the results above that. Based on Sebastian Moeller=E2=80=99s feedback: https://lists.bufferbloat.net/pipermail/bloat/2020-November/015963.html * We considered doing a bidirectional saturating load, but decided to skip on implementing it for now. * It=E2=80=99s definitely something we= =E2=80=99d like to experiment with more in the future. * We added a =E2=80=9Cwarming up=E2=80=9D period as well as a =E2=80=9Cdr= aining=E2=80=9D period to help fill and empty the buffer. We haven=E2=80=99t added the option for an extended test, but have this on our list of backlog changes to make in the future. Based on Y=E2=80=99s feedback (link): https://lists.bufferbloat.net/pipermail/bloat/2020-November/015962.html * We actually ended up removing the grades, but we explained our criteria for the new table in the FAQ. Based on Greg White's feedback (shared privately): * We added an FAQ answer explaining jitter and how we measure it. We=E2=80=99d love for you all to play with the new version of the tool and send over any feedback you might have. We=E2=80=99re going to be in a featu= re freeze before launch but we'd love to get any bugs sorted out. We'll likely put this project aside after we iron out a last round of bugs and launch, and turn back to working on projects that help us pay the bills, but we definitely hope to revisit and improve the tool over time. Best, Sina, Arshan, and Sam.