From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 329EF3B29D for ; Thu, 25 Feb 2021 05:49:57 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1614250195; bh=9IbjwI3NO8GsnoQyDuMbV9aRjPSfdJbfK4FNIl5Yobc=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=F4kzo9BWb0uLfnnOJV5c6Kiq++uLmqUTsK3zZeDvKuTEXsrpqXqe4+Cw0yDe3+agp co7fTZSgnROV4bmXOMJlWHaIDRw9PPAUvRkoJ9QqAxeK+KBhTgyPZWaKeWQ4veWt64 oaTYZUzVsSTqU3vrCEURKdd2uj8jDEhn3hyvKH64= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.250.106] ([134.76.241.253]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1MPog5-1lRgvI1da7-00MpFo; Thu, 25 Feb 2021 11:49:55 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.17\)) From: Sebastian Moeller In-Reply-To: Date: Thu, 25 Feb 2021 11:49:54 +0100 Cc: bloat , sam@waveform.com Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Sina Khanifar X-Mailer: Apple Mail (2.3445.104.17) X-Provags-ID: V03:K1:7hm8H0ayCWOlrJrchemyJ7Q4MnrnoZjeaxFMK1+Z8b+szsvSH0D CIVQLYMkaFxqATyRjDHQ95WE7Wy/b88VpFUgfFyuFpXoKyXGsEBlkboLYUmqdTjnhuo9VbZ kwSHyF8hjLVT9B984Md/f/lEOoXH/NpUO3P5XRPdKlmGQwOSeoM2rCYYQUqsXhm/FB/iuP0 x/mPKVw2adHzybroM+SWA== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:LeCkz/iAP+w=:IiNYQdeRIuGVbC1EyiOOr+ j0wYCGYujg34vs8VNjcoOl+vo02L/xCmVewC17Zo9yFh3s1lS244Q4ufBH7C6irybnZ5MoMXe VzV53dD/RSZepPKKztstMoXjnAx26IJkvfQM8vqs8UgIBkxGjIXr5loWjNTm0Ke39Xy4F1zSe qTAgXn77ItoaM24RVQTJ9g5weLG4o7tntSh8gN5vN2cWXAPu15xO9q+qFGWpT7sWtuinjAUw1 SpjPLxe8QwRAjZppRFSq1hqCHB6GsAuwgDdhoy1cVw+V30zmxS3dRYBX5bB/Wm4HucxW1pG3o eWZc9Voih5m9I7JhLf7PCEQjmcemCjllNLOfHlH8kcZqxiUseRD55yxVVPUCVY2+CqJjtPzr7 4xBUeCohFg/GtfBE5/0OZmaOvS6O7U5CbPtm3+DuQDDWWdC+/UJXPZU9m0ArXQst0QwUaTy2E gVHVhMtmsAqqD4OwLeGxYQiw/lzvVoggYNjOYx7TXzZpxmJ7jws5qPvoxd/RYULyGeCTg4wTF T9M6k29WlVpqujy0FtxNJDjvS1j0KGhc2IVRc01/x+vpdfJrtPSS08clW9bLwkepreXSLUgjK 6DnMmbdYVRjP5yuXvyVodywyut1B6LPxn5o6eWUcV3P4kiSppQvmW9vM+zZT3kPrUeYYIv8Sj eGzwQzpUWDlJiauVQlrz9R//Uk99s9Y12Zvt+jWPZjwEy4xTAGrnGqnmVK56PNAeY/agAOAMq HgDu+ZzrOwSHFsCgzAqnNS+NbTQhGUZGdRSz+7wst2UNUl6duagMCxM054WjO86MG7Mq8OP+N TgR1AsdhOw/fiuVQ7DvkJOhdix3bV6AOZXRu9aJ6Z2qnefJlQ6x5QkcGeilVcIn/R8GjP0sLP wSy1MjuIHrwsyEuKhmYg== Subject: Re: [Bloat] Updated Bufferbloat Test X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2021 10:49:57 -0000 Hi Sina, great work! I took the liberty to advertise this test already for some = weeks, because even in its still evolving developing state it was/is = already producubg interesting actionable results. Thanks foe fixing the = latency numbers for (desktop) Safari. More below. > On Feb 24, 2021, at 19:22, Sina Khanifar wrote: >=20 > Hi all, >=20 > A couple of months ago my co-founder Sam posted an early beta of the > Bufferbloat test that we=E2=80=99ve been working on, and Dave also = linked to > it a couple of weeks ago. >=20 > Thank you all so much for your feedback - we almost entirely > redesigned the tool and the UI based on the comments we received. > We=E2=80=99re almost ready to launch the tool officially today at this = URL, > but wanted to show it to the list in case anyone finds any last bugs > that we might have overlooked: >=20 > https://www.waveform.com/tools/bufferbloat >=20 > If you find a bug, please share the "Share Your Results" link with us > along with what happened. We capture some debugging information on the > backend, and having a share link allows us to diagnose any issues. [SM] not a bug, more of a feature request, could you add = information on whether the test ran over IPv6 or IPv4, and which = browser/user agent was involved (nothing too deep, just desktop/mobile = and firefox/chrome/safari/brave/...) as well as the date and time of the = test? All of these can help to interpret the test results. >=20 > This is really more of a passion project than anything else for us =E2=80= =93 > we don=E2=80=99t anticipate we=E2=80=99ll try to commercialize it or = anything like > that. We're very thankful for all the work the folks on this list have > done to identify and fix bufferbloat, and hope this is a useful > contribution. I=E2=80=99ve personally been very frustrated by = bufferbloat on a > range of devices, and decided it might be helpful to build another > bufferbloat test when the DSLReports test was down at some point last > year. >=20 > Our goals with this project were: > * To build a second solid bufferbloat test in case DSLReports goes = down again. > * Build a test where bufferbloat is front and center as the primary > purpose of the test, rather than just a feature. > * Try to explain bufferbloat and its effect on a user's connection > as clearly as possible for a lay audience. >=20 > A few notes: > * On the backend, we=E2=80=99re using Cloudflare=E2=80=99s CDN to = perform the actual > download and upload speed test. I know John Graham-Cunning has posted > to this list in the past; if he or anyone from Cloudflare sees this, > we=E2=80=99d love some help. Our Cloudflare Workers are being > bandwidth-throttled due to having a non-enterprise grade account. > We=E2=80=99ve worked around this in a kludgy way, but we=E2=80=99d = love to get it > resolved. [SM] I think this was a decent decision, as it seems your tests = has less issues even filling 1Gbps links than most others. > * We have lots of ideas for improvements, e.g. simultaneous > upload/downloads, trying different file size chunks, time-series > latency graphs, using WebRTC to test UDP traffic etc, but in the > interest of getting things launched we're sticking with the current > featureset. [SM] Reasonable trade-off, and hopefully potential for pleasant = surprises in the future ;) > * There are a lot of browser-specific workarounds that we had to > implement, and latency itself is measured in different ways on > Safari/Webkit vs Chromium/Firefox due to limitations of the > PerformanceTiming APIs. You may notice that latency is different on > different browsers, however the actual bufferbloat (relative increase > in latency) should be pretty consistent. >=20 > In terms of some of the changes we made based on the feedback we > receive on this list: >=20 > Based on Toke=E2=80=99s feedback: > = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015960.html > = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015976.html > * We changed the way the speed tests run to show an instantaneous > speed as the test is being run. [SM] Great, if only so it feels comparable to "other" = speedtests. > * We moved the bufferbloat grade into the main results box. [SM] +1; that helps set the mood ;) > * We tried really hard to get as close to saturating gigabit > connections as possible. We redesigned completely the way we chunk > files, added a =E2=80=9Cwarming up=E2=80=9D period, and spent quite a = bit optimizing > our code to minimize CPU usage, as we found that was often the > limiting factor to our speed test results. > * We changed the shield grades altogether and went through a few > different iterations of how to show the effect of bufferbloat on > connectivity, and ended up with a =E2=80=9Ctable view=E2=80=9D to try = to show the > effect that bufferbloat specifically is having on the connection > (compared to when the connection is unloaded). > * We now link from the results table view to the FAQ where the > conditions for each type of connection are explained. > * We also changed the way we measure latency and now use the faster > of either Google=E2=80=99s CDN or Cloudflare at any given location. = We=E2=80=99re also > using the WebTiming APIs to get a more accurate latency number, though > this does not work on some mobile browsers (e.g. iOS Safari) and as a > result we show a higher latency on mobile devices. Since our test is > less a test of absolute latency and more a test of relative latency > with and without load, we felt this was workable. > * Our jitter is now an average (was previously RMS). > * The =E2=80=9Cbefore you start=E2=80=9D text was rewritten and moved = above the start button. > * We now spell out upload and download instead of having arrows. > * We hugely reduced the number of cross-site scripts. I was a bit > embarrassed by this if I=E2=80=99m honest - I spent a long time = building web > tools for the EFF, where we almost never allowed any cross-site > scripts. * Our site is hosted on Shopify, and adding any features via > their app store ends up adding a whole lot of gunk. But we uninstalled > some apps, rewrote our template, and ended up removing a whole lot of > the gunk. There=E2=80=99s still plenty of room for improvement, but it = should > be a lot better than before. >=20 > Based on Dave Collier-Brown=E2=80=99s feedback: > = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015966.html > * We replaced the =E2=80=9Cunloaded=E2=80=9D and =E2=80=9Cloaded=E2=80=9D= language with =E2=80=9Cunloaded=E2=80=9D > and then =E2=80=9Cdownload active=E2=80=9D and =E2=80=9Cupload = active.=E2=80=9D In the grade box we > indicate that, for example, =E2=80=9CYour latency increased moderately = under > load.=E2=80=9D > * We tried to generally make it easier for non-techie folks to > understand by emphasizing the grade and adding the table showing how > bufferbloat affects some commonly-used services. > * We didn=E2=80=99t really change the candle charts too much - = they=E2=80=99re > mostly just to give a basic visual - we focused more on the actual > meat of the results above that. >=20 > Based on Sebastian Moeller=E2=80=99s feedback: > = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015963.html > * We considered doing a bidirectional saturating load, but decided > to skip on implementing it for now. * It=E2=80=99s definitely = something we=E2=80=99d > like to experiment with more in the future. > * We added a =E2=80=9Cwarming up=E2=80=9D period as well as a = =E2=80=9Cdraining=E2=80=9D period to > help fill and empty the buffer. We haven=E2=80=99t added the option = for an > extended test, but have this on our list of backlog changes to make in > the future. >=20 > Based on Y=E2=80=99s feedback (link): > = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015962.html > * We actually ended up removing the grades, but we explained our > criteria for the new table in the FAQ. >=20 > Based on Greg White's feedback (shared privately): > * We added an FAQ answer explaining jitter and how we measure it. [SM] "There are a number of different waus of measuring and defining = jitter. For the purpose of this test, we calculate jitter by taking the = average of the deviations from the mean latency." Small typo "waus" instead of "ways". Best Regards Sebastian >=20 > We=E2=80=99d love for you all to play with the new version of the tool = and > send over any feedback you might have. We=E2=80=99re going to be in a = feature > freeze before launch but we'd love to get any bugs sorted out. We'll > likely put this project aside after we iron out a last round of bugs > and launch, and turn back to working on projects that help us pay the > bills, but we definitely hope to revisit and improve the tool over > time. >=20 > Best, >=20 > Sina, Arshan, and Sam. > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat