From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 961E83B2A4 for ; Thu, 25 Feb 2021 16:15:05 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1614287704; bh=45+4QQiEYNX+hPKCgsQN4bz7+0q3eBoyKmHxSaN88hM=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=dY+8BqO9llbFxEe5Iq1fyYVP59mqFef9mToE8RuAMwpcTvD+iaIxW4g+r2TB+Fp/6 Gi8Ljzb+fbEH/+KjHNYQ2jdUB4XPFb9xqHqmyBMTAWqTEVbBc3zdkyuK0RVxOfaTJ6 AdsoIJNH+nerWAQrrWbq8uOB00yAocCkVkVY4gQo= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.42.229] ([77.8.105.209]) by mail.gmx.net (mrgmx104 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MwQXN-1m7lbu36eL-00sPq2; Thu, 25 Feb 2021 22:15:03 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.17\)) From: Sebastian Moeller In-Reply-To: Date: Thu, 25 Feb 2021 22:15:02 +0100 Cc: bloat , sam@waveform.com Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Sina Khanifar X-Mailer: Apple Mail (2.3445.104.17) X-Provags-ID: V03:K1:vKGgGcwZRCcjlWOHsjN6mdMH/PkfCGIJ/uhlasAzFH0JEwXTXAv wzAlGidlH4XGJF0Vm+h2LEWw2E7+vwwzCXAVpZJ8cm+ku4NE2vFRY++eWj3H9h43l0sDPE9 4pEKJ+E/DYOsKw07OjTPViyrc10iQ9t8jPZ44YlDjk8hs5QJ7iPk5mPavtUhBM9ViydGItg JOoAtvllOImCuTUUHQkAA== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:E8bYytJ2HF0=:KOnyPtq5CJTvs0nD53TtdJ 0sigqeeoydsADdBM5p5lagy/BC8jZujXi5ejWM8wjKpAYeaRW3Ra2Im18xrxOvSRhpMYtjyHP 73HHkso3cLM2eV68CIFIwVjA8Y416oVrfcVwqLOPqxaYtZx2AXfDfYtMOfzyeK5w5M+CWAhrT JU06rz0X8+APSOx3o3jqvMmEMsqr9GgVRlBr2N/R4WS7A03zR6a6JyPNlUl1DtecpQkwBNmr5 pNzUex0pw3bhUUEAV51QTvfhRsHD4+BJN6HQQ5+OVsOiV0X6OwiQSHfQ+SFmH2qLdavxBoxhd RNM81xjo4WXRk7hEw6fn5GPJUlgJrdTpwDCaj6XRa2iyq6/bpf/nzR/ecX7h6mh/4FW+10CRu MlwWveo4dTa3zjrj1921/dBHzdfRROUkp/qIeE9Gb5wYsHg6JdfwMSUtLOjCI1C0oYMLoQVvk u+q5Q2w6R4KmxQ+2N6le66yRyiiO0KcZFEA7ufRRr8xaJz7tWVNuabFSdECQFgiF2whu27bDE QMeL34hR347W6oxVegbwed8OZVNfZ/zPjMWGz6Vf4vp8zCLzhdkiUnvJXFSwDfMpQgl8wTlMK oxBbxhpuhgCyf+Z/k+oMj640xYFv2AbQRag5ESPHFtnuTi6IFb4YdhY7Y8jw15o91tqgBonMF AH0anFJnnMo0jd2G7zAFzv5ESeBVSOmEFUSHMFwD8VUbxow79eNRswS8l3jLjQlSiONWHPLAJ 4AOUf6K2hNoHT6hV012exJsdNw+ojBgqClNqdplFcDjn6ZmBR6mPwMCdrhGWaoA+qfHsktCgZ XzJIb20On//z9W8LCtD+XgD+Hr3CYjTUdnJFos3TFIvLAM9LcypHH1GLz8hFgHM9KAT0jw3dl r0o+m1PpD/IdtpolaFdw== Subject: Re: [Bloat] Updated Bufferbloat Test X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2021 21:15:05 -0000 Hi Sina, most excellent! While I concur with Simon that "keeping it simple" is = the right approach, would it be an option to embed the details link into = the results page? Best Regards Sebastian > On Feb 25, 2021, at 21:50, Sina Khanifar wrote: >=20 >> = https://bufferbloat.waveform.workers.dev/test-results?test-id=3D6fc7dd95-8= bfa-4b76-b141-ed423b6580a9 >=20 > One quick edit, I just changed the route to these, the debug data is > now available at: >=20 > = https://bufferbloat.waveform.com/test-results?test-id=3D6fc7dd95-8bfa-4b76= -b141-ed423b6580a9 >=20 >=20 > On Thu, Feb 25, 2021 at 12:41 PM Sina Khanifar = wrote: >>=20 >> Hi Sebastian! >>=20 >>> [SM] not a bug, more of a feature request, could you add information = on whether the test ran over IPv6 or IPv4, and which browser/user agent = was involved (nothing too deep, just desktop/mobile and = firefox/chrome/safari/brave/...) as well as the date and time of the = test? All of these can help to interpret the test results. >>=20 >> We actually collect all this data, it's just a little bit hidden. If >> you take the test-id from the end of the URL and put it at the end of >> a URL like this: >>=20 >> = https://bufferbloat.waveform.workers.dev/test-results?test-id=3D6fc7dd95-8= bfa-4b76-b141-ed423b6580a9 >>=20 >> You'll get a whole bunch of extra info, including useragent, a linux >> timestamp, and a bunch of other fun stuff :). We'll consider = surfacing >> this more at some point in the future though! >>=20 >>> Small typo "waus" instead of "ways". >>=20 >> Thanks for catching this! A fix is in the works :). >>=20 >> On Thu, Feb 25, 2021 at 2:49 AM Sebastian Moeller = wrote: >>>=20 >>> Hi Sina, >>>=20 >>> great work! I took the liberty to advertise this test already for = some weeks, because even in its still evolving developing state it = was/is already producubg interesting actionable results. Thanks foe = fixing the latency numbers for (desktop) Safari. More below. >>>=20 >>>=20 >>>> On Feb 24, 2021, at 19:22, Sina Khanifar wrote: >>>>=20 >>>> Hi all, >>>>=20 >>>> A couple of months ago my co-founder Sam posted an early beta of = the >>>> Bufferbloat test that we=E2=80=99ve been working on, and Dave also = linked to >>>> it a couple of weeks ago. >>>>=20 >>>> Thank you all so much for your feedback - we almost entirely >>>> redesigned the tool and the UI based on the comments we received. >>>> We=E2=80=99re almost ready to launch the tool officially today at = this URL, >>>> but wanted to show it to the list in case anyone finds any last = bugs >>>> that we might have overlooked: >>>>=20 >>>> https://www.waveform.com/tools/bufferbloat >>>>=20 >>>> If you find a bug, please share the "Share Your Results" link with = us >>>> along with what happened. We capture some debugging information on = the >>>> backend, and having a share link allows us to diagnose any issues. >>>=20 >>> [SM] not a bug, more of a feature request, could you add = information on whether the test ran over IPv6 or IPv4, and which = browser/user agent was involved (nothing too deep, just desktop/mobile = and firefox/chrome/safari/brave/...) as well as the date and time of the = test? All of these can help to interpret the test results. >>>=20 >>>=20 >>>>=20 >>>> This is really more of a passion project than anything else for us = =E2=80=93 >>>> we don=E2=80=99t anticipate we=E2=80=99ll try to commercialize it = or anything like >>>> that. We're very thankful for all the work the folks on this list = have >>>> done to identify and fix bufferbloat, and hope this is a useful >>>> contribution. I=E2=80=99ve personally been very frustrated by = bufferbloat on a >>>> range of devices, and decided it might be helpful to build another >>>> bufferbloat test when the DSLReports test was down at some point = last >>>> year. >>>>=20 >>>> Our goals with this project were: >>>> * To build a second solid bufferbloat test in case DSLReports goes = down again. >>>> * Build a test where bufferbloat is front and center as the primary >>>> purpose of the test, rather than just a feature. >>>> * Try to explain bufferbloat and its effect on a user's connection >>>> as clearly as possible for a lay audience. >>>>=20 >>>> A few notes: >>>> * On the backend, we=E2=80=99re using Cloudflare=E2=80=99s CDN to = perform the actual >>>> download and upload speed test. I know John Graham-Cunning has = posted >>>> to this list in the past; if he or anyone from Cloudflare sees = this, >>>> we=E2=80=99d love some help. Our Cloudflare Workers are being >>>> bandwidth-throttled due to having a non-enterprise grade account. >>>> We=E2=80=99ve worked around this in a kludgy way, but we=E2=80=99d = love to get it >>>> resolved. >>>=20 >>> [SM] I think this was a decent decision, as it seems your = tests has less issues even filling 1Gbps links than most others. >>>=20 >>>=20 >>>> * We have lots of ideas for improvements, e.g. simultaneous >>>> upload/downloads, trying different file size chunks, time-series >>>> latency graphs, using WebRTC to test UDP traffic etc, but in the >>>> interest of getting things launched we're sticking with the current >>>> featureset. >>>=20 >>> [SM] Reasonable trade-off, and hopefully potential for = pleasant surprises in the future ;) >>>=20 >>>> * There are a lot of browser-specific workarounds that we had to >>>> implement, and latency itself is measured in different ways on >>>> Safari/Webkit vs Chromium/Firefox due to limitations of the >>>> PerformanceTiming APIs. You may notice that latency is different on >>>> different browsers, however the actual bufferbloat (relative = increase >>>> in latency) should be pretty consistent. >>>>=20 >>>> In terms of some of the changes we made based on the feedback we >>>> receive on this list: >>>>=20 >>>> Based on Toke=E2=80=99s feedback: >>>> = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015960.html >>>> = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015976.html >>>> * We changed the way the speed tests run to show an instantaneous >>>> speed as the test is being run. >>>=20 >>> [SM] Great, if only so it feels comparable to "other" = speedtests. >>>=20 >>>=20 >>>> * We moved the bufferbloat grade into the main results box. >>>=20 >>> [SM] +1; that helps set the mood ;) >>>=20 >>>> * We tried really hard to get as close to saturating gigabit >>>> connections as possible. We redesigned completely the way we chunk >>>> files, added a =E2=80=9Cwarming up=E2=80=9D period, and spent quite = a bit optimizing >>>> our code to minimize CPU usage, as we found that was often the >>>> limiting factor to our speed test results. >>>> * We changed the shield grades altogether and went through a few >>>> different iterations of how to show the effect of bufferbloat on >>>> connectivity, and ended up with a =E2=80=9Ctable view=E2=80=9D to = try to show the >>>> effect that bufferbloat specifically is having on the connection >>>> (compared to when the connection is unloaded). >>>> * We now link from the results table view to the FAQ where the >>>> conditions for each type of connection are explained. >>>> * We also changed the way we measure latency and now use the faster >>>> of either Google=E2=80=99s CDN or Cloudflare at any given location. = We=E2=80=99re also >>>> using the WebTiming APIs to get a more accurate latency number, = though >>>> this does not work on some mobile browsers (e.g. iOS Safari) and as = a >>>> result we show a higher latency on mobile devices. Since our test = is >>>> less a test of absolute latency and more a test of relative latency >>>> with and without load, we felt this was workable. >>>> * Our jitter is now an average (was previously RMS). >>>> * The =E2=80=9Cbefore you start=E2=80=9D text was rewritten and = moved above the start button. >>>> * We now spell out upload and download instead of having arrows. >>>> * We hugely reduced the number of cross-site scripts. I was a bit >>>> embarrassed by this if I=E2=80=99m honest - I spent a long time = building web >>>> tools for the EFF, where we almost never allowed any cross-site >>>> scripts. * Our site is hosted on Shopify, and adding any features = via >>>> their app store ends up adding a whole lot of gunk. But we = uninstalled >>>> some apps, rewrote our template, and ended up removing a whole lot = of >>>> the gunk. There=E2=80=99s still plenty of room for improvement, but = it should >>>> be a lot better than before. >>>>=20 >>>> Based on Dave Collier-Brown=E2=80=99s feedback: >>>> = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015966.html >>>> * We replaced the =E2=80=9Cunloaded=E2=80=9D and =E2=80=9Cloaded=E2=80= =9D language with =E2=80=9Cunloaded=E2=80=9D >>>> and then =E2=80=9Cdownload active=E2=80=9D and =E2=80=9Cupload = active.=E2=80=9D In the grade box we >>>> indicate that, for example, =E2=80=9CYour latency increased = moderately under >>>> load.=E2=80=9D >>>> * We tried to generally make it easier for non-techie folks to >>>> understand by emphasizing the grade and adding the table showing = how >>>> bufferbloat affects some commonly-used services. >>>> * We didn=E2=80=99t really change the candle charts too much - = they=E2=80=99re >>>> mostly just to give a basic visual - we focused more on the actual >>>> meat of the results above that. >>>>=20 >>>> Based on Sebastian Moeller=E2=80=99s feedback: >>>> = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015963.html >>>> * We considered doing a bidirectional saturating load, but decided >>>> to skip on implementing it for now. * It=E2=80=99s definitely = something we=E2=80=99d >>>> like to experiment with more in the future. >>>> * We added a =E2=80=9Cwarming up=E2=80=9D period as well as a = =E2=80=9Cdraining=E2=80=9D period to >>>> help fill and empty the buffer. We haven=E2=80=99t added the option = for an >>>> extended test, but have this on our list of backlog changes to make = in >>>> the future. >>>>=20 >>>> Based on Y=E2=80=99s feedback (link): >>>> = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015962.html >>>> * We actually ended up removing the grades, but we explained our >>>> criteria for the new table in the FAQ. >>>>=20 >>>> Based on Greg White's feedback (shared privately): >>>> * We added an FAQ answer explaining jitter and how we measure it. >>>=20 >>> [SM] "There are a number of different waus of measuring and defining = jitter. For the purpose of this test, we calculate jitter by taking the = average of the deviations from the mean latency." >>>=20 >>> Small typo "waus" instead of "ways". >>>=20 >>> Best Regards >>> Sebastian >>>=20 >>>=20 >>>>=20 >>>> We=E2=80=99d love for you all to play with the new version of the = tool and >>>> send over any feedback you might have. We=E2=80=99re going to be in = a feature >>>> freeze before launch but we'd love to get any bugs sorted out. = We'll >>>> likely put this project aside after we iron out a last round of = bugs >>>> and launch, and turn back to working on projects that help us pay = the >>>> bills, but we definitely hope to revisit and improve the tool over >>>> time. >>>>=20 >>>> Best, >>>>=20 >>>> Sina, Arshan, and Sam. >>>> _______________________________________________ >>>> Bloat mailing list >>>> Bloat@lists.bufferbloat.net >>>> https://lists.bufferbloat.net/listinfo/bloat >>>=20