From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id DA2473B29E for ; Fri, 26 Feb 2021 14:58:15 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1614369494; bh=G3SvkKBXAyb8hCuzclWSfET55uXMqYu7q3OLi4Z6meI=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=M+BWQ9mjnZpXsohVQIL1JF5X6MAl+9qom0CCT7xHXAF+mseuGzi31p3OswCM4j8+G H4pqxaFtTNlbB3s22QxbNC94DRnsUiFJtXUsay6Liq5EBkPGUrtsdsPT+807oGQkcA gUWxGZ8JzBd0WFTJe4SrC6OHIRtwb8Z1FyEphNCo= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.42.229] ([77.6.2.187]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1M89Gt-1lBaNh0Oyh-005Hlj; Fri, 26 Feb 2021 20:58:14 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.17\)) From: Sebastian Moeller In-Reply-To: Date: Fri, 26 Feb 2021 20:58:13 +0100 Cc: bloat , sam@waveform.com Content-Transfer-Encoding: quoted-printable Message-Id: <26728229-74E3-4D40-B230-F274207AEA68@gmx.de> References: To: Sina Khanifar X-Mailer: Apple Mail (2.3445.104.17) X-Provags-ID: V03:K1:n7UyaZJI2WGAU3m0chUjKkf8TNqDByOHb1RbvJyKi9BLrE9K2bH p4+oa9UJY9D2vPDFCpJ4p0IUdPHbvnIlf/pcXPYo29v6tcy89rJUW2I5YXfMMcyp2vhdvve fSjdjfypX3VIEZM2xoCaZuy40BdNNBzGvmGXb/jXtDOGWmu4+BUZc5BZZTRGFkCqLI8IQgF LJACglERyFP7QUXL5aroQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:MKMTkEVD6p4=:V0vTyus8sH+gDKKL2K/9IE jvcfA6D3gXbpqH3w20VC8KicTXjGU/Vmonzai78+Vcu1ZcDjpWWMiI+Sahm8eaXBYeam1w6qk Tp1Q39zonIFOv6GpvrXVW+vkXakrw2DVEx6hcWbFZZhoch/8RLdcysHhOQ4ZZaii+7puJoW2G FuiPuGhNMb9fIhkKK8R7zE5IdEovyQ1DRhK46SARw7HwADqs37jh19I90Gc0JSKy9o21qSgap zuMi5upQR5GfA4tameusm5IHEPKhtqx78SHhMeLhi96Vs/entsbr2KXl2ad7Tcp9bZlsI8sO/ LbMmrOd+Qdci9vlyTERSvwZif5CgHRCHMaxjDX4y4+rNwGD0cWgFbLKljSNDpzYIRKKfbImYJ ZGVahwFOVs65TVRoQSAXRKx5JTUZxmNP3ad9uE8fxk/zbI8e1dOWkunc8vwH4fy7a4MYjJ6bR IrMOni1YUP4lcfrLbd+e11WzAMbbM/wCCkNuXuTMjIy7SLWZO7ZAkQ0OkXuMxkOSFlHRDWx0X 6XLWsuTp1SJhe6olJPNhsHUCqp+hIRt8stiZrKQ1HQYgTyebyzmRo9mme2o/n8p9mXYcrDQH5 pqRUEvPLLhQadtb1VBJCDoCmAnKA9HNE+sbI2VmwMplqbiFNcDR+C6E/Fw22asuUgnDjhbztb AphF5TBhNfDmoqK7ScvIjb7/aKXuHnyoZmVaJBNEfPnCoM0NTMko4tjEX3RGikX33/3jlPJ2K 4BoRtBf/7MepL0llwUHwQHm/+KM9IYcc6vwIjdE/TB1t+lpOZI+LEYSEXTAlfRtJWtzYBNonS VMF714aVHeagdJR4LXFPyscV9iadd9cehvcSnQ6kbUHK8wqC9HPQY3ve/5/tR3ZdjgqlGmeMp Cpp9EG6uewmxKIUgW1bg== Subject: Re: [Bloat] Updated Bufferbloat Test X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2021 19:58:16 -0000 Hi Sina, > On Feb 26, 2021, at 19:41, Sina Khanifar wrote: >=20 > Just a quick follow-up question @Sebastian @Simon: >=20 > We're thinking of implementing a "download" button that would download > the same data currently shown in the JSON view but instead as a CSV. > Would that work? Sure, I guess. Why CVS though, the data ist not really that = table like with multiple different record types / key value pairs. I = think that json looks like a good match, no? Best Regards Sebastian >=20 > Best, >=20 > Sina. >=20 > On Fri, Feb 26, 2021 at 12:23 AM Sina Khanifar = wrote: >>=20 >>> would it be an option to embed the details link into the results = page? >>=20 >>> Having more detail available but not shown by default on the main = page might keep the geeks happy and make diagnosis easier. >>=20 >> Will give this a bit of thought and see if we can make it happen! >>=20 >> On Thu, Feb 25, 2021 at 1:15 PM Sebastian Moeller = wrote: >>>=20 >>> Hi Sina, >>>=20 >>> most excellent! While I concur with Simon that "keeping it simple" = is the right approach, would it be an option to embed the details link = into the results page? >>>=20 >>> Best Regards >>> Sebastian >>>=20 >>>=20 >>>=20 >>>> On Feb 25, 2021, at 21:50, Sina Khanifar wrote: >>>>=20 >>>>> = https://bufferbloat.waveform.workers.dev/test-results?test-id=3D6fc7dd95-8= bfa-4b76-b141-ed423b6580a9 >>>>=20 >>>> One quick edit, I just changed the route to these, the debug data = is >>>> now available at: >>>>=20 >>>> = https://bufferbloat.waveform.com/test-results?test-id=3D6fc7dd95-8bfa-4b76= -b141-ed423b6580a9 >>>>=20 >>>>=20 >>>> On Thu, Feb 25, 2021 at 12:41 PM Sina Khanifar = wrote: >>>>>=20 >>>>> Hi Sebastian! >>>>>=20 >>>>>> [SM] not a bug, more of a feature request, could you add = information on whether the test ran over IPv6 or IPv4, and which = browser/user agent was involved (nothing too deep, just desktop/mobile = and firefox/chrome/safari/brave/...) as well as the date and time of the = test? All of these can help to interpret the test results. >>>>>=20 >>>>> We actually collect all this data, it's just a little bit hidden. = If >>>>> you take the test-id from the end of the URL and put it at the end = of >>>>> a URL like this: >>>>>=20 >>>>> = https://bufferbloat.waveform.workers.dev/test-results?test-id=3D6fc7dd95-8= bfa-4b76-b141-ed423b6580a9 >>>>>=20 >>>>> You'll get a whole bunch of extra info, including useragent, a = linux >>>>> timestamp, and a bunch of other fun stuff :). We'll consider = surfacing >>>>> this more at some point in the future though! >>>>>=20 >>>>>> Small typo "waus" instead of "ways". >>>>>=20 >>>>> Thanks for catching this! A fix is in the works :). >>>>>=20 >>>>> On Thu, Feb 25, 2021 at 2:49 AM Sebastian Moeller = wrote: >>>>>>=20 >>>>>> Hi Sina, >>>>>>=20 >>>>>> great work! I took the liberty to advertise this test already for = some weeks, because even in its still evolving developing state it = was/is already producubg interesting actionable results. Thanks foe = fixing the latency numbers for (desktop) Safari. More below. >>>>>>=20 >>>>>>=20 >>>>>>> On Feb 24, 2021, at 19:22, Sina Khanifar = wrote: >>>>>>>=20 >>>>>>> Hi all, >>>>>>>=20 >>>>>>> A couple of months ago my co-founder Sam posted an early beta of = the >>>>>>> Bufferbloat test that we=E2=80=99ve been working on, and Dave = also linked to >>>>>>> it a couple of weeks ago. >>>>>>>=20 >>>>>>> Thank you all so much for your feedback - we almost entirely >>>>>>> redesigned the tool and the UI based on the comments we = received. >>>>>>> We=E2=80=99re almost ready to launch the tool officially today = at this URL, >>>>>>> but wanted to show it to the list in case anyone finds any last = bugs >>>>>>> that we might have overlooked: >>>>>>>=20 >>>>>>> https://www.waveform.com/tools/bufferbloat >>>>>>>=20 >>>>>>> If you find a bug, please share the "Share Your Results" link = with us >>>>>>> along with what happened. We capture some debugging information = on the >>>>>>> backend, and having a share link allows us to diagnose any = issues. >>>>>>=20 >>>>>> [SM] not a bug, more of a feature request, could you add = information on whether the test ran over IPv6 or IPv4, and which = browser/user agent was involved (nothing too deep, just desktop/mobile = and firefox/chrome/safari/brave/...) as well as the date and time of the = test? All of these can help to interpret the test results. >>>>>>=20 >>>>>>=20 >>>>>>>=20 >>>>>>> This is really more of a passion project than anything else for = us =E2=80=93 >>>>>>> we don=E2=80=99t anticipate we=E2=80=99ll try to commercialize = it or anything like >>>>>>> that. We're very thankful for all the work the folks on this = list have >>>>>>> done to identify and fix bufferbloat, and hope this is a useful >>>>>>> contribution. I=E2=80=99ve personally been very frustrated by = bufferbloat on a >>>>>>> range of devices, and decided it might be helpful to build = another >>>>>>> bufferbloat test when the DSLReports test was down at some point = last >>>>>>> year. >>>>>>>=20 >>>>>>> Our goals with this project were: >>>>>>> * To build a second solid bufferbloat test in case DSLReports = goes down again. >>>>>>> * Build a test where bufferbloat is front and center as the = primary >>>>>>> purpose of the test, rather than just a feature. >>>>>>> * Try to explain bufferbloat and its effect on a user's = connection >>>>>>> as clearly as possible for a lay audience. >>>>>>>=20 >>>>>>> A few notes: >>>>>>> * On the backend, we=E2=80=99re using Cloudflare=E2=80=99s CDN = to perform the actual >>>>>>> download and upload speed test. I know John Graham-Cunning has = posted >>>>>>> to this list in the past; if he or anyone from Cloudflare sees = this, >>>>>>> we=E2=80=99d love some help. Our Cloudflare Workers are being >>>>>>> bandwidth-throttled due to having a non-enterprise grade = account. >>>>>>> We=E2=80=99ve worked around this in a kludgy way, but we=E2=80=99d= love to get it >>>>>>> resolved. >>>>>>=20 >>>>>> [SM] I think this was a decent decision, as it seems your = tests has less issues even filling 1Gbps links than most others. >>>>>>=20 >>>>>>=20 >>>>>>> * We have lots of ideas for improvements, e.g. simultaneous >>>>>>> upload/downloads, trying different file size chunks, time-series >>>>>>> latency graphs, using WebRTC to test UDP traffic etc, but in the >>>>>>> interest of getting things launched we're sticking with the = current >>>>>>> featureset. >>>>>>=20 >>>>>> [SM] Reasonable trade-off, and hopefully potential for = pleasant surprises in the future ;) >>>>>>=20 >>>>>>> * There are a lot of browser-specific workarounds that we had to >>>>>>> implement, and latency itself is measured in different ways on >>>>>>> Safari/Webkit vs Chromium/Firefox due to limitations of the >>>>>>> PerformanceTiming APIs. You may notice that latency is different = on >>>>>>> different browsers, however the actual bufferbloat (relative = increase >>>>>>> in latency) should be pretty consistent. >>>>>>>=20 >>>>>>> In terms of some of the changes we made based on the feedback we >>>>>>> receive on this list: >>>>>>>=20 >>>>>>> Based on Toke=E2=80=99s feedback: >>>>>>> = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015960.html >>>>>>> = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015976.html >>>>>>> * We changed the way the speed tests run to show an = instantaneous >>>>>>> speed as the test is being run. >>>>>>=20 >>>>>> [SM] Great, if only so it feels comparable to "other" = speedtests. >>>>>>=20 >>>>>>=20 >>>>>>> * We moved the bufferbloat grade into the main results box. >>>>>>=20 >>>>>> [SM] +1; that helps set the mood ;) >>>>>>=20 >>>>>>> * We tried really hard to get as close to saturating gigabit >>>>>>> connections as possible. We redesigned completely the way we = chunk >>>>>>> files, added a =E2=80=9Cwarming up=E2=80=9D period, and spent = quite a bit optimizing >>>>>>> our code to minimize CPU usage, as we found that was often the >>>>>>> limiting factor to our speed test results. >>>>>>> * We changed the shield grades altogether and went through a few >>>>>>> different iterations of how to show the effect of bufferbloat on >>>>>>> connectivity, and ended up with a =E2=80=9Ctable view=E2=80=9D = to try to show the >>>>>>> effect that bufferbloat specifically is having on the connection >>>>>>> (compared to when the connection is unloaded). >>>>>>> * We now link from the results table view to the FAQ where the >>>>>>> conditions for each type of connection are explained. >>>>>>> * We also changed the way we measure latency and now use the = faster >>>>>>> of either Google=E2=80=99s CDN or Cloudflare at any given = location. We=E2=80=99re also >>>>>>> using the WebTiming APIs to get a more accurate latency number, = though >>>>>>> this does not work on some mobile browsers (e.g. iOS Safari) and = as a >>>>>>> result we show a higher latency on mobile devices. Since our = test is >>>>>>> less a test of absolute latency and more a test of relative = latency >>>>>>> with and without load, we felt this was workable. >>>>>>> * Our jitter is now an average (was previously RMS). >>>>>>> * The =E2=80=9Cbefore you start=E2=80=9D text was rewritten and = moved above the start button. >>>>>>> * We now spell out upload and download instead of having arrows. >>>>>>> * We hugely reduced the number of cross-site scripts. I was a = bit >>>>>>> embarrassed by this if I=E2=80=99m honest - I spent a long time = building web >>>>>>> tools for the EFF, where we almost never allowed any cross-site >>>>>>> scripts. * Our site is hosted on Shopify, and adding any = features via >>>>>>> their app store ends up adding a whole lot of gunk. But we = uninstalled >>>>>>> some apps, rewrote our template, and ended up removing a whole = lot of >>>>>>> the gunk. There=E2=80=99s still plenty of room for improvement, = but it should >>>>>>> be a lot better than before. >>>>>>>=20 >>>>>>> Based on Dave Collier-Brown=E2=80=99s feedback: >>>>>>> = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015966.html >>>>>>> * We replaced the =E2=80=9Cunloaded=E2=80=9D and =E2=80=9Cloaded=E2= =80=9D language with =E2=80=9Cunloaded=E2=80=9D >>>>>>> and then =E2=80=9Cdownload active=E2=80=9D and =E2=80=9Cupload = active.=E2=80=9D In the grade box we >>>>>>> indicate that, for example, =E2=80=9CYour latency increased = moderately under >>>>>>> load.=E2=80=9D >>>>>>> * We tried to generally make it easier for non-techie folks to >>>>>>> understand by emphasizing the grade and adding the table showing = how >>>>>>> bufferbloat affects some commonly-used services. >>>>>>> * We didn=E2=80=99t really change the candle charts too much - = they=E2=80=99re >>>>>>> mostly just to give a basic visual - we focused more on the = actual >>>>>>> meat of the results above that. >>>>>>>=20 >>>>>>> Based on Sebastian Moeller=E2=80=99s feedback: >>>>>>> = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015963.html >>>>>>> * We considered doing a bidirectional saturating load, but = decided >>>>>>> to skip on implementing it for now. * It=E2=80=99s definitely = something we=E2=80=99d >>>>>>> like to experiment with more in the future. >>>>>>> * We added a =E2=80=9Cwarming up=E2=80=9D period as well as a = =E2=80=9Cdraining=E2=80=9D period to >>>>>>> help fill and empty the buffer. We haven=E2=80=99t added the = option for an >>>>>>> extended test, but have this on our list of backlog changes to = make in >>>>>>> the future. >>>>>>>=20 >>>>>>> Based on Y=E2=80=99s feedback (link): >>>>>>> = https://lists.bufferbloat.net/pipermail/bloat/2020-November/015962.html >>>>>>> * We actually ended up removing the grades, but we explained our >>>>>>> criteria for the new table in the FAQ. >>>>>>>=20 >>>>>>> Based on Greg White's feedback (shared privately): >>>>>>> * We added an FAQ answer explaining jitter and how we measure = it. >>>>>>=20 >>>>>> [SM] "There are a number of different waus of measuring and = defining jitter. For the purpose of this test, we calculate jitter by = taking the average of the deviations from the mean latency." >>>>>>=20 >>>>>> Small typo "waus" instead of "ways". >>>>>>=20 >>>>>> Best Regards >>>>>> Sebastian >>>>>>=20 >>>>>>=20 >>>>>>>=20 >>>>>>> We=E2=80=99d love for you all to play with the new version of = the tool and >>>>>>> send over any feedback you might have. We=E2=80=99re going to be = in a feature >>>>>>> freeze before launch but we'd love to get any bugs sorted out. = We'll >>>>>>> likely put this project aside after we iron out a last round of = bugs >>>>>>> and launch, and turn back to working on projects that help us = pay the >>>>>>> bills, but we definitely hope to revisit and improve the tool = over >>>>>>> time. >>>>>>>=20 >>>>>>> Best, >>>>>>>=20 >>>>>>> Sina, Arshan, and Sam. >>>>>>> _______________________________________________ >>>>>>> Bloat mailing list >>>>>>> Bloat@lists.bufferbloat.net >>>>>>> https://lists.bufferbloat.net/listinfo/bloat >>>>>>=20 >>>=20