From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 8EB463B2A4 for ; Fri, 26 Feb 2021 03:24:08 -0500 (EST) Received: by mail-ed1-x52e.google.com with SMTP id c23so3457044edr.13 for ; Fri, 26 Feb 2021 00:24:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=waveform.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=RU3wQyqS+zUr9I08RTfp2abfKfLDQkpxjaPEsOdXvGk=; b=hYKTW+y988QVdt5JO8MN2JxoAyPwqcqf9div40xAI/wvgNae3qQKzpeFOowLamHUj9 QdZ91I211xvbnPgc/YIbjYgzzUTMd86gNncMzqRiiiFrhj/s5dTi5/7oJJxT6vpZWfaS FJmxPjEYLpenCjXyyzRiRWkjcLy1uo06QKUc6kb0k/BxcuTgYpa6isNRusplGh8pKyNw idPbrLh1h3GAaw/ZbmZl4FMb28AmvZH9/nrHpm36EixCzmRpPDPofPTgsDqreeICS3Fi GAn1lkeo2ExTwqWlutePQHdyWM5mGlM2cg+ufdK0odiCYw3Iz8WY543meI/AdgUm9ePU eUWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=RU3wQyqS+zUr9I08RTfp2abfKfLDQkpxjaPEsOdXvGk=; b=Up0VCMQtF6lFvG2aF/NelrLKPax5x7q8tH9zXJsoI9bQcM1/MYc/pYRvuSCSzn2UVR tOUCzzAid+AjQDViBh7QcdAMSB3ZS0w8FxDCpMOiybhbdlZgyEvrvFkEVpSOWAy9u+hN Uqz573cs6AHDefOoU1KIr7vK/8mM/V/FaNb6klNh7mJwzy0fQIgks5JG/R25QymrF4Wc aSTZWfBXf6ROJNGq0LALvz45d6CpwBH9SntdKVMVKJb6ZQj6k7Kz6/kcCPSatgX39+Ig k/Y5Rzja9iLaTQZLcdL8UFUlyPXnRErN31NUKeNQqiDJ80nfTIVlyeeTPZfoKVOPX9vn +ayA== X-Gm-Message-State: AOAM5332cl6kyutMJHxODcgXl6+f3k4xbwLWCZhl732xgprOs+l8EKEq 1L0/tQANjrel8sj78bOoZwGh7LMs0XvfijBGvbHog/LhEF5hlIEzXUGAa+TQa3REqD9/jygNc9J e9UtL0o0AONLwtj+xq9jOP6XIi4eeRfwFzsUH5PbwDEPivcPpExX2u7xlOiku9JymxO6X9NJW4A 3gb3d46sg= X-Google-Smtp-Source: ABdhPJzS3QcmTqhzSEcSeQRYByg/q7m8/yNz4bgXToNffqdGDW3gdmItpMrti7qaYkRx0qJXhkIixg== X-Received: by 2002:a05:6402:3553:: with SMTP id f19mr2031508edd.271.1614327847278; Fri, 26 Feb 2021 00:24:07 -0800 (PST) Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com. [209.85.218.42]) by smtp.gmail.com with ESMTPSA id q1sm4665664ejt.65.2021.02.26.00.24.06 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 26 Feb 2021 00:24:06 -0800 (PST) Received: by mail-ej1-f42.google.com with SMTP id do6so13340720ejc.3 for ; Fri, 26 Feb 2021 00:24:06 -0800 (PST) X-Received: by 2002:a17:906:2898:: with SMTP id o24mr2017775ejd.215.1614327846566; Fri, 26 Feb 2021 00:24:06 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Sina Khanifar Date: Fri, 26 Feb 2021 00:23:55 -0800 X-Gmail-Original-Message-ID: Message-ID: To: Sebastian Moeller Cc: bloat , sam@waveform.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Bloat] Updated Bufferbloat Test X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2021 08:24:08 -0000 > would it be an option to embed the details link into the results page? > Having more detail available but not shown by default on the main page mi= ght keep the geeks happy and make diagnosis easier. Will give this a bit of thought and see if we can make it happen! On Thu, Feb 25, 2021 at 1:15 PM Sebastian Moeller wrote: > > Hi Sina, > > most excellent! While I concur with Simon that "keeping it simple" is the= right approach, would it be an option to embed the details link into the r= esults page? > > Best Regards > Sebastian > > > > > On Feb 25, 2021, at 21:50, Sina Khanifar wrote: > > > >> https://bufferbloat.waveform.workers.dev/test-results?test-id=3D6fc7dd= 95-8bfa-4b76-b141-ed423b6580a9 > > > > One quick edit, I just changed the route to these, the debug data is > > now available at: > > > > https://bufferbloat.waveform.com/test-results?test-id=3D6fc7dd95-8bfa-4= b76-b141-ed423b6580a9 > > > > > > On Thu, Feb 25, 2021 at 12:41 PM Sina Khanifar wrot= e: > >> > >> Hi Sebastian! > >> > >>> [SM] not a bug, more of a feature request, could you add information = on whether the test ran over IPv6 or IPv4, and which browser/user agent was= involved (nothing too deep, just desktop/mobile and firefox/chrome/safari/= brave/...) as well as the date and time of the test? All of these can help = to interpret the test results. > >> > >> We actually collect all this data, it's just a little bit hidden. If > >> you take the test-id from the end of the URL and put it at the end of > >> a URL like this: > >> > >> https://bufferbloat.waveform.workers.dev/test-results?test-id=3D6fc7dd= 95-8bfa-4b76-b141-ed423b6580a9 > >> > >> You'll get a whole bunch of extra info, including useragent, a linux > >> timestamp, and a bunch of other fun stuff :). We'll consider surfacing > >> this more at some point in the future though! > >> > >>> Small typo "waus" instead of "ways". > >> > >> Thanks for catching this! A fix is in the works :). > >> > >> On Thu, Feb 25, 2021 at 2:49 AM Sebastian Moeller wr= ote: > >>> > >>> Hi Sina, > >>> > >>> great work! I took the liberty to advertise this test already for som= e weeks, because even in its still evolving developing state it was/is alre= ady producubg interesting actionable results. Thanks foe fixing the latency= numbers for (desktop) Safari. More below. > >>> > >>> > >>>> On Feb 24, 2021, at 19:22, Sina Khanifar wrote: > >>>> > >>>> Hi all, > >>>> > >>>> A couple of months ago my co-founder Sam posted an early beta of the > >>>> Bufferbloat test that we=E2=80=99ve been working on, and Dave also l= inked to > >>>> it a couple of weeks ago. > >>>> > >>>> Thank you all so much for your feedback - we almost entirely > >>>> redesigned the tool and the UI based on the comments we received. > >>>> We=E2=80=99re almost ready to launch the tool officially today at th= is URL, > >>>> but wanted to show it to the list in case anyone finds any last bugs > >>>> that we might have overlooked: > >>>> > >>>> https://www.waveform.com/tools/bufferbloat > >>>> > >>>> If you find a bug, please share the "Share Your Results" link with u= s > >>>> along with what happened. We capture some debugging information on t= he > >>>> backend, and having a share link allows us to diagnose any issues. > >>> > >>> [SM] not a bug, more of a feature request, could you add infor= mation on whether the test ran over IPv6 or IPv4, and which browser/user ag= ent was involved (nothing too deep, just desktop/mobile and firefox/chrome/= safari/brave/...) as well as the date and time of the test? All of these ca= n help to interpret the test results. > >>> > >>> > >>>> > >>>> This is really more of a passion project than anything else for us = =E2=80=93 > >>>> we don=E2=80=99t anticipate we=E2=80=99ll try to commercialize it or= anything like > >>>> that. We're very thankful for all the work the folks on this list ha= ve > >>>> done to identify and fix bufferbloat, and hope this is a useful > >>>> contribution. I=E2=80=99ve personally been very frustrated by buffer= bloat on a > >>>> range of devices, and decided it might be helpful to build another > >>>> bufferbloat test when the DSLReports test was down at some point las= t > >>>> year. > >>>> > >>>> Our goals with this project were: > >>>> * To build a second solid bufferbloat test in case DSLReports goes d= own again. > >>>> * Build a test where bufferbloat is front and center as the primary > >>>> purpose of the test, rather than just a feature. > >>>> * Try to explain bufferbloat and its effect on a user's connection > >>>> as clearly as possible for a lay audience. > >>>> > >>>> A few notes: > >>>> * On the backend, we=E2=80=99re using Cloudflare=E2=80=99s CDN to pe= rform the actual > >>>> download and upload speed test. I know John Graham-Cunning has poste= d > >>>> to this list in the past; if he or anyone from Cloudflare sees this, > >>>> we=E2=80=99d love some help. Our Cloudflare Workers are being > >>>> bandwidth-throttled due to having a non-enterprise grade account. > >>>> We=E2=80=99ve worked around this in a kludgy way, but we=E2=80=99d l= ove to get it > >>>> resolved. > >>> > >>> [SM] I think this was a decent decision, as it seems your test= s has less issues even filling 1Gbps links than most others. > >>> > >>> > >>>> * We have lots of ideas for improvements, e.g. simultaneous > >>>> upload/downloads, trying different file size chunks, time-series > >>>> latency graphs, using WebRTC to test UDP traffic etc, but in the > >>>> interest of getting things launched we're sticking with the current > >>>> featureset. > >>> > >>> [SM] Reasonable trade-off, and hopefully potential for pleasan= t surprises in the future ;) > >>> > >>>> * There are a lot of browser-specific workarounds that we had to > >>>> implement, and latency itself is measured in different ways on > >>>> Safari/Webkit vs Chromium/Firefox due to limitations of the > >>>> PerformanceTiming APIs. You may notice that latency is different on > >>>> different browsers, however the actual bufferbloat (relative increas= e > >>>> in latency) should be pretty consistent. > >>>> > >>>> In terms of some of the changes we made based on the feedback we > >>>> receive on this list: > >>>> > >>>> Based on Toke=E2=80=99s feedback: > >>>> https://lists.bufferbloat.net/pipermail/bloat/2020-November/015960.h= tml > >>>> https://lists.bufferbloat.net/pipermail/bloat/2020-November/015976.h= tml > >>>> * We changed the way the speed tests run to show an instantaneous > >>>> speed as the test is being run. > >>> > >>> [SM] Great, if only so it feels comparable to "other" speedtes= ts. > >>> > >>> > >>>> * We moved the bufferbloat grade into the main results box. > >>> > >>> [SM] +1; that helps set the mood ;) > >>> > >>>> * We tried really hard to get as close to saturating gigabit > >>>> connections as possible. We redesigned completely the way we chunk > >>>> files, added a =E2=80=9Cwarming up=E2=80=9D period, and spent quite = a bit optimizing > >>>> our code to minimize CPU usage, as we found that was often the > >>>> limiting factor to our speed test results. > >>>> * We changed the shield grades altogether and went through a few > >>>> different iterations of how to show the effect of bufferbloat on > >>>> connectivity, and ended up with a =E2=80=9Ctable view=E2=80=9D to tr= y to show the > >>>> effect that bufferbloat specifically is having on the connection > >>>> (compared to when the connection is unloaded). > >>>> * We now link from the results table view to the FAQ where the > >>>> conditions for each type of connection are explained. > >>>> * We also changed the way we measure latency and now use the faster > >>>> of either Google=E2=80=99s CDN or Cloudflare at any given location. = We=E2=80=99re also > >>>> using the WebTiming APIs to get a more accurate latency number, thou= gh > >>>> this does not work on some mobile browsers (e.g. iOS Safari) and as = a > >>>> result we show a higher latency on mobile devices. Since our test is > >>>> less a test of absolute latency and more a test of relative latency > >>>> with and without load, we felt this was workable. > >>>> * Our jitter is now an average (was previously RMS). > >>>> * The =E2=80=9Cbefore you start=E2=80=9D text was rewritten and move= d above the start button. > >>>> * We now spell out upload and download instead of having arrows. > >>>> * We hugely reduced the number of cross-site scripts. I was a bit > >>>> embarrassed by this if I=E2=80=99m honest - I spent a long time buil= ding web > >>>> tools for the EFF, where we almost never allowed any cross-site > >>>> scripts. * Our site is hosted on Shopify, and adding any features vi= a > >>>> their app store ends up adding a whole lot of gunk. But we uninstall= ed > >>>> some apps, rewrote our template, and ended up removing a whole lot o= f > >>>> the gunk. There=E2=80=99s still plenty of room for improvement, but = it should > >>>> be a lot better than before. > >>>> > >>>> Based on Dave Collier-Brown=E2=80=99s feedback: > >>>> https://lists.bufferbloat.net/pipermail/bloat/2020-November/015966.h= tml > >>>> * We replaced the =E2=80=9Cunloaded=E2=80=9D and =E2=80=9Cloaded=E2= =80=9D language with =E2=80=9Cunloaded=E2=80=9D > >>>> and then =E2=80=9Cdownload active=E2=80=9D and =E2=80=9Cupload acti= ve.=E2=80=9D In the grade box we > >>>> indicate that, for example, =E2=80=9CYour latency increased moderate= ly under > >>>> load.=E2=80=9D > >>>> * We tried to generally make it easier for non-techie folks to > >>>> understand by emphasizing the grade and adding the table showing how > >>>> bufferbloat affects some commonly-used services. > >>>> * We didn=E2=80=99t really change the candle charts too much - they= =E2=80=99re > >>>> mostly just to give a basic visual - we focused more on the actual > >>>> meat of the results above that. > >>>> > >>>> Based on Sebastian Moeller=E2=80=99s feedback: > >>>> https://lists.bufferbloat.net/pipermail/bloat/2020-November/015963.h= tml > >>>> * We considered doing a bidirectional saturating load, but decided > >>>> to skip on implementing it for now. * It=E2=80=99s definitely someth= ing we=E2=80=99d > >>>> like to experiment with more in the future. > >>>> * We added a =E2=80=9Cwarming up=E2=80=9D period as well as a =E2=80= =9Cdraining=E2=80=9D period to > >>>> help fill and empty the buffer. We haven=E2=80=99t added the option = for an > >>>> extended test, but have this on our list of backlog changes to make = in > >>>> the future. > >>>> > >>>> Based on Y=E2=80=99s feedback (link): > >>>> https://lists.bufferbloat.net/pipermail/bloat/2020-November/015962.h= tml > >>>> * We actually ended up removing the grades, but we explained our > >>>> criteria for the new table in the FAQ. > >>>> > >>>> Based on Greg White's feedback (shared privately): > >>>> * We added an FAQ answer explaining jitter and how we measure it. > >>> > >>> [SM] "There are a number of different waus of measuring and defining = jitter. For the purpose of this test, we calculate jitter by taking the ave= rage of the deviations from the mean latency." > >>> > >>> Small typo "waus" instead of "ways". > >>> > >>> Best Regards > >>> Sebastian > >>> > >>> > >>>> > >>>> We=E2=80=99d love for you all to play with the new version of the to= ol and > >>>> send over any feedback you might have. We=E2=80=99re going to be in = a feature > >>>> freeze before launch but we'd love to get any bugs sorted out. We'll > >>>> likely put this project aside after we iron out a last round of bugs > >>>> and launch, and turn back to working on projects that help us pay th= e > >>>> bills, but we definitely hope to revisit and improve the tool over > >>>> time. > >>>> > >>>> Best, > >>>> > >>>> Sina, Arshan, and Sam. > >>>> _______________________________________________ > >>>> Bloat mailing list > >>>> Bloat@lists.bufferbloat.net > >>>> https://lists.bufferbloat.net/listinfo/bloat > >>> >