From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ot1-x331.google.com (mail-ot1-x331.google.com [IPv6:2607:f8b0:4864:20::331]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 92E513B29D for ; Thu, 25 Feb 2021 01:21:24 -0500 (EST) Received: by mail-ot1-x331.google.com with SMTP id b16so4729530otq.1 for ; Wed, 24 Feb 2021 22:21:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=FzI8SAGdpUHr0vDhtBxysNQeTmvnSLFDKQO9m87ayuk=; b=txhZzP3YE6/iheR+4n4Dl3/WIUnprYu3mzUO2IdfIasZyahTOqFBMYOUS0VcbCrEES oF8B9NCRMkaM5AKT2YdHD2gXn6ugHNnPNGNjpRXsFxbU8Qjrv8oX0tCjpqQBrx0m7UH8 EPO3bex130MPa5KmUURTP0RZkvQuTmK6/H0Q6Nj9MiYAQIY2J8v3d8hqRSLFne3U3Lie bGMllEmprlqyyVakdnGSnC0lZHxx7f61vqr+6MBtcSLVULXGUAUI0ZB22tMS+cFN3JM7 RtxxPfd4acgSVG4J9vEE0itkIeRARFMn3gBQQsDHvhv9yTh80/7kMFdtnMZEJr8nnSEG J9bA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=FzI8SAGdpUHr0vDhtBxysNQeTmvnSLFDKQO9m87ayuk=; b=J9CpawoZAMtdWGUa1hju7S8kaFDvKPZK3SIlrA2AiDzvnRJVGowFtpaTdB8CiZxI6I f9Y8frl1i6OnOPbL9ar44ajXGxz3E/dkPSkho5B6ZnmCRRrOIrekZTql/LYDM5MCDTZC 3LFBPnc93mog+Z88kzs2mb6omxl7JlNXDV6ohS+w53PQ75QciuYbZYkX3XgH0fVdORTk +Nm0GHmU6OcvZVl9aMHio7SMh3ft6bwH8OuQOoBKX4klJ9QGpB4im7pxDehoKz0QRGV9 c6x63r+OKNQYV7Q7TDkWmkNqkYOUOFOhr3KFo3gd+T0GRqWouRUhpkDlgJzuoaoMRpcI /CPQ== X-Gm-Message-State: AOAM530Qm+knaT2sx0KgLA0cauDAK4VU8Qo/ZZa1p1c/Gdd/UxodjWm9 ATDqZp6Fq/KC+385OpFenyAlLZWPWxs5F/TsN7w= X-Google-Smtp-Source: ABdhPJwKvM4xm+iLNRTdfkfW4tg5ksGQaNOMI0kaTHluUzpy9Y2L7VuHKeucrbTYmt0qXP3ZFSvNdX2TCUq/OgJIPYg= X-Received: by 2002:a05:6830:3152:: with SMTP id c18mr1099403ots.191.1614234083632; Wed, 24 Feb 2021 22:21:23 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Sina Khanifar Date: Wed, 24 Feb 2021 22:21:12 -0800 Message-ID: To: davecb@spamcop.net, dave.collier-brown@indexexchange.com Cc: bloat Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Thu, 25 Feb 2021 05:32:39 -0500 Subject: Re: [Bloat] Updated Bufferbloat Test X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2021 06:21:24 -0000 Hmm, I'm not quite sure what you mean here - not all cable modems are equal, surely? Some cable modems will run at 42 Gsps but others will run at 0.1 Gsps. I'm not sure a comparison to a global average would be helpful? I think our "grading" system is meant to give at least some indication of "how good am I versus what's normal?" - if you're getting anything less than an A, there's clearly room for improvement. On Wed, Feb 24, 2021 at 5:16 PM David Collier-Brown w= rote: > > I like it, but the next thing I wonder of is "how good am I versus > what's normal?" > > Let's say cable modems run at 42 Giga-somethings per second and I have a > cable modem. If I get 40 Gsps, or 95%, is that good or bad? > > Doing that as a little horizaontal graph might be a good approach, so > you can see if you land in the range around 42 up, 2 down, better or > much worse... > > --dave > > > > On 2021-02-24 1:22 p.m., Sina Khanifar wrote: > > Hi all, > > > > A couple of months ago my co-founder Sam posted an early beta of the > > Bufferbloat test that we=E2=80=99ve been working on, and Dave also link= ed to > > it a couple of weeks ago. > > > > Thank you all so much for your feedback - we almost entirely > > redesigned the tool and the UI based on the comments we received. > > We=E2=80=99re almost ready to launch the tool officially today at this = URL, > > but wanted to show it to the list in case anyone finds any last bugs > > that we might have overlooked: > > > > https://www.waveform.com/tools/bufferbloat > > > > If you find a bug, please share the "Share Your Results" link with us > > along with what happened. We capture some debugging information on the > > backend, and having a share link allows us to diagnose any issues. > > > > This is really more of a passion project than anything else for us =E2= =80=93 > > we don=E2=80=99t anticipate we=E2=80=99ll try to commercialize it or an= ything like > > that. We're very thankful for all the work the folks on this list have > > done to identify and fix bufferbloat, and hope this is a useful > > contribution. I=E2=80=99ve personally been very frustrated by bufferblo= at on a > > range of devices, and decided it might be helpful to build another > > bufferbloat test when the DSLReports test was down at some point last > > year. > > > > Our goals with this project were: > > * To build a second solid bufferbloat test in case DSLReports goes > > down again. > > * Build a test where bufferbloat is front and center as the primary > > purpose of the test, rather than just a feature. > > * Try to explain bufferbloat and its effect on a user's connection > > as clearly as possible for a lay audience. > > > > A few notes: > > * On the backend, we=E2=80=99re using Cloudflare=E2=80=99s CDN to perfo= rm the actual > > download and upload speed test. I know John Graham-Cunning has posted > > to this list in the past; if he or anyone from Cloudflare sees this, > > we=E2=80=99d love some help. Our Cloudflare Workers are being > > bandwidth-throttled due to having a non-enterprise grade account. > > We=E2=80=99ve worked around this in a kludgy way, but we=E2=80=99d love= to get it > > resolved. > > * We have lots of ideas for improvements, e.g. simultaneous > > upload/downloads, trying different file size chunks, time-series > > latency graphs, using WebRTC to test UDP traffic etc, but in the > > interest of getting things launched we're sticking with the current > > featureset. > > * There are a lot of browser-specific workarounds that we had to > > implement, and latency itself is measured in different ways on > > Safari/Webkit vs Chromium/Firefox due to limitations of the > > PerformanceTiming APIs. You may notice that latency is different on > > different browsers, however the actual bufferbloat (relative increase > > in latency) should be pretty consistent. > > > > In terms of some of the changes we made based on the feedback we > > receive on this list: > > > > Based on Toke=E2=80=99s feedback: > > https://lists.bufferbloat.net/pipermail/bloat/2020-November/015960.html > > https://lists.bufferbloat.net/pipermail/bloat/2020-November/015976.html > > * We changed the way the speed tests run to show an instantaneous > > speed as the test is being run. > > * We moved the bufferbloat grade into the main results box. > > * We tried really hard to get as close to saturating gigabit > > connections as possible. We redesigned completely the way we chunk > > files, added a =E2=80=9Cwarming up=E2=80=9D period, and spent quite a b= it optimizing > > our code to minimize CPU usage, as we found that was often the > > limiting factor to our speed test results. > > * We changed the shield grades altogether and went through a few > > different iterations of how to show the effect of bufferbloat on > > connectivity, and ended up with a =E2=80=9Ctable view=E2=80=9D to try t= o show the > > effect that bufferbloat specifically is having on the connection > > (compared to when the connection is unloaded). > > * We now link from the results table view to the FAQ where the > > conditions for each type of connection are explained. > > * We also changed the way we measure latency and now use the faster > > of either Google=E2=80=99s CDN or Cloudflare at any given location. We= =E2=80=99re also > > using the WebTiming APIs to get a more accurate latency number, though > > this does not work on some mobile browsers (e.g. iOS Safari) and as a > > result we show a higher latency on mobile devices. Since our test is > > less a test of absolute latency and more a test of relative latency > > with and without load, we felt this was workable. > > * Our jitter is now an average (was previously RMS). > > * The =E2=80=9Cbefore you start=E2=80=9D text was rewritten and moved a= bove the start > > button. > > * We now spell out upload and download instead of having arrows. > > * We hugely reduced the number of cross-site scripts. I was a bit > > embarrassed by this if I=E2=80=99m honest - I spent a long time buildin= g web > > tools for the EFF, where we almost never allowed any cross-site > > scripts. * Our site is hosted on Shopify, and adding any features via > > their app store ends up adding a whole lot of gunk. But we uninstalled > > some apps, rewrote our template, and ended up removing a whole lot of > > the gunk. There=E2=80=99s still plenty of room for improvement, but it = should > > be a lot better than before. > > > > Based on Dave Collier-Brown=E2=80=99s feedback: > > https://lists.bufferbloat.net/pipermail/bloat/2020-November/015966.html > > * We replaced the =E2=80=9Cunloaded=E2=80=9D and =E2=80=9Cloaded=E2=80= =9D language with =E2=80=9Cunloaded=E2=80=9D > > and then =E2=80=9Cdownload active=E2=80=9D and =E2=80=9Cupload active.= =E2=80=9D In the grade box we > > indicate that, for example, =E2=80=9CYour latency increased moderately = under > > load.=E2=80=9D > > * We tried to generally make it easier for non-techie folks to > > understand by emphasizing the grade and adding the table showing how > > bufferbloat affects some commonly-used services. > > * We didn=E2=80=99t really change the candle charts too much - they=E2= =80=99re > > mostly just to give a basic visual - we focused more on the actual > > meat of the results above that. > > > > Based on Sebastian Moeller=E2=80=99s feedback: > > https://lists.bufferbloat.net/pipermail/bloat/2020-November/015963.html > > * We considered doing a bidirectional saturating load, but decided > > to skip on implementing it for now. * It=E2=80=99s definitely something= we=E2=80=99d > > like to experiment with more in the future. > > * We added a =E2=80=9Cwarming up=E2=80=9D period as well as a =E2=80=9C= draining=E2=80=9D period to > > help fill and empty the buffer. We haven=E2=80=99t added the option for= an > > extended test, but have this on our list of backlog changes to make in > > the future. > > > > Based on Y=E2=80=99s feedback (link): > > https://lists.bufferbloat.net/pipermail/bloat/2020-November/015962.html > > * We actually ended up removing the grades, but we explained our > > criteria for the new table in the FAQ. > > > > Based on Greg White's feedback (shared privately): > > * We added an FAQ answer explaining jitter and how we measure it. > > > > We=E2=80=99d love for you all to play with the new version of the tool = and > > send over any feedback you might have. We=E2=80=99re going to be in a f= eature > > freeze before launch but we'd love to get any bugs sorted out. We'll > > likely put this project aside after we iron out a last round of bugs > > and launch, and turn back to working on projects that help us pay the > > bills, but we definitely hope to revisit and improve the tool over > > time. > > > > Best, > > > > Sina, Arshan, and Sam. > > _______________________________________________ > > Bloat mailing list > > Bloat@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/bloat > > -- > David Collier-Brown, | Always do right. This will gratify > System Programmer and Author | some people and astonish the rest > davecb@spamcop.net | -- Mark Twain > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat