From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 1467D3B2A4 for ; Thu, 25 Feb 2021 15:51:06 -0500 (EST) Received: by mail-ej1-x634.google.com with SMTP id do6so11127489ejc.3 for ; Thu, 25 Feb 2021 12:51:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=waveform.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=vmPLQP74z8gMGcq3AdaEtJWxg3kg/0eBA2C+QkuAH+0=; b=GFfZ1BGxIv7HU+WIt/EiuIqp9bDLqdhROwnpZJPKz3y9EA3MQzKQvo6xQDkREdmuio Y+/sMRv6ex5cQQf9foLkG8qJXMu5cHfzxcuqLG8nfwbK1L5JWphgCmzEzbIjIcf5psQE aumcbOnThVj4AfdIDU43+X73KOSpPNV6ofmJ8NCfgMgeAnGCd1+Cdhnhed1dmW2i0INO iWqpa3Kt3Bcdp4eYVvl4Ou6y5e/vDZDLw7DF9IcwXPEeOCwAgsfDAk6TBVmMVhysoinZ F/juFl9/X/K/zysgxQSeMb//l0VMnZcwz51/ybsCfPfqiENQD+gbk6rOHNYJOpEGJZMB X2xA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=vmPLQP74z8gMGcq3AdaEtJWxg3kg/0eBA2C+QkuAH+0=; b=mpYSVU3QX1mHD063PShf3+OKJ7NZm37avP5OOy+RZ09vXTd7pevg0LZAC0+AAUlenI 8GgBp7FykWLRBxeSoLvV3ReVe3SBER7+NH8fDnsvsPiuFnX4xEsQocvezwmDpZ1vhSkO DGsASp9XKTHHlohseGExHEHR8UZ29DG7PonZM83JnUR9PkLHwKzjfeySgqwcBk1T1HEf JzmpOMlRosO8K5FEcjIMkRk7cYnUBRAQJv3GyEddOXQR/CCZAjR2/kHKgjzO5301En8v ded939/EMkeGNY7KKA5uy6lJTG+KhqrZguMvMygsVR3VXUwpIGBU2AUVp2vxAuvj5etW RDJQ== X-Gm-Message-State: AOAM531loSLIQfiWFRyHRtd2+FBYyQBG9PWVUZIi1csy/InMj4mUb3p0 CrVjsAoi4HaNFVK26O7+iqwDYuLB9/4zI52JF5jiTFpCrbVyT7j/gAcLTxKR5uy2JuMwc+zBiXc V/iOAWltSJRsW6Kobe0VNGf0StUaTjMYPnZFhCw+d0kPJ1V8mNZEukFTNxTqyi8amF71OvLZW5v l9S5zfyPw= X-Google-Smtp-Source: ABdhPJxqYKSvVC8JAqQyJY0W/2B/ruR5flkZ1f+4Wlk43HDpi6Ati2DDNr4InRme5XYseqq98X7ddA== X-Received: by 2002:a17:906:380b:: with SMTP id v11mr4656714ejc.183.1614286264765; Thu, 25 Feb 2021 12:51:04 -0800 (PST) Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com. [209.85.208.53]) by smtp.gmail.com with ESMTPSA id bz25sm3727394ejc.97.2021.02.25.12.51.04 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 25 Feb 2021 12:51:04 -0800 (PST) Received: by mail-ed1-f53.google.com with SMTP id h19so8570601edb.9 for ; Thu, 25 Feb 2021 12:51:04 -0800 (PST) X-Received: by 2002:a05:6402:17d6:: with SMTP id s22mr1820921edy.232.1614286264173; Thu, 25 Feb 2021 12:51:04 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Sina Khanifar Date: Thu, 25 Feb 2021 12:50:52 -0800 X-Gmail-Original-Message-ID: Message-ID: To: Sebastian Moeller Cc: bloat , sam@waveform.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Bloat] Updated Bufferbloat Test X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2021 20:51:06 -0000 > https://bufferbloat.waveform.workers.dev/test-results?test-id=3D6fc7dd95-= 8bfa-4b76-b141-ed423b6580a9 One quick edit, I just changed the route to these, the debug data is now available at: https://bufferbloat.waveform.com/test-results?test-id=3D6fc7dd95-8bfa-4b76-= b141-ed423b6580a9 On Thu, Feb 25, 2021 at 12:41 PM Sina Khanifar wrote: > > Hi Sebastian! > > > [SM] not a bug, more of a feature request, could you add information on= whether the test ran over IPv6 or IPv4, and which browser/user agent was i= nvolved (nothing too deep, just desktop/mobile and firefox/chrome/safari/br= ave/...) as well as the date and time of the test? All of these can help to= interpret the test results. > > We actually collect all this data, it's just a little bit hidden. If > you take the test-id from the end of the URL and put it at the end of > a URL like this: > > https://bufferbloat.waveform.workers.dev/test-results?test-id=3D6fc7dd95-= 8bfa-4b76-b141-ed423b6580a9 > > You'll get a whole bunch of extra info, including useragent, a linux > timestamp, and a bunch of other fun stuff :). We'll consider surfacing > this more at some point in the future though! > > > Small typo "waus" instead of "ways". > > Thanks for catching this! A fix is in the works :). > > On Thu, Feb 25, 2021 at 2:49 AM Sebastian Moeller wrote= : > > > > Hi Sina, > > > > great work! I took the liberty to advertise this test already for some = weeks, because even in its still evolving developing state it was/is alread= y producubg interesting actionable results. Thanks foe fixing the latency n= umbers for (desktop) Safari. More below. > > > > > > > On Feb 24, 2021, at 19:22, Sina Khanifar wrote: > > > > > > Hi all, > > > > > > A couple of months ago my co-founder Sam posted an early beta of the > > > Bufferbloat test that we=E2=80=99ve been working on, and Dave also li= nked to > > > it a couple of weeks ago. > > > > > > Thank you all so much for your feedback - we almost entirely > > > redesigned the tool and the UI based on the comments we received. > > > We=E2=80=99re almost ready to launch the tool officially today at thi= s URL, > > > but wanted to show it to the list in case anyone finds any last bugs > > > that we might have overlooked: > > > > > > https://www.waveform.com/tools/bufferbloat > > > > > > If you find a bug, please share the "Share Your Results" link with us > > > along with what happened. We capture some debugging information on th= e > > > backend, and having a share link allows us to diagnose any issues. > > > > [SM] not a bug, more of a feature request, could you add inform= ation on whether the test ran over IPv6 or IPv4, and which browser/user age= nt was involved (nothing too deep, just desktop/mobile and firefox/chrome/s= afari/brave/...) as well as the date and time of the test? All of these can= help to interpret the test results. > > > > > > > > > > This is really more of a passion project than anything else for us = =E2=80=93 > > > we don=E2=80=99t anticipate we=E2=80=99ll try to commercialize it or = anything like > > > that. We're very thankful for all the work the folks on this list hav= e > > > done to identify and fix bufferbloat, and hope this is a useful > > > contribution. I=E2=80=99ve personally been very frustrated by bufferb= loat on a > > > range of devices, and decided it might be helpful to build another > > > bufferbloat test when the DSLReports test was down at some point last > > > year. > > > > > > Our goals with this project were: > > > * To build a second solid bufferbloat test in case DSLReports goes d= own again. > > > * Build a test where bufferbloat is front and center as the primary > > > purpose of the test, rather than just a feature. > > > * Try to explain bufferbloat and its effect on a user's connection > > > as clearly as possible for a lay audience. > > > > > > A few notes: > > > * On the backend, we=E2=80=99re using Cloudflare=E2=80=99s CDN to pe= rform the actual > > > download and upload speed test. I know John Graham-Cunning has posted > > > to this list in the past; if he or anyone from Cloudflare sees this, > > > we=E2=80=99d love some help. Our Cloudflare Workers are being > > > bandwidth-throttled due to having a non-enterprise grade account. > > > We=E2=80=99ve worked around this in a kludgy way, but we=E2=80=99d lo= ve to get it > > > resolved. > > > > [SM] I think this was a decent decision, as it seems your tests= has less issues even filling 1Gbps links than most others. > > > > > > > * We have lots of ideas for improvements, e.g. simultaneous > > > upload/downloads, trying different file size chunks, time-series > > > latency graphs, using WebRTC to test UDP traffic etc, but in the > > > interest of getting things launched we're sticking with the current > > > featureset. > > > > [SM] Reasonable trade-off, and hopefully potential for pleasant= surprises in the future ;) > > > > > * There are a lot of browser-specific workarounds that we had to > > > implement, and latency itself is measured in different ways on > > > Safari/Webkit vs Chromium/Firefox due to limitations of the > > > PerformanceTiming APIs. You may notice that latency is different on > > > different browsers, however the actual bufferbloat (relative increase > > > in latency) should be pretty consistent. > > > > > > In terms of some of the changes we made based on the feedback we > > > receive on this list: > > > > > > Based on Toke=E2=80=99s feedback: > > > https://lists.bufferbloat.net/pipermail/bloat/2020-November/015960.ht= ml > > > https://lists.bufferbloat.net/pipermail/bloat/2020-November/015976.ht= ml > > > * We changed the way the speed tests run to show an instantaneous > > > speed as the test is being run. > > > > [SM] Great, if only so it feels comparable to "other" speedtest= s. > > > > > > > * We moved the bufferbloat grade into the main results box. > > > > [SM] +1; that helps set the mood ;) > > > > > * We tried really hard to get as close to saturating gigabit > > > connections as possible. We redesigned completely the way we chunk > > > files, added a =E2=80=9Cwarming up=E2=80=9D period, and spent quite a= bit optimizing > > > our code to minimize CPU usage, as we found that was often the > > > limiting factor to our speed test results. > > > * We changed the shield grades altogether and went through a few > > > different iterations of how to show the effect of bufferbloat on > > > connectivity, and ended up with a =E2=80=9Ctable view=E2=80=9D to try= to show the > > > effect that bufferbloat specifically is having on the connection > > > (compared to when the connection is unloaded). > > > * We now link from the results table view to the FAQ where the > > > conditions for each type of connection are explained. > > > * We also changed the way we measure latency and now use the faster > > > of either Google=E2=80=99s CDN or Cloudflare at any given location. W= e=E2=80=99re also > > > using the WebTiming APIs to get a more accurate latency number, thoug= h > > > this does not work on some mobile browsers (e.g. iOS Safari) and as a > > > result we show a higher latency on mobile devices. Since our test is > > > less a test of absolute latency and more a test of relative latency > > > with and without load, we felt this was workable. > > > * Our jitter is now an average (was previously RMS). > > > * The =E2=80=9Cbefore you start=E2=80=9D text was rewritten and move= d above the start button. > > > * We now spell out upload and download instead of having arrows. > > > * We hugely reduced the number of cross-site scripts. I was a bit > > > embarrassed by this if I=E2=80=99m honest - I spent a long time build= ing web > > > tools for the EFF, where we almost never allowed any cross-site > > > scripts. * Our site is hosted on Shopify, and adding any features via > > > their app store ends up adding a whole lot of gunk. But we uninstalle= d > > > some apps, rewrote our template, and ended up removing a whole lot of > > > the gunk. There=E2=80=99s still plenty of room for improvement, but i= t should > > > be a lot better than before. > > > > > > Based on Dave Collier-Brown=E2=80=99s feedback: > > > https://lists.bufferbloat.net/pipermail/bloat/2020-November/015966.ht= ml > > > * We replaced the =E2=80=9Cunloaded=E2=80=9D and =E2=80=9Cloaded=E2= =80=9D language with =E2=80=9Cunloaded=E2=80=9D > > > and then =E2=80=9Cdownload active=E2=80=9D and =E2=80=9Cupload activ= e.=E2=80=9D In the grade box we > > > indicate that, for example, =E2=80=9CYour latency increased moderatel= y under > > > load.=E2=80=9D > > > * We tried to generally make it easier for non-techie folks to > > > understand by emphasizing the grade and adding the table showing how > > > bufferbloat affects some commonly-used services. > > > * We didn=E2=80=99t really change the candle charts too much - they= =E2=80=99re > > > mostly just to give a basic visual - we focused more on the actual > > > meat of the results above that. > > > > > > Based on Sebastian Moeller=E2=80=99s feedback: > > > https://lists.bufferbloat.net/pipermail/bloat/2020-November/015963.ht= ml > > > * We considered doing a bidirectional saturating load, but decided > > > to skip on implementing it for now. * It=E2=80=99s definitely somethi= ng we=E2=80=99d > > > like to experiment with more in the future. > > > * We added a =E2=80=9Cwarming up=E2=80=9D period as well as a =E2=80= =9Cdraining=E2=80=9D period to > > > help fill and empty the buffer. We haven=E2=80=99t added the option f= or an > > > extended test, but have this on our list of backlog changes to make i= n > > > the future. > > > > > > Based on Y=E2=80=99s feedback (link): > > > https://lists.bufferbloat.net/pipermail/bloat/2020-November/015962.ht= ml > > > * We actually ended up removing the grades, but we explained our > > > criteria for the new table in the FAQ. > > > > > > Based on Greg White's feedback (shared privately): > > > * We added an FAQ answer explaining jitter and how we measure it. > > > > [SM] "There are a number of different waus of measuring and defining ji= tter. For the purpose of this test, we calculate jitter by taking the avera= ge of the deviations from the mean latency." > > > > Small typo "waus" instead of "ways". > > > > Best Regards > > Sebastian > > > > > > > > > > We=E2=80=99d love for you all to play with the new version of the too= l and > > > send over any feedback you might have. We=E2=80=99re going to be in a= feature > > > freeze before launch but we'd love to get any bugs sorted out. We'll > > > likely put this project aside after we iron out a last round of bugs > > > and launch, and turn back to working on projects that help us pay the > > > bills, but we definitely hope to revisit and improve the tool over > > > time. > > > > > > Best, > > > > > > Sina, Arshan, and Sam. > > > _______________________________________________ > > > Bloat mailing list > > > Bloat@lists.bufferbloat.net > > > https://lists.bufferbloat.net/listinfo/bloat > >