From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 0633621F5BC for ; Fri, 25 Jul 2014 07:17:48 -0700 (PDT) Received: from hms-beagle.lan ([134.2.89.70]) by mail.gmx.com (mrgmx102) with ESMTPSA (Nemesis) id 0LpPg1-1WYboJ3xqf-00f7Hg; Fri, 25 Jul 2014 16:17:43 +0200 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Sebastian Moeller In-Reply-To: Date: Fri, 25 Jul 2014 16:17:39 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <66FF8435-C8A5-4596-B43A-EC12D537D49E@gmx.de> References: <03292B76-5273-4912-BB18-90E95C16A9F5@pnsol.com> To: Neil Davies X-Mailer: Apple Mail (2.1878.6) X-Provags-ID: V03:K0:ANy+Inmm9QbDtfkYh+0zB2kSMHOeHtghALVcm++CiZUghvV4B21 aCKZgifWrhJ4gF/Iwu5SfgKnpc1wmGOYAGPnYbEowRT/rmowbqTi1+8cxdqvaG8ChpJI28F M3RQ8HBoDvdil1oMZb9uCcq7FV/kfKlmjl5RVWtlWnbnMq/G7buAQhiakso+AZUmQ4aUHbL GuV9X4cZMOjhyzzeJa3RQ== Cc: cerowrt-devel , bloat Subject: Re: [Cerowrt-devel] [Bloat] Check out www.speedof.me - no Flash X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2014 14:17:49 -0000 Hi Neil, On Jul 25, 2014, at 14:24 , Neil Davies wrote: > Rich >=20 > I have a deep worry over this style of single point measurement - and = hence speed - as an appropriate measure. But how do you propose to measure the (bottleneck) link capacity = then? It turns out for current CPE and CMTS/DSLAM equipment one = typically can not relay on good QoE out of the box, since typically = these devices do not use their (largish) buffers wisely. Instead the = current remedy is to take back control over the bottleneck link by = shaping the actually sent traffic to stay below the hardware link = capacity thereby avoiding feeling the consequences of the = over-buffering. But to do this is is quite helpful to get an educated = guess what the bottleneck links capacity actually is. And for that = purpose a speediest seems useful. > We know, and have evidence, that throughput/utilisation is not a good = proxy for the network delivering suitable quality of experience. We work = with organisation (Telco=92s, large system integrators etc) where we = spend a lot of time having to =93undo=94 the consequences of =93maximising= speed=94. Just like there is more to life than work, there is more to = QoE than speed. >=20 > For more specific comments see inline >=20 > On 25 Jul 2014, at 13:09, Rich Brown wrote: >=20 >> Neil, >>=20 >> Thanks for the note and the observations. My thoughts: >>=20 >> 1) I note that speedof.me does seem to overstate the speed results. = At my home, it reports 5.98mbps down, and 638kbps up, while = betterspeedtest.sh shows 5.49/0.61 mbps. (speedtest.net gives numbers = similar to the betterspeedtest.net script.) >>=20 >> 2) I think we're in agreement about the peak upload rate that you = point out is too high. Their measurement code runs in the browser. It = seems likely that the browser pumps out a few big packets before getting = flow control information, thus giving the impression that they can send = at a higher rate. This comports with the obvious decay that ramps toward = the long-term rate.=20 >=20 > I think that its simpler than that, it is measuring the rate at which = it can push packets out the interface - its real time rate is precisely = that - it can not be the rate being reported by the far end, it can = never exceed the limiting link. The long term average (if it is like = other speed testers we=92ve had to look into) is being measured at the = TCP/IP SDU level by measuring the difference in time between the first = and last timestamps of data stream and dividing that into the total data = sent. Their =93over-estimate=94 is because there are packets buffered in = the CPE that have left the machine but not arrived at the far end. Testing from an openwrt router located at a = high-symmetric-bandwidth location shows that speedof.me does not scale = higher than ~ 130 Mbps server to client and ~15Mbps client to server (on = the same connection I can get 130Mbps S2C and ~80Mbps C2S, so the = asymmetry in the speedof.me results is not caused by my local = environment).=20 @Rich and Dave, this probably means that for the upper end of = fiber and cable and VDSL connections speed of.me is not going to be a = reliable speed measure=85 Side note www.sppedtest.net shows ~100Mbps S2C = and ~100Mbps C2S, so might be better suited to high-upload links... >=20 >>=20 >> 3) But that long-term speed should be at or below the theoretical = long-term rate, not above it.=20 >=20 > Agreed, but in this case knowing the sync rate already defines that = maximum. I fully agree, but for ADSL the sync rate also contains a lot of = encapsulation, so the maximum achievable TCP rate is at best ~90% of = link rate. Note for cerowrt=92s SQM system the link rate is exactly the = right number to start out with at that system can take the encapsulation = into account. But even then it is somewhat unintuitive to deduce the = expected good-put from the link rate. >=20 >>=20 >> Two experiments for you to try: >>=20 >> a) What does betterspeedtest.sh show? (It's in the latest CeroWrt, in = /usr/lib/CeroWrtScripts, or get it from github: = https://github.com/richb-hanover/CeroWrtScripts ) >>=20 >> b) What does www.speedtest.net show? >>=20 >> I will add your question (about the inaccuracy) to the note that I = want to send out to speedof.me this weekend. I will also ask that they = include min/max latency measurements to their test, and an option to = send for > 10 seconds to minimize any effect of PowerBoost=85 I think they do already, at least for the download bandwidth; = they start with 128Kb and keep doubling the file size until a file takes = longer than 8 seconds to transfer, they only claim to report the numbers = from that last transferred file, so worst case with a stable link and a = bandwidth > 16kbps ;), it has taken at least 12 seconds (4 plus 8) of = measuring before the end of the plot, so the bandwidth of at least the = last half of the download plot should be representative even assuming = power boost. Caveat, I assume that power boost will not be reset by the = transient lack of data transfer between the differently sized files (but = since it should involve the same IPs and port# why should power boost = reset itself?). Best Regards Sebastian >>=20 >> Best regards, >>=20 >> Rich >>=20 >>=20 >>=20 >> On Jul 25, 2014, at 5:10 AM, Neil Davies = wrote: >>=20 >>> Rich >>>=20 >>> You may want to check how accurate they are to start. >>>=20 >>> I just ran a =93speed test=94 on my line (which I have complete = control and visibility over the various network elements) and it reports = an average =93speed=94 (in the up direction) that is in excess of the = capacity of the line, it reports the maximum rate at nearly twice the = best possible rate of the ADSL connection. >>>=20 >>> Doesn=92t matter how pretty it is, if its not accurate it is of no = use. This is rather ironic as the web site claims it is the =93smartest = and most accurate=94! >>>=20 >>> Neil >>>=20 >>> >>>=20 >>> PS pretty clear to me what mistake they=92ve made in the measurement = process - its to do with incorrect inference and hence missing the = buffering effects. >>>=20 >>> On 20 Jul 2014, at 14:19, Rich Brown = wrote: >>>=20 >>>> Doc Searls = (http://blogs.law.harvard.edu/doc/2014/07/20/the-cliff-peronal-clouds-need= -to-climb/) mentioned in passing that he uses a new speed test website. = I checked it out, and it was very cool=85 >>>>=20 >>>> www.speedof.me is an all-HTML5 website that seems to make accurate = measurements of the up and download speeds of your internet connection. = It=92s also very attractive, and the real-time plots of the speed show = interesting info. (screen shot at: http://richb-hanover.com/speedof-me/) >>>>=20 >>>> Now if we could get them to a) allow longer/bigger tests to = circumvent PowerBoost, and b) include a latency measurement so people = could point out their bufferbloated equipment.=20 >>>>=20 >>>> I'm going to send them a note. Anything else I should add? >>>>=20 >>>> Rich >>>> _______________________________________________ >>>> Bloat mailing list >>>> Bloat@lists.bufferbloat.net >>>> https://lists.bufferbloat.net/listinfo/bloat >>>=20 >>=20 >=20 > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat