<html><head></head><body>It's important to note that modern browsers have direct access to TCP Up and Down connections using Web sockets from javascript threads. It is quite feasible to drive such connections at near wire speeds in both directions... I've done it in my own experiments. A knowledgeable network testing expert should be able to create a javascript library that can be used by any gui...so beauty and measurement quality can be improved by experts in the relevant fields.<br clear="none">
<br clear="none">
My experiments are a bit dated but if someone wants advice on how please ask. I'm flat out on my day job for the next month.<br clear="none"><br clear="none"><div class="gmail_quote">On Jul 25, 2014, Sebastian Moeller <moeller0@gmx.de> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<pre class="k10mail">Hi Neil,<br clear="none"><br clear="none"><br clear="none">On Jul 25, 2014, at 14:24 , Neil Davies <Neil.Davies@pnsol.com> wrote:<br clear="none"><br clear="none"></pre><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">Rich<br clear="none"><br clear="none">I have a deep worry over this style of single point measurement - and hence speed - as an appropriate measure.</blockquote><br clear="none"> But how do you propose to measure the (bottleneck) link capacity then? It turns out for current CPE and CMTS/DSLAM equipment one typically can not relay on good QoE out of the box, since typically these devices do not use their (largish) buffers wisely. Instead the current remedy is to take back control over the bottleneck link by shaping the actually sent traffic to stay below the hardware link capacity thereby avoiding feeling the consequences of the over-buffering. But to do this is is quite helpful to get an educated guess what the bottleneck links capacity actually is. And for that purpose a speediest seems useful.<br clear="none"><br clear="none"><br clear="none"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">We know, and have evidence, that throughput/utilisation is not a good proxy for the network delivering suitable quality of experience. We work with organisation (Telco’s, large system integrators etc) where we spend a lot of time having to “undo” the consequences of “maximising speed”. Just like there is more to life than work, there is more to QoE than speed.<br clear="none"><br clear="none">For more specific comments see inline<br clear="none"><br clear="none">On 25 Jul 2014, at 13:09, Rich Brown <richb.hanover@gmail.com> wrote:<br clear="none"><br clear="none"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #ad7fa8; padding-left: 1ex;">Neil,<br clear="none"><br clear="none">Thanks for the note and the observations. My thoughts:<br clear="none"><br clear="none">1) I note that <a shape="rect" href="http://speedof.me">speedof.me</a> does seem to overstate the speed results. At my home, it reports 5.98mbps down, and 638kbps up, while <a shape="rect" href="http://betterspeedtest.sh">betterspeedtest.sh</a> shows 5.49/0.61 mbps. (<a shape="rect" href="http://speedtest.net">speedtest.net</a> gives numbers similar to the <a shape="rect" href="http://betterspeedtest.net">betterspeedtest.net</a> script.)<br clear="none"><br clear="none">2) I think we're in agreement about the peak upload rate that you point out is too high. Their measurement code runs in the browser. It seems likely that the browser pumps out a few big packets before getting flow control information, thus giving the impression that they can send at a higher rate. This comports with the obvious decay that ramps toward the long-term rate. </blockquote><br clear="none">I think that its simpler than that, it is measuring the rate at which it can push packets out the interface - its real time rate is precisely that - it can not be the rate being reported by the far end, it can never exceed the limiting link. The long term average (if it is like other speed testers we’ve had to look into) is being measured at the TCP/IP SDU level by measuring the difference in time between the first and last timestamps of data stream and dividing that into the total data sent. Their “over-estimate” is because there are packets buffered in the CPE that have left the machine but not arrived at the far end.</blockquote><br clear="none"> Testing from an openwrt router located at a high-symmetric-bandwidth location shows that <a shape="rect" href="http://speedof.me">speedof.me</a> does not scale higher than ~ 130 Mbps server to client and ~15Mbps client to server (on the same connection I can get 130Mbps S2C and ~80Mbps C2S, so the asymmetry in the <a shape="rect" href="http://speedof.me">speedof.me</a> results is not caused by my local environment). <br clear="none"> @Rich and Dave, this probably means that for the upper end of fiber and cable and VDSL connections speed <a shape="rect" href="http://of.me">of.me</a> is not going to be a reliable speed measure… Side note <a shape="rect" href="http://www.sppedtest.net">www.sppedtest.net</a> shows ~100Mbps S2C and ~100Mbps C2S, so might be better suited to high-upload links...<br clear="none"><br clear="none"><br clear="none"><br clear="none"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #ad7fa8; padding-left: 1ex;">3) But that long-term speed should be at or below the theoretical long-term rate, not above it. </blockquote><br clear="none">Agreed, but in this case knowing the sync rate already defines that maximum.</blockquote><br clear="none"> I fully agree, but for ADSL the sync rate also contains a lot of encapsulation, so the maximum achievable TCP rate is at best ~90% of link rate. Note for cerowrt’s SQM system the link rate is exactly the right number to start out with at that system can take the encapsulation into account. But even then it is somewhat unintuitive to deduce the expected good-put from the link rate.<br clear="none"><br clear="none"><br clear="none"><br clear="none"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #ad7fa8; padding-left: 1ex;">Two experiments for you to try:<br clear="none"><br clear="none">a) What does <a shape="rect" href="http://betterspeedtest.sh">betterspeedtest.sh</a> show? (It's in the latest CeroWrt, in /usr/lib/CeroWrtScripts, or get it from github: <a shape="rect" href="https://github.com/richb-hanover/CeroWrtScripts">https://github.com/richb-hanover/CeroWrtScripts</a> )<br clear="none"><br clear="none">b) What does <a shape="rect" href="http://www.speedtest.net">www.speedtest.net</a> show?<br clear="none"><br clear="none">I will add your question (about the inaccuracy) to the note that I want to send out to <a shape="rect" href="http://speedof.me">speedof.me</a> this weekend. I will also ask that they include min/max latency measurements to their test, and an option to send for > 10 seconds to minimize any effect of PowerBoost…</blockquote><br clear="none"></blockquote> I think they do already, at least for the download bandwidth; they start with 128Kb and keep doubling the file size until a file takes longer than 8 seconds to transfer, they only claim to report the numbers from that last transferred file, so worst case with a stable link and a bandwidth > 16kbps ;), it has taken at least 12 seconds (4 plus 8) of measuring before the end of the plot, so the bandwidth of at least the last half of the download plot should be representative even assuming power boost. Caveat, I assume that power boost will not be reset by the transient lack of data transfer between the differently sized files (but since it should involve the same IPs and port# why should power boost reset itself?).<br clear="none"><br clear="none">Best Regards<br clear="none"> Sebastian<br clear="none"><br clear="none"><br clear="none"><br clear="none"><br clear="none"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #ad7fa8; padding-left: 1ex;">Best regards,<br clear="none"><br clear="none">Rich<br clear="none"><br clear="none"><br clear="none"><br clear="none">On Jul 25, 2014, at 5:10 AM, Neil Davies <neil.davies@pnsol.com> wrote:<br clear="none"><br clear="none"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #8ae234; padding-left: 1ex;">Rich<br clear="none"><br clear="none">You may want to check how accurate they are to start.<br clear="none"><br clear="none">I just ran a “speed test” on my line (which I have complete control and visibility over the various network elements) and it reports an average “speed” (in the up direction) that is in excess of the capacity of the line, it reports the maximum rate at nearly twice the best possible rate of the ADSL connection.<br clear="none"><br clear="none">Doesn’t matter how pretty it is, if its not accurate it is of no use. This is rather ironic as the web site claims it is the “smartest and most accurate”!<br clear="none"><br clear="none">Neil<br clear="none"><br clear="none"><speedof_me_14-07-25.png><br clear="none"><br clear="none">PS pretty clear to me what mistake they’ve made in the measurement process - its to do with incorrect inference and hence missing the buffering effects.<br clear="none"><br clear="none">On 20 Jul 2014, at 14:19, Rich Brown <richb.hanover@gmail.com> wrote:<br clear="none"><br clear="none"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #fcaf3e; padding-left: 1ex;">Doc Searls (<a shape="rect" href="http://blogs.law.harvard.edu/doc/2014/07/20/the-cliff-peronal-clouds-need-to-climb">http://blogs.law.harvard.edu/doc/2014/07/20/the-cliff-peronal-clouds-need-to-climb</a>/) mentioned in passing that he uses a new speed test website. I checked it out, and it was very cool…<br clear="none"><br clear="none"><a shape="rect" href="http://www.speedof.me">www.speedof.me</a> is an all-HTML5 website that seems to make accurate measurements of the up and download speeds of your internet connection. It’s also very attractive, and the real-time plots of the speed show interesting info. (screen shot at: <a shape="rect" href="http://richb-hanover.com/speedof-me">http://richb-hanover.com/speedof-me</a>/)<br clear="none"><br clear="none">Now if we could get them to a) allow longer/bigger tests to circumvent PowerBoost, and b) include a latency measurement so people could point out their bufferbloated equipment. <br clear="none"><br clear="none">I'm going to send them a note. Anything else I should add?<br clear="none"><br clear="none">Rich<br clear="none"><hr><br clear="none">Bloat mailing list<br clear="none">Bloat@lists.bufferbloat.net<br clear="none"><a shape="rect" href="https://lists.bufferbloat.net/listinfo/bloat">https://lists.bufferbloat.net/listinfo/bloat</a></blockquote><br clear="none"><br clear="none"><br clear="none"></blockquote></blockquote><hr><br clear="none">Bloat mailing list<br clear="none">Bloat@lists.bufferbloat.net<br clear="none"><a shape="rect" href="https://lists.bufferbloat.net/listinfo/bloat">https://lists.bufferbloat.net/listinfo/bloat</a></blockquote><br clear="none"><hr><br clear="none">Cerowrt-devel mailing list<br clear="none">Cerowrt-devel@lists.bufferbloat.net<br clear="none"><a shape="rect" href="https://lists.bufferbloat.net/listinfo/cerowrt-devel">https://lists.bufferbloat.net/listinfo/cerowrt-devel</a><br clear="none"></blockquote></div><br clear="none">-- Sent from my Android device with <b><a shape="rect" href="https://play.google.com/store/apps/details?id=com.onegravity.k10.pro2">K-@ Mail</a></b>. Please excuse my brevity.</body></html>