[Cerowrt-devel] [Bloat] Check out www.speedof.me - no Flash

Fri Jul 25 10:25:16 EDT 2014

You may find the following useful background reading on the state of the
art in network measurement, and a primer on ΔQ (which is the property we
wish to measure).

First, start with this presentation: Network performance optimisation using
high-fidelity measures
<http://www.slideshare.net/mgeddes/network-performance-optimisation-using-highfidelity-measures>
Then read this one to decompose ΔQ into G, S and V: Fundamentals of network
performance engineering
<http://www.slideshare.net/mgeddes/fundamentals-of-network-performance-engineering-18548490>
Then read this one to get a bit more sense on what ΔQ is about: Introduction
to ΔQ and Network Performance Science (extracts)
<http://www.slideshare.net/mgeddes/introduction-to-q-extracts>

Then read these essays:

Foundation of Network Science
<http://www.martingeddes.com/think-tank/foundation-network-science/>
How to do network performance chemistry
<http://www.martingeddes.com/think-tank/network-performance-chemistry/>
How to X-ray a telecoms network
<http://www.martingeddes.com/x-ray-telecoms-network/>
There is no quality in averages: IPX case study
<http://www.martingeddes.com/ipx-quality-averages/>

Martin

*For fresh thinking about telecoms sign up for my free newsletter
<http://eepurl.com/dSkfz> or visit the Geddes Think Tank
<http://www.martingeddes.com/think-tank/>.*
LinkedIn <http://www.linkedin.com/in/mgeddes> Twitter
<https://twitter.com/martingeddes> Mobile: +44 7957 499219 Skype: mgeddes
Martin Geddes Consulting Ltd, Incorporated in Scotland, number SC275827 VAT
Number: 859 5634 72 Registered office: 17-19 East London Street, Edinburgh,
EH7 4BN

On 25 July 2014 15:17, Sebastian Moeller <moeller0 at gmx.de> wrote:

> Hi Neil,
>
>
> On Jul 25, 2014, at 14:24 , Neil Davies <Neil.Davies at pnsol.com> wrote:
>
> > Rich
> >
> > I have a deep worry over this style of single point measurement - and
> hence speed - as an appropriate measure.
>
>         But how do you propose to measure the (bottleneck) link capacity
> then? It turns out for current CPE and CMTS/DSLAM equipment one typically
> can not relay on good QoE out of the box, since typically these devices do
> not use their (largish) buffers wisely. Instead the current remedy is to
> take back control over the bottleneck link by shaping the actually sent
> traffic to stay below the hardware link capacity thereby avoiding feeling
> the consequences of the over-buffering. But to do this is is quite helpful
> to get an educated guess what the bottleneck links capacity actually is.
> And for that purpose a speediest seems useful.
>
>
> > We know, and have evidence, that throughput/utilisation is not a good
> proxy for the network delivering suitable quality of experience. We work
> with organisation (Telco’s, large system integrators etc) where we spend a
> lot of time having to “undo” the consequences of “maximising speed”. Just
> like there is more to life than work, there is more to QoE than speed.
> >
> > For more specific comments see inline
> >
> > On 25 Jul 2014, at 13:09, Rich Brown <richb.hanover at gmail.com> wrote:
> >
> >> Neil,
> >>
> >> Thanks for the note and the observations. My thoughts:
> >>
> >> 1) I note that speedof.me does seem to overstate the speed results. At
> my home, it reports 5.98mbps down, and 638kbps up, while betterspeedtest.sh
> shows 5.49/0.61 mbps. (speedtest.net gives numbers similar to the
> betterspeedtest.net script.)
> >>
> >> 2) I think we're in agreement about the peak upload rate that you point
> out is too high. Their measurement code runs in the browser. It seems
> likely that the browser pumps out a few big packets before getting flow
> control information, thus giving the impression that they can send at a
> higher rate. This comports with the obvious decay that ramps toward the
> long-term rate.
> >
> > I think that its simpler than that, it is measuring the rate at which it
> can push packets out the interface - its real time rate is precisely that -
> it can not be the rate being reported by the far end, it can never exceed
> the limiting link. The long term average (if it is like other speed testers
> we’ve had to look into) is being measured at the TCP/IP SDU level by
> measuring the difference in time between the first and last timestamps of
> data stream and dividing that into the total data sent. Their
> “over-estimate” is because there are packets buffered in the CPE that have
> left the machine but not arrived at the far end.
>
>         Testing from an openwrt router located at a
> high-symmetric-bandwidth location shows that speedof.me does not scale
> higher than ~ 130 Mbps server to client and ~15Mbps client to server (on
> the same connection I can get 130Mbps S2C and ~80Mbps C2S, so the asymmetry
> in the speedof.me results is not caused by my local environment).
>         @Rich and Dave, this probably means that for the upper end of
> fiber and cable and VDSL connections speed of.me is not going to be a
> reliable speed measure… Side note www.sppedtest.net shows ~100Mbps S2C
> and ~100Mbps C2S, so might be better suited to high-upload links...
>
> >
> >>
> >> 3) But that long-term speed should be at or below the theoretical
> long-term rate, not above it.
> >
> > Agreed, but in this case knowing the sync rate already defines that
> maximum.
>
>         I fully agree, but for ADSL the sync rate also contains a lot of
> encapsulation, so the maximum achievable TCP rate is at best ~90% of link
> rate. Note for cerowrt’s SQM system the link rate is exactly the right
> number to start out with at that system can take the encapsulation into
> account. But even then it is somewhat unintuitive to deduce the expected
> good-put from the link rate.
>
> >
> >>
> >> Two experiments for you to try:
> >>
> >> a) What does betterspeedtest.sh show? (It's in the latest CeroWrt, in
> /usr/lib/CeroWrtScripts, or get it from github:
> https://github.com/richb-hanover/CeroWrtScripts )
> >>
> >> b) What does www.speedtest.net show?
> >>
> >> I will add your question (about the inaccuracy) to the note that I want
> to send out to speedof.me this weekend. I will also ask that they include
> min/max latency measurements to their test, and an option to send for > 10
> seconds to minimize any effect of PowerBoost…
>
>         I think they do already, at least for the download bandwidth; they
> start with 128Kb and keep doubling the file size until a file takes longer
> than 8 seconds to transfer, they only claim to report the numbers from that
> last transferred file, so worst case with a stable link and a bandwidth >
> 16kbps ;), it has taken at least 12 seconds (4 plus 8) of measuring before
> the end of the plot, so the bandwidth of at least the last half of the
> download plot should be representative even assuming power boost. Caveat, I
> assume that power boost will not be reset by the transient lack of data
> transfer between the differently sized files (but since it should involve
> the same IPs and port# why should power boost reset itself?).
>
> Best Regards
>         Sebastian
>
>
>
> >>
> >> Best regards,
> >>
> >> Rich
> >>
> >>
> >>
> >> On Jul 25, 2014, at 5:10 AM, Neil Davies <neil.davies at pnsol.com> wrote:
> >>
> >>> Rich
> >>>
> >>> You may want to check how accurate they are to start.
> >>>
> >>> I just ran a “speed test” on my line (which I have complete control
> and visibility over the various network elements) and it reports an average
> “speed” (in the up direction) that is in excess of the capacity of the
> line, it reports the maximum rate at nearly twice the best possible rate of
> the ADSL connection.
> >>>
> >>> Doesn’t matter how pretty it is, if its not accurate it is of no use.
> This is rather ironic as the web site claims it is the “smartest and most
> accurate”!
> >>>
> >>> Neil
> >>>
> >>> <speedof_me_14-07-25.png>
> >>>
> >>> PS pretty clear to me what mistake they’ve made in the measurement
> process - its to do with incorrect inference and hence missing the
> buffering effects.
> >>>
> >>> On 20 Jul 2014, at 14:19, Rich Brown <richb.hanover at gmail.com> wrote:
> >>>
> >>>> Doc Searls (
> http://blogs.law.harvard.edu/doc/2014/07/20/the-cliff-peronal-clouds-need-to-climb/)
> mentioned in passing that he uses a new speed test website. I checked it
> out, and it was very cool…
> >>>>
> >>>> www.speedof.me is an all-HTML5 website that seems to make accurate
> measurements of the up and download speeds of your internet connection.
> It’s also very attractive, and the real-time plots of the speed show
> interesting info. (screen shot at: http://richb-hanover.com/speedof-me/)
> >>>>
> >>>> Now if we could get them to a) allow longer/bigger tests to
> circumvent PowerBoost, and b) include a latency measurement so people could
> point out their bufferbloated equipment.
> >>>>
> >>>> I'm going to send them a note. Anything else I should add?
> >>>>
> >>>> Rich
> >>>> _______________________________________________
> >>>> Bloat mailing list
> >>>> Bloat at lists.bufferbloat.net
> >>>> https://lists.bufferbloat.net/listinfo/bloat
> >>>
> >>
> >
> > _______________________________________________
> > Bloat mailing list
> > Bloat at lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat
>
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/cerowrt-devel/attachments/20140725/9cfae006/attachment-0001.html>