From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp137.dfw.emailsrvr.com (smtp137.dfw.emailsrvr.com [67.192.241.137]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by huchra.bufferbloat.net (Postfix) with ESMTPS id D3C0821F5C2 for ; Fri, 25 Jul 2014 08:05:23 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp26.relay.dfw1a.emailsrvr.com (SMTP Server) with ESMTP id AEDF22800F4; Fri, 25 Jul 2014 11:05:22 -0400 (EDT) X-Virus-Scanned: OK Received: by smtp26.relay.dfw1a.emailsrvr.com (Authenticated sender: dpreed-AT-reed.com) with ESMTPSA id 709122800D5; Fri, 25 Jul 2014 11:05:21 -0400 (EDT) X-Sender-Id: dpreed@reed.com Received: from [10.163.89.145] (255.sub-70-197-13.myvzw.com [70.197.13.255]) (using TLSv1 with cipher DHE-RSA-AES256-SHA) by 0.0.0.0:465 (trex/5.2.10); Fri, 25 Jul 2014 15:05:22 GMT User-Agent: K-@ Mail for Android X-Priority: 3 In-Reply-To: <66FF8435-C8A5-4596-B43A-EC12D537D49E@gmx.de> References: <03292B76-5273-4912-BB18-90E95C16A9F5@pnsol.com> <66FF8435-C8A5-4596-B43A-EC12D537D49E@gmx.de> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----E9EHCW93JNV5Q5W7GZGJWN9Y2L493V" Content-Transfer-Encoding: 7bit From: "David P. Reed" Date: Fri, 25 Jul 2014 08:05:19 -0700 To: Sebastian Moeller ,Neil Davies Message-ID: Cc: cerowrt-devel , bloat Subject: Re: [Cerowrt-devel] [Bloat] Check out www.speedof.me - no Flash X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2014 15:05:24 -0000 ------E9EHCW93JNV5Q5W7GZGJWN9Y2L493V Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable It's important to note that modern browsers have direct access to TCP Up an= d Down connections using Web sockets from javascript threads=2E It is quite= feasible to drive such connections at near wire speeds in both directions= =2E=2E=2E I've done it in my own experiments=2E A knowledgeable network tes= ting expert should be able to create a javascript library that can be used = by any gui=2E=2E=2Eso beauty and measurement quality can be improved by exp= erts in the relevant fields=2E My experiments are a bit dated but if someo= ne wants advice on how please ask=2E I'm flat out on my day job for the nex= t month=2E On Jul 25, 2014, Sebastian Moeller wrote: >= Hi Neil, > > >On Jul 25, 2014, at 14:24 , Neil Davies wrote: > >> Rich >> >> I have a deep worry over this style of sing= le point measurement - and >hence speed - as an appropriate measure=2E > > = But how do you propose to measure the (bottleneck) link capacity then? >It = turns out for current CPE and CMTS/DSLAM equipment one typically can >not r= elay on good QoE out of the box, since typically these devices do >not use = their (largish) buffers wisely=2E Instead the current remedy is >to take ba= ck control over the bottleneck link by shaping the actually >sent traffic t= o stay below the hardware link capacity thereby avoiding >feeling the conse= quences of the over-buffering=2E But to do this is is >quite helpful to get= an educated guess what the bottleneck links >capacity actually is=2E And f= or that purpose a speediest seems useful=2E > > >> We know, and have eviden= ce, that throughput/utilisation is not a good >proxy for the network delive= ring suitable quality of experience=2E We >work with organisation (Telco=E2= =80=99s, large system integrators etc) where we >spend a lot of time having= to =E2=80=9Cundo=E2=80=9D the consequences of =E2=80=9Cmaximising >speed= =E2=80=9D=2E Just like there is more to life than work, there is more to Qo= E >than speed=2E >> >> For more specific comments see inline >> >> On 25 = Jul 2014, at 13:09, Rich Brown wrote: >> >>>= Neil, >>> >>> Thanks for the note and the observations=2E My thoughts: >>= > >>> 1) I note that speedof=2Eme does seem to overstate the speed results= =2E >At my home, it reports 5=2E98mbps down, and 638kbps up, while >betters= peedtest=2Esh shows 5=2E49/0=2E61 mbps=2E (speedtest=2Enet gives numbers >s= imilar to the betterspeedtest=2Enet script=2E) >>> >>> 2) I think we're in= agreement about the peak upload rate that you >point out is too high=2E Th= eir measurement code runs in the browser=2E It >seems likely that the brows= er pumps out a few big packets before >getting flow control information, th= us giving the impression that they >can send at a higher rate=2E This compo= rts with the obvious decay that >ramps toward the long-term rate=2E >> >>= I think that its simpler than that, it is measuring the rate at which >it = can push packets out the interface - its real time rate is precisely >that = - it can not be the rate being reported by the far end, it can >never excee= d the limiting link=2E The long term average (if it is like >other speed te= sters we=E2=80=99ve had to look into) is being measured at the >TCP/IP SDU = level by measuring the difference in time between the first >and last times= tamps of data stream and dividing that into the total >data sent=2E Their = =E2=80=9Cover-estimate=E2=80=9D is because there are packets buffered >in t= he CPE that have left the machine but not arrived at the far end=2E > > Tes= ting from an openwrt router located at a high-symmetric-bandwidth >location= shows that speedof=2Eme does not scale higher than ~ 130 Mbps >server to c= lient and ~15Mbps client to server (on the same connection I >can get 130Mb= ps S2C and ~80Mbps C2S, so the asymmetry in the speedof=2Eme >results is no= t caused by my local environment)=2E > @Rich and Dave, this probably means= that for the upper end of fiber >and cable and VDSL connections speed of= =2Eme is not going to be a >reliable speed measure=E2=80=A6 Side note www= =2Esppedtest=2Enet shows ~100Mbps S2C >and ~100Mbps C2S, so might be better= suited to high-upload links=2E=2E=2E > >> >>> >>> 3) But that long-term = speed should be at or below the theoretical >long-term rate, not above it= =2E >> >> Agreed, but in this case knowing the sync rate already defines = that >maximum=2E > > I fully agree, but for ADSL the sync rate also contain= s a lot of >encapsulation, so the maximum achievable TCP rate is at best ~9= 0% of >link rate=2E Note for cerowrt=E2=80=99s SQM system the link rate is = exactly the >right number to start out with at that system can take the >en= capsulation into account=2E But even then it is somewhat unintuitive to >de= duce the expected good-put from the link rate=2E > >> >>> >>> Two experim= ents for you to try: >>> >>> a) What does betterspeedtest=2Esh show? (It's= in the latest CeroWrt, >in /usr/lib/CeroWrtScripts, or get it from github:= >https://github=2Ecom/richb-hanover/CeroWrtScripts ) >>> >>> b) What does= www=2Espeedtest=2Enet show? >>> >>> I will add your question (about the i= naccuracy) to the note that I >want to send out to speedof=2Eme this weeken= d=2E I will also ask that they >include min/max latency measurements to the= ir test, and an option to >send for > 10 seconds to minimize any effect of = PowerBoost=E2=80=A6 > > I think they do already, at least for the download = bandwidth; they >start with 128Kb and keep doubling the file size until a f= ile takes >longer than 8 seconds to transfer, they only claim to report the= >numbers from that last transferred file, so worst case with a stable >lin= k and a bandwidth > 16kbps ;), it has taken at least 12 seconds (4 >plus 8)= of measuring before the end of the plot, so the bandwidth of at >least the= last half of the download plot should be representative even >assuming pow= er boost=2E Caveat, I assume that power boost will not be >reset by the tra= nsient lack of data transfer between the differently >sized files (but sinc= e it should involve the same IPs and port# why >should power boost reset it= self?)=2E > >Best Regards > Sebastian > > > >>> >>> Best regards, >>> >>>= Rich >>> >>> >>> >>> On Jul 25, 2014, at 5:10 AM, Neil Davies >wrote: >>> >>>> Rich >>>> >>>> You may want to check = how accurate they are to start=2E >>>> >>>> I just ran a =E2=80=9Cspeed te= st=E2=80=9D on my line (which I have complete control >and visibility over = the various network elements) and it reports an >average =E2=80=9Cspeed=E2= =80=9D (in the up direction) that is in excess of the capacity >of the line= , it reports the maximum rate at nearly twice the best >possible rate of th= e ADSL connection=2E >>>> >>>> Doesn=E2=80=99t matter how pretty it is, if= its not accurate it is of no >use=2E This is rather ironic as the web site= claims it is the =E2=80=9Csmartest >and most accurate=E2=80=9D! >>>> >>>>= Neil >>>> >>>> >>>> >>>> PS pretty clear to = me what mistake they=E2=80=99ve made in the measurement >process - its to d= o with incorrect inference and hence missing the >buffering effects=2E >>>>= >>>> On 20 Jul 2014, at 14:19, Rich Brown >= wrote: >>>> >>>>> Doc Searls >(http://blogs=2Elaw=2Eharvard=2Eedu/doc/2014= /07/20/the-cliff-peronal-clouds-need-to-climb/) >mentioned in passing that = he uses a new speed test website=2E I checked >it out, and it was very cool= =E2=80=A6 >>>>> >>>>> www=2Espeedof=2Eme is an all-HTML5 website that seem= s to make accurate >measurements of the up and download speeds of your inte= rnet connection=2E >It=E2=80=99s also very attractive, and the real-time pl= ots of the speed show >interesting info=2E (screen shot at: >http://richb-h= anover=2Ecom/speedof-me/) >>>>> >>>>> Now if we could get them to a) allow= longer/bigger tests to >circumvent PowerBoost, and b) include a latency me= asurement so people >could point out their bufferbloated equipment=2E >>>>= > >>>>> I'm going to send them a note=2E Anything else I should add? >>>>>= >>>>> Rich >>>>> _______________________________________________ >>>>> Bl= oat mailing list >>>>> Bloat@lists=2Ebufferbloat=2Enet >>>>> https://lists= =2Ebufferbloat=2Enet/listinfo/bloat >>>> >>> >> >> _____________________= __________________________ >> Bloat mailing list >> Bloat@lists=2Ebufferblo= at=2Enet >> https://lists=2Ebufferbloat=2Enet/listinfo/bloat > >___________= ____________________________________ >Cerowrt-devel mailing list >Cerowrt-d= evel@lists=2Ebufferbloat=2Enet >https://lists=2Ebufferbloat=2Enet/listinfo/= cerowrt-devel -- Sent from my Android device with K-@ Mail=2E Please excus= e my brevity=2E ------E9EHCW93JNV5Q5W7GZGJWN9Y2L493V Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable It's important to note that modern browsers have d= irect access to TCP Up and Down connections using Web sockets from javascri= pt threads=2E It is quite feasible to drive such connections at near wire s= peeds in both directions=2E=2E=2E I've done it in my own experiments=2E A k= nowledgeable network testing expert should be able to create a javascript l= ibrary that can be used by any gui=2E=2E=2Eso beauty and measurement qualit= y can be improved by experts in the relevant fields=2E
<= br clear=3D"none"> My experiments are a bit dated but if someone wants advi= ce on how please ask=2E I'm flat out on my day job for the next month=2E
On Jul 25, 20= 14, Sebastian Moeller <moeller0@gmx=2Ede> wrote:
Hi Neil,


On Jul 25, 2014, at 14=
:24 , Neil Davies <Neil=2EDavies@pnsol=2Ecom> wrote:

Rich

I have a deep worry over this st= yle of single point measurement - and hence speed - as an appropriate measu= re=2E

But how do you propose to measure the= (bottleneck) link capacity then? It turns out for current CPE and CMTS/DSL= AM equipment one typically can not relay on good QoE out of the box, since = typically these devices do not use their (largish) buffers wisely=2E Instea= d the current remedy is to take back control over the bottleneck link by sh= aping the actually sent traffic to stay below the hardware link capacity th= ereby avoiding feeling the consequences of the over-buffering=2E But to do = this is is quite helpful to get an educated guess what the bottleneck links= capacity actually is=2E And for that purpose a speediest seems useful=2E

We know, and have evidence, that throughput/ut= ilisation is not a good proxy for the network delivering suitable quality o= f experience=2E We work with organisation (Telco’s, large system inte= grators etc) where we spend a lot of time having to “undo” the = consequences of “maximising speed”=2E Just like there is more t= o life than work, there is more to QoE than speed=2E

For more specific comments see inline

On 25 Jul 2014, at 13:09, Rich Brown <richb=2Ehanover@gma= il=2Ecom> wrote:

Neil,

= Thanks for the note and the observations=2E My thoughts:
=
1) I note that speedof=2Eme does seem to overstate the speed results=2E At my h= ome, it reports 5=2E98mbps down, and 638kbps up, while betterspeedtest=2Esh shows 5=2E49/0= =2E61 mbps=2E (speedtest= =2Enet gives numbers similar to the betterspeedtest=2Enet script=2E)

2) I think we're in agreement about the peak upload rat= e that you point out is too high=2E Their measurement code runs in the brow= ser=2E It seems likely that the browser pumps out a few big packets before = getting flow control information, thus giving the impression that they can = send at a higher rate=2E This comports with the obvious decay that ramps to= ward the long-term rate=2E

I think that its= simpler than that, it is measuring the rate at which it can push packets o= ut the interface - its real time rate is precisely that - it can not be the= rate being reported by the far end, it can never exceed the limiting link= =2E The long term average (if it is like other speed testers we’ve ha= d to look into) is being measured at the TCP/IP SDU level by measuring the = difference in time between the first and last timestamps of data stream and= dividing that into the total data sent=2E Their “over-estimate”= ; is because there are packets buffered in the CPE that have left the machi= ne but not arrived at the far end=2E

Testin= g from an openwrt router located at a high-symmetric-bandwidth location sho= ws that speedof=2Eme doe= s not scale higher than ~ 130 Mbps server to client and ~15Mbps client to s= erver (on the same connection I can get 130Mbps S2C and ~80Mbps C2S, so the= asymmetry in the speedof=2E= me results is not caused by my local environment)=2E
@Rich and Dave, this probably means that for the upper end of fiber and c= able and VDSL connections speed o= f=2Eme is not going to be a reliable speed measure… Side note www=2Esppedtest=2Enet<= /a> shows ~100Mbps S2C and ~100Mbps C2S, so might be better suited to high-= upload links=2E=2E=2E



3) But that long-term speed sh= ould be at or below the theoretical long-term rate, not above it=2E
Agreed, but in this case knowing the sync rate alr= eady defines that maximum=2E

I fully agree,= but for ADSL the sync rate also contains a lot of encapsulation, so the ma= ximum achievable TCP rate is at best ~90% of link rate=2E Note for cerowrt&= #8217;s SQM system the link rate is exactly the right number to start out w= ith at that system can take the encapsulation into account=2E But even then= it is somewhat unintuitive to deduce the expected good-put from the link r= ate=2E



Two experiments for you to try:

a) What does betterspeedtest=2Esh show? (It's in the latest CeroW= rt, in /usr/lib/CeroWrtScripts, or get it from github: https://github=2Ec= om/richb-hanover/CeroWrtScripts )

= b) What does www=2E= speedtest=2Enet show?

I will add y= our question (about the inaccuracy) to the note that I want to send out to = speedof=2Eme this weeken= d=2E I will also ask that they include min/max latency measurements to thei= r test, and an option to send for > 10 seconds to minimize any effect of= PowerBoost…

I think the= y do already, at least for the download bandwidth; they start with 128Kb an= d keep doubling the file size until a file takes longer than 8 seconds to t= ransfer, they only claim to report the numbers from that last transferred f= ile, so worst case with a stable link and a bandwidth > 16kbps ;), it ha= s taken at least 12 seconds (4 plus 8) of measuring before the end of the p= lot, so the bandwidth of at least the last half of the download plot should= be representative even assuming power boost=2E Caveat, I assume that power= boost will not be reset by the transient lack of data transfer between the= differently sized files (but since it should involve the same IPs and port= # why should power boost reset itself?)=2E

Best Regards
Sebastian




Best regards,

Rich



On J= ul 25, 2014, at 5:10 AM, Neil Davies <neil=2Edavies@pnsol=2Ecom> wrot= e:

Rich

You may want to che= ck how accurate they are to start=2E

I= just ran a “speed test” on my line (which I have complete cont= rol and visibility over the various network elements) and it reports an ave= rage “speed” (in the up direction) that is in excess of the cap= acity of the line, it reports the maximum rate at nearly twice the best pos= sible rate of the ADSL connection=2E

D= oesn’t matter how pretty it is, if its not accurate it is of no use= =2E This is rather ironic as the web site claims it is the “smartest = and most accurate”!

Neil

<speedof_me_14-07-25=2Epng>

PS pretty clear to me what mistake they’ve = made in the measurement process - its to do with incorrect inference and he= nce missing the buffering effects=2E

O= n 20 Jul 2014, at 14:19, Rich Brown <richb=2Ehanover@gmail=2Ecom> wro= te:

Doc Searls (http://= blogs=2Elaw=2Eharvard=2Eedu/doc/2014/07/20/the-cliff-peronal-clouds-need-to= -climb/) mentioned in passing that he uses a new speed test website=2E = I checked it out, and it was very cool…

www=2Espeedo= f=2Eme is an all-HTML5 website that seems to make accurate measurements= of the up and download speeds of your internet connection=2E It’s al= so very attractive, and the real-time plots of the speed show interesting i= nfo=2E (screen shot at: http://richb-hanover=2Ecom/speedof-me/)

Now if we could get them to a) allow longer/bigger test= s to circumvent PowerBoost, and b) include a latency measurement so people = could point out their bufferbloated equipment=2E

I'm going to send them a note=2E Anything else I should add?
Rich

Bloat mailing list
Bloat@lists=2Ebufferbloat=2Enet<= br clear=3D"none">https://lists=2Ebufferbloat=2Enet/listinfo/bloat



Bloat mailing list
Bloat@lists=2Ebufferbloat=2Enet
https://lists=2Ebuffe= rbloat=2Enet/listinfo/bloat


Cerowrt-devel mailing list
Cerowrt-devel@lists= =2Ebufferbloat=2Enet
https://lists=2Ebufferbloat= =2Enet/listinfo/cerowrt-devel

-- Sent from my Android device with K-@ Mail=2E Please excuse my brevity=2E ------E9EHCW93JNV5Q5W7GZGJWN9Y2L493V--