From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id D959A3B25E for ; Sat, 27 Aug 2016 19:39:27 -0400 (EDT) Received: by mail-wm0-x231.google.com with SMTP id o80so40330606wme.1 for ; Sat, 27 Aug 2016 16:39:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=MBD2KWZSHERCAnUHCDrronGtWk9m0+PMf5PTSKVxD8o=; b=CGLE+q+iCsAqsV/X06RNuKlySJeA2b1cLEX3G1ID952qQaxFWbC3mddWFGj0Lxd9UZ kZd76Q6/rr0W3WXlRAP6P0UJWIFqmeMEwG8Cv0JHB6NkaFmXKo7ZWXxHqteHRxHmjHM5 o3zUL14a9zvj5uYGuUpE57Qgdt4EzkrN26UGKR8UypDPUoTPdC3ON7so9r/T2ch3c+ON jELbEOuLWuJ+ZNKWZj0mIJtZA8lS8OZheaeApQO//wrX9MV9B+QN9dPsmgD/Km7jBZQ/ Y80zabqPWgpaAF4cCqm0NZr3SFsaL5a/7iOK2vWB0ppD9iAH87f3hksfHasamvkW+Tpb cvTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=MBD2KWZSHERCAnUHCDrronGtWk9m0+PMf5PTSKVxD8o=; b=RBGbgIAV78Fg6XzN7le8/XYc5AriuxfXbZIMq3VAMGCvDC0/0IupGpHOfQbeT1gE4C GVAOxuQjLCr9SPMvZMpuJesFU+MrPILy0hT7sKNe5paqwsHSs8JND4VNPPZX4WKmWaxh gwDIlAAwRAHOPwUpTtxEokdnNcCWtu7bwn4Klk/PvkW1XIVlm83aQlLuSOZKqICDbNZQ El1ty7PgsYrJwrgWF/WZUo7R+XRqZytN/HlijmpRbJ8Y5xbKcgTXbivBYWkNuvvIjicT K7Cr5fCN+5z5AzwrfDeZrYo1tCQk2tkiwGOyOBRoi05xR54S8PbR9o0ZUTa1tIaOUHqh 8XNg== X-Gm-Message-State: AE9vXwPCnIaJrNe3bOTqU03DuETciq1pItPgCZdY6v0o9nH22WSawPzGXcitGKJQT+o1+C8SreVMVY9li8JH5g== X-Received: by 10.195.12.77 with SMTP id eo13mr9605653wjd.142.1472341166805; Sat, 27 Aug 2016 16:39:26 -0700 (PDT) MIME-Version: 1.0 Sender: justinbeech@gmail.com Received: by 10.28.220.8 with HTTP; Sat, 27 Aug 2016 16:39:26 -0700 (PDT) In-Reply-To: <8971f9f1-a79b-ce49-f7c4-660a355fdd43@pollere.com> References: <05056A85-894D-477F-A10D-C912C7D52C2F@gmail.com> <448a25be-160f-701e-f56b-b3a33d49cff8@pollere.com> <63403b38-94fa-1670-908c-e14573f822a8@gmail.com> <8971f9f1-a79b-ce49-f7c4-660a355fdd43@pollere.com> From: jb Date: Sun, 28 Aug 2016 09:39:26 +1000 X-Google-Sender-Auth: 3O73Yg9iEniPLJ-NQX1QBzVwhjM Message-ID: To: Kathleen Nichols Cc: Alan Jenkins , bloat Content-Type: multipart/alternative; boundary=047d7bd9172a3c5fe1053b162418 Subject: Re: [Bloat] Bufferbloat test measurements X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2016 23:39:28 -0000 --047d7bd9172a3c5fe1053b162418 Content-Type: text/plain; charset=UTF-8 Hi Kathleen, On Sun, Aug 28, 2016 at 3:37 AM, Kathleen Nichols wrote: > In-line below. Only for geeks. > > On 8/27/16 9:03 AM, Alan Jenkins wrote: > > > > That's the simplest measure of bufferbloat though :). > > Don't I know! :) Have spent a couple of years figuring out how to measure > experienced delay... > > > > Do you have a criticism in terms of dslreports.com? I think it's fairly > > transparent, showing idle v.s. download v.s. upload. The headline > > figures are an average, and you can look at all the data points. (You > > can increase the measurement frequency if you're specifically interested > > in that). > > "Criticism" seems too harsh. The uplink and downlink speed stuff is great > and agrees with all other measures. The "bufferbloat" grade is, I think, > sort > of confusing. Also, it's not clear where the queue the test builds up is > located. > It could be in ISP or home. I haven't found any cases where the queue builds up in the ISP yet. The people who have got grades and do something about it, always fix them by fixing their home router/modem either by aggressively introducing the kinds of stacks developed here in this list, or simply applying more rough fixes such as capping speeds. When they do that, their grade goes to A+ and the latency does not rise during the fastest transfer rates their connections run. Which is, after all, the easiest to demonstrate issue with excess buffer sizes. If you consider the ISP relative to an individuals connection, is it unlikely that said individuals contribution running their (necessarily capped) service at full speed can fill shared, much higher speed, buffers within the ISP infrastructure assuming the perfect world where ISPs are not over-subscribed.. > So, I ran the test while I was also streaming a > Netflix video. Under the column "RTT/jitter Avg" the test lists values that > range from 654 to 702 with +/- 5.2 to 20.8 ms (for the four servers). I > couldn't > figure out what that means. The jitter variation is a foot note, and not something that determines the grade. I use it to discover connections that are poor or lossy but not to draw any conclusions about the buffer bloat grade. The entire buffer bloat grade is the data in the graph that ties together the latency experienced during idle, upload and download. If you turn on 'hires' buffer bloat in preferences you get finer grained sampling of that latency. > I can see the delay ramp up over the test > (the video > stream also follows that ramp though it's normal delay ranges from about > 1ms to > about 40ms). If I took the RT delay experienced by the packets to/from > those servers, > I got median values between 391 and 429 ms. The IQRs were about 240ms to > 536-616ms. The maximum values where all just over 700ms, agreeing with > the dslreports > number. But that number is listed as an average so I don't understand? > Also what is the > jitter. I looked around for info on the numbers but I didn't find it. > Probably I just totally > missed some obvious thing to click on. > Again, the grade for buffer bloat is based on the latency over the course of the (hopefully) full speed download v the full speed upload vs the latency during idle. The latency is measured to a different and uninvolved server. Generally if a user does not use their connection to full capacity, latency is acceptable (but obviously the buffers are still there, lurking). Bursts and so on can discover them again but the effects are transient. > > But the thing is, I've been doing a lot of monitoring of my link and I > don't normally see > those kinds of delays. In the note I put out a bit ago, there is > definitely this bursting > behavior that is particularly indulged in by netflix and google but when > our downstream > bandwidth went up, those bursts no longer caused such long transient > delays (okay, duh). > So, I'm not sure who should be tagged as the "responsible party" for the > grade that the > test gives. Nor am I convinced that users have to assume that means they > are going to see > those kinds of delays. But this is a general issue with active measurement. > The responsible party is almost always the modem, especially DSL modems, that have buffers that are completely wrong for the speed set by the product or ISP. For instance I still get an "F" grade because during upload, with my very poor 1 megabit upload DSL sync, there is an enormous buffer and the latency rises to over a second making typing anything impossible. The modem tends to be the gatekeeper for the narrowest part of the flow of data and so is the first to be revealed as the culprit by the tests. You can't really fix any issues with buffer bloat without first sizing that right. Then, after that, there may be other things to correct. >From looking at the data, the amount of excess latency tends to drop with the higher product speed. So people on fiber and the faster cable connections don't see nearly such high latency up-lifts. I imagine to some extent this is because cable modem firmware has buffers sized for what they expect is the highest speed connections are, and so the faster products tend to operate in that area where it isn't as severe. Although one could argue that a 10ms increase on a gigabit connection is just as bothersome as a 1000ms increase on a 1megabit upload like mine, and both deserve an "F" grade, the fact is the consumer with the gigabit connection is never going to notice that kind of latency increase unless they're running a stock-trading program at home! There is various information on how to fix an "F" grade on the site, and it is always a challenge because it tends to boil down to : get rid of your ISP provided modem or pressure your ISP to introduce better home equipment because it is bound to be to the root cause of the worst of it, and you can't fix the rest without fixing that first. And that is often a difficult message to deliver. > It's not a criticism of the test, but maybe of the presentation of the > result. > > Kathie > > > > [random selection from Google] > > http://www.dslreports.com/speedtest/419540 > > http://forum.kitz.co.uk/index.php?topic=15620.0 > > > > It's more aggressive than a single file download, but that's not > > deliberately to exacerbate bufferbloat. It's just designed to measure > > performance of prolonged downloads / streaming, in a competitively short > > test. "For our busy lives", as the overused saying goes. > > > > (The initial summary only gives a grade. The figure wouldn't be one of > > the headlines their ISP advertises. Saying "100ms" would confuse > > people. And the tests they're used to / compare with, show idle latency > > instead.) > > > > On 27/08/16 16:19, Kathleen Nichols wrote: > >> Yeah. > >> > >> I admit to muddying the waters because I think of the size of a buffer > as > >> being in megabytes and the size of a queue (latency) as being in > >> milliseconds. I think the tests attempt to measure the worst possible > >> latency/queue that can occur on a path. > >> > >> On 8/27/16 4:46 AM, Rich Brown wrote: > >>> It has always been my intent to define bufferbloat as *latency*. The > >>> first sentence on www.bufferbloat.net says, "Bufferbloat is the > >>> undesirable latency that comes from a router or other network > >>> equipment buffering too much data." > >>> > >>> That definition focuses on observable/measurable values. It sidesteps > >>> objections I've seen on the forums, "How could $TEST measure the size > >>> of buffers?" > >>> > >>> So what matters is whether the buffers (of any size) are filling up. > >>> _______________________________________________ Bloat mailing list > >>> Bloat@lists.bufferbloat.net > >>> https://lists.bufferbloat.net/listinfo/bloat > >>> > >> _______________________________________________ > >> Bloat mailing list > >> Bloat@lists.bufferbloat.net > >> https://lists.bufferbloat.net/listinfo/bloat > > > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > --047d7bd9172a3c5fe1053b162418 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Kathleen,

On Sun, Aug 28, 2016 at 3:37 AM, Kathleen Nichols <nich= ols@pollere.com> wrote:
In-= line below. Only for geeks.

On 8/27/16 9:03 AM, Alan Jenkins wrote:
>
> That's the simplest measure of bufferbloat though :).

Don't I know! :)=C2=A0 Have spent a couple of years figuring out= how to measure
experienced delay...
>
> Do you have a criticism in terms of dslreports.com?=C2=A0 I think it&#= 39;s fairly
> transparent, showing idle v.s. download v.s. upload.=C2=A0 The headlin= e
> figures are an average, and you can look at all the data points.=C2=A0= (You
> can increase the measurement frequency if you're specifically inte= rested
> in that).

"Criticism" seems too harsh. The uplink and downlink speed= stuff is great
and agrees with all other measures. The "bufferbloat" grade is, I= think,
sort
of confusing. Also, it's not clear where the queue the test builds up i= s
located.
It could be in ISP or home.

I haven't = found any cases where the queue builds up in the ISP yet.
The peo= ple who have got grades and do something about it, always fix them
by fixing their home router/modem either by aggressively introducing the = kinds
of stacks developed here in this list, or simply applying m= ore rough fixes such
as capping speeds. When they do that, their = grade goes to A+ and the latency
does not rise during the fastest= transfer rates their connections run. Which is,
after all, the e= asiest to demonstrate issue with excess buffer sizes.
If you cons= ider the ISP relative to an individuals connection, is it unlikely
that said individuals contribution running their (necessarily capped) ser= vice at
full speed can fill shared, much higher speed, buffers wi= thin the ISP infrastructure
assuming the perfect world where ISPs= are not over-subscribed..
=C2=A0
So, I ran the test while I was also streaming a
Netflix video. Under the column "RTT/jitter Avg" the test lists v= alues that
range from 654 to 702 with +/- 5.2 to 20.8 ms (for the four servers). I
couldn't
figure out what that means.

The jitter var= iation is a foot note, and not something that determines the
grad= e. I use it to discover connections that are poor or lossy but not=C2=A0
to draw any conclusions about the buffer bloat grade. The entire bu= ffer bloat
grade is the data in the graph that ties together the = latency experienced
during idle, upload and download. If you turn= on 'hires' buffer bloat in preferences
you get finer gra= ined sampling of that latency.
=C2=A0
I can see the delay ramp up over the test
(the video
stream also follows that ramp though it's normal delay ranges from abou= t
1ms to
about 40ms). If I took the RT delay experienced by the packets to/from
those servers,
I got median values between 391 and 429 ms. The IQRs were about 240ms to 536-616ms. The maximum values where all just over 700ms, agreeing with
the dslreports
number. But that number is listed as an average so I don't understand?<= br> Also what is the
jitter. I looked around for info on the numbers but I didn't find it. Probably I just totally
missed some obvious thing to click on.

= Again, the grade for buffer bloat is based on the latency over the course o= f
the (hopefully) full speed download v the full speed upload vs = the latency
during idle. The latency is measured to a different a= nd uninvolved server.

Generally if a user does not= use their connection to full capacity, latency
is acceptable (bu= t obviously the buffers are still there, lurking). Bursts and
so = on can discover them again but the effects are transient.
=C2=A0<= /div>

But the thing is, I've been doing a lot of monitoring of my link and I<= br> don't normally see
those kinds of delays. In the note I put out a bit ago, there is
definitely this bursting
behavior that is particularly indulged in by netflix and google but when our downstream
bandwidth went up, those bursts no longer caused such long transient
delays (okay, duh).
So, I'm not sure who should be tagged as the "responsible party&qu= ot; for the
grade that the
test gives. Nor am I convinced that users have to assume that means they are going to see
those kinds of delays. But this is a general issue with active measurement.=

The responsible party is almost always= the modem, especially DSL modems,
that have buffers that are com= pletely wrong for the speed set by the product
or ISP. For instan= ce I still get an "F" grade because during upload, with my
<= div>very poor 1 megabit upload DSL sync, there is an enormous buffer and th= e
latency rises to over a second making typing anything impossibl= e. The modem
tends to be the gatekeeper for the narrowest part of= the flow of data and so
is the first to be revealed as the culpr= it by the tests. You can't really fix any
issues with buffer = bloat without first sizing that right. Then, after that, there
ma= y be other things to correct.

From looking at the = data, the amount of excess latency tends to drop with
the higher = product speed. So people on fiber and the faster cable connections
don't see nearly such high latency up-lifts. I imagine to some extent= this is=C2=A0
because cable modem firmware has buffers sized for= what they expect is
the highest speed connections are, and so th= e faster products tend to operate in
that area where it isn't= as severe. Although one could argue that a 10ms increase
on a gi= gabit connection is just as bothersome as a 1000ms increase on a 1megabit
upload like mine, and both deserve an "F" grade, the fac= t is the consumer=C2=A0
with the gigabit connection is never goin= g to notice that kind of latency increase
unless they're runn= ing a stock-trading program at home!

There is vari= ous information on how to fix an "F" grade on the site, and it is= always
a challenge because it tends to boil down to : get rid of= your ISP provided modem
or pressure your ISP to introduce better= home equipment because it is bound to
be to the root cause of th= e worst of =C2=A0it, and you can't fix the rest without fixing that
first. And that is often a difficult message to deliver.
<= br>

It's not a criticism of the test, but maybe of the presentation of the<= br> result.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 Kathie
>
> [random selection from Google]
> http://www.dslreports.com/speedtest/419540 > http://forum.kitz.co.uk/index.php?topic= =3D15620.0
>
> It's more aggressive than a single file download, but that's n= ot
> deliberately to exacerbate bufferbloat.=C2=A0 It's just designed t= o measure
> performance of prolonged downloads / streaming, in a competitively sho= rt
> test.=C2=A0 "For our busy lives", as the overused saying goe= s.
>
> (The initial summary only gives a grade.=C2=A0 The figure wouldn't= be one of
> the headlines their ISP advertises.=C2=A0 Saying "100ms" wou= ld confuse
> people.=C2=A0 And the tests they're used to / compare with, show i= dle latency
> instead.)
>
> On 27/08/16 16:19, Kathleen Nichols wrote:
>> Yeah.
>>
>> I admit to muddying the waters because I think of the size of a bu= ffer as
>> being in megabytes and the size of a queue (latency) as being in >> milliseconds. I think the tests attempt to measure the worst possi= ble
>> latency/queue that can occur on a path.
>>
>> On 8/27/16 4:46 AM, Rich Brown wrote:
>>> It has always been my intent to define bufferbloat as *latency= *. The
>>> first sentence on www.bufferbloat.net says, "Buffe= rbloat is the
>>> undesirable latency that comes from a router or other network<= br> >>> equipment buffering too much data."
>>>
>>> That definition focuses on observable/measurable values. It si= desteps
>>> objections I've seen on the forums, "How could $TEST = measure the size
>>> of buffers?"
>>>
>>> So what matters is whether the buffers (of any size) are filli= ng up.
>>> _______________________________________________ Bloat mai= ling list
>>> Bloat@lists.buf= ferbloat.net
>>> https://lists.bufferbloat.net/listin= fo/bloat
>>>
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferb= loat.net
>> https://lists.bufferbloat.net/listinfo/blo= at
>

_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net<= /a>
https://lists.bufferbloat.net/listinfo/bloat

--047d7bd9172a3c5fe1053b162418--