[Bloat] Bufferbloat test measurements

Sat Aug 27 19:39:26 EDT 2016

Hi Kathleen,

On Sun, Aug 28, 2016 at 3:37 AM, Kathleen Nichols <nichols at pollere.com>
wrote:

> In-line below. Only for geeks.
>
> On 8/27/16 9:03 AM, Alan Jenkins wrote:
> >
> > That's the simplest measure of bufferbloat though :).
>
> Don't I know! :)  Have spent a couple of years figuring out how to measure
> experienced delay...
> >
> > Do you have a criticism in terms of dslreports.com?  I think it's fairly
> > transparent, showing idle v.s. download v.s. upload.  The headline
> > figures are an average, and you can look at all the data points.  (You
> > can increase the measurement frequency if you're specifically interested
> > in that).
>
> "Criticism" seems too harsh. The uplink and downlink speed stuff is great
> and agrees with all other measures. The "bufferbloat" grade is, I think,
> sort
> of confusing. Also, it's not clear where the queue the test builds up is
> located.
> It could be in ISP or home.

I haven't found any cases where the queue builds up in the ISP yet.
The people who have got grades and do something about it, always fix them
by fixing their home router/modem either by aggressively introducing the
kinds
of stacks developed here in this list, or simply applying more rough fixes
such
as capping speeds. When they do that, their grade goes to A+ and the latency
does not rise during the fastest transfer rates their connections run.
Which is,
after all, the easiest to demonstrate issue with excess buffer sizes.
If you consider the ISP relative to an individuals connection, is it
unlikely
that said individuals contribution running their (necessarily capped)
service at
full speed can fill shared, much higher speed, buffers within the ISP
infrastructure
assuming the perfect world where ISPs are not over-subscribed..

> So, I ran the test while I was also streaming a
> Netflix video. Under the column "RTT/jitter Avg" the test lists values that
> range from 654 to 702 with +/- 5.2 to 20.8 ms (for the four servers). I
> couldn't
> figure out what that means.

The jitter variation is a foot note, and not something that determines the
grade. I use it to discover connections that are poor or lossy but not
to draw any conclusions about the buffer bloat grade. The entire buffer
bloat
grade is the data in the graph that ties together the latency experienced
during idle, upload and download. If you turn on 'hires' buffer bloat in
preferences
you get finer grained sampling of that latency.

> I can see the delay ramp up over the test
> (the video
> stream also follows that ramp though it's normal delay ranges from about
> 1ms to
> about 40ms). If I took the RT delay experienced by the packets to/from
> those servers,
> I got median values between 391 and 429 ms. The IQRs were about 240ms to
> 536-616ms. The maximum values where all just over 700ms, agreeing with
> the dslreports
> number. But that number is listed as an average so I don't understand?
> Also what is the
> jitter. I looked around for info on the numbers but I didn't find it.
> Probably I just totally
> missed some obvious thing to click on.
>

Again, the grade for buffer bloat is based on the latency over the course of
the (hopefully) full speed download v the full speed upload vs the latency
during idle. The latency is measured to a different and uninvolved server.

Generally if a user does not use their connection to full capacity, latency
is acceptable (but obviously the buffers are still there, lurking). Bursts
and
so on can discover them again but the effects are transient.

>
> But the thing is, I've been doing a lot of monitoring of my link and I
> don't normally see
> those kinds of delays. In the note I put out a bit ago, there is
> definitely this bursting
> behavior that is particularly indulged in by netflix and google but when
> our downstream
> bandwidth went up, those bursts no longer caused such long transient
> delays (okay, duh).
> So, I'm not sure who should be tagged as the "responsible party" for the
> grade that the
> test gives. Nor am I convinced that users have to assume that means they
> are going to see
> those kinds of delays. But this is a general issue with active measurement.
>

The responsible party is almost always the modem, especially DSL modems,
that have buffers that are completely wrong for the speed set by the product
or ISP. For instance I still get an "F" grade because during upload, with my
very poor 1 megabit upload DSL sync, there is an enormous buffer and the
latency rises to over a second making typing anything impossible. The modem
tends to be the gatekeeper for the narrowest part of the flow of data and so
is the first to be revealed as the culprit by the tests. You can't really
fix any
issues with buffer bloat without first sizing that right. Then, after that,
there
may be other things to correct.

>From looking at the data, the amount of excess latency tends to drop with
the higher product speed. So people on fiber and the faster cable
connections
don't see nearly such high latency up-lifts. I imagine to some extent this
is
because cable modem firmware has buffers sized for what they expect is
the highest speed connections are, and so the faster products tend to
operate in
that area where it isn't as severe. Although one could argue that a 10ms
increase
on a gigabit connection is just as bothersome as a 1000ms increase on a
1megabit
upload like mine, and both deserve an "F" grade, the fact is the consumer
with the gigabit connection is never going to notice that kind of latency
increase
unless they're running a stock-trading program at home!

There is various information on how to fix an "F" grade on the site, and it
is always
a challenge because it tends to boil down to : get rid of your ISP provided
modem
or pressure your ISP to introduce better home equipment because it is bound
to
be to the root cause of the worst of  it, and you can't fix the rest
without fixing that
first. And that is often a difficult message to deliver.

> It's not a criticism of the test, but maybe of the presentation of the
> result.
>
>         Kathie
> >
> > [random selection from Google]
> > http://www.dslreports.com/speedtest/419540
> > http://forum.kitz.co.uk/index.php?topic=15620.0
> >
> > It's more aggressive than a single file download, but that's not
> > deliberately to exacerbate bufferbloat.  It's just designed to measure
> > performance of prolonged downloads / streaming, in a competitively short
> > test.  "For our busy lives", as the overused saying goes.
> >
> > (The initial summary only gives a grade.  The figure wouldn't be one of
> > the headlines their ISP advertises.  Saying "100ms" would confuse
> > people.  And the tests they're used to / compare with, show idle latency
> > instead.)
> >
> > On 27/08/16 16:19, Kathleen Nichols wrote:
> >> Yeah.
> >>
> >> I admit to muddying the waters because I think of the size of a buffer
> as
> >> being in megabytes and the size of a queue (latency) as being in
> >> milliseconds. I think the tests attempt to measure the worst possible
> >> latency/queue that can occur on a path.
> >>
> >> On 8/27/16 4:46 AM, Rich Brown wrote:
> >>> It has always been my intent to define bufferbloat as *latency*. The
> >>> first sentence on www.bufferbloat.net says, "Bufferbloat is the
> >>> undesirable latency that comes from a router or other network
> >>> equipment buffering too much data."
> >>>
> >>> That definition focuses on observable/measurable values. It sidesteps
> >>> objections I've seen on the forums, "How could $TEST measure the size
> >>> of buffers?"
> >>>
> >>> So what matters is whether the buffers (of any size) are filling up.
> >>> _______________________________________________ Bloat mailing list
> >>> Bloat at lists.bufferbloat.net
> >>> https://lists.bufferbloat.net/listinfo/bloat
> >>>
> >> _______________________________________________
> >> Bloat mailing list
> >> Bloat at lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/bloat
> >
>
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/bloat/attachments/20160828/4b7e707f/attachment-0002.html>