[Bloat] grading bloat better

[Bloat] grading bloat better

[Dave Taht]

This has major bloat happening at the end of the upload test. Which
worries me - here, at a gbit.

http://www.dslreports.com/speedtest/5284047

-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Bloat] grading bloat better

[jb]

[-- Attachment #1: Type: text/plain, Size: 1182 bytes --]

A while ago I changed from mean to median with the reasoning being that one
spike to a crazy level was not representative of bloat but instead
representative of a network stall or other anomaly. Graphs that were nearly
all good samples with one outlier were being unfairly graded poorly.

But this example has the opposite issue - the median of this set of samples
is the first half where everything is ok. Hence the good score. Using a
mean would be correct for this sample.
What should happen is to throw away a couple (max) outliers first, then do
a mean to avoid punishing the results that come in as good but include one
errant measurement.

thanks
-Justin

On Wed, Oct 12, 2016 at 11:16 PM, Dave Taht <dave.taht@gmail.com> wrote:

> This has major bloat happening at the end of the upload test. Which
> worries me - here, at a gbit.
>
> http://www.dslreports.com/speedtest/5284047
>
> --
> Dave Täht
> Let's go make home routers and wifi faster! With better software!
> http://blog.cerowrt.org
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

[-- Attachment #2: Type: text/html, Size: 1931 bytes --]

Re: [Bloat] grading bloat better

[jb]

[-- Attachment #1: Type: text/plain, Size: 1511 bytes --]

Actually I think the concept I need is the trimmed mean.
throwing away the highest couple of values (lowest couple are not to be
thrown away because they can't be errant).
It isn't perfect but it would help.

On Thu, Oct 13, 2016 at 11:39 AM, jb <justin@dslr.net> wrote:

> A while ago I changed from mean to median with the reasoning being that
> one spike to a crazy level was not representative of bloat but instead
> representative of a network stall or other anomaly. Graphs that were nearly
> all good samples with one outlier were being unfairly graded poorly.
>
> But this example has the opposite issue - the median of this set of
> samples is the first half where everything is ok. Hence the good score.
> Using a mean would be correct for this sample.
> What should happen is to throw away a couple (max) outliers first, then do
> a mean to avoid punishing the results that come in as good but include one
> errant measurement.
>
> thanks
> -Justin
>
> On Wed, Oct 12, 2016 at 11:16 PM, Dave Taht <dave.taht@gmail.com> wrote:
>
>> This has major bloat happening at the end of the upload test. Which
>> worries me - here, at a gbit.
>>
>> http://www.dslreports.com/speedtest/5284047
>>
>> --
>> Dave Täht
>> Let's go make home routers and wifi faster! With better software!
>> http://blog.cerowrt.org
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>>
>
>

[-- Attachment #2: Type: text/html, Size: 2606 bytes --]

Re: [Bloat] grading bloat better

[jb]

[-- Attachment #1: Type: text/plain, Size: 1708 bytes --]

It is done
under the trimmed mean method, that would be a "C" grade result.



On Thu, Oct 13, 2016 at 11:46 AM, jb <justin@dslr.net> wrote:

> Actually I think the concept I need is the trimmed mean.
> throwing away the highest couple of values (lowest couple are not to be
> thrown away because they can't be errant).
> It isn't perfect but it would help.
>
> On Thu, Oct 13, 2016 at 11:39 AM, jb <justin@dslr.net> wrote:
>
>> A while ago I changed from mean to median with the reasoning being that
>> one spike to a crazy level was not representative of bloat but instead
>> representative of a network stall or other anomaly. Graphs that were nearly
>> all good samples with one outlier were being unfairly graded poorly.
>>
>> But this example has the opposite issue - the median of this set of
>> samples is the first half where everything is ok. Hence the good score.
>> Using a mean would be correct for this sample.
>> What should happen is to throw away a couple (max) outliers first, then
>> do a mean to avoid punishing the results that come in as good but include
>> one errant measurement.
>>
>> thanks
>> -Justin
>>
>> On Wed, Oct 12, 2016 at 11:16 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>
>>> This has major bloat happening at the end of the upload test. Which
>>> worries me - here, at a gbit.
>>>
>>> http://www.dslreports.com/speedtest/5284047
>>>
>>> --
>>> Dave Täht
>>> Let's go make home routers and wifi faster! With better software!
>>> http://blog.cerowrt.org
>>> _______________________________________________
>>> Bloat mailing list
>>> Bloat@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/bloat
>>>
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 3200 bytes --]

Re: [Bloat] grading bloat better

[Dave Taht]

Thank you very much for the explanation and the fix. I am confronted
by the dsltestreports stuff every day on my search for bufferbloat. I
don't consider it annoying, but as a chance to spot check!

...

I still might quibble, but a trimmed mean makes more sense than just a mean.

Problem I always have is bloat is biased always towards the end of a test. Here,
at 1gbit, it took nearly 20 seconds to start going boom. Maybe we need
to invent a new distribution (The bloat distribution? The TCP
distribution)...

You are getting towards a big dataset now. (has it been a year yet?)
Got anyone lined up for a paper on it? I'd still love it if one day
someone could take all the data you are filtering out, and plot
that....

I imagine the user's test result is cached and not subject to these
modifications?

On Wed, Oct 12, 2016 at 5:57 PM, jb <justin@dslr.net> wrote:
> It is done
> under the trimmed mean method, that would be a "C" grade result.
>
>
>
> On Thu, Oct 13, 2016 at 11:46 AM, jb <justin@dslr.net> wrote:
>>
>> Actually I think the concept I need is the trimmed mean.
>> throwing away the highest couple of values (lowest couple are not to be
>> thrown away because they can't be errant).
>> It isn't perfect but it would help.
>>
>> On Thu, Oct 13, 2016 at 11:39 AM, jb <justin@dslr.net> wrote:
>>>
>>> A while ago I changed from mean to median with the reasoning being that
>>> one spike to a crazy level was not representative of bloat but instead
>>> representative of a network stall or other anomaly. Graphs that were nearly
>>> all good samples with one outlier were being unfairly graded poorly.
>>>
>>> But this example has the opposite issue - the median of this set of
>>> samples is the first half where everything is ok. Hence the good score.
>>> Using a mean would be correct for this sample.
>>> What should happen is to throw away a couple (max) outliers first, then
>>> do a mean to avoid punishing the results that come in as good but include
>>> one errant measurement.
>>>
>>> thanks
>>> -Justin
>>>
>>> On Wed, Oct 12, 2016 at 11:16 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>>>
>>>> This has major bloat happening at the end of the upload test. Which
>>>> worries me - here, at a gbit.
>>>>
>>>> http://www.dslreports.com/speedtest/5284047
>>>>
>>>> --
>>>> Dave Täht
>>>> Let's go make home routers and wifi faster! With better software!
>>>> http://blog.cerowrt.org
>>>> _______________________________________________
>>>> Bloat mailing list
>>>> Bloat@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/bloat
>>>
>>>
>>
>



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Bloat] grading bloat better

[Jonathan Morton]

> On 13 Oct, 2016, at 06:22, Dave Taht <dave.taht@gmail.com> wrote:
> 
> I still might quibble, but a trimmed mean makes more sense than just a mean.
> 
> Problem I always have is bloat is biased always towards the end of a test. Here,
> at 1gbit, it took nearly 20 seconds to start going boom. Maybe we need
> to invent a new distribution (The bloat distribution? The TCP
> distribution)...

Perhaps the 90th percentile would be relevant - the median is simply the 50th percentile.  Taking the 90th avoids counting true flukes, but also captures the idea that intermittently high latency is in fact noticeable.

 - Jonathan Morton

Re: [Bloat] grading bloat better

[jb]

[-- Attachment #1: Type: text/plain, Size: 3359 bytes --]

Is this classic buffer bloat on 50 megabit cable modem?

https://www.dslreports.com/forum/r31035315-Weird-speed-test-results-It-falls-off-right-at-the-end

by extending the download duration to 30 seconds, what looks like
a speed "fall-off at the end" reveals two complete stall/recoveries, and
associated
high latency during the download phase.

On Thu, Oct 13, 2016 at 2:22 PM, Dave Taht <dave.taht@gmail.com> wrote:

> Thank you very much for the explanation and the fix. I am confronted
> by the dsltestreports stuff every day on my search for bufferbloat. I
> don't consider it annoying, but as a chance to spot check!
>
> ...
>
> I still might quibble, but a trimmed mean makes more sense than just a
> mean.
>
> Problem I always have is bloat is biased always towards the end of a test.
> Here,
> at 1gbit, it took nearly 20 seconds to start going boom. Maybe we need
> to invent a new distribution (The bloat distribution? The TCP
> distribution)...
>
> You are getting towards a big dataset now. (has it been a year yet?)
> Got anyone lined up for a paper on it? I'd still love it if one day
> someone could take all the data you are filtering out, and plot
> that....
>
> I imagine the user's test result is cached and not subject to these
> modifications?
>
> On Wed, Oct 12, 2016 at 5:57 PM, jb <justin@dslr.net> wrote:
> > It is done
> > under the trimmed mean method, that would be a "C" grade result.
> >
> >
> >
> > On Thu, Oct 13, 2016 at 11:46 AM, jb <justin@dslr.net> wrote:
> >>
> >> Actually I think the concept I need is the trimmed mean.
> >> throwing away the highest couple of values (lowest couple are not to be
> >> thrown away because they can't be errant).
> >> It isn't perfect but it would help.
> >>
> >> On Thu, Oct 13, 2016 at 11:39 AM, jb <justin@dslr.net> wrote:
> >>>
> >>> A while ago I changed from mean to median with the reasoning being that
> >>> one spike to a crazy level was not representative of bloat but instead
> >>> representative of a network stall or other anomaly. Graphs that were
> nearly
> >>> all good samples with one outlier were being unfairly graded poorly.
> >>>
> >>> But this example has the opposite issue - the median of this set of
> >>> samples is the first half where everything is ok. Hence the good score.
> >>> Using a mean would be correct for this sample.
> >>> What should happen is to throw away a couple (max) outliers first, then
> >>> do a mean to avoid punishing the results that come in as good but
> include
> >>> one errant measurement.
> >>>
> >>> thanks
> >>> -Justin
> >>>
> >>> On Wed, Oct 12, 2016 at 11:16 PM, Dave Taht <dave.taht@gmail.com>
> wrote:
> >>>>
> >>>> This has major bloat happening at the end of the upload test. Which
> >>>> worries me - here, at a gbit.
> >>>>
> >>>> http://www.dslreports.com/speedtest/5284047
> >>>>
> >>>> --
> >>>> Dave Täht
> >>>> Let's go make home routers and wifi faster! With better software!
> >>>> http://blog.cerowrt.org
> >>>> _______________________________________________
> >>>> Bloat mailing list
> >>>> Bloat@lists.bufferbloat.net
> >>>> https://lists.bufferbloat.net/listinfo/bloat
> >>>
> >>>
> >>
> >
>
>
>
> --
> Dave Täht
> Let's go make home routers and wifi faster! With better software!
> http://blog.cerowrt.org
>

[-- Attachment #2: Type: text/html, Size: 5040 bytes --]

Re: [Bloat] grading bloat better

[Dave Taht]

On Fri, Oct 14, 2016 at 3:48 PM, jb <justin@dslr.net> wrote:
> Is this classic buffer bloat on 50 megabit cable modem?

Looks like it.

> https://www.dslreports.com/forum/r31035315-Weird-speed-test-results-It-falls-off-right-at-the-end
>
> by extending the download duration to 30 seconds, what looks like
> a speed "fall-off at the end" reveals two complete stall/recoveries, and
> associated
> high latency during the download phase.
>
> On Thu, Oct 13, 2016 at 2:22 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>
>> Thank you very much for the explanation and the fix. I am confronted
>> by the dsltestreports stuff every day on my search for bufferbloat. I
>> don't consider it annoying, but as a chance to spot check!
>>
>> ...
>>
>> I still might quibble, but a trimmed mean makes more sense than just a
>> mean.
>>
>> Problem I always have is bloat is biased always towards the end of a test.
>> Here,
>> at 1gbit, it took nearly 20 seconds to start going boom. Maybe we need
>> to invent a new distribution (The bloat distribution? The TCP
>> distribution)...
>>
>> You are getting towards a big dataset now. (has it been a year yet?)
>> Got anyone lined up for a paper on it? I'd still love it if one day
>> someone could take all the data you are filtering out, and plot
>> that....
>>
>> I imagine the user's test result is cached and not subject to these
>> modifications?
>>
>> On Wed, Oct 12, 2016 at 5:57 PM, jb <justin@dslr.net> wrote:
>> > It is done
>> > under the trimmed mean method, that would be a "C" grade result.
>> >
>> >
>> >
>> > On Thu, Oct 13, 2016 at 11:46 AM, jb <justin@dslr.net> wrote:
>> >>
>> >> Actually I think the concept I need is the trimmed mean.
>> >> throwing away the highest couple of values (lowest couple are not to be
>> >> thrown away because they can't be errant).
>> >> It isn't perfect but it would help.
>> >>
>> >> On Thu, Oct 13, 2016 at 11:39 AM, jb <justin@dslr.net> wrote:
>> >>>
>> >>> A while ago I changed from mean to median with the reasoning being
>> >>> that
>> >>> one spike to a crazy level was not representative of bloat but instead
>> >>> representative of a network stall or other anomaly. Graphs that were
>> >>> nearly
>> >>> all good samples with one outlier were being unfairly graded poorly.
>> >>>
>> >>> But this example has the opposite issue - the median of this set of
>> >>> samples is the first half where everything is ok. Hence the good
>> >>> score.
>> >>> Using a mean would be correct for this sample.
>> >>> What should happen is to throw away a couple (max) outliers first,
>> >>> then
>> >>> do a mean to avoid punishing the results that come in as good but
>> >>> include
>> >>> one errant measurement.
>> >>>
>> >>> thanks
>> >>> -Justin
>> >>>
>> >>> On Wed, Oct 12, 2016 at 11:16 PM, Dave Taht <dave.taht@gmail.com>
>> >>> wrote:
>> >>>>
>> >>>> This has major bloat happening at the end of the upload test. Which
>> >>>> worries me - here, at a gbit.
>> >>>>
>> >>>> http://www.dslreports.com/speedtest/5284047
>> >>>>
>> >>>> --
>> >>>> Dave Täht
>> >>>> Let's go make home routers and wifi faster! With better software!
>> >>>> http://blog.cerowrt.org
>> >>>> _______________________________________________
>> >>>> Bloat mailing list
>> >>>> Bloat@lists.bufferbloat.net
>> >>>> https://lists.bufferbloat.net/listinfo/bloat
>> >>>
>> >>>
>> >>
>> >
>>
>>
>>
>> --
>> Dave Täht
>> Let's go make home routers and wifi faster! With better software!
>> http://blog.cerowrt.org
>
>



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Bloat] grading bloat better

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 1184 bytes --]

Yes, it looks like typical head-of-line blocking with a multi-second
buffer.  To smooth it out, you would need an averaging window wider than
the effective (bloated) RTT.

A tell-tale symptom is that the width of the gap, in which little or no
progress is made at the application layer, is approximately the measured
RTT at the left edge of the valley (or, equivalently, the peak RTT). It
takes that long for the retransmitted packets to traverse the stuffed
queue, even though the sender also reduces its send rate at the same time.

The latter should cause the RTT to reduce somewhat after the right edge of
the valley - this is part of the classic sawtooth.  Another tell-tale is
therefore that the valleys are regularly spaced on a lengthened test,
though there are other possible causes of that.

Your user should be able to reproduce the effect on a big, single stream
download, by watching the progress and noting that it periodically stalls
and then jumps ahead.  The average progress rate remains 50Mbps, but the
instantaneous progress rate regularly falls to zero.

I do suggest that you try to detect this case and explain the above facts
automatically.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 1297 bytes --]