[Bloat] dslreports bufferbloat tests

[Bloat] dslreports bufferbloat tests

[Dave Taht]

3.5m samples thus far. Wow.

I have not been running it much recently but I keep hoping that more
and more samples would start appearing above the green line, and maybe
I'm fooling myself, but they are.

http://www.dslreports.com/speedtest/results/bufferbloat?up=1

Tested cake today on a linksys 1200ac:

cake: http://www.dslreports.com/speedtest/3527810

vs

native: http://www.dslreports.com/speedtest/3527835

Does anyone know what the "quality" portion of dslreport's metric means?

--
Dave Täht
Let's go make home routers and wifi faster! With better software!
https://www.gofundme.com/savewifi

Re: [Bloat] dslreports bufferbloat tests

[Brandon Applegate]

[-- Attachment #1: Type: text/plain, Size: 658 bytes --]


> On Apr 5, 2016, at 9:04 PM, Dave Taht <dave.taht@gmail.com> wrote:
> 
> Does anyone know what the "quality" portion of dslreport's metric means?

Basically - packet loss.

https://www.dslreports.com/faq/17930

—
Quality Grades

Quality refers to average detected packet loss / re-transmit percentages during download phase. The higher the packet loss / re-transmit percentage the more inefficient the connection is, and a very poor result may be indicative of congestion, inside wiring issues or other problems that need addressing.

1% or less - A+
2.5% or less - A
3% or less - B
5% or less - C
12% or less - D
over 12% - F
—


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]

Re: [Bloat] dslreports bufferbloat tests

[Dave Taht]

On Tue, Apr 5, 2016 at 6:33 PM, Brandon Applegate <brandon@burn.net> wrote:
>
>> On Apr 5, 2016, at 9:04 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>
>> Does anyone know what the "quality" portion of dslreport's metric means?
>
> Basically - packet loss.
>
> https://www.dslreports.com/faq/17930

Sigh. I ranted. I might rant harder.

http://blog.cerowrt.org/post/bufferbloat_vs_quality/

>
> —
> Quality Grades
>
> Quality refers to average detected packet loss / re-transmit percentages during download phase. The higher the packet loss / re-transmit percentage the more inefficient the connection is, and a very poor result may be indicative of congestion, inside wiring issues or other problems that need addressing.
>
> 1% or less - A+
> 2.5% or less - A
> 3% or less - B
> 5% or less - C
> 12% or less - D
> over 12% - F
> —
>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

Re: [Bloat] dslreports bufferbloat tests

[jb]

I take your point regarding Quality

but both your examples in the post show A+ quality?

I'm thinking that packet loss significant enough to show as a "C" or
worse is mostly a bad situation
even if avoiding all packet loss - by using huge buffers - is
definitely a disaster..


On Wed, Apr 6, 2016 at 12:21 PM, Dave Taht <dave.taht@gmail.com> wrote:
> On Tue, Apr 5, 2016 at 6:33 PM, Brandon Applegate <brandon@burn.net> wrote:
>>
>>> On Apr 5, 2016, at 9:04 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>>
>>> Does anyone know what the "quality" portion of dslreport's metric means?
>>
>> Basically - packet loss.
>>
>> https://www.dslreports.com/faq/17930
>
> Sigh. I ranted. I might rant harder.
>
> http://blog.cerowrt.org/post/bufferbloat_vs_quality/
>
>>
>> —
>> Quality Grades
>>
>> Quality refers to average detected packet loss / re-transmit percentages during download phase. The higher the packet loss / re-transmit percentage the more inefficient the connection is, and a very poor result may be indicative of congestion, inside wiring issues or other problems that need addressing.
>>
>> 1% or less - A+
>> 2.5% or less - A
>> 3% or less - B
>> 5% or less - C
>> 12% or less - D
>> over 12% - F
>> —
>>
>>
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] dslreports bufferbloat tests

[Dave Taht]

On Tue, Apr 5, 2016 at 8:19 PM, jb <justinbeech@gmail.com> wrote:
> I take your point regarding Quality

Thx! I am not grumpy at you in particular, but at a world that
continues to view packet loss as completely undesirable (I was at
several ietf meetings like that yesterday, and 2 days ago the
FCC's "nutrition labels for broadband" got a lot of press:
http://arstechnica.com/business/2016/04/fccs-nutrition-labels-for-broadband-show-speed-caps-and-hidden-fees/
)

Does anyone know how these nutrition labels are calculated?

> but both your examples in the post show A+ quality?

Well, yes... :) I did both of those with ecn on. (I did also get an A+
with ecn off)

"desirable" packet loss, varies based on the number of flows, the
targetted queuing delay, the bandwidth of the link, the tcp, and the
RTT. It rapidly drops into the low percentage points for this
particular test,for those particular parameters.

as for:

"Quality Grades

Quality refers to average detected packet loss / re-transmit
percentages during download phase. The higher the packet loss /
re-transmit percentage the more inefficient the connection is, and a
very poor result may be indicative of congestion, inside wiring issues
or other problems that need addressing.

"1% or less - A+
2.5% or less - A
3% or less - B
5% or less - C
12% or less - D
over 12% - F"

What I specifically objected to was this formula for calculating the
grade. After stewing about it a while (um, er, *years*, now), I
realized last night that with a little work, now that we know what
aqms such as pie and fq_codel can achieve, that we could, indeed, get
a desirable range of "packet loss" for X flows, Y RTT, and Z
bandwidth.

btw: Does your test have the ability to track "CE" marks? That would
be like a "gold star" affixed to the test report. (latest IoS has some
support for ecn now by default)

> I'm thinking that packet loss significant enough to show as a "C" or
> worse is mostly a bad situation

I pointed at a case where 25% packet loss was good here.

http://localhost:1313/post/rtt_fair_on_wifi/

I am tempted to build on this theme, because it is not intuitive that
desirable loss is a curve - a lot at low rates, but at higher rates,
much less loss occurs and and really high rates one loss hurts...

> even if avoiding all packet loss - by using huge buffers - is
> definitely a disaster..

Yes. 0 and major bufferbloat would be an F grade for me, for "Quality" :)

In other news, I sure wish the cable modems out there had followed
these guidelines at least.

http://www.cablelabs.com/wp-content/uploads/specdocs/CM-GL-Buffer-V01-110915.pdf

>
>
> On Wed, Apr 6, 2016 at 12:21 PM, Dave Taht <dave.taht@gmail.com> wrote:
>> On Tue, Apr 5, 2016 at 6:33 PM, Brandon Applegate <brandon@burn.net> wrote:
>>>
>>>> On Apr 5, 2016, at 9:04 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>>>
>>>> Does anyone know what the "quality" portion of dslreport's metric means?
>>>
>>> Basically - packet loss.
>>>
>>> https://www.dslreports.com/faq/17930
>>
>> Sigh. I ranted. I might rant harder.
>>
>> http://blog.cerowrt.org/post/bufferbloat_vs_quality/
>>
>>>
>>> —
>>> Quality Grades
>>>
>>> Quality refers to average detected packet loss / re-transmit percentages during download phase. The higher the packet loss / re-transmit percentage the more inefficient the connection is, and a very poor result may be indicative of congestion, inside wiring issues or other problems that need addressing.
>>>
>>> 1% or less - A+
>>> 2.5% or less - A
>>> 3% or less - B
>>> 5% or less - C
>>> 12% or less - D
>>> over 12% - F
>>> —
>>>
>>>
>>> _______________________________________________
>>> Bloat mailing list
>>> Bloat@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/bloat
>>>
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] dslreports bufferbloat tests

[Kelvin Edmison]

I think people focus on packet loss so much because the term is short, seemingly conveys a lot of info and is easy to measure. 

Given that know how to measure bloat, it strikes me that what is needed is a short marketing-style tag for bloat that can be put up against the term "packet loss". 

To kick off discussion, I'd suggest 'latency hit'. As in, man, I took a big latency hit when trying to get to zero packet loss.   If it's measured as milliseconds of delay above the minimum possible when buffers are empty, then we can start to get testers to test and publish results in both dimensions and start to drive discussion and awareness of how a little packet loss can drive your latency hit way down. 

Regards,
  Kelvin 

Sent from my iPhone

> On Apr 6, 2016, at 11:30 AM, Dave Taht <dave.taht@gmail.com> wrote:
> 
>> On Tue, Apr 5, 2016 at 8:19 PM, jb <justinbeech@gmail.com> wrote:
>> I take your point regarding Quality
> 
> Thx! I am not grumpy at you in particular, but at a world that
> continues to view packet loss as completely undesirable (I was at
> several ietf meetings like that yesterday, and 2 days ago the
> FCC's "nutrition labels for broadband" got a lot of press:
> http://arstechnica.com/business/2016/04/fccs-nutrition-labels-for-broadband-show-speed-caps-and-hidden-fees/
> )
> 
> Does anyone know how these nutrition labels are calculated?
> 
>> but both your examples in the post show A+ quality?
> 
> Well, yes... :) I did both of those with ecn on. (I did also get an A+
> with ecn off)
> 
> "desirable" packet loss, varies based on the number of flows, the
> targetted queuing delay, the bandwidth of the link, the tcp, and the
> RTT. It rapidly drops into the low percentage points for this
> particular test,for those particular parameters.
> 
> as for:
> 
> "Quality Grades
> 
> Quality refers to average detected packet loss / re-transmit
> percentages during download phase. The higher the packet loss /
> re-transmit percentage the more inefficient the connection is, and a
> very poor result may be indicative of congestion, inside wiring issues
> or other problems that need addressing.
> 
> "1% or less - A+
> 2.5% or less - A
> 3% or less - B
> 5% or less - C
> 12% or less - D
> over 12% - F"
> 
> What I specifically objected to was this formula for calculating the
> grade. After stewing about it a while (um, er, *years*, now), I
> realized last night that with a little work, now that we know what
> aqms such as pie and fq_codel can achieve, that we could, indeed, get
> a desirable range of "packet loss" for X flows, Y RTT, and Z
> bandwidth.
> 
> btw: Does your test have the ability to track "CE" marks? That would
> be like a "gold star" affixed to the test report. (latest IoS has some
> support for ecn now by default)
> 
>> I'm thinking that packet loss significant enough to show as a "C" or
>> worse is mostly a bad situation
> 
> I pointed at a case where 25% packet loss was good here.
> 
> http://localhost:1313/post/rtt_fair_on_wifi/
> 
> I am tempted to build on this theme, because it is not intuitive that
> desirable loss is a curve - a lot at low rates, but at higher rates,
> much less loss occurs and and really high rates one loss hurts...
> 
>> even if avoiding all packet loss - by using huge buffers - is
>> definitely a disaster..
> 
> Yes. 0 and major bufferbloat would be an F grade for me, for "Quality" :)
> 
> In other news, I sure wish the cable modems out there had followed
> these guidelines at least.
> 
> http://www.cablelabs.com/wp-content/uploads/specdocs/CM-GL-Buffer-V01-110915.pdf
> 
>> 
>> 
>>> On Wed, Apr 6, 2016 at 12:21 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>>> On Tue, Apr 5, 2016 at 6:33 PM, Brandon Applegate <brandon@burn.net> wrote:
>>>> 
>>>>> On Apr 5, 2016, at 9:04 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>>>> 
>>>>> Does anyone know what the "quality" portion of dslreport's metric means?
>>>> 
>>>> Basically - packet loss.
>>>> 
>>>> https://www.dslreports.com/faq/17930
>>> 
>>> Sigh. I ranted. I might rant harder.
>>> 
>>> http://blog.cerowrt.org/post/bufferbloat_vs_quality/
>>> 
>>>> 
>>>> —
>>>> Quality Grades
>>>> 
>>>> Quality refers to average detected packet loss / re-transmit percentages during download phase. The higher the packet loss / re-transmit percentage the more inefficient the connection is, and a very poor result may be indicative of congestion, inside wiring issues or other problems that need addressing.
>>>> 
>>>> 1% or less - A+
>>>> 2.5% or less - A
>>>> 3% or less - B
>>>> 5% or less - C
>>>> 12% or less - D
>>>> over 12% - F
>>>> —
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Bloat mailing list
>>>> Bloat@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/bloat
>>> _______________________________________________
>>> Bloat mailing list
>>> Bloat@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/bloat
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] dslreports bufferbloat tests

[Livingood, Jason]

On 4/6/16, 1:08 PM, "Bloat on behalf of Kelvin Edmison"
<bloat-bounces@lists.bufferbloat.net on behalf of kelvin@edmison.net>
wrote:


>I think people focus on packet loss so much because the term is short,
>seemingly conveys a lot of info and is easy to measure.
>
>Given that know how to measure bloat, it strikes me that what is needed
>is a short marketing-style tag for bloat that can be put up against the
>term "packet loss".

The BITAG has discussed this issue and debated whether to take up a work
item on it, because it is clear that it is not always the case that more
(or any) packet loss is necessarily bad. And Dave Clark and Steve Bauer
have done some good analysis basically saying packet loss measurements
don¹t really correlate the broadband quality and more work is needed.

My sincere hope is that the new FCC broadband label does not encourage
behavior that will lead to a goal of 0% loss, which would quite obviously
be bad in many ways from a performance and quality standpoint and run
counter to all the good AQM-related work.

Jason

Re: [Bloat] dslreports bufferbloat tests

[Livingood, Jason]

On 4/6/16, 1:08 PM, "Bloat on behalf of Kelvin Edmison"
<bloat-bounces@lists.bufferbloat.net on behalf of kelvin@edmison.net>
wrote:


>I think people focus on packet loss so much because the term is short,
>seemingly conveys a lot of info and is easy to measure.
>
>Given that know how to measure bloat, it strikes me that what is needed
>is a short marketing-style tag for bloat that can be put up against the
>term "packet loss".

The BITAG has discussed this issue and debated whether to take up a work
item on it, because it is clear that it is not always the case that more
(or any) packet loss is necessarily bad. And Dave Clark and Steve Bauer
have done some good analysis basically saying packet loss measurements
don¹t really correlate the broadband quality and more work is needed.

My sincere hope is that the new FCC broadband label does not encourage
behavior that will lead to a goal of 0% loss, which would quite obviously
be bad in many ways from a performance and quality standpoint and run
counter to all the good AQM-related work.

Jason

Re: [Bloat] dslreports bufferbloat tests

[jb]

I'm running experimental captures on one of the speed test servers..
once in while an IP appears to have traffic control doing something.

For example..

DONE 71.126.39.162:62753
  Packets: 272
  Data outbound: 200340
  Data inbound: 435
  3-way initial RTT 641.267061233521 ms
  RTTs 908 909 910 921 922 924 935 938 939 940 1167 1170 1169 1174
1187 1188 1186 1187 1187 1193 1205 1207 1196 1197 1201 1203 1206 1207
1213 1214 1373 1374 1375 1377 1380 1384 1396 1397 1384 1385 1392 1394
1386 1390 1393 1396 1398 1401 1408 1413 1406 1408 1408 1412 1417 1429
1429 1430 1434 1435 1435 1436 1436 1450 1450 1451 1445 1446 1456 1457
1584 1585 1595 1596 1596 1598 1597 1608 1605 1607 1610 1613 1599 1608
1608 1622 1622 1623 1623 1634 1627 1628 1628 1630 1639 1637 1634 1641
1640 1638 1628 1634 1631 1633 1638 1635 1633 1625  ms
  Timestamps not offered
  inbound TCP cwr_seen
  inbound TCP ece_seen
  inbound IP ecn_capable

So this particular snap from the first 200 packets of a download there
are poor RTTs but it also picked up the IP ecn_capable flag (but not
the IP ecn congestion flag) and it picked up the TCP ece and cwr flags
on.

Before I go and collect statistics, does anyone have any input on what
else can be combed out of the captures? the RTTs I'm measuring are
calculated (ignoring selective acks) as the gap from the data segment
to the first ack that includes it.
In this case tcp timestamps was not offered by the client, if it was,
I suppose there is more RTT information to be gotten.

I suppose I should be analysing the moving window during the transfer
as well, to pick up on what the ece/cwr is actually doing.

I could just run tcptrace on each complete dump.
but I'm trying to only isolate the most important information.

thanks
-Justin

On Fri, Apr 8, 2016 at 4:23 AM, Livingood, Jason
<Jason_Livingood@comcast.com> wrote:
> On 4/6/16, 1:08 PM, "Bloat on behalf of Kelvin Edmison"
> <bloat-bounces@lists.bufferbloat.net on behalf of kelvin@edmison.net>
> wrote:
>
>
>>I think people focus on packet loss so much because the term is short,
>>seemingly conveys a lot of info and is easy to measure.
>>
>>Given that know how to measure bloat, it strikes me that what is needed
>>is a short marketing-style tag for bloat that can be put up against the
>>term "packet loss".
>
> The BITAG has discussed this issue and debated whether to take up a work
> item on it, because it is clear that it is not always the case that more
> (or any) packet loss is necessarily bad. And Dave Clark and Steve Bauer
> have done some good analysis basically saying packet loss measurements
> don¹t really correlate the broadband quality and more work is needed.
>
> My sincere hope is that the new FCC broadband label does not encourage
> behavior that will lead to a goal of 0% loss, which would quite obviously
> be bad in many ways from a performance and quality standpoint and run
> counter to all the good AQM-related work.
>
> Jason
>

Re: [Bloat] dslreports bufferbloat tests

[Dave Taht]

if you would like me to generate a few tests from my location(s),
I could (for example) send codel, pie, fq_codel'd flows along with
"normal" behaviors....

On Tue, Apr 19, 2016 at 11:06 PM, jb <justin@dslr.net> wrote:
> I'm running experimental captures on one of the speed test servers..
> once in while an IP appears to have traffic control doing something.
>
> For example..
>
> DONE 71.126.39.162:62753
>   Packets: 272
>   Data outbound: 200340
>   Data inbound: 435
>   3-way initial RTT 641.267061233521 ms
>   RTTs 908 909 910 921 922 924 935 938 939 940 1167 1170 1169 1174
> 1187 1188 1186 1187 1187 1193 1205 1207 1196 1197 1201 1203 1206 1207
> 1213 1214 1373 1374 1375 1377 1380 1384 1396 1397 1384 1385 1392 1394
> 1386 1390 1393 1396 1398 1401 1408 1413 1406 1408 1408 1412 1417 1429
> 1429 1430 1434 1435 1435 1436 1436 1450 1450 1451 1445 1446 1456 1457
> 1584 1585 1595 1596 1596 1598 1597 1608 1605 1607 1610 1613 1599 1608
> 1608 1622 1622 1623 1623 1634 1627 1628 1628 1630 1639 1637 1634 1641
> 1640 1638 1628 1634 1631 1633 1638 1635 1633 1625  ms
>   Timestamps not offered
>   inbound TCP cwr_seen
>   inbound TCP ece_seen
>   inbound IP ecn_capable
>
> So this particular snap from the first 200 packets of a download there
> are poor RTTs but it also picked up the IP ecn_capable flag (but not
> the IP ecn congestion flag) and it picked up the TCP ece and cwr flags
> on.
>
> Before I go and collect statistics, does anyone have any input on what
> else can be combed out of the captures? the RTTs I'm measuring are
> calculated (ignoring selective acks) as the gap from the data segment
> to the first ack that includes it.
> In this case tcp timestamps was not offered by the client, if it was,
> I suppose there is more RTT information to be gotten.
>
> I suppose I should be analysing the moving window during the transfer
> as well, to pick up on what the ece/cwr is actually doing.
>
> I could just run tcptrace on each complete dump.
> but I'm trying to only isolate the most important information.
>
> thanks
> -Justin
>
> On Fri, Apr 8, 2016 at 4:23 AM, Livingood, Jason
> <Jason_Livingood@comcast.com> wrote:
>> On 4/6/16, 1:08 PM, "Bloat on behalf of Kelvin Edmison"
>> <bloat-bounces@lists.bufferbloat.net on behalf of kelvin@edmison.net>
>> wrote:
>>
>>
>>>I think people focus on packet loss so much because the term is short,
>>>seemingly conveys a lot of info and is easy to measure.
>>>
>>>Given that know how to measure bloat, it strikes me that what is needed
>>>is a short marketing-style tag for bloat that can be put up against the
>>>term "packet loss".
>>
>> The BITAG has discussed this issue and debated whether to take up a work
>> item on it, because it is clear that it is not always the case that more
>> (or any) packet loss is necessarily bad. And Dave Clark and Steve Bauer
>> have done some good analysis basically saying packet loss measurements
>> don¹t really correlate the broadband quality and more work is needed.
>>
>> My sincere hope is that the new FCC broadband label does not encourage
>> behavior that will lead to a goal of 0% loss, which would quite obviously
>> be bad in many ways from a performance and quality standpoint and run
>> counter to all the good AQM-related work.
>>
>> Jason
>>



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Bloat] dslreports bufferbloat tests

[Jonathan Morton]

> On 20 Apr, 2016, at 09:06, jb <justin@dslr.net> wrote:
> 
> So this particular snap from the first 200 packets of a download there
> are poor RTTs but it also picked up the IP ecn_capable flag (but not
> the IP ecn congestion flag) and it picked up the TCP ece and cwr flags
> on.

The TCP ECE and CWR flags are always set during ECN negotiation.  If there are no CE marks subsequently, they will not appear again once the data phase begins.  You can therefore distinguish between ECN capability of the host and the network.

For a “download” test, your sender will see ECE flags inbound and generate CWR flags outbound *during the data phase* only if there is an ECN aware AQM on the path at the bottleneck.  It will not see the CE marks themselves, as those are only present between the bottleneck and the receiver.

For an “upload" test, your receiver will see the inbound CE marks from an AQM active on the bottleneck, respond to them with ECE flags, and receive CWR flags in reply.  Again, only during the data phase.

 - Jonathan Morton

Re: [Bloat] dslreports bufferbloat tests

[jb]

thanks

And yes Dave, some logged tests (if we pre-arrange what day, and you
tell me the UTCs) using this particular server would be good, I'll be
in touch on that, I have to setup a capture that just isolates your
IP.

So Is there anything different, or detectable, on the server side for
such a fully congestion aware connection, but no congestion is
encountered?
So on upload phase, I can see CE marks come in, but during download
phase there is nothing interesting to detect and log?

On Wed, Apr 20, 2016 at 10:43 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
>
>> On 20 Apr, 2016, at 09:06, jb <justin@dslr.net> wrote:
>>
>> So this particular snap from the first 200 packets of a download there
>> are poor RTTs but it also picked up the IP ecn_capable flag (but not
>> the IP ecn congestion flag) and it picked up the TCP ece and cwr flags
>> on.
>
> The TCP ECE and CWR flags are always set during ECN negotiation.  If there are no CE marks subsequently, they will not appear again once the data phase begins.  You can therefore distinguish between ECN capability of the host and the network.
>
> For a “download” test, your sender will see ECE flags inbound and generate CWR flags outbound *during the data phase* only if there is an ECN aware AQM on the path at the bottleneck.  It will not see the CE marks themselves, as those are only present between the bottleneck and the receiver.
>
> For an “upload" test, your receiver will see the inbound CE marks from an AQM active on the bottleneck, respond to them with ECE flags, and receive CWR flags in reply.  Again, only during the data phase.
>
>  - Jonathan Morton
>

Re: [Bloat] dslreports bufferbloat tests

[Jonathan Morton]

> On 21 Apr, 2016, at 01:53, jb <justin@dslr.net> wrote:
> 
> So Is there anything different, or detectable, on the server side for
> such a fully congestion aware connection, but no congestion is
> encountered?

This would imply that your server was unable to saturate the connection.  You should be able to infer this case by comparing the measured throughput to your server’s capacity.

> So on upload phase, I can see CE marks come in, but during download
> phase there is nothing interesting to detect and log?

Since the router handling the queue for the bottleneck link will likely be different in each direction (in the simple case, one is at the ISP and the other is in the subscriber’s home), it’s entirely possible for one direction to be fully ECN-aware and AQMed at the bottleneck, and the other to be completely bloated.  In both cases, the traffic will negotiate ECN (or not) the same way.

Additionally, even if there are ECN-aware, AQMed queues on the path, it’s the bottleneck queue that counts.  It’s possible to get some sporadic CE marks on a path where the major bottleneck link is bloated, if the traffic passes through an ECN-aware pinch point which is transiently congested.

 - Jonathan Morton

Re: [Bloat] dslreports bufferbloat tests

[jb]

Regarding picking up advanced congestion management in a result, it
would be possible by adding a concurrent tcpdump on each test server -
running all the time and filtering for the appropriate bits.

But am I just looking for "ECN capable" flags originating from a given
public IP?
or am I filtering just for CE marks (11), indicating there was some
active queue management actually going on -- and only that would be
worth mentioning?

thanks


On Thu, Apr 7, 2016 at 1:30 AM, Dave Taht <dave.taht@gmail.com> wrote:
> On Tue, Apr 5, 2016 at 8:19 PM, jb <justinbeech@gmail.com> wrote:
>> I take your point regarding Quality
>
> Thx! I am not grumpy at you in particular, but at a world that
> continues to view packet loss as completely undesirable (I was at
> several ietf meetings like that yesterday, and 2 days ago the
> FCC's "nutrition labels for broadband" got a lot of press:
> http://arstechnica.com/business/2016/04/fccs-nutrition-labels-for-broadband-show-speed-caps-and-hidden-fees/
> )
>
> Does anyone know how these nutrition labels are calculated?
>
>> but both your examples in the post show A+ quality?
>
> Well, yes... :) I did both of those with ecn on. (I did also get an A+
> with ecn off)
>
> "desirable" packet loss, varies based on the number of flows, the
> targetted queuing delay, the bandwidth of the link, the tcp, and the
> RTT. It rapidly drops into the low percentage points for this
> particular test,for those particular parameters.
>
> as for:
>
> "Quality Grades
>
> Quality refers to average detected packet loss / re-transmit
> percentages during download phase. The higher the packet loss /
> re-transmit percentage the more inefficient the connection is, and a
> very poor result may be indicative of congestion, inside wiring issues
> or other problems that need addressing.
>
> "1% or less - A+
> 2.5% or less - A
> 3% or less - B
> 5% or less - C
> 12% or less - D
> over 12% - F"
>
> What I specifically objected to was this formula for calculating the
> grade. After stewing about it a while (um, er, *years*, now), I
> realized last night that with a little work, now that we know what
> aqms such as pie and fq_codel can achieve, that we could, indeed, get
> a desirable range of "packet loss" for X flows, Y RTT, and Z
> bandwidth.
>
> btw: Does your test have the ability to track "CE" marks? That would
> be like a "gold star" affixed to the test report. (latest IoS has some
> support for ecn now by default)
>
>> I'm thinking that packet loss significant enough to show as a "C" or
>> worse is mostly a bad situation
>
> I pointed at a case where 25% packet loss was good here.
>
> http://localhost:1313/post/rtt_fair_on_wifi/
>
> I am tempted to build on this theme, because it is not intuitive that
> desirable loss is a curve - a lot at low rates, but at higher rates,
> much less loss occurs and and really high rates one loss hurts...
>
>> even if avoiding all packet loss - by using huge buffers - is
>> definitely a disaster..
>
> Yes. 0 and major bufferbloat would be an F grade for me, for "Quality" :)
>
> In other news, I sure wish the cable modems out there had followed
> these guidelines at least.
>
> http://www.cablelabs.com/wp-content/uploads/specdocs/CM-GL-Buffer-V01-110915.pdf
>
>>
>>
>> On Wed, Apr 6, 2016 at 12:21 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>> On Tue, Apr 5, 2016 at 6:33 PM, Brandon Applegate <brandon@burn.net> wrote:
>>>>
>>>>> On Apr 5, 2016, at 9:04 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>>>>
>>>>> Does anyone know what the "quality" portion of dslreport's metric means?
>>>>
>>>> Basically - packet loss.
>>>>
>>>> https://www.dslreports.com/faq/17930
>>>
>>> Sigh. I ranted. I might rant harder.
>>>
>>> http://blog.cerowrt.org/post/bufferbloat_vs_quality/
>>>
>>>>
>>>> —
>>>> Quality Grades
>>>>
>>>> Quality refers to average detected packet loss / re-transmit percentages during download phase. The higher the packet loss / re-transmit percentage the more inefficient the connection is, and a very poor result may be indicative of congestion, inside wiring issues or other problems that need addressing.
>>>>
>>>> 1% or less - A+
>>>> 2.5% or less - A
>>>> 3% or less - B
>>>> 5% or less - C
>>>> 12% or less - D
>>>> over 12% - F
>>>> —
>>>>
>>>>
>>>> _______________________________________________
>>>> Bloat mailing list
>>>> Bloat@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/bloat
>>>>
>>> _______________________________________________
>>> Bloat mailing list
>>> Bloat@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] dslreports bufferbloat tests

[Jonathan Morton]

> On 7 Apr, 2016, at 05:06, jb <justin@dslr.net> wrote:
> 
> But am I just looking for "ECN capable" flags originating from a given
> public IP?
> or am I filtering just for CE marks (11), indicating there was some
> active queue management actually going on -- and only that would be
> worth mentioning?

The latter.  CE marks indicate that the bottleneck router is using AQM with ECN support.  ECT marks only indicate that the traffic supports ECN, not that it’s actually doing any good.

Beware however that one or two ISPs are known to set CE on all traffic, indiscriminately.  You should be able to detect this by looking for CE on SYN packets, which should never be there (ECN negotiation is initiated through TCP flags, not the IP TOS byte).  Report this to the user as a problem.

 - Jonathan Morton

Re: [Bloat] dslreports bufferbloat tests

[Livingood, Jason]

On 4/6/16, 12:30 PM, "Bloat on behalf of Dave Taht"
<bloat-bounces@lists.bufferbloat.net on behalf of dave.taht@gmail.com>
wrote:

>Does anyone know how these nutrition labels are calculated?

This is a tough question because AFAIK the FCC has not provided too much
guidance. But I believe we can use the data from the FCC¹s SamKnows (MBA)
panel and that is considered a ³safe harbor². So my guess is any ISP
covered by the FCC MBA platform will use the SamKnows data to be covered
by the safe harbor. But at my company not all of our tiers are covered but
the FCC MBA program, so we and others have a gap to fill.

For tiers of service (speeds) and ISPs not covered by the FCC MBA platform
then one has to develop a methodology and document it and be prepared to
defend it with the FCC. And it will need to be documented in a reference
linked to from the disclosure.

My guess, though IANAL, is that the non-MBA numbers ISPs use will be
refined over time through the back-and-forth of complaint & inquiries
until it becomes more clear what the smaller number of acceptable
measurement methodologies are, and this will take quite some time (and
perhaps be a bit messy).

We shall see!

Jason