[Bloat] Invisibility of bufferbloat and its remedies

[Bloat] Invisibility of bufferbloat and its remedies

[dpreed]

https://www.cordcuttersnews.com/3-easy-tips-to-fix-constant-buffering/

It's distressing how little the tech press understands the real problem.

Of course, cable companies like Charter and ATT who have mostly DOCSIS 2 gear deployed can't admit to their plant being bloat-causing.

In fact it protects their cable business against cord cutters.

And the solution is needed in the CMTS once neighbors all start becoming heavier users, because it is a shared buffering pool with no fairness or bloat protection.

Still, routers with queue management that reduce bloat would help a lot, if "buffering" is seen frequently under load.

So why isn't anyone talking about this problem after at least a decade of knowing it, and knowing how to fix it?

I blame IETF members, individually and collectively. If ietf exists for any reason other than as a boondoggle for world travel, it's for resolving issues like this one.

Re: [Bloat] Invisibility of bufferbloat and its remedies

[Dave Taht]

On Mon, Jun 18, 2018 at 9:26 AM, dpreed@deepplum.com
<dpreed@deepplum.com> wrote:
> https://www.cordcuttersnews.com/3-easy-tips-to-fix-constant-buffering/
>
> It's distressing how little the tech press understands the real problem.

Yea, that one is pretty sad.

It still remains a field of active academic research:

https://scholar.google.com/scholar?as_ylo=2018&q=bufferbloat&hl=en&as_sdt=0,5

>
> Of course, cable companies like Charter and ATT who have mostly DOCSIS 2 gear deployed can't admit to their plant being bloat-causing.
>
> In fact it protects their cable business against cord cutters.

Lacking competition in general, doesn't help.

What I am actually more frustrated about is the network neutrality
advocates A) conflating "buffering" with malfeasance, rather than a
technical problem
and B) Using politics rather than technology to attempt to achieve
their goals. If *only* a few prominent members of that side of affairs
"got" that some better technology, deployed, might solve some of their
problems and make the internet better for everyone, we'd make more
progress.

fq_codel is a uniquely "american" algorithm, in that it gives the
"little guy" a little boost until it achieves parity with everyone
else.

>
> And the solution is needed in the CMTS once neighbors all start becoming heavier users, because it is a shared buffering pool with no fairness or bloat protection.

My principle observation is that with the changes in traffic patterns
in the last decade, and the predominance of application-rate limited
streaming, that most
folk are merely forced into a bandwidth tier that is less rarely
annoying. This does not of course solve the corporate gateway problems
very well, nor does it truly kill it dead, but until that day when
"the right stuff" is readily available, and more informed demand
exists.

I was sad to see recently a cisco white paper that even ignored their
own work on pie.

> Still, routers with queue management that reduce bloat would help a lot, if "buffering" is seen frequently under load.
>
> So why isn't anyone talking about this problem after at least a decade of knowing it, and knowing how to fix it?
>
> I blame IETF members, individually and collectively. If ietf exists for any reason other than as a boondoggle for world travel, it's for resolving issues like this one.

Heh. I have essentially abandoned the IETF as the inmates are running
the asylum, and trying to continue to make our points there was
seemingly fruitless
- and out of my budget. I'd rather stay home and get better code out
the door. Or come up with some other set of orgs to annoy into paying
attention.

I would not mind going to another IETF meeting to give a preso (on,
say, cake), but I'm unwilling to front the funds or time anymore.


>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Bloat] Invisibility of bufferbloat and its remedies

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 6294 bytes --]


I have been one of the most prominent advocates of network neutrality. I'm constantly informing my friends and the press about "buffering" not being related to neutrality at all.
 
I think that's the best we can do.
 
Packet neutrality is actually a key part of the Internet's design, pushing control mechanisms to the edge, with a minimum of "intelligence" in the routers/switches/gateways. In particular, "content-based" and "endpoint-address-based" targeted throttling was never how the Internet Protocol layer was supposed to work. That's a fundamental result of the "end-to-end argument" applied to congestion management. I've spent a lot of time on that issue technically. The original discussions going back before Van Jacobson's early work, up to RED and ECN, all are based on providing congestion signalling sufficient to cause endpoints competing for resources to adapt their behavior cooperatively in real time, while maintaining minimal latency under load.
 
Latency under load is the crucial metric across the entire IP layer, endpoints and routers. It's clear that minimizing latency under load has to be achieved by avoiding buffering insite the network, moving it to the source (and destination).
 
I've given this lecture to policy people a lot. In fact, deliberate creation of "bloat" is a technical strategy that has been used in the past to destroy VoIP and other real-time communicaitons. Microsoft was caught doing it decades ago, as were some other conflicted communicaitons providers. They could selectively delay small packets using DPI, while letting FTPs get full speed. That's one of the reasons we coined the idea of Network Neutrality.
 
But radical right wingers of the sort that blossom in the paranoid world of the dark net started arguing that the routers should have political freedom to do whatever they damn well pleased with packets, because routers are people just like corporations, and a "free market" is the solution to everything.
 
Well, technically, the Internet doesn't work if their is not some mechanism for eliminiating lag under load by eliminating queueing delay in bottleneck nodes.
 
That's ultimately what Network Neutrality is about.  There's a lot of other crap being pushed by folks who pile on to the Network Neutrality discussion. People want to "fix prices" for example, arguing that profits are bad. Those guys are not the problem.
 
The problem is that the vertically integrated monopolists want to claim that the Internet should be subject to Deep Packet Inspection at every router, designed to charge rents based on content of the packets and who is the original sender or destination of the packet - that is, charging Netflix or NBC Universal packets nothing, and charging IPFS packets 100x as much.
 
So, no, the Network Neutrality people are NOT the problem with Bufferbloat.
 
Comcast, on the other hand, has been slow-rolling DOCSIS 3.0, because their customers on DOCSIS 2.0 are just ordering faster service tiers to overcome the Bufferbloat in their DOCSIS 2 CMTS's.
-----Original Message-----
From: "Dave Taht" <dave.taht@gmail.com>
Sent: Monday, June 18, 2018 3:07pm
To: "dpreed@deepplum.com" <dpreed@deepplum.com>
Cc: cerowrt-devel@lists.bufferbloat.net, "bloat" <bloat@lists.bufferbloat.net>
Subject: Re: Invisibility of bufferbloat and its remedies



On Mon, Jun 18, 2018 at 9:26 AM, dpreed@deepplum.com
<dpreed@deepplum.com> wrote:
> https://www.cordcuttersnews.com/3-easy-tips-to-fix-constant-buffering/
>
> It's distressing how little the tech press understands the real problem.

Yea, that one is pretty sad.

It still remains a field of active academic research:

https://scholar.google.com/scholar?as_ylo=2018&q=bufferbloat&hl=en&as_sdt=0,5

>
> Of course, cable companies like Charter and ATT who have mostly DOCSIS 2 gear deployed can't admit to their plant being bloat-causing.
>
> In fact it protects their cable business against cord cutters.

Lacking competition in general, doesn't help.

What I am actually more frustrated about is the network neutrality
advocates A) conflating "buffering" with malfeasance, rather than a
technical problem
and B) Using politics rather than technology to attempt to achieve
their goals. If *only* a few prominent members of that side of affairs
"got" that some better technology, deployed, might solve some of their
problems and make the internet better for everyone, we'd make more
progress.

fq_codel is a uniquely "american" algorithm, in that it gives the
"little guy" a little boost until it achieves parity with everyone
else.

>
> And the solution is needed in the CMTS once neighbors all start becoming heavier users, because it is a shared buffering pool with no fairness or bloat protection.

My principle observation is that with the changes in traffic patterns
in the last decade, and the predominance of application-rate limited
streaming, that most
folk are merely forced into a bandwidth tier that is less rarely
annoying. This does not of course solve the corporate gateway problems
very well, nor does it truly kill it dead, but until that day when
"the right stuff" is readily available, and more informed demand
exists.

I was sad to see recently a cisco white paper that even ignored their
own work on pie.

> Still, routers with queue management that reduce bloat would help a lot, if "buffering" is seen frequently under load.
>
> So why isn't anyone talking about this problem after at least a decade of knowing it, and knowing how to fix it?
>
> I blame IETF members, individually and collectively. If ietf exists for any reason other than as a boondoggle for world travel, it's for resolving issues like this one.

Heh. I have essentially abandoned the IETF as the inmates are running
the asylum, and trying to continue to make our points there was
seemingly fruitless
- and out of my budget. I'd rather stay home and get better code out
the door. Or come up with some other set of orgs to annoy into paying
attention.

I would not mind going to another IETF meeting to give a preso (on,
say, cake), but I'm unwilling to front the funds or time anymore.


>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

[-- Attachment #2: Type: text/html, Size: 8984 bytes --]

Re: [Bloat] Invisibility of bufferbloat and its remedies

[Jonathan Morton]

> On 19 Jun, 2018, at 1:43 am, dpreed@deepplum.com wrote:
> 
> So, no, the Network Neutrality people are NOT the problem with Bufferbloat.

No, but I think it's fair to point towards corporate greed and political ignorance as common causes of both problems.

 - Jonathan Morton

Re: [Bloat] Invisibility of bufferbloat and its remedies

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 2106 bytes --]


No doubt.
 
However, sometimes one can get powerful economic forces to  support good ideas.
 
The entire telecom industry was going after the Internet as a concept fiercely in the days of "dialup" Internet access. Trying to get the government to allow them to price it out of existence, trying to argue that ISDN couldn't be deployed, trying to argue that "selective content" (AOL) was better than the dangerous open Internet full of kiddy porn and drug cartels, whereas Ma Bell et al. would deliver clean and wholesome content only, if the government would just allow them to build the National Inofmration Superhighway the way it "should be engineered".
 
Yet, the Internet community routed around all of this, by showing hugely valuable new ideas that were available instantly, and a vibrant ecosystem of innovators working for users, not for the big companies.
 
It's still reasonable to continute that path.  But it is worth remembering that when Venture Capital joined in, things started to go awry. @Home (done by Milo Medin - funded by Kleiner Perkins, now at Google) was conceived as a closed, walled garden, instituting "web caching" that was supposedly "good for users", whie at the same time breaking the WWW protocols needed for evolution of the Internet, and instituting systematic port blocking that prevented anyone from creating servers.
 
But Medin and Kleiner were failures, not getting that openness was at the core of the Internet.
 
-----Original Message-----
From: "Jonathan Morton" <chromatix99@gmail.com>
Sent: Monday, June 18, 2018 7:17pm
To: "dpreed@deepplum.com" <dpreed@deepplum.com>
Cc: "Dave Taht" <dave.taht@gmail.com>, cerowrt-devel@lists.bufferbloat.net, "bloat" <bloat@lists.bufferbloat.net>
Subject: Re: [Bloat] Invisibility of bufferbloat and its remedies



> On 19 Jun, 2018, at 1:43 am, dpreed@deepplum.com wrote:
> 
> So, no, the Network Neutrality people are NOT the problem with Bufferbloat.

No, but I think it's fair to point towards corporate greed and political ignorance as common causes of both problems.

 - Jonathan Morton


[-- Attachment #2: Type: text/html, Size: 3627 bytes --]

Re: [Bloat] Invisibility of bufferbloat and its remedies

[Dave Taht]

I will be in D.C., presenting https://www.cs.kau.se/tohojo/cake/ - next week.

If there are people to see, asses to kick or kiss, I'm up for it, let
me know. Presently I just plan to give my talk and get the heck out of
dodge.

One of cake's "minor" features is the *perfect* defeat of the htb
based shaper in cable modems. If you know the set-rate on the modem,
you
just set it to the same thing and get vastly superior performance to
docsis 3.1, pie, or the sqm-scripts.

On Mon, Jun 18, 2018 at 3:43 PM, dpreed@deepplum.com
<dpreed@deepplum.com> wrote:
> I have been one of the most prominent advocates of network neutrality. I'm
> constantly informing my friends and the press about "buffering" not being
> related to neutrality at all.
>
>
>
> I think that's the best we can do.
>
>
>
> Packet neutrality is actually a key part of the Internet's design, pushing
> control mechanisms to the edge, with a minimum of "intelligence" in the
> routers/switches/gateways. In particular, "content-based" and
> "endpoint-address-based" targeted throttling was never how the Internet
> Protocol layer was supposed to work. That's a fundamental result of the
> "end-to-end argument" applied to congestion management. I've spent a lot of
> time on that issue technically. The original discussions going back before
> Van Jacobson's early work, up to RED and ECN, all are based on providing
> congestion signalling sufficient to cause endpoints competing for resources
> to adapt their behavior cooperatively in real time, while maintaining
> minimal latency under load.
>
>
>
> Latency under load is the crucial metric across the entire IP layer,
> endpoints and routers. It's clear that minimizing latency under load has to
> be achieved by avoiding buffering insite the network, moving it to the
> source (and destination).
>
>
>
> I've given this lecture to policy people a lot. In fact, deliberate creation
> of "bloat" is a technical strategy that has been used in the past to destroy
> VoIP and other real-time communicaitons. Microsoft was caught doing it
> decades ago, as were some other conflicted communicaitons providers. They
> could selectively delay small packets using DPI, while letting FTPs get full
> speed. That's one of the reasons we coined the idea of Network Neutrality.
>
>
>
> But radical right wingers of the sort that blossom in the paranoid world of
> the dark net started arguing that the routers should have political freedom
> to do whatever they damn well pleased with packets, because routers are
> people just like corporations, and a "free market" is the solution to
> everything.
>
>
>
> Well, technically, the Internet doesn't work if their is not some mechanism
> for eliminiating lag under load by eliminating queueing delay in bottleneck
> nodes.
>
>
>
> That's ultimately what Network Neutrality is about.  There's a lot of other
> crap being pushed by folks who pile on to the Network Neutrality discussion.
> People want to "fix prices" for example, arguing that profits are bad. Those
> guys are not the problem.
>
>
>
> The problem is that the vertically integrated monopolists want to claim that
> the Internet should be subject to Deep Packet Inspection at every router,
> designed to charge rents based on content of the packets and who is the
> original sender or destination of the packet - that is, charging Netflix or
> NBC Universal packets nothing, and charging IPFS packets 100x as much.
>
>
>
> So, no, the Network Neutrality people are NOT the problem with Bufferbloat.
>
>
>
> Comcast, on the other hand, has been slow-rolling DOCSIS 3.0, because their
> customers on DOCSIS 2.0 are just ordering faster service tiers to overcome
> the Bufferbloat in their DOCSIS 2 CMTS's.
>
> -----Original Message-----
> From: "Dave Taht" <dave.taht@gmail.com>
> Sent: Monday, June 18, 2018 3:07pm
> To: "dpreed@deepplum.com" <dpreed@deepplum.com>
> Cc: cerowrt-devel@lists.bufferbloat.net, "bloat"
> <bloat@lists.bufferbloat.net>
> Subject: Re: Invisibility of bufferbloat and its remedies
>
> On Mon, Jun 18, 2018 at 9:26 AM, dpreed@deepplum.com
> <dpreed@deepplum.com> wrote:
>> https://www.cordcuttersnews.com/3-easy-tips-to-fix-constant-buffering/
>>
>> It's distressing how little the tech press understands the real problem.
>
> Yea, that one is pretty sad.
>
> It still remains a field of active academic research:
>
> https://scholar.google.com/scholar?as_ylo=2018&q=bufferbloat&hl=en&as_sdt=0,5
>
>>
>> Of course, cable companies like Charter and ATT who have mostly DOCSIS 2
>> gear deployed can't admit to their plant being bloat-causing.
>>
>> In fact it protects their cable business against cord cutters.
>
> Lacking competition in general, doesn't help.
>
> What I am actually more frustrated about is the network neutrality
> advocates A) conflating "buffering" with malfeasance, rather than a
> technical problem
> and B) Using politics rather than technology to attempt to achieve
> their goals. If *only* a few prominent members of that side of affairs
> "got" that some better technology, deployed, might solve some of their
> problems and make the internet better for everyone, we'd make more
> progress.
>
> fq_codel is a uniquely "american" algorithm, in that it gives the
> "little guy" a little boost until it achieves parity with everyone
> else.
>
>>
>> And the solution is needed in the CMTS once neighbors all start becoming
>> heavier users, because it is a shared buffering pool with no fairness or
>> bloat protection.
>
> My principle observation is that with the changes in traffic patterns
> in the last decade, and the predominance of application-rate limited
> streaming, that most
> folk are merely forced into a bandwidth tier that is less rarely
> annoying. This does not of course solve the corporate gateway problems
> very well, nor does it truly kill it dead, but until that day when
> "the right stuff" is readily available, and more informed demand
> exists.
>
> I was sad to see recently a cisco white paper that even ignored their
> own work on pie.
>
>> Still, routers with queue management that reduce bloat would help a lot,
>> if "buffering" is seen frequently under load.
>>
>> So why isn't anyone talking about this problem after at least a decade of
>> knowing it, and knowing how to fix it?
>>
>> I blame IETF members, individually and collectively. If ietf exists for
>> any reason other than as a boondoggle for world travel, it's for resolving
>> issues like this one.
>
> Heh. I have essentially abandoned the IETF as the inmates are running
> the asylum, and trying to continue to make our points there was
> seemingly fruitless
> - and out of my budget. I'd rather stay home and get better code out
> the door. Or come up with some other set of orgs to annoy into paying
> attention.
>
> I would not mind going to another IETF meeting to give a preso (on,
> say, cake), but I'm unwilling to front the funds or time anymore.
>
>
>>
>
>
>
> --
>
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Bloat] Invisibility of bufferbloat and its remedies

[Dave Taht]

Well, I ranted: http://blog.cerowrt.org/post/net_neutrality_isps/

I am thinking a string of shorter pieces aimed directly at customers,
reviewers, politicians, etc might do a bit more good than the
gargantuan things jim tends to do.

On Mon, Jun 18, 2018 at 6:46 PM, Dave Taht <dave.taht@gmail.com> wrote:
> I will be in D.C., presenting https://www.cs.kau.se/tohojo/cake/ - next week.
>
> If there are people to see, asses to kick or kiss, I'm up for it, let
> me know. Presently I just plan to give my talk and get the heck out of
> dodge.
>
> One of cake's "minor" features is the *perfect* defeat of the htb
> based shaper in cable modems. If you know the set-rate on the modem,
> you
> just set it to the same thing and get vastly superior performance to
> docsis 3.1, pie, or the sqm-scripts.
>
> On Mon, Jun 18, 2018 at 3:43 PM, dpreed@deepplum.com
> <dpreed@deepplum.com> wrote:
>> I have been one of the most prominent advocates of network neutrality. I'm
>> constantly informing my friends and the press about "buffering" not being
>> related to neutrality at all.
>>
>>
>>
>> I think that's the best we can do.
>>
>>
>>
>> Packet neutrality is actually a key part of the Internet's design, pushing
>> control mechanisms to the edge, with a minimum of "intelligence" in the
>> routers/switches/gateways. In particular, "content-based" and
>> "endpoint-address-based" targeted throttling was never how the Internet
>> Protocol layer was supposed to work. That's a fundamental result of the
>> "end-to-end argument" applied to congestion management. I've spent a lot of
>> time on that issue technically. The original discussions going back before
>> Van Jacobson's early work, up to RED and ECN, all are based on providing
>> congestion signalling sufficient to cause endpoints competing for resources
>> to adapt their behavior cooperatively in real time, while maintaining
>> minimal latency under load.
>>
>>
>>
>> Latency under load is the crucial metric across the entire IP layer,
>> endpoints and routers. It's clear that minimizing latency under load has to
>> be achieved by avoiding buffering insite the network, moving it to the
>> source (and destination).
>>
>>
>>
>> I've given this lecture to policy people a lot. In fact, deliberate creation
>> of "bloat" is a technical strategy that has been used in the past to destroy
>> VoIP and other real-time communicaitons. Microsoft was caught doing it
>> decades ago, as were some other conflicted communicaitons providers. They
>> could selectively delay small packets using DPI, while letting FTPs get full
>> speed. That's one of the reasons we coined the idea of Network Neutrality.
>>
>>
>>
>> But radical right wingers of the sort that blossom in the paranoid world of
>> the dark net started arguing that the routers should have political freedom
>> to do whatever they damn well pleased with packets, because routers are
>> people just like corporations, and a "free market" is the solution to
>> everything.
>>
>>
>>
>> Well, technically, the Internet doesn't work if their is not some mechanism
>> for eliminiating lag under load by eliminating queueing delay in bottleneck
>> nodes.
>>
>>
>>
>> That's ultimately what Network Neutrality is about.  There's a lot of other
>> crap being pushed by folks who pile on to the Network Neutrality discussion.
>> People want to "fix prices" for example, arguing that profits are bad. Those
>> guys are not the problem.
>>
>>
>>
>> The problem is that the vertically integrated monopolists want to claim that
>> the Internet should be subject to Deep Packet Inspection at every router,
>> designed to charge rents based on content of the packets and who is the
>> original sender or destination of the packet - that is, charging Netflix or
>> NBC Universal packets nothing, and charging IPFS packets 100x as much.
>>
>>
>>
>> So, no, the Network Neutrality people are NOT the problem with Bufferbloat.
>>
>>
>>
>> Comcast, on the other hand, has been slow-rolling DOCSIS 3.0, because their
>> customers on DOCSIS 2.0 are just ordering faster service tiers to overcome
>> the Bufferbloat in their DOCSIS 2 CMTS's.
>>
>> -----Original Message-----
>> From: "Dave Taht" <dave.taht@gmail.com>
>> Sent: Monday, June 18, 2018 3:07pm
>> To: "dpreed@deepplum.com" <dpreed@deepplum.com>
>> Cc: cerowrt-devel@lists.bufferbloat.net, "bloat"
>> <bloat@lists.bufferbloat.net>
>> Subject: Re: Invisibility of bufferbloat and its remedies
>>
>> On Mon, Jun 18, 2018 at 9:26 AM, dpreed@deepplum.com
>> <dpreed@deepplum.com> wrote:
>>> https://www.cordcuttersnews.com/3-easy-tips-to-fix-constant-buffering/
>>>
>>> It's distressing how little the tech press understands the real problem.
>>
>> Yea, that one is pretty sad.
>>
>> It still remains a field of active academic research:
>>
>> https://scholar.google.com/scholar?as_ylo=2018&q=bufferbloat&hl=en&as_sdt=0,5
>>
>>>
>>> Of course, cable companies like Charter and ATT who have mostly DOCSIS 2
>>> gear deployed can't admit to their plant being bloat-causing.
>>>
>>> In fact it protects their cable business against cord cutters.
>>
>> Lacking competition in general, doesn't help.
>>
>> What I am actually more frustrated about is the network neutrality
>> advocates A) conflating "buffering" with malfeasance, rather than a
>> technical problem
>> and B) Using politics rather than technology to attempt to achieve
>> their goals. If *only* a few prominent members of that side of affairs
>> "got" that some better technology, deployed, might solve some of their
>> problems and make the internet better for everyone, we'd make more
>> progress.
>>
>> fq_codel is a uniquely "american" algorithm, in that it gives the
>> "little guy" a little boost until it achieves parity with everyone
>> else.
>>
>>>
>>> And the solution is needed in the CMTS once neighbors all start becoming
>>> heavier users, because it is a shared buffering pool with no fairness or
>>> bloat protection.
>>
>> My principle observation is that with the changes in traffic patterns
>> in the last decade, and the predominance of application-rate limited
>> streaming, that most
>> folk are merely forced into a bandwidth tier that is less rarely
>> annoying. This does not of course solve the corporate gateway problems
>> very well, nor does it truly kill it dead, but until that day when
>> "the right stuff" is readily available, and more informed demand
>> exists.
>>
>> I was sad to see recently a cisco white paper that even ignored their
>> own work on pie.
>>
>>> Still, routers with queue management that reduce bloat would help a lot,
>>> if "buffering" is seen frequently under load.
>>>
>>> So why isn't anyone talking about this problem after at least a decade of
>>> knowing it, and knowing how to fix it?
>>>
>>> I blame IETF members, individually and collectively. If ietf exists for
>>> any reason other than as a boondoggle for world travel, it's for resolving
>>> issues like this one.
>>
>> Heh. I have essentially abandoned the IETF as the inmates are running
>> the asylum, and trying to continue to make our points there was
>> seemingly fruitless
>> - and out of my budget. I'd rather stay home and get better code out
>> the door. Or come up with some other set of orgs to annoy into paying
>> attention.
>>
>> I would not mind going to another IETF meeting to give a preso (on,
>> say, cake), but I'm unwilling to front the funds or time anymore.
>>
>>
>>>
>>
>>
>>
>> --
>>
>> Dave Täht
>> CEO, TekLibre, LLC
>> http://www.teklibre.com
>> Tel: 1-669-226-2619
>
>
>
> --
>
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Bloat] Invisibility of bufferbloat and its remedies

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 8221 bytes --]


good rant!
 
-----Original Message-----
From: "Dave Taht" <dave.taht@gmail.com>
Sent: Tuesday, June 19, 2018 3:33pm
To: "dpreed@deepplum.com" <dpreed@deepplum.com>
Cc: cerowrt-devel@lists.bufferbloat.net, "bloat" <bloat@lists.bufferbloat.net>
Subject: Re: Invisibility of bufferbloat and its remedies



Well, I ranted: http://blog.cerowrt.org/post/net_neutrality_isps/

I am thinking a string of shorter pieces aimed directly at customers,
reviewers, politicians, etc might do a bit more good than the
gargantuan things jim tends to do.

On Mon, Jun 18, 2018 at 6:46 PM, Dave Taht <dave.taht@gmail.com> wrote:
> I will be in D.C., presenting https://www.cs.kau.se/tohojo/cake/ - next week.
>
> If there are people to see, asses to kick or kiss, I'm up for it, let
> me know. Presently I just plan to give my talk and get the heck out of
> dodge.
>
> One of cake's "minor" features is the *perfect* defeat of the htb
> based shaper in cable modems. If you know the set-rate on the modem,
> you
> just set it to the same thing and get vastly superior performance to
> docsis 3.1, pie, or the sqm-scripts.
>
> On Mon, Jun 18, 2018 at 3:43 PM, dpreed@deepplum.com
> <dpreed@deepplum.com> wrote:
>> I have been one of the most prominent advocates of network neutrality. I'm
>> constantly informing my friends and the press about "buffering" not being
>> related to neutrality at all.
>>
>>
>>
>> I think that's the best we can do.
>>
>>
>>
>> Packet neutrality is actually a key part of the Internet's design, pushing
>> control mechanisms to the edge, with a minimum of "intelligence" in the
>> routers/switches/gateways. In particular, "content-based" and
>> "endpoint-address-based" targeted throttling was never how the Internet
>> Protocol layer was supposed to work. That's a fundamental result of the
>> "end-to-end argument" applied to congestion management. I've spent a lot of
>> time on that issue technically. The original discussions going back before
>> Van Jacobson's early work, up to RED and ECN, all are based on providing
>> congestion signalling sufficient to cause endpoints competing for resources
>> to adapt their behavior cooperatively in real time, while maintaining
>> minimal latency under load.
>>
>>
>>
>> Latency under load is the crucial metric across the entire IP layer,
>> endpoints and routers. It's clear that minimizing latency under load has to
>> be achieved by avoiding buffering insite the network, moving it to the
>> source (and destination).
>>
>>
>>
>> I've given this lecture to policy people a lot. In fact, deliberate creation
>> of "bloat" is a technical strategy that has been used in the past to destroy
>> VoIP and other real-time communicaitons. Microsoft was caught doing it
>> decades ago, as were some other conflicted communicaitons providers. They
>> could selectively delay small packets using DPI, while letting FTPs get full
>> speed. That's one of the reasons we coined the idea of Network Neutrality.
>>
>>
>>
>> But radical right wingers of the sort that blossom in the paranoid world of
>> the dark net started arguing that the routers should have political freedom
>> to do whatever they damn well pleased with packets, because routers are
>> people just like corporations, and a "free market" is the solution to
>> everything.
>>
>>
>>
>> Well, technically, the Internet doesn't work if their is not some mechanism
>> for eliminiating lag under load by eliminating queueing delay in bottleneck
>> nodes.
>>
>>
>>
>> That's ultimately what Network Neutrality is about. There's a lot of other
>> crap being pushed by folks who pile on to the Network Neutrality discussion.
>> People want to "fix prices" for example, arguing that profits are bad. Those
>> guys are not the problem.
>>
>>
>>
>> The problem is that the vertically integrated monopolists want to claim that
>> the Internet should be subject to Deep Packet Inspection at every router,
>> designed to charge rents based on content of the packets and who is the
>> original sender or destination of the packet - that is, charging Netflix or
>> NBC Universal packets nothing, and charging IPFS packets 100x as much.
>>
>>
>>
>> So, no, the Network Neutrality people are NOT the problem with Bufferbloat.
>>
>>
>>
>> Comcast, on the other hand, has been slow-rolling DOCSIS 3.0, because their
>> customers on DOCSIS 2.0 are just ordering faster service tiers to overcome
>> the Bufferbloat in their DOCSIS 2 CMTS's.
>>
>> -----Original Message-----
>> From: "Dave Taht" <dave.taht@gmail.com>
>> Sent: Monday, June 18, 2018 3:07pm
>> To: "dpreed@deepplum.com" <dpreed@deepplum.com>
>> Cc: cerowrt-devel@lists.bufferbloat.net, "bloat"
>> <bloat@lists.bufferbloat.net>
>> Subject: Re: Invisibility of bufferbloat and its remedies
>>
>> On Mon, Jun 18, 2018 at 9:26 AM, dpreed@deepplum.com
>> <dpreed@deepplum.com> wrote:
>>> https://www.cordcuttersnews.com/3-easy-tips-to-fix-constant-buffering/
>>>
>>> It's distressing how little the tech press understands the real problem.
>>
>> Yea, that one is pretty sad.
>>
>> It still remains a field of active academic research:
>>
>> https://scholar.google.com/scholar?as_ylo=2018&q=bufferbloat&hl=en&as_sdt=0,5
>>
>>>
>>> Of course, cable companies like Charter and ATT who have mostly DOCSIS 2
>>> gear deployed can't admit to their plant being bloat-causing.
>>>
>>> In fact it protects their cable business against cord cutters.
>>
>> Lacking competition in general, doesn't help.
>>
>> What I am actually more frustrated about is the network neutrality
>> advocates A) conflating "buffering" with malfeasance, rather than a
>> technical problem
>> and B) Using politics rather than technology to attempt to achieve
>> their goals. If *only* a few prominent members of that side of affairs
>> "got" that some better technology, deployed, might solve some of their
>> problems and make the internet better for everyone, we'd make more
>> progress.
>>
>> fq_codel is a uniquely "american" algorithm, in that it gives the
>> "little guy" a little boost until it achieves parity with everyone
>> else.
>>
>>>
>>> And the solution is needed in the CMTS once neighbors all start becoming
>>> heavier users, because it is a shared buffering pool with no fairness or
>>> bloat protection.
>>
>> My principle observation is that with the changes in traffic patterns
>> in the last decade, and the predominance of application-rate limited
>> streaming, that most
>> folk are merely forced into a bandwidth tier that is less rarely
>> annoying. This does not of course solve the corporate gateway problems
>> very well, nor does it truly kill it dead, but until that day when
>> "the right stuff" is readily available, and more informed demand
>> exists.
>>
>> I was sad to see recently a cisco white paper that even ignored their
>> own work on pie.
>>
>>> Still, routers with queue management that reduce bloat would help a lot,
>>> if "buffering" is seen frequently under load.
>>>
>>> So why isn't anyone talking about this problem after at least a decade of
>>> knowing it, and knowing how to fix it?
>>>
>>> I blame IETF members, individually and collectively. If ietf exists for
>>> any reason other than as a boondoggle for world travel, it's for resolving
>>> issues like this one.
>>
>> Heh. I have essentially abandoned the IETF as the inmates are running
>> the asylum, and trying to continue to make our points there was
>> seemingly fruitless
>> - and out of my budget. I'd rather stay home and get better code out
>> the door. Or come up with some other set of orgs to annoy into paying
>> attention.
>>
>> I would not mind going to another IETF meeting to give a preso (on,
>> say, cake), but I'm unwilling to front the funds or time anymore.
>>
>>
>>>
>>
>>
>>
>> --
>>
>> Dave Täht
>> CEO, TekLibre, LLC
>> http://www.teklibre.com
>> Tel: 1-669-226-2619
>
>
>
> --
>
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

[-- Attachment #2: Type: text/html, Size: 10715 bytes --]

Re: [Bloat] [Cerowrt-devel] Invisibility of bufferbloat and its remedies

[valdis.kletnieks]

On Mon, 18 Jun 2018 18:46:18 -0700, Dave Taht said:

> One of cake's "minor" features is the *perfect* defeat of the htb
> based shaper in cable modems. If you know the set-rate on the modem,
> you just set it to the same thing and get vastly superior performance to
> docsis 3.1, pie, or the sqm-scripts.

Do we have a good cookbook on how to determine the set-rate?

Re: [Bloat] [Cerowrt-devel] Invisibility of bufferbloat and its remedies

[Sebastian Moeller]

Well,

On June 19, 2018 10:34:07 PM GMT+02:00, valdis.kletnieks@vt.edu wrote:
>On Mon, 18 Jun 2018 18:46:18 -0700, Dave Taht said:
>
>> One of cake's "minor" features is the *perfect* defeat of the htb
>> based shaper in cable modems. If you know the set-rate on the modem,
>> you just set it to the same thing and get vastly superior performance
>to
>> docsis 3.1, pie, or the sqm-scripts.
>
>Do we have a good cookbook on how to determine the set-rate?

With cable <DOCSIS 3.1 one can Snoop the management frames that will also carry the bandwidth definitions. Or with a compromised cable modem one could dump the DOCSIS config file that also contains that information. Or ask the ISP. Short of any of that one could run a umber of speedtests over a 24 hour period and simply extrapolate from the measured goodput to the required 'gross' DOCSIS shaper rate:

Measured TCP/ipv4 goodput * ((1518)/(1500-20-20)) = lower bound gross bandwidth estimate

DOCSIS employs a shaper to limit user's exceeding the contracted rates, that per DOCSIS standard assumes 1518 bytes of accountable raw frame size. Tangent that 1518 obviously is a 'lie' in that it does not include all per packet overhead in use on a DOCSIS carrier, but from an end-user perspective that difference is immaterial...

I realize that you probably wanted something simpler and more accurate, sorry to disappoint.

Best Regards
        Sebastian



>_______________________________________________
>Cerowrt-devel mailing list
>Cerowrt-devel@lists.bufferbloat.net
>https://lists.bufferbloat.net/listinfo/cerowrt-devel

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[Bloat] (no subject)

[Sebastian Moeller]

Hi all,


> On Jun 20, 2018, at 01:32, Sebastian Moeller <moeller0@gmx.de> wrote:
> 
> Well,
> 
> On June 19, 2018 10:34:07 PM GMT+02:00, valdis.kletnieks@vt.edu wrote:
>> On Mon, 18 Jun 2018 18:46:18 -0700, Dave Taht said:
>> 
>>> One of cake's "minor" features is the *perfect* defeat of the htb
>>> based shaper in cable modems. If you know the set-rate on the modem,
>>> you just set it to the same thing and get vastly superior performance
>> to
>>> docsis 3.1, pie, or the sqm-scripts.
>> 
>> Do we have a good cookbook on how to determine the set-rate?
> 
> With cable <DOCSIS 3.1 one can Snoop the management frames that will also carry the bandwidth definitions. Or with a compromised cable modem one could dump the DOCSIS config file that also contains that information. Or ask the ISP. Short of any of that one could run a umber of speedtests over a 24 hour period and simply extrapolate from the measured goodput to the required 'gross' DOCSIS shaper rate:
> 
> Measured TCP/ipv4 goodput * ((1518)/(1500-20-20)) = lower bound gross bandwidth estimate

Addendum: when running speedtests on cable for the purpose of estimating the "true" docsis shaper goodput one needs to take care to not be fooled by transient bandwidth allowances like powerboost but rather one needs to find the sustainable stable  maximum bandwidth; so fun all around. (And one needs to account for the correct overhead in the equation above so in the IPv6 case the (1500-20-20) needs to be replaced by (1500-40-20))

Best Regards
	Sebastian


> 
> DOCSIS employs a shaper to limit user's exceeding the contracted rates, that per DOCSIS standard assumes 1518 bytes of accountable raw frame size. Tangent that 1518 obviously is a 'lie' in that it does not include all per packet overhead in use on a DOCSIS carrier, but from an end-user perspective that difference is immaterial...
> 
> I realize that you probably wanted something simpler and more accurate, sorry to disappoint.
> 
> Best Regards
>        Sebastian
> 
> 
> 
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 
> -- 
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Bloat] (no subject)

[Kathleen Nichols]

On 6/19/18 11:56 PM, Sebastian Moeller wrote:

> 
> Addendum: when running speedtests on cable for the purpose of estimating the "true" docsis shaper goodput one needs to take care to not be fooled by transient bandwidth allowances like powerboost but rather one needs to find the sustainable stable  maximum bandwidth; so fun all around. (And one needs to account for the correct overhead in the equation above so in the IPv6 case the (1500-20-20) needs to be replaced by (1500-40-20))
> 

This is a good argument for *monitoring* the link rather than
snapshotting it.

	Kathie

Re: [Bloat] (no subject)

[Kathleen Nichols]

On 6/19/18 11:56 PM, Sebastian Moeller wrote:

> Addendum: when running speedtests on cable for the purpose of
> estimating the "true" docsis shaper goodput one needs to take care to
> not be fooled by transient bandwidth allowances like powerboost but
> rather one needs to find the sustainable stable  maximum bandwidth;
> so fun all around. (And one needs to account for the correct overhead
> in the equation above so in the IPv6 case the (1500-20-20) needs to
> be replaced by (1500-40-20))


This is a good argument for *monitoring* the link rather than
snapshotting it.

	Kathie

Re: [Bloat] [Cerowrt-devel] Invisibility of bufferbloat and its remedies

[Jonathan Morton]

> On 19 Jun, 2018, at 11:34 pm, valdis.kletnieks@vt.edu wrote:
> 
> Do we have a good cookbook on how to determine the set-rate?

On DSL, the sync rates in each direction should usually be readable from the modem; they are typically reported on the router's status page.  The advertised rate is less reliable, to say the least.

On wireless links, all bets are off - even with stationary endpoints, link capacity varies wildly over time.  This needs to be solved in the radio-modems.

If it's wifi, however, a link-rate-independent solution now exists for certain hardware, and there's nothing theoretically stopping something similar being put into future HardMAC implementations.  If we get the choice of hardware, naturally we choose wisely.

 - Jonathan Morton

Re: [Bloat] [Cerowrt-devel] Invisibility of bufferbloat and its remedies

[Sebastian Moeller]

On June 20, 2018 1:41:19 AM GMT+02:00, Jonathan Morton <chromatix99@gmail.com> wrote:
>> On 19 Jun, 2018, at 11:34 pm, valdis.kletnieks@vt.edu wrote:
>> 
>> Do we have a good cookbook on how to determine the set-rate?
>
>On DSL, the sync rates in each direction should usually be readable
>from the modem; they are typically reported on the router's status
>page.  The advertised rate is less reliable, to say the least.

Well many ISPs nowadays employ additional traffic shapers upstream of the dslam/MSAN, and it is this shaper's set-rate we would need to have. All we know is that this needs to be <= sync but the details differ. Also we need information about per packet overhead which again is not easily available for end-users. IMHO our best bet would be to get regulatory agancies to force ISPs to make this information easily available to end-users so that customers can actually compare different offers realistically, but I am not holding my breath for this to happen in my lifetime...

Best Regards
         Sebastian

>
>On wireless links, all bets are off - even with stationary endpoints,
>link capacity varies wildly over time.  This needs to be solved in the
>radio-modems.
>
>If it's wifi, however, a link-rate-independent solution now exists for
>certain hardware, and there's nothing theoretically stopping something
>similar being put into future HardMAC implementations.  If we get the
>choice of hardware, naturally we choose wisely.
>
> - Jonathan Morton
>
>_______________________________________________
>Bloat mailing list
>Bloat@lists.bufferbloat.net
>https://lists.bufferbloat.net/listinfo/bloat

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [Bloat] [Cerowrt-devel] Invisibility of bufferbloat and its remedies

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 877 bytes --]



> On 20 Jun 2018, at 00:41, Jonathan Morton <chromatix99@gmail.com> wrote:
> 
>> On 19 Jun, 2018, at 11:34 pm, valdis.kletnieks@vt.edu wrote:
>> 
>> Do we have a good cookbook on how to determine the set-rate?
> 
> On DSL, the sync rates in each direction should usually be readable from the modem; they are typically reported on the router's status page.  The advertised rate is less reliable, to say the least.

I’ve been experimenting with this hack in sqm-scripts to do just that https://github.com/ldir-EDB0/sqm-scripts/commit/a71ab1c9acd75d6e9254360e832ac7b6f9514e88 originally on my parents DGN3500 and currently on my on BT HomeHub5a test line.

It fails to work if the ISP does rate banding (BT 20CN does this, BT 21CN doesn’t) where downstream is limited to a rate below downstream sync rate.  I guess a lookup table could be implemented.

Kevin

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Bloat] [Cerowrt-devel] Invisibility of bufferbloat and its remedies

[Sebastian Moeller]

Hi Kevin,


> On Jun 20, 2018, at 09:12, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
> 
> 
> 
>> On 20 Jun 2018, at 00:41, Jonathan Morton <chromatix99@gmail.com> wrote:
>> 
>>> On 19 Jun, 2018, at 11:34 pm, valdis.kletnieks@vt.edu wrote:
>>> 
>>> Do we have a good cookbook on how to determine the set-rate?
>> 
>> On DSL, the sync rates in each direction should usually be readable from the modem; they are typically reported on the router's status page.  The advertised rate is less reliable, to say the least.
> 
> I’ve been experimenting with this hack in sqm-scripts to do just that https://github.com/ldir-EDB0/sqm-scripts/commit/a71ab1c9acd75d6e9254360e832ac7b6f9514e88 originally on my parents DGN3500 and currently on my on BT HomeHub5a test line.

	Clever (I see how you chiseled data_rates() into shape here, respect)! Even though I believe that this is not pppoa specific and should probably check whether /lib/functions/lantiq_dsl.sh exists. Actually this code will also work on VDSL2 links... (and we should also be able to extract the encapsulation atm or ptm).

if [ "pppoa-wan" = "$IFACE" ]; then
	if [ -f "/lib/functions/lantiq_dsl.sh" ] : then
		...
	fi
fi

But would it not be simpler to call /etc/init.d/dsl_control status | grep -e "Data Rate:" or somesuch? Cureently openwrt only supports lantiq/intel modems, but if broadcom modems should ever be supported I venture a guess they will not use /lib/functions/lantiq_dsl.sh to generate the stats output ;) (not that there is a guarantee that dsl_control would exost and generate compatible output).


But I believe that this is not that helpful as a mode to automatically set the bandwidth*, as I assume that most ISPs will shape the downstream bandwidth upstream of the DSLAM (if just to avoid having a DDOS against one user taking down the whole DSLAM). In my case my ISP even shapes the upstream, which is somewhat more puzzling. It looks like a cool way to deal with variable sync (either after a re-sync due to say DLM action or due to SRA) so how about polishing this a bit and including this as pure informational line in the log? 


*) if this is to be made automatic by say allowing to scrape bandwidth from the modem we would need additional setting for setting the shaper percentage. I wonder whether all of this is worth it though, given that the number of users running sqm on devices in control of the dsl-modem is going to be miniscule, no?

> 
> It fails to work if the ISP does rate banding (BT 20CN does this, BT 21CN doesn’t) where downstream is limited to a rate below downstream sync rate.  I guess a lookup table could be implemented.

	I predict that a lookup table is going to be constantly out of date, especially since at least my ISP is a moving target in both the shaper settings as well as the actual overheads (on the plus side they started to send the applicable net bandwidth as part of the pppoe negotiations (but failed to document how those rates are actually to be interpreted ;) win some loose some)). 


Final thought, how about just using this on the luci side to give hints about the sync bandwidth in the GUI (like displaying the value of either sync for xdsl or the speed for ethernet devices (speed in ethtool parlance, so 10Mb/s, 100Mb/s, ...)) that will not be as smooth as your solution, but should also be more robust against doing the wrmg thing automatically? Or am I overcomplicating things again.

Final final thought ;) since lantig_dsl.sh is quite openwrt specific, maybe we should do the automatic mode inside of the equally openwrt specific /usr/lib/sqm/run instead of in start-sqm that will might also be executed on other platforms?


Best Regards
	Sebastian

> 
> Kevin
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Cerowrt-devel] Invisibility of bufferbloat and its remedies

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 4622 bytes --]



> On 20 Jun 2018, at 09:07, Sebastian Moeller <moeller0@gmx.de> wrote:
> 
> Hi Kevin,
> 
> 
>> On Jun 20, 2018, at 09:12, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
>> 
>> 
>> 
>>> On 20 Jun 2018, at 00:41, Jonathan Morton <chromatix99@gmail.com> wrote:
>>> 
>>>> On 19 Jun, 2018, at 11:34 pm, valdis.kletnieks@vt.edu wrote:
>>>> 
>>>> Do we have a good cookbook on how to determine the set-rate?
>>> 
>>> On DSL, the sync rates in each direction should usually be readable from the modem; they are typically reported on the router's status page.  The advertised rate is less reliable, to say the least.
>> 
>> I’ve been experimenting with this hack in sqm-scripts to do just that https://github.com/ldir-EDB0/sqm-scripts/commit/a71ab1c9acd75d6e9254360e832ac7b6f9514e88 originally on my parents DGN3500 and currently on my on BT HomeHub5a test line.
> 
> 	Clever (I see how you chiseled data_rates() into shape here, respect)! Even though I believe that this is not pppoa specific and should probably check whether /lib/functions/lantiq_dsl.sh exists. Actually this code will also work on VDSL2 links... (and we should also be able to extract the encapsulation atm or ptm).
> 
> if [ "pppoa-wan" = "$IFACE" ]; then
> 	if [ -f "/lib/functions/lantiq_dsl.sh" ] : then
> 		...
> 	fi
> fi
> 
> But would it not be simpler to call /etc/init.d/dsl_control status | grep -e "Data Rate:" or somesuch? Cureently openwrt only supports lantiq/intel modems, but if broadcom modems should ever be supported I venture a guess they will not use /lib/functions/lantiq_dsl.sh to generate the stats output ;) (not that there is a guarantee that dsl_control would exost and generate compatible output).
> 
> 
> But I believe that this is not that helpful as a mode to automatically set the bandwidth*, as I assume that most ISPs will shape the downstream bandwidth upstream of the DSLAM (if just to avoid having a DDOS against one user taking down the whole DSLAM). In my case my ISP even shapes the upstream, which is somewhat more puzzling. It looks like a cool way to deal with variable sync (either after a re-sync due to say DLM action or due to SRA) so how about polishing this a bit and including this as pure informational line in the log?
> 
> 
> *) if this is to be made automatic by say allowing to scrape bandwidth from the modem we would need additional setting for setting the shaper percentage. I wonder whether all of this is worth it though, given that the number of users running sqm on devices in control of the dsl-modem is going to be miniscule, no?
> 
>> 
>> It fails to work if the ISP does rate banding (BT 20CN does this, BT 21CN doesn’t) where downstream is limited to a rate below downstream sync rate.  I guess a lookup table could be implemented.
> 
> 	I predict that a lookup table is going to be constantly out of date, especially since at least my ISP is a moving target in both the shaper settings as well as the actual overheads (on the plus side they started to send the applicable net bandwidth as part of the pppoe negotiations (but failed to document how those rates are actually to be interpreted ;) win some loose some)).
> 
> 
> Final thought, how about just using this on the luci side to give hints about the sync bandwidth in the GUI (like displaying the value of either sync for xdsl or the speed for ethernet devices (speed in ethtool parlance, so 10Mb/s, 100Mb/s, ...)) that will not be as smooth as your solution, but should also be more robust against doing the wrmg thing automatically? Or am I overcomplicating things again.

What part of ‘hack’ didn’t you understand? ;-)

Luci is a ‘bad’ idea in that setting it there effectively makes the rate fixed again…. and line resyncs do not remain static, so it needs to be a hotplug type solution.

The commit isn’t perfect…it was never meant to make it into the wild…I explored 2 years or so ago to see if something like that could be done but was foiled by a BT 20CN banding plan.  But the seed of an idea is there.  To grow it needs watering by someone who can code.

Incidentally luci sqm scripts typo (and possibly worse/not intended) "Create log file for this SQM instance under /var/run/sqm/${Inerface_name}.debug.log. Make sure to delete log files manually.”

‘Inerface_name’. - and I’m not sure if that isn’t supposed to be expanded to the actual interface name in use - I tried fixing the typo but when the name expansion still didn’t work and I had no idea how to fix that…. I got distracted by something else :-)





[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Bloat] [Cerowrt-devel] Invisibility of bufferbloat and its remedies

[Sebastian Moeller]

Hi Kevin,


> On Jun 20, 2018, at 11:15, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
> 
> 
> 
>> On 20 Jun 2018, at 09:07, Sebastian Moeller <moeller0@gmx.de> wrote:
>> 
>> Hi Kevin,
>> 
>> 
>>> On Jun 20, 2018, at 09:12, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
>>> 
>>> 
>>> 
>>>> On 20 Jun 2018, at 00:41, Jonathan Morton <chromatix99@gmail.com> wrote:
>>>> 
>>>>> On 19 Jun, 2018, at 11:34 pm, valdis.kletnieks@vt.edu wrote:
>>>>> 
>>>>> Do we have a good cookbook on how to determine the set-rate?
>>>> 
>>>> On DSL, the sync rates in each direction should usually be readable from the modem; they are typically reported on the router's status page.  The advertised rate is less reliable, to say the least.
>>> 
>>> I’ve been experimenting with this hack in sqm-scripts to do just that https://github.com/ldir-EDB0/sqm-scripts/commit/a71ab1c9acd75d6e9254360e832ac7b6f9514e88 originally on my parents DGN3500 and currently on my on BT HomeHub5a test line.
>> 
>> 	Clever (I see how you chiseled data_rates() into shape here, respect)! Even though I believe that this is not pppoa specific and should probably check whether /lib/functions/lantiq_dsl.sh exists. Actually this code will also work on VDSL2 links... (and we should also be able to extract the encapsulation atm or ptm).
>> 
>> if [ "pppoa-wan" = "$IFACE" ]; then
>> 	if [ -f "/lib/functions/lantiq_dsl.sh" ] : then
>> 		...
>> 	fi
>> fi
>> 
>> But would it not be simpler to call /etc/init.d/dsl_control status | grep -e "Data Rate:" or somesuch? Cureently openwrt only supports lantiq/intel modems, but if broadcom modems should ever be supported I venture a guess they will not use /lib/functions/lantiq_dsl.sh to generate the stats output ;) (not that there is a guarantee that dsl_control would exost and generate compatible output).
>> 
>> 
>> But I believe that this is not that helpful as a mode to automatically set the bandwidth*, as I assume that most ISPs will shape the downstream bandwidth upstream of the DSLAM (if just to avoid having a DDOS against one user taking down the whole DSLAM). In my case my ISP even shapes the upstream, which is somewhat more puzzling. It looks like a cool way to deal with variable sync (either after a re-sync due to say DLM action or due to SRA) so how about polishing this a bit and including this as pure informational line in the log?
>> 
>> 
>> *) if this is to be made automatic by say allowing to scrape bandwidth from the modem we would need additional setting for setting the shaper percentage. I wonder whether all of this is worth it though, given that the number of users running sqm on devices in control of the dsl-modem is going to be miniscule, no?
>> 
>>> 
>>> It fails to work if the ISP does rate banding (BT 20CN does this, BT 21CN doesn’t) where downstream is limited to a rate below downstream sync rate.  I guess a lookup table could be implemented.
>> 
>> 	I predict that a lookup table is going to be constantly out of date, especially since at least my ISP is a moving target in both the shaper settings as well as the actual overheads (on the plus side they started to send the applicable net bandwidth as part of the pppoe negotiations (but failed to document how those rates are actually to be interpreted ;) win some loose some)).
>> 
>> 
>> Final thought, how about just using this on the luci side to give hints about the sync bandwidth in the GUI (like displaying the value of either sync for xdsl or the speed for ethernet devices (speed in ethtool parlance, so 10Mb/s, 100Mb/s, ...)) that will not be as smooth as your solution, but should also be more robust against doing the wrmg thing automatically? Or am I overcomplicating things again.
> 
> What part of ‘hack’ didn’t you understand? ;-)

	I guess your hack have a higher quality than my release patches ;)


> 
> Luci is a ‘bad’ idea in that setting it there effectively makes the rate fixed again….

	Sure.


> and line resyncs do not remain static, so it needs to be a hotplug type solution.

	from my experience sync rates are typically quite robust, not on the kbit but close enough given the typical reduction in bandwidth for the shaper to be active. But here we are just introducing DLM, so I guess I have a somewhat naive and optimistic view based on my ISPs conservative profiles and enforced high SNR-margins... 

> 
> The commit isn’t perfect…it was never meant to make it into the wild…I explored 2 years or so ago to see if something like that could be done but was foiled by a BT 20CN banding plan.  But the seed of an idea is there.  To grow it needs watering by someone who can code.

	Nah, there are always folks that will fix the implementation (whom am I kidding, it typically is Toke who picks up the slack ;) ) the challenge is to get the algorithm right. Between your UK link(s?) and my single DTAG link we might be able to come up with something. The issue remains though that this still leaves the overhead to be accounted for and that the number of users that will have a chance to exercise this feature will be small.
On the other hand maybe changing the GUI to set bandwidth and shaper-percentage as two independent values might be more in line with our recommendations (set bandwidth to 90% of the contracted bandwidth), what do you think?


> 
> Incidentally luci sqm scripts typo (and possibly worse/not intended) "Create log file for this SQM instance under /var/run/sqm/${Inerface_name}.debug.log. Make sure to delete log files manually.”

	Oops, thanks that is my typo, fixes and pushed upstream.

> 
> ‘Inerface_name’. - and I’m not sure if that isn’t supposed to be expanded to the actual interface name in use - I tried fixing the typo but when the name expansion still didn’t work and I had no idea how to fix that…. I got distracted by something else :-)

	No, that is not supposed to auto expand, I just happen to like the shell syntax to denote that something is a variable...

Best Regards
	Sebastian


> 
> 
> 
> 

Re: [Bloat] [Cerowrt-devel] Invisibility of bufferbloat and its remedies

[Jan Ceuleers]

On 20/06/18 01:41, Jonathan Morton wrote:
>> On 19 Jun, 2018, at 11:34 pm, valdis.kletnieks@vt.edu wrote:
>>
>> Do we have a good cookbook on how to determine the set-rate?
> 
> On DSL, the sync rates in each direction should usually be readable from the modem; they are typically reported on the router's status page.  The advertised rate is less reliable, to say the least.

I'm afraid that my DSL modem doesn't belong to the "usually" case. I
have a BBox3 from Proximus (Belgium). Proximus buys these from two
manufacturers: Technicolor and Sagemcom. I have the Sagemcom variant.

Proximus went out of their way to avoid that end users be able to query
the modem for its connection parameters. There is still an indirect way
to do so in the Sagemcom variant (and so I know that my modem provides
50Mbit/s down and 10Mbit/s up) but for marketing reasons Proxinus would
prefer its users not to know. The reason is that their principal
competitor, cable operator Telenet (owned by Liberty Global) would win
the game based on raw link speed.

So whereas I have access to the information I need to be able to
configure my queues there are others who do not, and it would be good
for a generic method to exist that derives the correct parameters from
measurements.

Kathy put it more succinctly: there is a good case for monitoring.
Question is what to monitor and how to set/modify the qdisc parameters.

Jan

Re: [Bloat] [Cerowrt-devel] Invisibility of bufferbloat and its remedies

[Sebastian Moeller]

Hi Jan,


> On Jun 20, 2018, at 17:57, Jan Ceuleers <jan.ceuleers@gmail.com> wrote:
> 
> On 20/06/18 01:41, Jonathan Morton wrote:
>>> On 19 Jun, 2018, at 11:34 pm, valdis.kletnieks@vt.edu wrote:
>>> 
>>> Do we have a good cookbook on how to determine the set-rate?
>> 
>> On DSL, the sync rates in each direction should usually be readable from the modem; they are typically reported on the router's status page.  The advertised rate is less reliable, to say the least.
> 
> I'm afraid that my DSL modem doesn't belong to the "usually" case. I
> have a BBox3 from Proximus (Belgium). Proximus buys these from two
> manufacturers: Technicolor and Sagemcom. I have the Sagemcom variant.

	I believe the ITU standards actually mandate modems to supply a specific set of parameters upon query, maybe an snmp query on the modem' sLAN side might reveal something? (I fear that all the standards mandate is that the CPE tells the dslam and that communication might be shielded from endusers; I still hope that european regulators will at one point require ISPs to reveal all standard mandated values, but I am not holding my breath...)


> 
> Proximus went out of their way to avoid that end users be able to query
> the modem for its connection parameters. There is still an indirect way
> to do so in the Sagemcom variant (and so I know that my modem provides
> 50Mbit/s down and 10Mbit/s up) but for marketing reasons Proxinus would
> prefer its users not to know. The reason is that their principal
> competitor, cable operator Telenet (owned by Liberty Global) would win
> the game based on raw link speed.

	But they probably also win on user-measurable goodput and most likely have done so for at least a decade and still customers did not abandon DSL, so this logic, while probably true from the ISPs POV, is also flawed.


> 
> So whereas I have access to the information I need to be able to
> configure my queues there are others who do not, and it would be good
> for a generic method to exist that derives the correct parameters from
> measurements.


	Sure, I agree, as measuring will also take care of any potential shapers the ISP might employ upstream of the dslam. But it raises the question how this measurement can be done without requiring dedicated reference points somewhere in the network, which in turn means the actual measurement should only generate minimal traffic itself.

> 
> Kathy put it more succinctly: there is a good case for monitoring.

	Well that is exactly what Kevin proposed, his approach will query the modem about the sync rates whenever his device hotplugs (and on his line any change of sync rate is accompanied by a re-sync as his ISP does not user seamless rate adaptation/SRA I believe, so the hotplug approach will reget the actual sync values whenever a change might have occurred). Nice solution, but way to specialized to help a noticeable number of people, let alone making a dent into the problem.

> Question is what to monitor and how to set/modify the qdisc parameters.

	The what seems easy, bandwidth and latency ;) the how escapes me...

Best Regards
	Sebastian

> 
> Jan
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Cerowrt-devel] Invisibility of bufferbloat and its remedies

[Michael Richardson]

[-- Attachment #1: Type: text/plain, Size: 1780 bytes --]


dpreed@deepplum.com <dpreed@deepplum.com> wrote:
    > I blame IETF members, individually and collectively. If ietf exists for
    > any reason other than as a boondoggle for world travel, it's for
    > resolving issues like this one.

Slightly fair.

A thing that I tried to get ISOC's "deploy360" to do was to create a wall of
shame on bufferbloat five or eight years ago.  It was before the AQM WG did
any work, I think.  I wanted ISOC to collect the data, because I wanted it
visible at the CTO level.

At the very very least, I wanted a series of curated links to statements from
various equipment vendors about the problem.
I.e. www.example.com/bufferbloat.html for example in
{cisco,juniper,huawei,calix,...}

I asked many of my IETF contacts at these places if they knew who at their
company was dealing with it.  I also asked salespeople that I dealt with,
hoping to put pressure the other way.  I got nowhere.

I wanted, at the very least, to get them to acknowledge that there was a
problem, even if they didn't have an estimate on a solution, at least I could
point the salesperson at the page on their site, and say, "I'll buy when you
get me a delivery date".
A major client of mine lost three large (VoIP) customers because of hidden
bufferbloat in Bell Canada's LAN extension service... which "never lost any
packets", the sales people explained.  It was at a time when we
weren't quite using the term bufferbloat, and we didn't quite know how to
measure it in convincing ways.

I still think that this is a good idea.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        | network architect  [
]     mcr@sandelman.ca  http://www.sandelman.ca/        |   ruby on rails    [



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 464 bytes --]

Re: [Bloat] [Cerowrt-devel] Invisibility of bufferbloat and its remedies

[Dave Taht]

From a ton of memes at: https://me.me/t/buffering

"Dear youtube:

I can deal with ads
I can deal with buffer
But when ads buffer, I suffer"

and innumerable others, some NSFW

https://me.me/i/16133421

On Mon, Jun 18, 2018 at 1:59 PM, Michael Richardson <mcr@sandelman.ca> wrote:
>
> dpreed@deepplum.com <dpreed@deepplum.com> wrote:
>     > I blame IETF members, individually and collectively. If ietf exists for
>     > any reason other than as a boondoggle for world travel, it's for
>     > resolving issues like this one.
>
> Slightly fair.
>
> A thing that I tried to get ISOC's "deploy360" to do was to create a wall of
> shame on bufferbloat five or eight years ago.  It was before the AQM WG did
> any work, I think.  I wanted ISOC to collect the data, because I wanted it
> visible at the CTO level.
>
> At the very very least, I wanted a series of curated links to statements from
> various equipment vendors about the problem.
> I.e. www.example.com/bufferbloat.html for example in
> {cisco,juniper,huawei,calix,...}
>
> I asked many of my IETF contacts at these places if they knew who at their
> company was dealing with it.  I also asked salespeople that I dealt with,
> hoping to put pressure the other way.  I got nowhere.
>
> I wanted, at the very least, to get them to acknowledge that there was a
> problem, even if they didn't have an estimate on a solution, at least I could
> point the salesperson at the page on their site, and say, "I'll buy when you
> get me a delivery date".
> A major client of mine lost three large (VoIP) customers because of hidden
> bufferbloat in Bell Canada's LAN extension service... which "never lost any
> packets", the sales people explained.  It was at a time when we
> weren't quite using the term bufferbloat, and we didn't quite know how to
> measure it in convincing ways.
>
> I still think that this is a good idea.
>
> --
> ]               Never tell me the odds!                 | ipv6 mesh networks [
> ]   Michael Richardson, Sandelman Software Works        | network architect  [
> ]     mcr@sandelman.ca  http://www.sandelman.ca/        |   ruby on rails    [
>
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619