Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[David Collier-Brown]

dpreed@reed.com wrote
> Fixing buffer bloat allows the edge- and enterprise- networks to be more efficient,
> whereas not fixing it lets the edge networks move users up to more and more 
> expensive "plans" due to frustration and to sell much more gear into Enterprises 
> because they are easy marks for complex gadgets.

The new, snazzier Bell gear Bell flogged to my former across-the-street
neighbour is far more affected by bloat than the older, supposedly
slower stuff he had before. He promptly went back to "faster and cheaper".

The gaming world is also reputedly aware of some new offerings that are
great disappointments. They're sensitive to response time.

--dave
-- 
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
davecb@spamcop.net           |                      -- Mark Twain

[Bloat] fq_codel is two years old

[Rich Brown]

Folks,

I just noticed that the announcement for the first testable implementation of fq_codel was two days ago today - 14 May 2012. Build 3.3.6-2 was the first to include working fq_codel. https://lists.bufferbloat.net/pipermail/cerowrt-devel/2012-May/000233.html

Two years later, we see fq_codel being adopted lots of places. As more and more people/organizations come to understand the problem, and how straightforward the solution can be, we're beginning to win the battle to have a good Internet experience everywhere.

Thanks to Dave, Eric, Kathie, Van, and all the members of this list for their perseverance, testing, comments, and patience. Congratulations!

Best regards,

Rich

Re: [Bloat] fq_codel is two years old

[Kathleen Nichols]

Thanks, Rich.

And to show you what an amazing bit of work that first fq_codel was, I have
on my calendar that I first "exposed" CoDel to a small group in a
meeting room
and on the phone at ISC on April 24. It was really amazing to me to watch
something Van and I had been discussing (okay, arguing) about privately for
6 months and I'd been tinkering with be turned into real code on real
networks.
Jim Gettys is an incredible (and constructive) nagger, Eric Dumazet and
amazing
coder, and the entire open source community a really nifty group of folks.

Maybe someday we will actually update the first article with some of the
stuff
we got into the last internet draft....

	be the change,
		Kathie

On 5/14/14 2:01 PM, Rich Brown wrote:
> Folks,
> 
> I just noticed that the announcement for the first testable
> implementation of fq_codel was two days ago today - 14 May 2012.
> Build 3.3.6-2 was the first to include working fq_codel.
> https://lists.bufferbloat.net/pipermail/cerowrt-devel/2012-May/000233.html
>
>  Two years later, we see fq_codel being adopted lots of places. As
> more and more people/organizations come to understand the problem,
> and how straightforward the solution can be, we're beginning to win
> the battle to have a good Internet experience everywhere.
> 
> Thanks to Dave, Eric, Kathie, Van, and all the members of this list
> for their perseverance, testing, comments, and patience.
> Congratulations!
> 
> Best regards,
> 
> Rich _______________________________________________ Bloat mailing
> list Bloat@lists.bufferbloat.net 
> https://lists.bufferbloat.net/listinfo/bloat
> 

Re: [Bloat] fq_codel is two years old

[Dave Taht]

On Wed, May 14, 2014 at 3:32 PM, Kathleen Nichols <nichols@pollere.com> wrote:
>
> Thanks, Rich.
>
> And to show you what an amazing bit of work that first fq_codel was, I have
> on my calendar that I first "exposed" CoDel to a small group in a
> meeting room
> and on the phone at ISC on April 24.

And we had all sorts of trouble with the phone, (eric didn't hear
much) and we then spent hours and hours afterwards discussing wifi
instead of codel... it was too much to take in...

me, I'd started jumping up and down in excitement about 20 minutes
into kathies preso...

May 3rd, 2012 was the last 24 hr coding stint I think I'll ever have.

https://lists.bufferbloat.net/pipermail/codel/2012-May/000023.html

Ahh, the good ole days, when bufferbloat was first solved and we all
thought the internet would speed up overnight, and we were going to be
rock stars, invited to all the best parties, acquire fame and fortune,
and be awarded medals and given awards. Re-reading all this brought
back memories.... (heck, there's still a couple good ideas in that
thread left unimplemented)

https://lists.bufferbloat.net/pipermail/codel/2012-May/thread.html

It looks by may 5th we were getting in shape, and then there were a
few other issues along the way with the control law and so on... and
eric rewrote the whole thing and made it oodles faster and then as
best as I recall came up with fq_codel on saturday may 5th(?) -

Ah, I haven't had so much fun in in years. My life since then seems
like an endless string of meetings, politics, and bugfixing.

The code went from sim/paper, to code, to testing, to mainline linux
in another week. I wish more research went like that!

commit 76e3cc126bb223013a6b9a0e2a51238d1ef2e409
Author: Eric Dumazet <edumazet@google.com>
Date:   Thu May 10 07:51:25 2012 +0000

    codel: Controlled Delay AQM

Now, as I recall the story, eric came up with fq_codel on a saturday
afternoon, so I guess that was may 5th - cinco de mayo!

And that too, landed in mainline...

commit 4b549a2ef4bef9965d97cbd992ba67930cd3e0fe
Author: Eric Dumazet <edumazet@google.com>
Date:   Fri May 11 09:30:50 2012 +0000

    fq_codel: Fair Queue Codel AQM

let's not forget tom herbert & BQL

commit 75957ba36c05b979701e9ec64b37819adc12f830
Author: Tom Herbert <therbert@google.com>
Date:   Mon Nov 28 16:32:35 2011 +0000

    dql: Dynamic queue limits

    Implementation of dynamic queue limits (dql).  This is a libary which
    allows a queue limit to be dynamically managed.  The goal of dql is
    to set the queue limit, number of objects to the queue, to be minimized
    without allowing the queue to be starved.




> It was really amazing to me to watch
> something Van and I had been discussing (okay, arguing) about privately for
> 6 months and I'd been tinkering with be turned into real code on real
> networks.
> Jim Gettys is an incredible (and constructive) nagger, Eric Dumazet and
> amazing
> coder, and the entire open source community a really nifty group of folks.
>
> Maybe someday we will actually update the first article with some of the
> stuff
> we got into the last internet draft....
>
>         be the change,
>                 Kathie
>
> On 5/14/14 2:01 PM, Rich Brown wrote:
>> Folks,
>>
>> I just noticed that the announcement for the first testable
>> implementation of fq_codel was two days ago today - 14 May 2012.
>> Build 3.3.6-2 was the first to include working fq_codel.
>> https://lists.bufferbloat.net/pipermail/cerowrt-devel/2012-May/000233.html
>>
>>  Two years later, we see fq_codel being adopted lots of places. As
>> more and more people/organizations come to understand the problem,
>> and how straightforward the solution can be, we're beginning to win
>> the battle to have a good Internet experience everywhere.
>>
>> Thanks to Dave, Eric, Kathie, Van, and all the members of this list
>> for their perseverance, testing, comments, and patience.
>> Congratulations!
>>
>> Best regards,
>>
>> Rich _______________________________________________ Bloat mailing
>> list Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Dave Täht

NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 6278 bytes --]


Well done.  I'm optimistic for deployment everywhere, except CMTS's, the LTE and HSPA+ access networks, and all corporate firewall and intranet gear.
 
The solution du jour is to leave bufferbloat in place, while using the real fads: prioritization (diffserv once we have the "fast lanes" fully legal) and "software defined networking" appliances that use DPI for packet routing and traffic management.
 
Fixing buffer bloat allows the edge- and enterprise- networks to be more efficient, whereas not fixing it lets the edge networks move users up to more and more expensive "plans" due to frustration and to sell much more gear into Enterprises because they are easy marks for complex gadgets.
 
But maybe a few engineers who operate and design gear for such networks will overcome the incredible business biases against fixing this.
 
That's why all the efforts you guys have put forth are immensely worth it.  I think this is one of the best innovations in recent years (Bram Cohen's original BitTorrent is another, for fully decentralizing data delivery for the very first time in a brilliant way.) I will certainly push everywhere I can to see fq_codel deployed.
 
If there were a prize for brilliant projects, this would be top on my list.
 


On Wednesday, May 14, 2014 9:25pm, "Dave Taht" <dave.taht@gmail.com> said:



> On Wed, May 14, 2014 at 3:32 PM, Kathleen Nichols <nichols@pollere.com>
> wrote:
> >
> > Thanks, Rich.
> >
> > And to show you what an amazing bit of work that first fq_codel was, I have
> > on my calendar that I first "exposed" CoDel to a small group in a
> > meeting room
> > and on the phone at ISC on April 24.
> 
> And we had all sorts of trouble with the phone, (eric didn't hear
> much) and we then spent hours and hours afterwards discussing wifi
> instead of codel... it was too much to take in...
> 
> me, I'd started jumping up and down in excitement about 20 minutes
> into kathies preso...
> 
> May 3rd, 2012 was the last 24 hr coding stint I think I'll ever have.
> 
> https://lists.bufferbloat.net/pipermail/codel/2012-May/000023.html
> 
> Ahh, the good ole days, when bufferbloat was first solved and we all
> thought the internet would speed up overnight, and we were going to be
> rock stars, invited to all the best parties, acquire fame and fortune,
> and be awarded medals and given awards. Re-reading all this brought
> back memories.... (heck, there's still a couple good ideas in that
> thread left unimplemented)
> 
> https://lists.bufferbloat.net/pipermail/codel/2012-May/thread.html
> 
> It looks by may 5th we were getting in shape, and then there were a
> few other issues along the way with the control law and so on... and
> eric rewrote the whole thing and made it oodles faster and then as
> best as I recall came up with fq_codel on saturday may 5th(?) -
> 
> Ah, I haven't had so much fun in in years. My life since then seems
> like an endless string of meetings, politics, and bugfixing.
> 
> The code went from sim/paper, to code, to testing, to mainline linux
> in another week. I wish more research went like that!
> 
> commit 76e3cc126bb223013a6b9a0e2a51238d1ef2e409
> Author: Eric Dumazet <edumazet@google.com>
> Date:   Thu May 10 07:51:25 2012 +0000
> 
>     codel: Controlled Delay AQM
> 
> Now, as I recall the story, eric came up with fq_codel on a saturday
> afternoon, so I guess that was may 5th - cinco de mayo!
> 
> And that too, landed in mainline...
> 
> commit 4b549a2ef4bef9965d97cbd992ba67930cd3e0fe
> Author: Eric Dumazet <edumazet@google.com>
> Date:   Fri May 11 09:30:50 2012 +0000
> 
>     fq_codel: Fair Queue Codel AQM
> 
> let's not forget tom herbert & BQL
> 
> commit 75957ba36c05b979701e9ec64b37819adc12f830
> Author: Tom Herbert <therbert@google.com>
> Date:   Mon Nov 28 16:32:35 2011 +0000
> 
>     dql: Dynamic queue limits
> 
>     Implementation of dynamic queue limits (dql).  This is a libary which
>     allows a queue limit to be dynamically managed.  The goal of dql is
>     to set the queue limit, number of objects to the queue, to be minimized
>     without allowing the queue to be starved.
> 
> 
> 
> 
> > It was really amazing to me to watch
> > something Van and I had been discussing (okay, arguing) about privately for
> > 6 months and I'd been tinkering with be turned into real code on real
> > networks.
> > Jim Gettys is an incredible (and constructive) nagger, Eric Dumazet and
> > amazing
> > coder, and the entire open source community a really nifty group of folks.
> >
> > Maybe someday we will actually update the first article with some of the
> > stuff
> > we got into the last internet draft....
> >
> >         be the change,
> >                 Kathie
> >
> > On 5/14/14 2:01 PM, Rich Brown wrote:
> >> Folks,
> >>
> >> I just noticed that the announcement for the first testable
> >> implementation of fq_codel was two days ago today - 14 May 2012.
> >> Build 3.3.6-2 was the first to include working fq_codel.
> >>
> https://lists.bufferbloat.net/pipermail/cerowrt-devel/2012-May/000233.html
> >>
> >>  Two years later, we see fq_codel being adopted lots of places. As
> >> more and more people/organizations come to understand the problem,
> >> and how straightforward the solution can be, we're beginning to win
> >> the battle to have a good Internet experience everywhere.
> >>
> >> Thanks to Dave, Eric, Kathie, Van, and all the members of this list
> >> for their perseverance, testing, comments, and patience.
> >> Congratulations!
> >>
> >> Best regards,
> >>
> >> Rich _______________________________________________ Bloat mailing
> >> list Bloat@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/bloat
> >>
> >
> > _______________________________________________
> > Bloat mailing list
> > Bloat@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat
> 
> 
> 
> --
> Dave Täht
> 
> NSFW:
> https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>

[-- Attachment #2: Type: text/html, Size: 8139 bytes --]

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[Kathleen Nichols]

Gosh, that's high praise. And what's really neat is that this was such a
team effort
where we don't even necessarily know each other! What's perhaps bad is that
this was a "volunteer" effort, though that also is a strength. I'm not
sure the
answer is for everyone to work for Google.

On 5/15/14 6:47 AM, dpreed@reed.com wrote:
...
> 
> The solution du jour is to leave bufferbloat in place, while using the
> real fads: prioritization (diffserv once we have the "fast lanes" fully
> legal) and "software defined networking" appliances that use DPI for
> packet routing and traffic management.
>

I think diffserv could be used for good instead of evil.

>  
> 
> Fixing buffer bloat allows the edge- and enterprise- networks to be more
> efficient, whereas not fixing it lets the edge networks move users up to
> more and more expensive "plans" due to frustration and to sell much more
> gear into Enterprises because they are easy marks for complex gadgets.
> 
>  

The above is exactly what Van told me when he was trying to get me to
"paint the fence" of working on aqm again. (That volunteer effort again:
his employer at that time was not paying him to do this sort of thing, so
he had to interest someone to work with him.) I hope you people with big
platforms will continue to try to educate folks.

	Kathie

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 4741 bytes --]


diffserv is not evil.  However, there has never been a practical mechanism for defining the meaning of the different "levels of service" across vendors and Autonomous Systems.
 
The problem is that diffserv is framed as if there were a global "controller" of the Internet who could impose a precise standard.
 
Andrew Odlyzko once proposed that the Internet try a very simple experiment - invent "first class" and "second class" packets.  (just one bit of differentiation).  Then try to emulate so-called "paris metro pricing", which involves two things: making "first class" cost more than "second class", and providing a meaningful advantage for first class packet senders, while at the same time ensuring that "second class" was very good.  And they adjust the pricing daily or hourly to achieve a global goal: a) that the price of first class is high enough that most people don't use it, and b) that the payments for first class packets go to the points of maximum congestion in the form of funds for upgrades, and not to the points of non-congestion.
 
And then create a means to globally purchase some number of "first class upgrades" that could be either attached to packets one sends, or get from the endpoint you are sending to as a "credit".
 
The routers would do some sort of prioritization of "first class" packets over "second class" packets along the way, and count how many first class packets were processed compared to second class packets.
 
The "upgrades" would expire frequently (daily?) and the cost of purchasing them would be changed daily so that there is a meaningful difference in observed latency on an end-to-end basis.
 
Etc.
 
You can see that even a single distinguished class of service needs some deep economic thinking so that it works on an end-to-end basis across autonomous systems.  (15 classes of service might be definable, but deploying them across the world Internet in a way that the mean the "same" everywhere, and in a way that the differentiated payments actually have a *real* effect on observed service across a many-AS path is an exercise likely to create a market failure).
 
And in the end of the day, the problem is congestion, which is very non-linear.  There is almost no congestion at almost all places in the Internet at any particular time.  You can't fix congestion locally - you have to slow down the sources across all of the edge of the Internet, quickly.
 
Non-linear cost functions do not reach Pareto optimality in a bursty and unpredictable market.  So the folks who would win are those who benefit from extreme non-linearity and instability - "market speculators".
 
If we can't make a "two class" worldwide diffserv work, my sense is that there's no possible way the current diffserv could be made to work in a business and economic sense.
 
This is the fundamental engineering problem.  Not the writing down of standards and debating them in the IETF, which is silly if it's not a good engineering solution to the real problem of a highly diverse, rapidly evolving system whose use is unpredictable from week to week and minute to minute.
 
So diffserv has always been more or less a "science project" with no clear application, IMHO.


On Thursday, May 15, 2014 12:32pm, "Kathleen Nichols" <nichols@pollere.com> said:



> 
> Gosh, that's high praise. And what's really neat is that this was such a
> team effort
> where we don't even necessarily know each other! What's perhaps bad is that
> this was a "volunteer" effort, though that also is a strength. I'm not
> sure the
> answer is for everyone to work for Google.
> 
> On 5/15/14 6:47 AM, dpreed@reed.com wrote:
> ...
> >
> > The solution du jour is to leave bufferbloat in place, while using the
> > real fads: prioritization (diffserv once we have the "fast lanes" fully
> > legal) and "software defined networking" appliances that use DPI for
> > packet routing and traffic management.
> >
> 
> I think diffserv could be used for good instead of evil.
> 
> >
> >
> > Fixing buffer bloat allows the edge- and enterprise- networks to be more
> > efficient, whereas not fixing it lets the edge networks move users up to
> > more and more expensive "plans" due to frustration and to sell much more
> > gear into Enterprises because they are easy marks for complex gadgets.
> >
> >
> 
> The above is exactly what Van told me when he was trying to get me to
> "paint the fence" of working on aqm again. (That volunteer effort again:
> his employer at that time was not paying him to do this sort of thing, so
> he had to interest someone to work with him.) I hope you people with big
> platforms will continue to try to educate folks.
> 
> 	Kathie
>

[-- Attachment #2: Type: text/html, Size: 6255 bytes --]

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 2179 bytes --]

There is, I think, one good way to make Diffserv actually work. It does
require several steps in tandem.

Step one is to accept and admit that differential pricing based on scarcity
economics does not work on the internet. That's going to be tough to
swallow for the big commercial players.

Step two is to define service levels in such a way that asking for a bonus
in one category inherently requires taking a deficit in some other
category. This permits trusting the Diffserv field, wherever it happens to
come from.

That part is where the old TOS flags went wrong, because they tried to
define mutually exclusive characteristics of traffic orthogonally. It was
possible for traffic to request service that was simultaneously higher
bandwidth, higher reliability, lower latency, *and* cheaper than service
without any flags set. This was obviously nonsensical.

My suggested definition is a straight trade-off of priority for bandwidth.
If you want maximum bandwidth, you're going to have to put up with lower
priority relative to traffic which has effectively requested low latency,
which in turn will find itself throttled to some fraction of the available
bandwidth in return for that priority. It forces whoever is setting the
flags to make a genuine engineering trade-off, and happily it can trivially
be made compatible with the legacy Precedence interpretation of the
Diffserv field.

Codepoint 000000, naturally, corresponds to full bandwidth, minimum
priority traffic, and is the default.

To implement it, we're going to need a throttled priority queue. This
should be straightforward - a set of 64 TBFs with the special properties
that higher priority buckets refill more slowly, and that spending from a
bucket also spends the same amount from all lower-priority buckets. Then at
dequeue, take a packet from the highest priority queue with a positive
bucket and a waiting packet, then refill each bucket with the appropriate
fraction of the dequeued packet size. (Implementation detail: what to do if
no such packet exists; also, what fraction to use for each bucket.)
Naturally, each TBF can and should support a child qdisc such as fq_codel.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 2343 bytes --]

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[Michael Welzl]

[-- Attachment #1: Type: text/plain, Size: 3166 bytes --]

Hi,

I completely agree that this service differentiation was wrong from the beginning. The idea of using bits to implement a trade-off that doesn't involve prioritization between users is old (*), and has always been the right approach in my opinion.

At least one proposal in this direction is currently being made in the IETF, and I definitely think that this is the right way to go:   http://tools.ietf.org/html/draft-lai-tsvwg-normalizer-02

Cheers,
Michael



(*) - the oldest example I know of is the Alternative Best Effort Service http://infoscience.epfl.ch/record/223 , slightly newer we have e.g.:
http://people.networks.imdea.org/~sergey_gorinsky/pdf/RD_Services_SIGCOMM_2008.pdf



On 16. mai 2014, at 00:53, Jonathan Morton <chromatix99@gmail.com> wrote:

> There is, I think, one good way to make Diffserv actually work. It does require several steps in tandem.
> 
> Step one is to accept and admit that differential pricing based on scarcity economics does not work on the internet. That's going to be tough to swallow for the big commercial players.
> 
> Step two is to define service levels in such a way that asking for a bonus in one category inherently requires taking a deficit in some other category. This permits trusting the Diffserv field, wherever it happens to come from.
> 
> That part is where the old TOS flags went wrong, because they tried to define mutually exclusive characteristics of traffic orthogonally. It was possible for traffic to request service that was simultaneously higher bandwidth, higher reliability, lower latency, *and* cheaper than service without any flags set. This was obviously nonsensical.
> 
> My suggested definition is a straight trade-off of priority for bandwidth. If you want maximum bandwidth, you're going to have to put up with lower priority relative to traffic which has effectively requested low latency, which in turn will find itself throttled to some fraction of the available bandwidth in return for that priority. It forces whoever is setting the flags to make a genuine engineering trade-off, and happily it can trivially be made compatible with the legacy Precedence interpretation of the Diffserv field.
> 
> Codepoint 000000, naturally, corresponds to full bandwidth, minimum priority traffic, and is the default.
> 
> To implement it, we're going to need a throttled priority queue. This should be straightforward - a set of 64 TBFs with the special properties that higher priority buckets refill more slowly, and that spending from a bucket also spends the same amount from all lower-priority buckets. Then at dequeue, take a packet from the highest priority queue with a positive bucket and a waiting packet, then refill each bucket with the appropriate fraction of the dequeued packet size. (Implementation detail: what to do if no such packet exists; also, what fraction to use for each bucket.) Naturally, each TBF can and should support a child qdisc such as fq_codel.
> 
> - Jonathan Morton
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat


[-- Attachment #2: Type: text/html, Size: 4042 bytes --]

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[Jim Gettys]

[-- Attachment #1: Type: text/plain, Size: 2487 bytes --]

On Fri, May 16, 2014 at 10:52 AM, <Valdis.Kletnieks@vt.edu> wrote:

> On Thu, 15 May 2014 16:32:55 -0400, dpreed@reed.com said:
>
> > And in the end of the day, the problem is congestion, which is very
> > non-linear.  There is almost no congestion at almost all places in the
> Internet
> > at any particular time.  You can't fix congestion locally - you have to
> slow
> > down the sources across all of the edge of the Internet, quickly.
>
> There's a second very important point that somebody mentioned on the NANOG
> list a while ago:
>
> If the local router/net/link/whatever isn't congested, QoS cannot do
> anything
> to improve life for anybody.
>
> If there *is* congestion, QoS can only improve your service to the normal
> uncongested state - and it can *only do so by making somebody else's
> experience
> suck more*....
>
​The somebody else might be "you", in which life is much better.​  once you
have the concept of flows (at some level of abstraction), you can make more
sane choices.

​Personally, I've mostly been interested in QOS in the local network: as
"hints", for example, that it is worth more aggressive bidding for transmit
opportunities in WiFi, for example to ensure my VOIP, teleconferencing,
gaming, music playing and other actually real time packets get priority
over bulk data (which includes web traffic), and may need access to the
medium sooner than for routine applications or scavenger applications.

Whether it should have any use beyond the scope of the network that I
control is less than clear to me, for the reasons you state; having my
traffic screw up other people's traffic isn't high on my list of "good
ideas".

The other danger of QOS is that applications may "game" its use of QOS, to
get preferential treatment, so each network (and potentially hosts) need to
be able to control its own policy, and detect (and potentially punish)
transgressors.  Right now, we don't have those detectors or controls in
place (and how to inform naive users that their applications are asking for
priority service for no good reason) is another unanswered question.

This gaming danger (and a UI to enable policy to be set), make me think
it's something we're going to have to work through carefully.

- Jim


>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>

[-- Attachment #2: Type: text/html, Size: 3983 bytes --]

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 6206 bytes --]


I agree with you Jim about being careful with QoS.  That's why Andrew Odlyzko proposed the experiment with exactly two classes, and proposed it as an *experiment*. So many researchers and IETF members seem to think we should just turn on diffserv and everything will work great... I've seen very senior members of IETF actually propose diffserv become a provider-wide standard as soon as possible.  I suppose they have a couple of ns2 runs that show "nothing can go wrong". :-)
 
(that's why I'm so impressed by the fq_codel work - it's more than just simulation, but has been tested and more or less stressed in real life, yet it is quite simple).
 
I don't agree with the idea that switches alone can solve global system problems by themselves. That's why the original AIMD algorithms use packet drops as signals, but make the endpoints responsible for managing congestion.  The switches have nothing to do with the AIMD algorithm, they just create the control inputs.
 
So it is kind of telling that Valdis cites a totally "switch-centric" view from NANOG's perspective.  It's not the job of switches to manage congestion, just as it is not the job of endpoints to program switches.  There's a separation of concerns.
 
The simpler observation would be "if you are a switch, there is NOTHING you can do to stop congestion.  Even dropping packets doesn't ameliorate congestion.  However, if you are a switch there are some things you can tell the endpoints, in particular the receiving endpoints of flows traveling across the switch, about the local 'collision' of packets trying to get through the switch at the same time."
 
Since the Internet end-to-end protocols are "receiver controlled" (TCP's receive window is what controls the sender's flow, but it is set by the receiver), the locus of decision making is the collection of receivers.
 
Buffering is not the real issue - the issue is the frequency with which the packets of all the flows going through a particular switch "collide".  The control problem is to make that frequency of collision quite small.
 
The nice thing about packet drops is that collisions are remediated immediately, rather than creating sustained bottlenecks that increase the "collision cross section" of that switch, increasing the likelihood of collisions in the switch dramatically. Replacing a collided/dropped packet with a much smaller "token" that goes on to the receiver would keep the collision cross section from growing, but provide better samples of collision info to the receiver.  For fairness, you want all packets involved in a collision to carry information, and ideally all "near collisions" to also carry information about near collisions.
 
A collision in this is simply defined: a packet that enters a switch collides with any other packets that have not completed traversal of the switch when the packet arrives is considered to have collided with those packets.
 
You can expand packets' virtual time in the switch by thinking of them as "virtually still in the switch" for some number of bit times after they exit.   Then a "near collision" happens between a packet and any packets that are still virtually in the switch.  Near collisions are signals that can keep the system inside the "ballistic region" of the phase space.
 
(you can track near collisions by a little memory on each outbound link state - and even use Bloom Filters to quickly detect collisions, but that is for a different lecture).
 
Please steal this idea and develop it.
 
 
 


On Friday, May 16, 2014 12:06pm, "Jim Gettys" <jg@freedesktop.org> said:







On Fri, May 16, 2014 at 10:52 AM,  <[Valdis.Kletnieks@vt.edu](mailto:Valdis.Kletnieks@vt.edu)> wrote:

On Thu, 15 May 2014 16:32:55 -0400, [dpreed@reed.com](mailto:dpreed@reed.com) said:

 > And in the end of the day, the problem is congestion, which is very
 > non-linear.  There is almost no congestion at almost all places in the Internet
 > at any particular time.  You can't fix congestion locally - you have to slow
 > down the sources across all of the edge of the Internet, quickly.

There's a second very important point that somebody mentioned on the NANOG
 list a while ago:

 If the local router/net/link/whatever isn't congested, QoS cannot do anything
 to improve life for anybody.

 If there *is* congestion, QoS can only improve your service to the normal
 uncongested state - and it can *only do so by making somebody else's experience
 suck more*....

​The somebody else might be "you", in which life is much better.​  once you have the concept of flows (at some level of abstraction), you can make more sane choices.
​Personally, I've mostly been interested in QOS in the local network: as "hints", for example, that it is worth more aggressive bidding for transmit opportunities in WiFi, for example to ensure my VOIP, teleconferencing, gaming, music playing and other actually real time packets get priority over bulk data (which includes web traffic), and may need access to the medium sooner than for routine applications or scavenger applications.
Whether it should have any use beyond the scope of the network that I control is less than clear to me, for the reasons you state; having my traffic screw up other people's traffic isn't high on my list of "good ideas".
The other danger of QOS is that applications may "game" its use of QOS, to get preferential treatment, so each network (and potentially hosts) need to be able to control its own policy, and detect (and potentially punish) transgressors.  Right now, we don't have those detectors or controls in place (and how to inform naive users that their applications are asking for priority service for no good reason) is another unanswered question.
This gaming danger (and a UI to enable policy to be set), make me think it's something we're going to have to work through carefully.
- Jim

_______________________________________________
 Cerowrt-devel mailing list
[Cerowrt-devel@lists.bufferbloat.net](mailto:Cerowrt-devel@lists.bufferbloat.net)
[https://lists.bufferbloat.net/listinfo/cerowrt-devel](https://lists.bufferbloat.net/listinfo/cerowrt-devel)


[-- Attachment #2: Type: text/html, Size: 8393 bytes --]

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[David Lang]

[-- Attachment #1: Type: TEXT/Plain, Size: 6597 bytes --]

If you think "fast lanes" will actually increase performance for any traffic, 
you are dreaming.

the people looking for "fast lanes" are't trying to get traffic through any 
faster, they are looking to charge more for the traffic they are already 
passing.

David Lang

  On Thu, 15 May 2014, dpreed@reed.com wrote:

> Well done.  I'm optimistic for deployment everywhere, except CMTS's, the LTE and HSPA+ access networks, and all corporate firewall and intranet gear.
> 
> The solution du jour is to leave bufferbloat in place, while using the real fads: prioritization (diffserv once we have the "fast lanes" fully legal) and "software defined networking" appliances that use DPI for packet routing and traffic management.
> 
> Fixing buffer bloat allows the edge- and enterprise- networks to be more efficient, whereas not fixing it lets the edge networks move users up to more and more expensive "plans" due to frustration and to sell much more gear into Enterprises because they are easy marks for complex gadgets.
> 
> But maybe a few engineers who operate and design gear for such networks will overcome the incredible business biases against fixing this.
> 
> That's why all the efforts you guys have put forth are immensely worth it.  I think this is one of the best innovations in recent years (Bram Cohen's original BitTorrent is another, for fully decentralizing data delivery for the very first time in a brilliant way.) I will certainly push everywhere I can to see fq_codel deployed.
> 
> If there were a prize for brilliant projects, this would be top on my list.
> 
>
>
> On Wednesday, May 14, 2014 9:25pm, "Dave Taht" <dave.taht@gmail.com> said:
>
>
>
>> On Wed, May 14, 2014 at 3:32 PM, Kathleen Nichols <nichols@pollere.com>
>> wrote:
>> >
>> > Thanks, Rich.
>> >
>> > And to show you what an amazing bit of work that first fq_codel was, I have
>> > on my calendar that I first "exposed" CoDel to a small group in a
>> > meeting room
>> > and on the phone at ISC on April 24.
>> 
>> And we had all sorts of trouble with the phone, (eric didn't hear
>> much) and we then spent hours and hours afterwards discussing wifi
>> instead of codel... it was too much to take in...
>> 
>> me, I'd started jumping up and down in excitement about 20 minutes
>> into kathies preso...
>> 
>> May 3rd, 2012 was the last 24 hr coding stint I think I'll ever have.
>> 
>> https://lists.bufferbloat.net/pipermail/codel/2012-May/000023.html
>> 
>> Ahh, the good ole days, when bufferbloat was first solved and we all
>> thought the internet would speed up overnight, and we were going to be
>> rock stars, invited to all the best parties, acquire fame and fortune,
>> and be awarded medals and given awards. Re-reading all this brought
>> back memories.... (heck, there's still a couple good ideas in that
>> thread left unimplemented)
>> 
>> https://lists.bufferbloat.net/pipermail/codel/2012-May/thread.html
>> 
>> It looks by may 5th we were getting in shape, and then there were a
>> few other issues along the way with the control law and so on... and
>> eric rewrote the whole thing and made it oodles faster and then as
>> best as I recall came up with fq_codel on saturday may 5th(?) -
>> 
>> Ah, I haven't had so much fun in in years. My life since then seems
>> like an endless string of meetings, politics, and bugfixing.
>> 
>> The code went from sim/paper, to code, to testing, to mainline linux
>> in another week. I wish more research went like that!
>> 
>> commit 76e3cc126bb223013a6b9a0e2a51238d1ef2e409
>> Author: Eric Dumazet <edumazet@google.com>
>> Date:   Thu May 10 07:51:25 2012 +0000
>>
>>     codel: Controlled Delay AQM
>> 
>> Now, as I recall the story, eric came up with fq_codel on a saturday
>> afternoon, so I guess that was may 5th - cinco de mayo!
>> 
>> And that too, landed in mainline...
>> 
>> commit 4b549a2ef4bef9965d97cbd992ba67930cd3e0fe
>> Author: Eric Dumazet <edumazet@google.com>
>> Date:   Fri May 11 09:30:50 2012 +0000
>>
>>     fq_codel: Fair Queue Codel AQM
>> 
>> let's not forget tom herbert & BQL
>> 
>> commit 75957ba36c05b979701e9ec64b37819adc12f830
>> Author: Tom Herbert <therbert@google.com>
>> Date:   Mon Nov 28 16:32:35 2011 +0000
>>
>>     dql: Dynamic queue limits
>>
>>     Implementation of dynamic queue limits (dql).  This is a libary which
>>     allows a queue limit to be dynamically managed.  The goal of dql is
>>     to set the queue limit, number of objects to the queue, to be minimized
>>     without allowing the queue to be starved.
>> 
>> 
>> 
>> 
>> > It was really amazing to me to watch
>> > something Van and I had been discussing (okay, arguing) about privately for
>> > 6 months and I'd been tinkering with be turned into real code on real
>> > networks.
>> > Jim Gettys is an incredible (and constructive) nagger, Eric Dumazet and
>> > amazing
>> > coder, and the entire open source community a really nifty group of folks.
>> >
>> > Maybe someday we will actually update the first article with some of the
>> > stuff
>> > we got into the last internet draft....
>> >
>> >         be the change,
>> >                 Kathie
>> >
>> > On 5/14/14 2:01 PM, Rich Brown wrote:
>> >> Folks,
>> >>
>> >> I just noticed that the announcement for the first testable
>> >> implementation of fq_codel was two days ago today - 14 May 2012.
>> >> Build 3.3.6-2 was the first to include working fq_codel.
>> >>
>> https://lists.bufferbloat.net/pipermail/cerowrt-devel/2012-May/000233.html
>> >>
>> >>  Two years later, we see fq_codel being adopted lots of places. As
>> >> more and more people/organizations come to understand the problem,
>> >> and how straightforward the solution can be, we're beginning to win
>> >> the battle to have a good Internet experience everywhere.
>> >>
>> >> Thanks to Dave, Eric, Kathie, Van, and all the members of this list
>> >> for their perseverance, testing, comments, and patience.
>> >> Congratulations!
>> >>
>> >> Best regards,
>> >>
>> >> Rich _______________________________________________ Bloat mailing
>> >> list Bloat@lists.bufferbloat.net
>> >> https://lists.bufferbloat.net/listinfo/bloat
>> >>
>> >
>> > _______________________________________________
>> > Bloat mailing list
>> > Bloat@lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/bloat
>> 
>> 
>> 
>> --
>> Dave Täht
>> 
>> NSFW:
>> https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>

[-- Attachment #2: Type: TEXT/PLAIN, Size: 140 bytes --]

_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 8438 bytes --]


I don't think that at all. I suspect you know that. But if I confused you, let me assure you that my view of the proper operating point of the Internet as a whole is that the expected buffer queue on any switch anywhere in the Internet is < 1 datagram.
 
fq_codel is a good start, but it still requires letting buffer queueing increase.  However, mathematically, one need not have the queues build up to sustain the control loop that fq_codel creates.
 
I conjecture that one can create an equally effective congestion control mechanism as fq_codel without any standing queues being allowed to build up. (Someone should try the exercise of trying to prove that an optimal end-to-end feedback control system requires queueing delay to be imposed. I've tried and it's led me to the conjecture that one can always replace a standing queue with a finite memory of past activities - and if one does, the lack of a standing queue means that the algorithm is better than any that end up with a standing queue).
 
fq_codel could be redesigned into a queue-free fq_codel.


On Thursday, May 15, 2014 7:46pm, "David Lang" <david@lang.hm> said:



> If you think "fast lanes" will actually increase performance for any traffic,
> you are dreaming.
> 
> the people looking for "fast lanes" are't trying to get traffic through any
> faster, they are looking to charge more for the traffic they are already
> passing.
> 
> David Lang
> 
>   On Thu, 15 May 2014, dpreed@reed.com wrote:
> 
> > Well done.  I'm optimistic for deployment everywhere, except CMTS's, the LTE
> and HSPA+ access networks, and all corporate firewall and intranet gear.
> >
> > The solution du jour is to leave bufferbloat in place, while using the real
> fads: prioritization (diffserv once we have the "fast lanes" fully legal) and
> "software defined networking" appliances that use DPI for packet routing and
> traffic management.
> >
> > Fixing buffer bloat allows the edge- and enterprise- networks to be more
> efficient, whereas not fixing it lets the edge networks move users up to more and
> more expensive "plans" due to frustration and to sell much more gear into
> Enterprises because they are easy marks for complex gadgets.
> >
> > But maybe a few engineers who operate and design gear for such networks will
> overcome the incredible business biases against fixing this.
> >
> > That's why all the efforts you guys have put forth are immensely worth it.  I
> think this is one of the best innovations in recent years (Bram Cohen's original
> BitTorrent is another, for fully decentralizing data delivery for the very first
> time in a brilliant way.) I will certainly push everywhere I can to see fq_codel
> deployed.
> >
> > If there were a prize for brilliant projects, this would be top on my list.
> >
> >
> >
> > On Wednesday, May 14, 2014 9:25pm, "Dave Taht" <dave.taht@gmail.com>
> said:
> >
> >
> >
> >> On Wed, May 14, 2014 at 3:32 PM, Kathleen Nichols
> <nichols@pollere.com>
> >> wrote:
> >> >
> >> > Thanks, Rich.
> >> >
> >> > And to show you what an amazing bit of work that first fq_codel was,
> I have
> >> > on my calendar that I first "exposed" CoDel to a small group in a
> >> > meeting room
> >> > and on the phone at ISC on April 24.
> >>
> >> And we had all sorts of trouble with the phone, (eric didn't hear
> >> much) and we then spent hours and hours afterwards discussing wifi
> >> instead of codel... it was too much to take in...
> >>
> >> me, I'd started jumping up and down in excitement about 20 minutes
> >> into kathies preso...
> >>
> >> May 3rd, 2012 was the last 24 hr coding stint I think I'll ever have.
> >>
> >> https://lists.bufferbloat.net/pipermail/codel/2012-May/000023.html
> >>
> >> Ahh, the good ole days, when bufferbloat was first solved and we all
> >> thought the internet would speed up overnight, and we were going to be
> >> rock stars, invited to all the best parties, acquire fame and fortune,
> >> and be awarded medals and given awards. Re-reading all this brought
> >> back memories.... (heck, there's still a couple good ideas in that
> >> thread left unimplemented)
> >>
> >> https://lists.bufferbloat.net/pipermail/codel/2012-May/thread.html
> >>
> >> It looks by may 5th we were getting in shape, and then there were a
> >> few other issues along the way with the control law and so on... and
> >> eric rewrote the whole thing and made it oodles faster and then as
> >> best as I recall came up with fq_codel on saturday may 5th(?) -
> >>
> >> Ah, I haven't had so much fun in in years. My life since then seems
> >> like an endless string of meetings, politics, and bugfixing.
> >>
> >> The code went from sim/paper, to code, to testing, to mainline linux
> >> in another week. I wish more research went like that!
> >>
> >> commit 76e3cc126bb223013a6b9a0e2a51238d1ef2e409
> >> Author: Eric Dumazet <edumazet@google.com>
> >> Date:   Thu May 10 07:51:25 2012 +0000
> >>
> >>     codel: Controlled Delay AQM
> >>
> >> Now, as I recall the story, eric came up with fq_codel on a saturday
> >> afternoon, so I guess that was may 5th - cinco de mayo!
> >>
> >> And that too, landed in mainline...
> >>
> >> commit 4b549a2ef4bef9965d97cbd992ba67930cd3e0fe
> >> Author: Eric Dumazet <edumazet@google.com>
> >> Date:   Fri May 11 09:30:50 2012 +0000
> >>
> >>     fq_codel: Fair Queue Codel AQM
> >>
> >> let's not forget tom herbert & BQL
> >>
> >> commit 75957ba36c05b979701e9ec64b37819adc12f830
> >> Author: Tom Herbert <therbert@google.com>
> >> Date:   Mon Nov 28 16:32:35 2011 +0000
> >>
> >>     dql: Dynamic queue limits
> >>
> >>     Implementation of dynamic queue limits (dql).  This is a libary
> which
> >>     allows a queue limit to be dynamically managed.  The goal of dql is
> >>     to set the queue limit, number of objects to the queue, to be
> minimized
> >>     without allowing the queue to be starved.
> >>
> >>
> >>
> >>
> >> > It was really amazing to me to watch
> >> > something Van and I had been discussing (okay, arguing) about
> privately for
> >> > 6 months and I'd been tinkering with be turned into real code on
> real
> >> > networks.
> >> > Jim Gettys is an incredible (and constructive) nagger, Eric Dumazet
> and
> >> > amazing
> >> > coder, and the entire open source community a really nifty group of
> folks.
> >> >
> >> > Maybe someday we will actually update the first article with some of
> the
> >> > stuff
> >> > we got into the last internet draft....
> >> >
> >> >         be the change,
> >> >                 Kathie
> >> >
> >> > On 5/14/14 2:01 PM, Rich Brown wrote:
> >> >> Folks,
> >> >>
> >> >> I just noticed that the announcement for the first testable
> >> >> implementation of fq_codel was two days ago today - 14 May
> 2012.
> >> >> Build 3.3.6-2 was the first to include working fq_codel.
> >> >>
> >>
> https://lists.bufferbloat.net/pipermail/cerowrt-devel/2012-May/000233.html
> >> >>
> >> >>  Two years later, we see fq_codel being adopted lots of places.
> As
> >> >> more and more people/organizations come to understand the
> problem,
> >> >> and how straightforward the solution can be, we're beginning to
> win
> >> >> the battle to have a good Internet experience everywhere.
> >> >>
> >> >> Thanks to Dave, Eric, Kathie, Van, and all the members of this
> list
> >> >> for their perseverance, testing, comments, and patience.
> >> >> Congratulations!
> >> >>
> >> >> Best regards,
> >> >>
> >> >> Rich _______________________________________________ Bloat
> mailing
> >> >> list Bloat@lists.bufferbloat.net
> >> >> https://lists.bufferbloat.net/listinfo/bloat
> >> >>
> >> >
> >> > _______________________________________________
> >> > Bloat mailing list
> >> > Bloat@lists.bufferbloat.net
> >> > https://lists.bufferbloat.net/listinfo/bloat
> >>
> >>
> >>
> >> --
> >> Dave Täht
> >>
> >> NSFW:
> >>
> https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
> >> _______________________________________________
> >> Cerowrt-devel mailing list
> >> Cerowrt-devel@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> >>_______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

[-- Attachment #2: Type: text/html, Size: 11353 bytes --]

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[Dave Taht]

On Thu, May 15, 2014 at 6:01 PM,  <dpreed@reed.com> wrote:
> I don't think that at all. I suspect you know that. But if I confused you,
> let me assure you that my view of the proper operating point of the Internet
> as a whole is that the expected buffer queue on any switch anywhere in the
> Internet is < 1 datagram.
>
>
>
> fq_codel is a good start, but it still requires letting buffer queueing
> increase.  However, mathematically, one need not have the queues build up to
> sustain the control loop that fq_codel creates.
>
>
>
> I conjecture that one can create an equally effective congestion control
> mechanism as fq_codel without any standing queues being allowed to build up.
> (Someone should try the exercise of trying to prove that an optimal
> end-to-end feedback control system requires queueing delay to be imposed.
> I've tried and it's led me to the conjecture that one can always replace a
> standing queue with a finite memory of past activities - and if one does,
> the lack of a standing queue means that the algorithm is better than any
> that end up with a standing queue).

I like the Remy work too, and there is an increasing amount of information
about the path cached on hosts now -

but the problem is a finite amount of information
is needed per machine connecting per machine connecting to, which
gets to be a rather larger number, rather fast.

I think it is possible to get ever closer to a saner minimum, both from an aqm
side and from a e2e transport side. 0? not possible, due to random delays
from underlying reliability features at layer 2.

> fq_codel could be redesigned into a queue-free fq_codel.

I like that the ideas are getting used elsewhere. ;)

Exhibit A) current linux tcp + sch_fq + pacing. Wow. I mean,
seriously. Wow. It achieves the goal
of filling the pipe and has dramatically reduced packet loss and
retransmits. I hope they publish some stuff on their results soon.

B) If we could assume fq everywhere, we could develop much better
delay based e2e congestion control algos that could move back on
single points of RTT variance, not outrageously high values like
100ms.

>
>
>
>
> On Thursday, May 15, 2014 7:46pm, "David Lang" <david@lang.hm> said:
>
>> If you think "fast lanes" will actually increase performance for any
>> traffic,
>> you are dreaming.
>>
>> the people looking for "fast lanes" are't trying to get traffic through
>> any
>> faster, they are looking to charge more for the traffic they are already
>> passing.
>>
>> David Lang
>>
>> On Thu, 15 May 2014, dpreed@reed.com wrote:
>>
>> > Well done. I'm optimistic for deployment everywhere, except CMTS's, the
>> > LTE
>> and HSPA+ access networks, and all corporate firewall and intranet gear.
>> >
>> > The solution du jour is to leave bufferbloat in place, while using the
>> > real
>> fads: prioritization (diffserv once we have the "fast lanes" fully legal)
>> and
>> "software defined networking" appliances that use DPI for packet routing
>> and
>> traffic management.
>> >
>> > Fixing buffer bloat allows the edge- and enterprise- networks to be more
>> efficient, whereas not fixing it lets the edge networks move users up to
>> more and
>> more expensive "plans" due to frustration and to sell much more gear into
>> Enterprises because they are easy marks for complex gadgets.
>> >
>> > But maybe a few engineers who operate and design gear for such networks
>> > will
>> overcome the incredible business biases against fixing this.
>> >
>> > That's why all the efforts you guys have put forth are immensely worth
>> > it. I
>> think this is one of the best innovations in recent years (Bram Cohen's
>> original
>> BitTorrent is another, for fully decentralizing data delivery for the very
>> first
>> time in a brilliant way.) I will certainly push everywhere I can to see
>> fq_codel
>> deployed.
>> >
>> > If there were a prize for brilliant projects, this would be top on my
>> > list.
>> >
>> >
>> >
>> > On Wednesday, May 14, 2014 9:25pm, "Dave Taht" <dave.taht@gmail.com>
>> said:
>> >
>> >
>> >
>> >> On Wed, May 14, 2014 at 3:32 PM, Kathleen Nichols
>> <nichols@pollere.com>
>> >> wrote:
>> >> >
>> >> > Thanks, Rich.
>> >> >
>> >> > And to show you what an amazing bit of work that first fq_codel was,
>> I have
>> >> > on my calendar that I first "exposed" CoDel to a small group in a
>> >> > meeting room
>> >> > and on the phone at ISC on April 24.
>> >>
>> >> And we had all sorts of trouble with the phone, (eric didn't hear
>> >> much) and we then spent hours and hours afterwards discussing wifi
>> >> instead of codel... it was too much to take in...
>> >>
>> >> me, I'd started jumping up and down in excitement about 20 minutes
>> >> into kathies preso...
>> >>
>> >> May 3rd, 2012 was the last 24 hr coding stint I think I'll ever have.
>> >>
>> >> https://lists.bufferbloat.net/pipermail/codel/2012-May/000023.html
>> >>
>> >> Ahh, the good ole days, when bufferbloat was first solved and we all
>> >> thought the internet would speed up overnight, and we were going to be
>> >> rock stars, invited to all the best parties, acquire fame and fortune,
>> >> and be awarded medals and given awards. Re-reading all this brought
>> >> back memories.... (heck, there's still a couple good ideas in that
>> >> thread left unimplemented)
>> >>
>> >> https://lists.bufferbloat.net/pipermail/codel/2012-May/thread.html
>> >>
>> >> It looks by may 5th we were getting in shape, and then there were a
>> >> few other issues along the way with the control law and so on... and
>> >> eric rewrote the whole thing and made it oodles faster and then as
>> >> best as I recall came up with fq_codel on saturday may 5th(?) -
>> >>
>> >> Ah, I haven't had so much fun in in years. My life since then seems
>> >> like an endless string of meetings, politics, and bugfixing.
>> >>
>> >> The code went from sim/paper, to code, to testing, to mainline linux
>> >> in another week. I wish more research went like that!
>> >>
>> >> commit 76e3cc126bb223013a6b9a0e2a51238d1ef2e409
>> >> Author: Eric Dumazet <edumazet@google.com>
>> >> Date: Thu May 10 07:51:25 2012 +0000
>> >>
>> >> codel: Controlled Delay AQM
>> >>
>> >> Now, as I recall the story, eric came up with fq_codel on a saturday
>> >> afternoon, so I guess that was may 5th - cinco de mayo!
>> >>
>> >> And that too, landed in mainline...
>> >>
>> >> commit 4b549a2ef4bef9965d97cbd992ba67930cd3e0fe
>> >> Author: Eric Dumazet <edumazet@google.com>
>> >> Date: Fri May 11 09:30:50 2012 +0000
>> >>
>> >> fq_codel: Fair Queue Codel AQM
>> >>
>> >> let's not forget tom herbert & BQL
>> >>
>> >> commit 75957ba36c05b979701e9ec64b37819adc12f830
>> >> Author: Tom Herbert <therbert@google.com>
>> >> Date: Mon Nov 28 16:32:35 2011 +0000
>> >>
>> >> dql: Dynamic queue limits
>> >>
>> >> Implementation of dynamic queue limits (dql). This is a libary
>> which
>> >> allows a queue limit to be dynamically managed. The goal of dql is
>> >> to set the queue limit, number of objects to the queue, to be
>> minimized
>> >> without allowing the queue to be starved.
>> >>
>> >>
>> >>
>> >>
>> >> > It was really amazing to me to watch
>> >> > something Van and I had been discussing (okay, arguing) about
>> privately for
>> >> > 6 months and I'd been tinkering with be turned into real code on
>> real
>> >> > networks.
>> >> > Jim Gettys is an incredible (and constructive) nagger, Eric Dumazet
>> and
>> >> > amazing
>> >> > coder, and the entire open source community a really nifty group of
>> folks.
>> >> >
>> >> > Maybe someday we will actually update the first article with some of
>> the
>> >> > stuff
>> >> > we got into the last internet draft....
>> >> >
>> >> > be the change,
>> >> > Kathie
>> >> >
>> >> > On 5/14/14 2:01 PM, Rich Brown wrote:
>> >> >> Folks,
>> >> >>
>> >> >> I just noticed that the announcement for the first testable
>> >> >> implementation of fq_codel was two days ago today - 14 May
>> 2012.
>> >> >> Build 3.3.6-2 was the first to include working fq_codel.
>> >> >>
>> >>
>> https://lists.bufferbloat.net/pipermail/cerowrt-devel/2012-May/000233.html
>> >> >>
>> >> >> Two years later, we see fq_codel being adopted lots of places.
>> As
>> >> >> more and more people/organizations come to understand the
>> problem,
>> >> >> and how straightforward the solution can be, we're beginning to
>> win
>> >> >> the battle to have a good Internet experience everywhere.
>> >> >>
>> >> >> Thanks to Dave, Eric, Kathie, Van, and all the members of this
>> list
>> >> >> for their perseverance, testing, comments, and patience.
>> >> >> Congratulations!
>> >> >>
>> >> >> Best regards,
>> >> >>
>> >> >> Rich _______________________________________________ Bloat
>> mailing
>> >> >> list Bloat@lists.bufferbloat.net
>> >> >> https://lists.bufferbloat.net/listinfo/bloat
>> >> >>
>> >> >
>> >> > _______________________________________________
>> >> > Bloat mailing list
>> >> > Bloat@lists.bufferbloat.net
>> >> > https://lists.bufferbloat.net/listinfo/bloat
>> >>
>> >>
>> >>
>> >> --
>> >> Dave Täht
>> >>
>> >> NSFW:
>> >>
>>
>> https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
>> >> _______________________________________________
>> >> Cerowrt-devel mailing list
>> >> Cerowrt-devel@lists.bufferbloat.net
>> >> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>> >>_______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>>



-- 
Dave Täht

NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[David Lang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 9808 bytes --]

We are talking about different things then.

The "fast lane" I'm talking about is where ISPs want companies to pay them for 
bandwidth (in addition to the companies paying their own ISP for bandwidth and 
in addition to their users paying them for bandwidth)

As for your contention that an ideal Internet will have a buffer size of <1 
packet. That will work, but that will not fully utilize the network, because 
there will be jitter in the senders and some packet generation will be delayed, 
leaving the network with nothing to send in that timeslot.

If the network is not fully utilized, then fq_codel isn't needed, just send 
packets as they arrive. It's only as a particular link approaches full 
utilization that queues will build up and deciding what to do becomes 
significant (and fq_codel and similar will matter)

As for your thought of having an end-to-end feedback loop, the problem with that 
is that it will only work if the path between them is stable and not interfered 
with by other flows. In the Internet as we have it today, neither are true. The 
packets for your connection may travel over different paths, and congestion 
happens on a link-by-link basis with the combined packets of many connections, 
not end-to-end based on a single connection.

David Lang

On Thu, 15 May 2014, dpreed@reed.com wrote:

> I don't think that at all. I suspect you know that. But if I confused you, let 
> me assure you that my view of the proper operating point of the Internet as a 
> whole is that the expected buffer queue on any switch anywhere in the Internet 
> is < 1 datagram.
> 
> fq_codel is a good start, but it still requires letting buffer queueing 
> increase.  However, mathematically, one need not have the queues build up to 
> sustain the control loop that fq_codel creates.
> 
> I conjecture that one can create an equally effective congestion control 
> mechanism as fq_codel without any standing queues being allowed to build up. 
> (Someone should try the exercise of trying to prove that an optimal end-to-end 
> feedback control system requires queueing delay to be imposed. I've tried and 
> it's led me to the conjecture that one can always replace a standing queue 
> with a finite memory of past activities - and if one does, the lack of a 
> standing queue means that the algorithm is better than any that end up with a 
> standing queue).
> 
> fq_codel could be redesigned into a queue-free fq_codel.
>
>
> On Thursday, May 15, 2014 7:46pm, "David Lang" <david@lang.hm> said:
>
>
>
>> If you think "fast lanes" will actually increase performance for any traffic,
>> you are dreaming.
>> 
>> the people looking for "fast lanes" are't trying to get traffic through any
>> faster, they are looking to charge more for the traffic they are already
>> passing.
>> 
>> David Lang
>>
>>   On Thu, 15 May 2014, dpreed@reed.com wrote:
>> 
>> > Well done.  I'm optimistic for deployment everywhere, except CMTS's, the LTE
>> and HSPA+ access networks, and all corporate firewall and intranet gear.
>> >
>> > The solution du jour is to leave bufferbloat in place, while using the real
>> fads: prioritization (diffserv once we have the "fast lanes" fully legal) and
>> "software defined networking" appliances that use DPI for packet routing and
>> traffic management.
>> >
>> > Fixing buffer bloat allows the edge- and enterprise- networks to be more
>> efficient, whereas not fixing it lets the edge networks move users up to more and
>> more expensive "plans" due to frustration and to sell much more gear into
>> Enterprises because they are easy marks for complex gadgets.
>> >
>> > But maybe a few engineers who operate and design gear for such networks will
>> overcome the incredible business biases against fixing this.
>> >
>> > That's why all the efforts you guys have put forth are immensely worth it.  I
>> think this is one of the best innovations in recent years (Bram Cohen's original
>> BitTorrent is another, for fully decentralizing data delivery for the very first
>> time in a brilliant way.) I will certainly push everywhere I can to see fq_codel
>> deployed.
>> >
>> > If there were a prize for brilliant projects, this would be top on my list.
>> >
>> >
>> >
>> > On Wednesday, May 14, 2014 9:25pm, "Dave Taht" <dave.taht@gmail.com>
>> said:
>> >
>> >
>> >
>> >> On Wed, May 14, 2014 at 3:32 PM, Kathleen Nichols
>> <nichols@pollere.com>
>> >> wrote:
>> >> >
>> >> > Thanks, Rich.
>> >> >
>> >> > And to show you what an amazing bit of work that first fq_codel was,
>> I have
>> >> > on my calendar that I first "exposed" CoDel to a small group in a
>> >> > meeting room
>> >> > and on the phone at ISC on April 24.
>> >>
>> >> And we had all sorts of trouble with the phone, (eric didn't hear
>> >> much) and we then spent hours and hours afterwards discussing wifi
>> >> instead of codel... it was too much to take in...
>> >>
>> >> me, I'd started jumping up and down in excitement about 20 minutes
>> >> into kathies preso...
>> >>
>> >> May 3rd, 2012 was the last 24 hr coding stint I think I'll ever have.
>> >>
>> >> https://lists.bufferbloat.net/pipermail/codel/2012-May/000023.html
>> >>
>> >> Ahh, the good ole days, when bufferbloat was first solved and we all
>> >> thought the internet would speed up overnight, and we were going to be
>> >> rock stars, invited to all the best parties, acquire fame and fortune,
>> >> and be awarded medals and given awards. Re-reading all this brought
>> >> back memories.... (heck, there's still a couple good ideas in that
>> >> thread left unimplemented)
>> >>
>> >> https://lists.bufferbloat.net/pipermail/codel/2012-May/thread.html
>> >>
>> >> It looks by may 5th we were getting in shape, and then there were a
>> >> few other issues along the way with the control law and so on... and
>> >> eric rewrote the whole thing and made it oodles faster and then as
>> >> best as I recall came up with fq_codel on saturday may 5th(?) -
>> >>
>> >> Ah, I haven't had so much fun in in years. My life since then seems
>> >> like an endless string of meetings, politics, and bugfixing.
>> >>
>> >> The code went from sim/paper, to code, to testing, to mainline linux
>> >> in another week. I wish more research went like that!
>> >>
>> >> commit 76e3cc126bb223013a6b9a0e2a51238d1ef2e409
>> >> Author: Eric Dumazet <edumazet@google.com>
>> >> Date:   Thu May 10 07:51:25 2012 +0000
>> >>
>> >>     codel: Controlled Delay AQM
>> >>
>> >> Now, as I recall the story, eric came up with fq_codel on a saturday
>> >> afternoon, so I guess that was may 5th - cinco de mayo!
>> >>
>> >> And that too, landed in mainline...
>> >>
>> >> commit 4b549a2ef4bef9965d97cbd992ba67930cd3e0fe
>> >> Author: Eric Dumazet <edumazet@google.com>
>> >> Date:   Fri May 11 09:30:50 2012 +0000
>> >>
>> >>     fq_codel: Fair Queue Codel AQM
>> >>
>> >> let's not forget tom herbert & BQL
>> >>
>> >> commit 75957ba36c05b979701e9ec64b37819adc12f830
>> >> Author: Tom Herbert <therbert@google.com>
>> >> Date:   Mon Nov 28 16:32:35 2011 +0000
>> >>
>> >>     dql: Dynamic queue limits
>> >>
>> >>     Implementation of dynamic queue limits (dql).  This is a libary
>> which
>> >>     allows a queue limit to be dynamically managed.  The goal of dql is
>> >>     to set the queue limit, number of objects to the queue, to be
>> minimized
>> >>     without allowing the queue to be starved.
>> >>
>> >>
>> >>
>> >>
>> >> > It was really amazing to me to watch
>> >> > something Van and I had been discussing (okay, arguing) about
>> privately for
>> >> > 6 months and I'd been tinkering with be turned into real code on
>> real
>> >> > networks.
>> >> > Jim Gettys is an incredible (and constructive) nagger, Eric Dumazet
>> and
>> >> > amazing
>> >> > coder, and the entire open source community a really nifty group of
>> folks.
>> >> >
>> >> > Maybe someday we will actually update the first article with some of
>> the
>> >> > stuff
>> >> > we got into the last internet draft....
>> >> >
>> >> >         be the change,
>> >> >                 Kathie
>> >> >
>> >> > On 5/14/14 2:01 PM, Rich Brown wrote:
>> >> >> Folks,
>> >> >>
>> >> >> I just noticed that the announcement for the first testable
>> >> >> implementation of fq_codel was two days ago today - 14 May
>> 2012.
>> >> >> Build 3.3.6-2 was the first to include working fq_codel.
>> >> >>
>> >>
>> https://lists.bufferbloat.net/pipermail/cerowrt-devel/2012-May/000233.html
>> >> >>
>> >> >>  Two years later, we see fq_codel being adopted lots of places.
>> As
>> >> >> more and more people/organizations come to understand the
>> problem,
>> >> >> and how straightforward the solution can be, we're beginning to
>> win
>> >> >> the battle to have a good Internet experience everywhere.
>> >> >>
>> >> >> Thanks to Dave, Eric, Kathie, Van, and all the members of this
>> list
>> >> >> for their perseverance, testing, comments, and patience.
>> >> >> Congratulations!
>> >> >>
>> >> >> Best regards,
>> >> >>
>> >> >> Rich _______________________________________________ Bloat
>> mailing
>> >> >> list Bloat@lists.bufferbloat.net
>> >> >> https://lists.bufferbloat.net/listinfo/bloat
>> >> >>
>> >> >
>> >> > _______________________________________________
>> >> > Bloat mailing list
>> >> > Bloat@lists.bufferbloat.net
>> >> > https://lists.bufferbloat.net/listinfo/bloat
>> >>
>> >>
>> >>
>> >> --
>> >> Dave Täht
>> >>
>> >> NSFW:
>> >>
>> https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
>> >> _______________________________________________
>> >> Cerowrt-devel mailing list
>> >> Cerowrt-devel@lists.bufferbloat.net
>> >> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>> >>_______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>>

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 10750 bytes --]

Both you and Dave Taft misunderstood my idea about standing queues not being the right way to encode congestion in switches. I do not say there would be no buffers for jitter. Nor do I call for admission control. I just suggest that instead of deriving congestion from backlog measures (requiring that there be backlogs created and sustained) one can derive congestion measures without sustainng a backlog...

The result is ballistic flows, if you will. Analogous to ballistic electrons in materials.

On May 15, 2014, David Lang <david@lang.hm> wrote:
>We are talking about different things then.
>
>The "fast lane" I'm talking about is where ISPs want companies to pay
>them for 
>bandwidth (in addition to the companies paying their own ISP for
>bandwidth and 
>in addition to their users paying them for bandwidth)
>
>As for your contention that an ideal Internet will have a buffer size
>of <1 
>packet. That will work, but that will not fully utilize the network,
>because 
>there will be jitter in the senders and some packet generation will be
>delayed, 
>leaving the network with nothing to send in that timeslot.
>
>If the network is not fully utilized, then fq_codel isn't needed, just
>send 
>packets as they arrive. It's only as a particular link approaches full 
>utilization that queues will build up and deciding what to do becomes 
>significant (and fq_codel and similar will matter)
>
>As for your thought of having an end-to-end feedback loop, the problem
>with that 
>is that it will only work if the path between them is stable and not
>interfered 
>with by other flows. In the Internet as we have it today, neither are
>true. The 
>packets for your connection may travel over different paths, and
>congestion 
>happens on a link-by-link basis with the combined packets of many
>connections, 
>not end-to-end based on a single connection.
>
>David Lang
>
>On Thu, 15 May 2014, dpreed@reed.com wrote:
>
>> I don't think that at all. I suspect you know that. But if I confused
>you, let 
>> me assure you that my view of the proper operating point of the
>Internet as a 
>> whole is that the expected buffer queue on any switch anywhere in the
>Internet 
>> is < 1 datagram.
>> 
>> fq_codel is a good start, but it still requires letting buffer
>queueing 
>> increase.  However, mathematically, one need not have the queues
>build up to 
>> sustain the control loop that fq_codel creates.
>> 
>> I conjecture that one can create an equally effective congestion
>control 
>> mechanism as fq_codel without any standing queues being allowed to
>build up. 
>> (Someone should try the exercise of trying to prove that an optimal
>end-to-end 
>> feedback control system requires queueing delay to be imposed. I've
>tried and 
>> it's led me to the conjecture that one can always replace a standing
>queue 
>> with a finite memory of past activities - and if one does, the lack
>of a 
>> standing queue means that the algorithm is better than any that end
>up with a 
>> standing queue).
>> 
>> fq_codel could be redesigned into a queue-free fq_codel.
>>
>>
>> On Thursday, May 15, 2014 7:46pm, "David Lang" <david@lang.hm> said:
>>
>>
>>
>>> If you think "fast lanes" will actually increase performance for any
>traffic,
>>> you are dreaming.
>>> 
>>> the people looking for "fast lanes" are't trying to get traffic
>through any
>>> faster, they are looking to charge more for the traffic they are
>already
>>> passing.
>>> 
>>> David Lang
>>>
>>>   On Thu, 15 May 2014, dpreed@reed.com wrote:
>>> 
>>> > Well done.  I'm optimistic for deployment everywhere, except
>CMTS's, the LTE
>>> and HSPA+ access networks, and all corporate firewall and intranet
>gear.
>>> >
>>> > The solution du jour is to leave bufferbloat in place, while using
>the real
>>> fads: prioritization (diffserv once we have the "fast lanes" fully
>legal) and
>>> "software defined networking" appliances that use DPI for packet
>routing and
>>> traffic management.
>>> >
>>> > Fixing buffer bloat allows the edge- and enterprise- networks to
>be more
>>> efficient, whereas not fixing it lets the edge networks move users
>up to more and
>>> more expensive "plans" due to frustration and to sell much more gear
>into
>>> Enterprises because they are easy marks for complex gadgets.
>>> >
>>> > But maybe a few engineers who operate and design gear for such
>networks will
>>> overcome the incredible business biases against fixing this.
>>> >
>>> > That's why all the efforts you guys have put forth are immensely
>worth it.  I
>>> think this is one of the best innovations in recent years (Bram
>Cohen's original
>>> BitTorrent is another, for fully decentralizing data delivery for
>the very first
>>> time in a brilliant way.) I will certainly push everywhere I can to
>see fq_codel
>>> deployed.
>>> >
>>> > If there were a prize for brilliant projects, this would be top on
>my list.
>>> >
>>> >
>>> >
>>> > On Wednesday, May 14, 2014 9:25pm, "Dave Taht"
><dave.taht@gmail.com>
>>> said:
>>> >
>>> >
>>> >
>>> >> On Wed, May 14, 2014 at 3:32 PM, Kathleen Nichols
>>> <nichols@pollere.com>
>>> >> wrote:
>>> >> >
>>> >> > Thanks, Rich.
>>> >> >
>>> >> > And to show you what an amazing bit of work that first fq_codel
>was,
>>> I have
>>> >> > on my calendar that I first "exposed" CoDel to a small group in
>a
>>> >> > meeting room
>>> >> > and on the phone at ISC on April 24.
>>> >>
>>> >> And we had all sorts of trouble with the phone, (eric didn't hear
>>> >> much) and we then spent hours and hours afterwards discussing
>wifi
>>> >> instead of codel... it was too much to take in...
>>> >>
>>> >> me, I'd started jumping up and down in excitement about 20
>minutes
>>> >> into kathies preso...
>>> >>
>>> >> May 3rd, 2012 was the last 24 hr coding stint I think I'll ever
>have.
>>> >>
>>> >>
>https://lists.bufferbloat.net/pipermail/codel/2012-May/000023.html
>>> >>
>>> >> Ahh, the good ole days, when bufferbloat was first solved and we
>all
>>> >> thought the internet would speed up overnight, and we were going
>to be
>>> >> rock stars, invited to all the best parties, acquire fame and
>fortune,
>>> >> and be awarded medals and given awards. Re-reading all this
>brought
>>> >> back memories.... (heck, there's still a couple good ideas in
>that
>>> >> thread left unimplemented)
>>> >>
>>> >>
>https://lists.bufferbloat.net/pipermail/codel/2012-May/thread.html
>>> >>
>>> >> It looks by may 5th we were getting in shape, and then there were
>a
>>> >> few other issues along the way with the control law and so on...
>and
>>> >> eric rewrote the whole thing and made it oodles faster and then
>as
>>> >> best as I recall came up with fq_codel on saturday may 5th(?) -
>>> >>
>>> >> Ah, I haven't had so much fun in in years. My life since then
>seems
>>> >> like an endless string of meetings, politics, and bugfixing.
>>> >>
>>> >> The code went from sim/paper, to code, to testing, to mainline
>linux
>>> >> in another week. I wish more research went like that!
>>> >>
>>> >> commit 76e3cc126bb223013a6b9a0e2a51238d1ef2e409
>>> >> Author: Eric Dumazet <edumazet@google.com>
>>> >> Date:   Thu May 10 07:51:25 2012 +0000
>>> >>
>>> >>     codel: Controlled Delay AQM
>>> >>
>>> >> Now, as I recall the story, eric came up with fq_codel on a
>saturday
>>> >> afternoon, so I guess that was may 5th - cinco de mayo!
>>> >>
>>> >> And that too, landed in mainline...
>>> >>
>>> >> commit 4b549a2ef4bef9965d97cbd992ba67930cd3e0fe
>>> >> Author: Eric Dumazet <edumazet@google.com>
>>> >> Date:   Fri May 11 09:30:50 2012 +0000
>>> >>
>>> >>     fq_codel: Fair Queue Codel AQM
>>> >>
>>> >> let's not forget tom herbert & BQL
>>> >>
>>> >> commit 75957ba36c05b979701e9ec64b37819adc12f830
>>> >> Author: Tom Herbert <therbert@google.com>
>>> >> Date:   Mon Nov 28 16:32:35 2011 +0000
>>> >>
>>> >>     dql: Dynamic queue limits
>>> >>
>>> >>     Implementation of dynamic queue limits (dql).  This is a
>libary
>>> which
>>> >>     allows a queue limit to be dynamically managed.  The goal of
>dql is
>>> >>     to set the queue limit, number of objects to the queue, to be
>>> minimized
>>> >>     without allowing the queue to be starved.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> > It was really amazing to me to watch
>>> >> > something Van and I had been discussing (okay, arguing) about
>>> privately for
>>> >> > 6 months and I'd been tinkering with be turned into real code
>on
>>> real
>>> >> > networks.
>>> >> > Jim Gettys is an incredible (and constructive) nagger, Eric
>Dumazet
>>> and
>>> >> > amazing
>>> >> > coder, and the entire open source community a really nifty
>group of
>>> folks.
>>> >> >
>>> >> > Maybe someday we will actually update the first article with
>some of
>>> the
>>> >> > stuff
>>> >> > we got into the last internet draft....
>>> >> >
>>> >> >         be the change,
>>> >> >                 Kathie
>>> >> >
>>> >> > On 5/14/14 2:01 PM, Rich Brown wrote:
>>> >> >> Folks,
>>> >> >>
>>> >> >> I just noticed that the announcement for the first testable
>>> >> >> implementation of fq_codel was two days ago today - 14 May
>>> 2012.
>>> >> >> Build 3.3.6-2 was the first to include working fq_codel.
>>> >> >>
>>> >>
>>>
>https://lists.bufferbloat.net/pipermail/cerowrt-devel/2012-May/000233.html
>>> >> >>
>>> >> >>  Two years later, we see fq_codel being adopted lots of
>places.
>>> As
>>> >> >> more and more people/organizations come to understand the
>>> problem,
>>> >> >> and how straightforward the solution can be, we're beginning
>to
>>> win
>>> >> >> the battle to have a good Internet experience everywhere.
>>> >> >>
>>> >> >> Thanks to Dave, Eric, Kathie, Van, and all the members of this
>>> list
>>> >> >> for their perseverance, testing, comments, and patience.
>>> >> >> Congratulations!
>>> >> >>
>>> >> >> Best regards,
>>> >> >>
>>> >> >> Rich _______________________________________________ Bloat
>>> mailing
>>> >> >> list Bloat@lists.bufferbloat.net
>>> >> >> https://lists.bufferbloat.net/listinfo/bloat
>>> >> >>
>>> >> >
>>> >> > _______________________________________________
>>> >> > Bloat mailing list
>>> >> > Bloat@lists.bufferbloat.net
>>> >> > https://lists.bufferbloat.net/listinfo/bloat
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Dave Täht
>>> >>
>>> >> NSFW:
>>> >>
>>>
>https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
>>> >> _______________________________________________
>>> >> Cerowrt-devel mailing list
>>> >> Cerowrt-devel@lists.bufferbloat.net
>>> >> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>> >>_______________________________________________
>>> Bloat mailing list
>>> Bloat@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/bloat
>>>

-- Sent from my Android device with K-@ Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 20062 bytes --]

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[David Lang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 11593 bytes --]

Well, if the link isn't congested, why do you need to do anything to the traffic 
other than forward it? You have no way of knowing what paths the traffic is 
going to follow once it hits the next router, so you don't know which streams 
are independent of each other.

Now, if you are saying that fq_codel can be enhanced to gather stats even when 
there is no congestion so that it has a better idea of what to do once 
congestion starts, then you may have a point.

but fq_codel is very happy to run and do basically nothing if there is no 
congestion. It doesn't delay things to create a buffer.

David Lang

  On Thu, 15 May 2014, David P. Reed wrote:

> Both you and Dave Taft misunderstood my idea about standing queues not being the right way to encode congestion in switches. I do not say there would be no buffers for jitter. Nor do I call for admission control. I just suggest that instead of deriving congestion from backlog measures (requiring that there be backlogs created and sustained) one can derive congestion measures without sustainng a backlog...
>
> The result is ballistic flows, if you will. Analogous to ballistic electrons in materials.
>
> On May 15, 2014, David Lang <david@lang.hm> wrote:
>> We are talking about different things then.
>>
>> The "fast lane" I'm talking about is where ISPs want companies to pay
>> them for
>> bandwidth (in addition to the companies paying their own ISP for
>> bandwidth and
>> in addition to their users paying them for bandwidth)
>>
>> As for your contention that an ideal Internet will have a buffer size
>> of <1
>> packet. That will work, but that will not fully utilize the network,
>> because
>> there will be jitter in the senders and some packet generation will be
>> delayed,
>> leaving the network with nothing to send in that timeslot.
>>
>> If the network is not fully utilized, then fq_codel isn't needed, just
>> send
>> packets as they arrive. It's only as a particular link approaches full
>> utilization that queues will build up and deciding what to do becomes
>> significant (and fq_codel and similar will matter)
>>
>> As for your thought of having an end-to-end feedback loop, the problem
>> with that
>> is that it will only work if the path between them is stable and not
>> interfered
>> with by other flows. In the Internet as we have it today, neither are
>> true. The
>> packets for your connection may travel over different paths, and
>> congestion
>> happens on a link-by-link basis with the combined packets of many
>> connections,
>> not end-to-end based on a single connection.
>>
>> David Lang
>>
>> On Thu, 15 May 2014, dpreed@reed.com wrote:
>>
>>> I don't think that at all. I suspect you know that. But if I confused
>> you, let
>>> me assure you that my view of the proper operating point of the
>> Internet as a
>>> whole is that the expected buffer queue on any switch anywhere in the
>> Internet
>>> is < 1 datagram.
>>>
>>> fq_codel is a good start, but it still requires letting buffer
>> queueing
>>> increase.  However, mathematically, one need not have the queues
>> build up to
>>> sustain the control loop that fq_codel creates.
>>>
>>> I conjecture that one can create an equally effective congestion
>> control
>>> mechanism as fq_codel without any standing queues being allowed to
>> build up.
>>> (Someone should try the exercise of trying to prove that an optimal
>> end-to-end
>>> feedback control system requires queueing delay to be imposed. I've
>> tried and
>>> it's led me to the conjecture that one can always replace a standing
>> queue
>>> with a finite memory of past activities - and if one does, the lack
>> of a
>>> standing queue means that the algorithm is better than any that end
>> up with a
>>> standing queue).
>>>
>>> fq_codel could be redesigned into a queue-free fq_codel.
>>>
>>>
>>> On Thursday, May 15, 2014 7:46pm, "David Lang" <david@lang.hm> said:
>>>
>>>
>>>
>>>> If you think "fast lanes" will actually increase performance for any
>> traffic,
>>>> you are dreaming.
>>>>
>>>> the people looking for "fast lanes" are't trying to get traffic
>> through any
>>>> faster, they are looking to charge more for the traffic they are
>> already
>>>> passing.
>>>>
>>>> David Lang
>>>>
>>>>   On Thu, 15 May 2014, dpreed@reed.com wrote:
>>>>
>>>>> Well done.  I'm optimistic for deployment everywhere, except
>> CMTS's, the LTE
>>>> and HSPA+ access networks, and all corporate firewall and intranet
>> gear.
>>>>>
>>>>> The solution du jour is to leave bufferbloat in place, while using
>> the real
>>>> fads: prioritization (diffserv once we have the "fast lanes" fully
>> legal) and
>>>> "software defined networking" appliances that use DPI for packet
>> routing and
>>>> traffic management.
>>>>>
>>>>> Fixing buffer bloat allows the edge- and enterprise- networks to
>> be more
>>>> efficient, whereas not fixing it lets the edge networks move users
>> up to more and
>>>> more expensive "plans" due to frustration and to sell much more gear
>> into
>>>> Enterprises because they are easy marks for complex gadgets.
>>>>>
>>>>> But maybe a few engineers who operate and design gear for such
>> networks will
>>>> overcome the incredible business biases against fixing this.
>>>>>
>>>>> That's why all the efforts you guys have put forth are immensely
>> worth it.  I
>>>> think this is one of the best innovations in recent years (Bram
>> Cohen's original
>>>> BitTorrent is another, for fully decentralizing data delivery for
>> the very first
>>>> time in a brilliant way.) I will certainly push everywhere I can to
>> see fq_codel
>>>> deployed.
>>>>>
>>>>> If there were a prize for brilliant projects, this would be top on
>> my list.
>>>>>
>>>>>
>>>>>
>>>>> On Wednesday, May 14, 2014 9:25pm, "Dave Taht"
>> <dave.taht@gmail.com>
>>>> said:
>>>>>
>>>>>
>>>>>
>>>>>> On Wed, May 14, 2014 at 3:32 PM, Kathleen Nichols
>>>> <nichols@pollere.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Thanks, Rich.
>>>>>>>
>>>>>>> And to show you what an amazing bit of work that first fq_codel
>> was,
>>>> I have
>>>>>>> on my calendar that I first "exposed" CoDel to a small group in
>> a
>>>>>>> meeting room
>>>>>>> and on the phone at ISC on April 24.
>>>>>>
>>>>>> And we had all sorts of trouble with the phone, (eric didn't hear
>>>>>> much) and we then spent hours and hours afterwards discussing
>> wifi
>>>>>> instead of codel... it was too much to take in...
>>>>>>
>>>>>> me, I'd started jumping up and down in excitement about 20
>> minutes
>>>>>> into kathies preso...
>>>>>>
>>>>>> May 3rd, 2012 was the last 24 hr coding stint I think I'll ever
>> have.
>>>>>>
>>>>>>
>> https://lists.bufferbloat.net/pipermail/codel/2012-May/000023.html
>>>>>>
>>>>>> Ahh, the good ole days, when bufferbloat was first solved and we
>> all
>>>>>> thought the internet would speed up overnight, and we were going
>> to be
>>>>>> rock stars, invited to all the best parties, acquire fame and
>> fortune,
>>>>>> and be awarded medals and given awards. Re-reading all this
>> brought
>>>>>> back memories.... (heck, there's still a couple good ideas in
>> that
>>>>>> thread left unimplemented)
>>>>>>
>>>>>>
>> https://lists.bufferbloat.net/pipermail/codel/2012-May/thread.html
>>>>>>
>>>>>> It looks by may 5th we were getting in shape, and then there were
>> a
>>>>>> few other issues along the way with the control law and so on...
>> and
>>>>>> eric rewrote the whole thing and made it oodles faster and then
>> as
>>>>>> best as I recall came up with fq_codel on saturday may 5th(?) -
>>>>>>
>>>>>> Ah, I haven't had so much fun in in years. My life since then
>> seems
>>>>>> like an endless string of meetings, politics, and bugfixing.
>>>>>>
>>>>>> The code went from sim/paper, to code, to testing, to mainline
>> linux
>>>>>> in another week. I wish more research went like that!
>>>>>>
>>>>>> commit 76e3cc126bb223013a6b9a0e2a51238d1ef2e409
>>>>>> Author: Eric Dumazet <edumazet@google.com>
>>>>>> Date:   Thu May 10 07:51:25 2012 +0000
>>>>>>
>>>>>>     codel: Controlled Delay AQM
>>>>>>
>>>>>> Now, as I recall the story, eric came up with fq_codel on a
>> saturday
>>>>>> afternoon, so I guess that was may 5th - cinco de mayo!
>>>>>>
>>>>>> And that too, landed in mainline...
>>>>>>
>>>>>> commit 4b549a2ef4bef9965d97cbd992ba67930cd3e0fe
>>>>>> Author: Eric Dumazet <edumazet@google.com>
>>>>>> Date:   Fri May 11 09:30:50 2012 +0000
>>>>>>
>>>>>>     fq_codel: Fair Queue Codel AQM
>>>>>>
>>>>>> let's not forget tom herbert & BQL
>>>>>>
>>>>>> commit 75957ba36c05b979701e9ec64b37819adc12f830
>>>>>> Author: Tom Herbert <therbert@google.com>
>>>>>> Date:   Mon Nov 28 16:32:35 2011 +0000
>>>>>>
>>>>>>     dql: Dynamic queue limits
>>>>>>
>>>>>>     Implementation of dynamic queue limits (dql).  This is a
>> libary
>>>> which
>>>>>>     allows a queue limit to be dynamically managed.  The goal of
>> dql is
>>>>>>     to set the queue limit, number of objects to the queue, to be
>>>> minimized
>>>>>>     without allowing the queue to be starved.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> It was really amazing to me to watch
>>>>>>> something Van and I had been discussing (okay, arguing) about
>>>> privately for
>>>>>>> 6 months and I'd been tinkering with be turned into real code
>> on
>>>> real
>>>>>>> networks.
>>>>>>> Jim Gettys is an incredible (and constructive) nagger, Eric
>> Dumazet
>>>> and
>>>>>>> amazing
>>>>>>> coder, and the entire open source community a really nifty
>> group of
>>>> folks.
>>>>>>>
>>>>>>> Maybe someday we will actually update the first article with
>> some of
>>>> the
>>>>>>> stuff
>>>>>>> we got into the last internet draft....
>>>>>>>
>>>>>>>         be the change,
>>>>>>>                 Kathie
>>>>>>>
>>>>>>> On 5/14/14 2:01 PM, Rich Brown wrote:
>>>>>>>> Folks,
>>>>>>>>
>>>>>>>> I just noticed that the announcement for the first testable
>>>>>>>> implementation of fq_codel was two days ago today - 14 May
>>>> 2012.
>>>>>>>> Build 3.3.6-2 was the first to include working fq_codel.
>>>>>>>>
>>>>>>
>>>>
>> https://lists.bufferbloat.net/pipermail/cerowrt-devel/2012-May/000233.html
>>>>>>>>
>>>>>>>>  Two years later, we see fq_codel being adopted lots of
>> places.
>>>> As
>>>>>>>> more and more people/organizations come to understand the
>>>> problem,
>>>>>>>> and how straightforward the solution can be, we're beginning
>> to
>>>> win
>>>>>>>> the battle to have a good Internet experience everywhere.
>>>>>>>>
>>>>>>>> Thanks to Dave, Eric, Kathie, Van, and all the members of this
>>>> list
>>>>>>>> for their perseverance, testing, comments, and patience.
>>>>>>>> Congratulations!
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>>
>>>>>>>> Rich _______________________________________________ Bloat
>>>> mailing
>>>>>>>> list Bloat@lists.bufferbloat.net
>>>>>>>> https://lists.bufferbloat.net/listinfo/bloat
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Bloat mailing list
>>>>>>> Bloat@lists.bufferbloat.net
>>>>>>> https://lists.bufferbloat.net/listinfo/bloat
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Dave Täht
>>>>>>
>>>>>> NSFW:
>>>>>>
>>>>
>> https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
>>>>>> _______________________________________________
>>>>>> Cerowrt-devel mailing list
>>>>>> Cerowrt-devel@lists.bufferbloat.net
>>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>>>>> _______________________________________________
>>>> Bloat mailing list
>>>> Bloat@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/bloat
>>>>
>
> -- Sent from my Android device with K-@ Mail. Please excuse my brevity.

Re: [Bloat] [Cerowrt-devel] fq_codel is two years old

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 2246 bytes --]

I'll answer this way... The endpoints can use information to slow down as early as possible. That's the whole point of control loop tuning. The fundamental resonance of a control loop depends on its speed of draining and filling the storage element.

So you want to sample and deliver ASAP two things in a network that is trying to stay in a ballistic state. One is the aggregate instantaneous controlled (by TCP receiver windows and individual application demand) input rate affecting each switch and the other is the buffered amount on a path.

The two factors are not the same, and buffered amount wants to be minimized. For max throughput you need only be in ballistic-maximum buffering. Which is the phase when the bottleneck buffers always have a packet to send but no more (analogous to double buffering).

This is a phase in phase space. There can be waves of buffer expansion traveling in the medium at the critical phase boundary, but you don't want to allow a phase transition to backlogged because then the buffers begin to back up and the time to drain adds to the time to recover from sustained descent into colllapse.

All flow receivers ideally would be receiving the two sampled measures.

You are right that no action can be taken at a switch... but flow receivers can use earlier sampled measures to decide to take action more quickly to prevent incipient buffer growth.

Though the situation is different in a highway network, the phase when a jam has developed at an intersection is similar. You want a highway system to operate at the point where no sustained jams exist.

On May 15, 2014, David Lang <david@lang.hm> wrote:

Well, if the link isn't congested, why do you need to do anything to the traffic 
other than forward it? You have no way of knowing what paths the traffic is 
going to follow once it hits the next router, so you don't know which streams 
are independent of each other.

Now, if you are saying that fq_codel can be enhanced to gather stats even when 
there is no congestion so that it has a better idea of what to do once 
congestion starts, then you may have a point.

but fq_codel is very happy to run and do basically nothing if there is no 
congestion. It doesn't delay things to create a buffer.

[-- Attachment #2: Type: text/html, Size: 2361 bytes --]