[Bloat] fast lanes

Dave Taht dave.taht at gmail.com
Thu May 15 21:03:37 EDT 2014

On Thu, May 15, 2014 at 4:46 PM, David Lang <david at lang.hm> wrote:
> If you think "fast lanes" will actually increase performance for any
> traffic, you are dreaming.

I have a huge problem with the fast lane analogy. The bigger ISPs have
national backbones of their own that could be used to transport traffic.

If you are a netflix/google/CDN provider and you are shipping 10Gb/sec
into an ISPs (or more likely 100Gb+) local data center it makes sense
to co-locate one of your boxes inside their data center rather than go
through a third party peering with you both in that data center.

If there is regulation proposed it should be able achieving fair pricing
and availability for co-locating boxes in ISP's data centers, but that
doesn't seem
to be what the debate is about. If that ISP is both refusing to  upgrade
their peering to provide better service to their customers to meet demand,
and not allowing "competing" services into their datacenter, then that's a

ISPs certainly tended in the past toward locating critical latency and bandwidth
sensitive caching services inside their data center as it additionally
saved on interconnect costs. DNS servers are one example, many CDNs
are others. It was a joke to believe promises that a big media merger
would not lead to disputes of this sort over "competing" streaming

A side note: It's taken me a long time to finally realize what was
wrong with level3's recent blog posting here:


The loss chart on the right here that they show to support their
argument that the current interconnects are "incurring excessive delay
and loss" fails utterly to support their argument -


It SHOULD show diurnal variance, and doesn't. The loss rates it shows
are both consistent and quite low, compared to the bandwidth being
used up, (indicating more of a cabling problem? or excessive load on
the other switch? or?)

They are either 1) lying, 2) using the wrong chart, or 3) not in a
position to measure actual packet loss from this vantage point.

Anybody know? I am nominally on level3's side but the data the are
presenting doesn't support their argument here.

I'd have liked it if they published actual delays (some anecdotal
evidence says it's higher than 100ms and all someone has to do is
setup some smokepings between two interconnect points to find out
more), and whether or not they were using some AQM algorithm supported
on their (jupiter, I think) hardware to cut delays down to something

> the people looking for "fast lanes" are't trying to get traffic through any
> faster, they are looking to charge more for the traffic they are already
> passing.

Agreed. some big ISPs seem more awanting to spend money on M&A and
politics vs engineering or capex.

> David Lang
>  On Thu, 15 May 2014, dpreed at reed.com wrote:
>> Well done.  I'm optimistic for deployment everywhere, except CMTS's, the
>> LTE and HSPA+ access networks, and all corporate firewall and intranet gear.
>> The solution du jour is to leave bufferbloat in place, while using the
>> real fads: prioritization (diffserv once we have the "fast lanes" fully
>> legal) and "software defined networking" appliances that use DPI for packet
>> routing and traffic management.
>> Fixing buffer bloat allows the edge- and enterprise- networks to be more
>> efficient, whereas not fixing it lets the edge networks move users up to
>> more and more expensive "plans" due to frustration and to sell much more
>> gear into Enterprises because they are easy marks for complex gadgets.
>> But maybe a few engineers who operate and design gear for such networks
>> will overcome the incredible business biases against fixing this.
>> That's why all the efforts you guys have put forth are immensely worth it.
>> I think this is one of the best innovations in recent years (Bram Cohen's
>> original BitTorrent is another, for fully decentralizing data delivery for
>> the very first time in a brilliant way.) I will certainly push everywhere I
>> can to see fq_codel deployed.
>> If there were a prize for brilliant projects, this would be top on my
>> list.
>> On Wednesday, May 14, 2014 9:25pm, "Dave Taht" <dave.taht at gmail.com> said:
>>> On Wed, May 14, 2014 at 3:32 PM, Kathleen Nichols <nichols at pollere.com>
>>> wrote:
>>> >
>>> > Thanks, Rich.
>>> >
>>> > And to show you what an amazing bit of work that first fq_codel was, I
>>> > have
>>> > on my calendar that I first "exposed" CoDel to a small group in a
>>> > meeting room
>>> > and on the phone at ISC on April 24.
>>> And we had all sorts of trouble with the phone, (eric didn't hear
>>> much) and we then spent hours and hours afterwards discussing wifi
>>> instead of codel... it was too much to take in...
>>> me, I'd started jumping up and down in excitement about 20 minutes
>>> into kathies preso...
>>> May 3rd, 2012 was the last 24 hr coding stint I think I'll ever have.
>>> https://lists.bufferbloat.net/pipermail/codel/2012-May/000023.html
>>> Ahh, the good ole days, when bufferbloat was first solved and we all
>>> thought the internet would speed up overnight, and we were going to be
>>> rock stars, invited to all the best parties, acquire fame and fortune,
>>> and be awarded medals and given awards. Re-reading all this brought
>>> back memories.... (heck, there's still a couple good ideas in that
>>> thread left unimplemented)
>>> https://lists.bufferbloat.net/pipermail/codel/2012-May/thread.html
>>> It looks by may 5th we were getting in shape, and then there were a
>>> few other issues along the way with the control law and so on... and
>>> eric rewrote the whole thing and made it oodles faster and then as
>>> best as I recall came up with fq_codel on saturday may 5th(?) -
>>> Ah, I haven't had so much fun in in years. My life since then seems
>>> like an endless string of meetings, politics, and bugfixing.
>>> The code went from sim/paper, to code, to testing, to mainline linux
>>> in another week. I wish more research went like that!
>>> commit 76e3cc126bb223013a6b9a0e2a51238d1ef2e409
>>> Author: Eric Dumazet <edumazet at google.com>
>>> Date:   Thu May 10 07:51:25 2012 +0000
>>>     codel: Controlled Delay AQM
>>> Now, as I recall the story, eric came up with fq_codel on a saturday
>>> afternoon, so I guess that was may 5th - cinco de mayo!
>>> And that too, landed in mainline...
>>> commit 4b549a2ef4bef9965d97cbd992ba67930cd3e0fe
>>> Author: Eric Dumazet <edumazet at google.com>
>>> Date:   Fri May 11 09:30:50 2012 +0000
>>>     fq_codel: Fair Queue Codel AQM
>>> let's not forget tom herbert & BQL
>>> commit 75957ba36c05b979701e9ec64b37819adc12f830
>>> Author: Tom Herbert <therbert at google.com>
>>> Date:   Mon Nov 28 16:32:35 2011 +0000
>>>     dql: Dynamic queue limits
>>>     Implementation of dynamic queue limits (dql).  This is a libary which
>>>     allows a queue limit to be dynamically managed.  The goal of dql is
>>>     to set the queue limit, number of objects to the queue, to be
>>> minimized
>>>     without allowing the queue to be starved.
>>> > It was really amazing to me to watch
>>> > something Van and I had been discussing (okay, arguing) about privately
>>> > for
>>> > 6 months and I'd been tinkering with be turned into real code on real
>>> > networks.
>>> > Jim Gettys is an incredible (and constructive) nagger, Eric Dumazet and
>>> > amazing
>>> > coder, and the entire open source community a really nifty group of
>>> > folks.
>>> >
>>> > Maybe someday we will actually update the first article with some of
>>> > the
>>> > stuff
>>> > we got into the last internet draft....
>>> >
>>> >         be the change,
>>> >                 Kathie
>>> >
>>> > On 5/14/14 2:01 PM, Rich Brown wrote:
>>> >> Folks,
>>> >>
>>> >> I just noticed that the announcement for the first testable
>>> >> implementation of fq_codel was two days ago today - 14 May 2012.
>>> >> Build 3.3.6-2 was the first to include working fq_codel.
>>> >>
>>> https://lists.bufferbloat.net/pipermail/cerowrt-devel/2012-May/000233.html
>>> >>
>>> >>  Two years later, we see fq_codel being adopted lots of places. As
>>> >> more and more people/organizations come to understand the problem,
>>> >> and how straightforward the solution can be, we're beginning to win
>>> >> the battle to have a good Internet experience everywhere.
>>> >>
>>> >> Thanks to Dave, Eric, Kathie, Van, and all the members of this list
>>> >> for their perseverance, testing, comments, and patience.
>>> >> Congratulations!
>>> >>
>>> >> Best regards,
>>> >>
>>> >> Rich _______________________________________________ Bloat mailing
>>> >> list Bloat at lists.bufferbloat.net
>>> >> https://lists.bufferbloat.net/listinfo/bloat
>>> >>
>>> >
>>> > _______________________________________________
>>> > Bloat mailing list
>>> > Bloat at lists.bufferbloat.net
>>> > https://lists.bufferbloat.net/listinfo/bloat
>>> --
>>> Dave Täht
>>> NSFW:
>>> https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
>>> _______________________________________________
>>> Cerowrt-devel mailing list
>>> Cerowrt-devel at lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Dave Täht

NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article

More information about the Bloat mailing list