[Bloat] RED against bufferbloat

Wed Feb 25 12:57:54 EST 2015

On Wed, Feb 25, 2015 at 12:06 AM, Bob Briscoe <bob.briscoe at bt.com> wrote:
> Sahil,
>
> At 06:46 25/02/2015, Mikael Abrahamsson wrote:
>>
>> On Tue, 24 Feb 2015, sahil grover wrote:
>>
>>> (i) First of all,i want to know whether RED was implemented or not?
>>> if not then what were the reasons(major) ?
>>
>>
>> RED has been available on most platforms, but it was generally not turned
>> on. It also needs configuration from an operator, and it's hard to know how
>> to configure.
>
>
> About a decade ago my company (BT) widely deployed RED in the upstream
> 'head-end' of our global MPLS network, i.e. the likely bottleneck in the
> customer edge router where the customer's LAN traffic enters their access
> link. We deployed it as WRED, i.e. different configurations of RED across
> the various diffserv classes, in order to minimise queuing latency in all
> the classes, including the lowest priority class. A configuration calculator
> was developed to help the engineers during set up. We still use this setup
> successfuly today, including for all our particularly latency sensitive
> customers in the finance sector.

Thank you for more public detail on this than you made available before.

I note note that free.fr deployed SFQ also about a decade ago. And
DRR+fq_codel 3 years ago. And I am told that SQF deployed also in
various places. I remain bugged I have not managed to see a study that
actually looked for FQ´s  characteristic signatures on data at scale.

In polling various ISPs at various conferences through the world - I
realize this is unscientific - uknof had 3? 4? RED users - and it was
about 30% HFSC + SFQ for downstream shaping among the rest of the
room, and about 120 in the room.

nznog (about 150 in the room) - 0 RED, 1/3 "some form of FQ".
Certainly in their world of highly mixed RTTs on and off the island I
can see why FQ was preferred. I also found from poking on their
networks that they had an unbelievable amount of bad policers
configured, with one test showing it kicking in with about 3ms of
buffering. That was what decided me that I should probably get around
to finishing the kinder/gentler "bobbie" policer - not that I have any
hope it would deploy where needed in under 5 years.

> We did not deploy RED on our broadband platform (ie public Internet), altho
> in retrospect we should have done, because any AQM is much better than none.
> We're fixing that now.

Applause. I have always said, that if you can figure out how to deploy
RED, do so, and I believe I did pester the universe for better tools
to figure out how to better automagically configure it.

>>> (ii)Second, as we all know RED controls the  average queue size from
>>> growing.
>>> So it also controls delay in a way or  we can say  is a solution to
>>> bufferbloat problem. Then why it was not considered.
>>
>>
>> It was designed to fix "bufferbloat" long before the bufferbloat word was
>> even invented. It's just that in practice, it doesn't work very well. RED is
>> configured with a drop probability slope at certain buffer depths, and
>> that's it. It doesn't react or change depending on conditions. You have to
>> guess at configure-time.
>>
>> What we need are mechanisms that work better in real life and that are
>> adaptive.
>
>
> If you were prepared to read a paper, I would have suggested:
> "The New AQM Kids on the Block: An Experimental Evaluation of CoDel and PIE"
> <http://infocom2014.ieee-infocom.org/GI14-slides/GI14-s2-3.pdf>
>
> This compares CoDel and PIE against Adaptive RED, which was a variant of RED
> proposed by Sally Floyd & co-authors in 2001 and available since Linux
> kernel version 3.3. ARED addressed the configuration sensitivity problem of
> RED by adapting the parameters to link rate and load conditions.
>
> The paper convinced me that ARED is good enough (in the paper's simulations
> it was often better than PIE or CoDel), at least for links with fixed rate
> (or only occasionally varying rate like DSL).* This is important for us
> because it means we can consider deploying AQM by adding soft controls on
> top of the RED implementations we already have in existing equipment. This
> could reduce deployment completion time from decades to a few months.

ARED is easier to configure, but I gotta admit, after having lunch and
multiple conversations with the authors 2 years back that I had
expected the follow-on work addressing my concerns to come out long
before now.

I felt that paper was massively flawed on multiple accounts and while
I see it has now been pimped into multiple publications... (which I
now understand takes a lot of work, far more than doing the
experiments themselves...)

Nobody seems to have noticed that all it tested was:

"More realistic traffic types (here, only bulk TCP traffic) including
bursty traffic"

And that doing sane analysis of more latency sensitive traffic - like
web, webrtc, voip, etc - under load in either direction - what I am
perpetually harping on - was "reserved for future work. " I am still
waiting for competent papers tackling that , and increasingly bugged
by the stream of papers that still don´t address the problem or repeat
the experiments that kicked off the whole bufferbloat effort in the
first place. (and thus I have two rants on the codel list from
yesterday). Thankfully - I have had a chance to review a few papers
prepublication in the last few months and rejected them for a variety
of truly egregious flaws... so nobody else has to suffer through them.

I like toke´s stuff FQ vs AQM work much better in comparison - which
does tackle those subjects. FQ wins over AQM alone, FQ+AQM reduces the
impact of hash collisions. He has an enormous backing set of data that
wont fit into less than 5 papers, and sadly, the first core  paper -
needed for all the follow-ons covering things like ECN, or
improvements to TCP - hasn't found publication outside the ietf as yet
- although he did get a nice tour of stanford, caltech, google, and
ucsb out of it, and got the stanford and google talks filmed - and I
think got a couple changes to TCP made out of it... and there is still
a pending improvement or two to codel... so even unpublished it has
made more impact than most of the published stuff, combined.

At caltech, Kleinrock was there (toke: "what a nice guy!") and
kleinrock listened to the preso... went back to his office, pulled out
a tattered blue mimeographed sheet from 1962 or so talking about
generalized processor sharing, and said: "Yup. Only took 50 years..."

> * I'm not sure ARED would be able to cope with the rapidly changing rate of
> a wireless link tho.

It isn´t. Neither are codel or pie (unmodified), as they work on
packets not aggregates and TXOPs, nor are sensitive to retries, and
other forms of jitter on a wifi link, like powersave.

We have made a LOT of progress on fixing wifi lately on a variety of
fronts and have most of the infrastructure architecturally in place to
massively improve things there on the only two chipsets we can fix
(mt72, ath9k). (please note there is so much else wrong in wifi that
bufferbloat is only a piece of the overall list of things we plan to
fix) The first bit of code for that has landed for review in
linux-wireless - It is called minstrel-blues which does "coupled rate
control and tx power management" - and it is *wonderful*.

http://www.linuxplumbersconf.net/2014/ocw//system/presentations/2439/original/Minstrel-Blues%20@LinuxPlumbers%202014.pdf

There is no need to transmit at the highest power if your device is
right next to the AP.
We got some pretty good test results back from it last weekend.

There is another fix to minstrel andrew just did that improves tcp
performance by - well, still measuring - it looks GREAT.

There are about 6 more things coming up for wifi over the next few
months... which includes a modified version of a fq_codel like
algorithm - so long as more funding lands... more people show up to
help... and I stop paying attention to email.  I am done talking, I am
off doing.

> HTH
>
>
> Bob
>
>
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

-- 
Dave Täht
Let's make wifi fast, less jittery and reliable again!

https://plus.google.com/u/0/107942175615993706558/posts/TVX3o84jjmb