[Bloat] geoff huston's take on BBR

[Dave Taht]

On Tue, Jun 12, 2018 at 12:49 AM, Geoff Huston <gih at apnic.net> wrote:
>
>
>> On 12 Jun 2018, at 4:55 pm, Dave Taht <dave.taht at gmail.com> wrote:
>>
>> On Mon, Jun 11, 2018 at 10:58 PM, Kevin Darbyshire-Bryant
>> <kevin at darbyshire-bryant.me.uk> wrote:
>>>
>>>
>>>> On 11 Jun 2018, at 22:27, Dave Taht <dave.taht at gmail.com> wrote:
>>>>
>>>> https://ripe76.ripe.net/presentations/10-2018-05-15-bbr.pdf
>>>
>>> Fascinating!
>>>
>>> "       • BBR changes all those assumptions, and could potentially push many networks into sustained instability
>>>
>>>        • –  We cannot use the conventional network control mechanisms to regulate BBR flows
>>>
>>> • Selective packet drop just won’t create back pressure on the flow”
>>>
>>> And I keep on seeing questions on whether BBR understands ECN - if not…. well I think we see the results.
>>
>> I think geoff goofed in his testing of BBR, starting all flows at the
>> same time, thus syncing up their probing periods. Real traffic is not
>> correlated this way.
>> (I made the same mistake on my initial bbr testing)
>
> no - I started the flows at 10, 20 and 30 seconds after the initial flow started.

k.

>
>>
>> I do agree that bbr treats aqm drops as "noise", not backpressure. And
>> bbr scares me.
>>
>> I look forward very much to bbr one day soon doing some sort of sane,
>> conservative, response to ecn marks.
>>
>
>
> I’m not sure that I understand this comment.
>
> Part of the pressure going on here is the issue of whether the endpoints can and should trust the signals and.or manipulation that they get from the network infrastructure. BBR is using a different form of feedback to control its send rate. Essentially BBR is taking a delay variance measurement 1 / 8 of the time to adjust its internal model of the end-to-end delay bandwidth product (every 8th RTT). ECN provides a constant information flow, and this certainly matches the requirements of loss-based TCP, where every ACK contributes to the TCP flow dynamic, but it does not seem to me to be a good match to BBR’s requirements.

ECN CE is explicit. Someone on the path is saying "slow down". If BBR
merely responded like cubic to an ECN mark, it would be a win.

This of course doesn't handle the malicious middlebox or sender
problem, but neither does anything else.

I was pleased to see recently from
https://www.ietf.org/proceedings/98/slides/slides-98-maprg-tcp-ecn-experience-with-enabling-ecn-on-the-internet-padma-bhooma-00.pdf
that france was showing 6% CE markings on uplnks. Since free.fr
deployed fq_codel in 2011, I figure that's almost entirely from their
deployment. I was boggled at the argentine result (30%???), although I
know openwrt and derivatives with sqm is very popular there, that
can't explain all of that.

> The idea with BBR is to drive the network path such at the internal routers are sitting just at the initial onset of queuing. In theory ECN will not trigger at the onset of queuing, but will trigger later in the cycle of queue buildup.

a few ms of queue is not a problem.... we (in the fq_codel world) are
seeing 5ms queues and full throughput on ethernet, 15-20ms
on wifi. (and BBR and cubic share almost equally, all the time, at
least at the range of rates we've tested)

Wifi: https://arxiv.org/pdf/1703.00064.pdf
Cake shaper: https://arxiv.org/abs/1804.07617

There's a paper with the BBR vs cubic vs fq_codel result around here somewhere.

I am painfully aware that upgrading edge routers and other bottleneck
devices to have smart queue management is a long game...

... but I long ago gave up on pure end to end approaches.

>
>
>> PS having fq on the link makes cubic and bbr cohabitate just fine.
>> fq_codel vs bbr behavior was reasonable, though bbr lost a lot more
>> packets before finding a decent state.
>
> I guess that "It Depends" - my long delay experiments in the presentation referenced above showed cubic being completely crowded out by BBR.

Please give sqm or cake a shot.

>
> Geoff
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619