[Bloat] geoff huston's take on BBR

[Bless, Roland (TM)]

Hi Geoff,

see inline.

Am 12.06.2018 um 09:49 schrieb Geoff Huston:
>> On 12 Jun 2018, at 4:55 pm, Dave Taht <dave.taht at gmail.com> wrote:
>>
>> On Mon, Jun 11, 2018 at 10:58 PM, Kevin Darbyshire-Bryant
>> <kevin at darbyshire-bryant.me.uk> wrote:
>>>
>>>
>>>> On 11 Jun 2018, at 22:27, Dave Taht <dave.taht at gmail.com> wrote:
>>>>
>>>> https://ripe76.ripe.net/presentations/10-2018-05-15-bbr.pdf
>>>
>>> Fascinating!
>>>
>>> "       • BBR changes all those assumptions, and could potentially push many networks into sustained instability
>>>
>>>        • –  We cannot use the conventional network control mechanisms to regulate BBR flows
>>>
>>> • Selective packet drop just won’t create back pressure on the flow”
>>>
>>> And I keep on seeing questions on whether BBR understands ECN - if not…. well I think we see the results.
>>
>> I think geoff goofed in his testing of BBR, starting all flows at the
>> same time, thus syncing up their probing periods. Real traffic is not
>> correlated this way.
>> (I made the same mistake on my initial bbr testing)
> 
> no - I started the flows at 10, 20 and 30 seconds after the initial flow started.

This is nevertheless advantageous for BBR, since it performs its
ProbeRTT phase every 10s. So using 11, 23, and 34 seconds may
make a difference in convergence speed (that was at least observed
in our experiments).
>> I do agree that bbr treats aqm drops as "noise", not backpressure. And
>> bbr scares me.
>>
>> I look forward very much to bbr one day soon doing some sort of sane,
>> conservative, response to ecn marks.
>>
> 
> 
> I’m not sure that I understand this comment.
> 
> Part of the pressure going on here is the issue of whether the endpoints can and should trust the signals and.or manipulation that they get from the network infrastructure. BBR is using a different form of feedback to control its send rate. Essentially BBR is taking a delay variance measurement 1 / 8 of the time to adjust its internal model of the end-to-end delay bandwidth product (every 8th RTT). ECN provides a constant information flow, and this certainly matches the requirements of loss-based TCP, where every ACK contributes to the TCP flow dynamic, but it does not seem to me to be a good match to BBR’s requirements. 

I don't think that this characterization of BBR is completely accurate.
BBR is not measuring the "delay variance" (though it could) in ProbeBW,
but it measures the maximally achieved delivery rate while sending at an
increased data rate. So even after the onset of queueing, a BBR sender
will see a higher achieved delivery rate (at the cost of other competing
flows). BBR also calculates a BDP estimate, but it is only used to
calculated the "inflight cap". Especially the measured RTT is only used
in this calculation and nowhere else in BBRv1.0.

> The idea with BBR is to drive the network path such at the internal routers are sitting just at the initial onset of queuing. In theory ECN will not trigger at the onset of queuing, but will trigger later in the cycle of queue buildup.

Yep, but a BBR sender doesn't detect the onset of queuing and steadily
increases its amount of inflight data until the cap of 2BDP is reached.
In total, the queue can reach 1.5 BDP.

>> PS having fq on the link makes cubic and bbr cohabitate just fine.
>> fq_codel vs bbr behavior was reasonable, though bbr lost a lot more
>> packets before finding a decent state.
> 
> I guess that "It Depends" - my long delay experiments in the presentation referenced above showed cubic being completely crowded out by BBR.

Yes. This will always happen in case the bottleneck buffer is smaller
than 1 BDP (see slide 9 of
https://datatracker.ietf.org/meeting/100/materials/slides-100-iccrg-an-experimental-evaluation-of-bbr-congestion-control-00).

Regards,
 Roland