Re: [Cake] cake default target is too low for bbr?

[Jonathan Morton <chromatix99@gmail.com>]

> On 29 Apr, 2017, at 01:26, Andy Furniss <adf.lists@gmail.com> wrote:
> 
>>>> As I understand it, increase in RTT due to queueing of packets is
>>>> the main feedback mechanism for BBR. So dropping packets, which I
>>>> already consider harmful, is really harmful with BBR because
>>>> you're not telling the sender to slow down.

Actually, BBR considers mainly a measurement of “delivery rate”, and paces its sending to match that.  It does *not* primarily rely on a congestion window as most TCPs do; one is provided only as a safety net of last resort.

Measurements of RTT are mostly used for two purposes: to size the congestion window so that it doesn’t interfere with normal operation; and to estimate when “queue draining” is complete after a bandwidth probe cycle.

>>> If BBR does not slow down when packets are dropped, it's too
>>> hostile to use on a public network. The only way for a public
>>> network to respond to a flood of traffic higher than what it can
>>> handle is to drop packets (with a possible warning via ECN shortly
>>> before packets get dropped). If BBR doesn't slow down, it's just
>>> going to be wasting bandwidth.

>> No it isn't. Packet loss does not equal conguestion - it never did. Dropping packets to signal congestion is an ugly hack for implementations that are too dumb to understand any proper congestion
>> control mechanism.
> 
> Hmm, I bet a lot of carrier links are policed rather than smart queue.

Policing should theoretically produce a consistent delivery rate, which is what BBR needs to work effectively.  A wrinkle here is that most policers and shapers to date rely on a token-bucket algorithm which permits bursts at rates well above the policy, and BBR has to attempt to infer the policy rate from the medium-term behaviour.

> It also seems (OK one quick possibly flawed test), that bbr ignores ECN
> as well as drops in the sense that marked is just as high as dropped.

Yes, BBR ignores ECN, which I consider to be an unfortunate feature; it could quite reasonably be used to terminate bandwidth probes early, before they build up a significant queue (which then needs to be drained).

Cake works very well with BBR, provided it is deployed at the upstream end of the bottleneck link.  In this position, Cake happily absorbs the temporary standing queue caused by bandwidth probes, and the deficit-mode shaper means that BBR tends to see a consistent delivery rate, which it considers ideal.  In practice it matters little whether the BBR sender negotiates ECN or not, in this case.

When deployed at the downstream end of the bottleneck link, Cake works less well with BBR - but that’s true to some extent of all TCPs.  In ingress mode, at least, dropping packets effectively causes a reduction in the delivery rate, which should influence BBR more strongly to correct itself.  But if ECN is negotiated, these packet drops do not occur.  In both cases, the temporary standing queue collects in the upstream dumb queue, unless Cake is configured to a conservative enough margin below the bottleneck rate.  Cake does everything it reasonably can here, but the topology is fundamentally unfavourable.

 - Jonathan Morton