Re: [Cerowrt-devel] BBR congestion control algorithm for TCP in net-next

[Dave Taht <dave.taht@gmail.com>]

On Wed, Sep 21, 2016 at 3:15 AM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
> On Wed, 21 Sep 2016, Dave Taht wrote:
>
>> I dunno, I'm just reading tea leaves here!
>>
>> can't wait for the paper!
>
>
> +1.
>
> I would like to understand how BBR interacts with a window-fully-open
> classic TCP session and FIFO induced delay that is in steady-state before
> the BBR session starts.

I did a fairly comprehensive string of tests today, comparing it at
20Mbits, 48ms RTT, to cubic and competing with cubic, against a byte
fifo of 256k, pie, cake, cake flowblind, and fq_codel.

The flent datasets are now at:
http://blog.cerowrt.org/flent/bbr-comprehensive.tgz

You will need the latest flent from git to plot the new
tcp_4up_squarewave test. That test starts 1 BBR flow, then 3 seconds
later, cubic, 3 seconds after that, bbr again, 3 seconds after cubic
again.

It's 4am here, and we've also been busy with something else due today,
but some observations:

* respecting ecn appears unimplemented, so if you compare cubic with
ecn against bbr not respecting it, bad things happen. I bug reported
this to the bbr mailing list. To some extent cake's and pie's overload
on ecn drop mechanism helps. fq_codel/cake with fq is fine, so long as
there are no collisions.

* It seriously outcompetes cubic, particularly on the single queue
aqms. fq_codel is fine. I need to take apart the captures to see how
well it is behaving in this case. My general hope was that with fq in
place, anything that was delay based worked better as it was only
competing against itself.

* It does the right things against bfifo when multiple streams are
present, but in the presence of reverse traffic, cubic starves. (the
dataset for this is kind of wrong in that the legend of the
"smackdown" chart claims BBR is on on the download, but it was
unimplemented in that direction, I only had it running on the upload.)

Did I mention fq* was fine? :)

I'm going to bed.

> So let's say I have 100ms of lightspeed-in-fiber RTT, and I am then running
> a file transfer with some other TCP algorithm which is sitting there, window
> fully open, creating an additional 100ms of stupid-router-FIFO-buffering
> delay.
>
> So new BBR TCP session comes along, sees 200ms of RTT, and starts sending. I
> guess the classic TCP algorithm still keeps its window fully open, and
> doesn't care that RTT now increased to 210ms by the BBR flow packets.

I see evidence of the classic latecomer advantage but it subsides in 10-20 sec:

http://blog.cerowrt.org/flent/bbr-comprehensive/latecomer_advantage.svg

>
> Now what? BBR flow sees increased latency, and backs off, right? So how much
> of the bandwidth will each flow get? How do these interact?

Plot it in flent.

>
> --
> Mikael Abrahamsson    email: swmike@swm.pp.se



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org