[Bloat] BBR implementations, knobs to turn?

[Neal Cardwell]

On Tue, Nov 17, 2020 at 10:08 AM Jesper Dangaard Brouer
<brouer at redhat.com> wrote:
>
> On Tue, 17 Nov 2020 10:05:24 +0000 <erik.taraldsen at telenor.com> wrote:
>
> > Thank you for the response Neal
>
> Yes. And it is impressive how many highly qualified people are on the
> bufferbloat list.
>
> > old_hw # uname -r
> > 5.3.0-64-generic
> > (Ubuntu 19.10 on xenon workstation, integrated network card, 1Gbit
> > GPON access.  Used as proof of concept from the lab at work)
> >
> >
> > new_hw # uname -r
> > 4.18.0-193.19.1.el8_2.x86_64
> > (Centos 8.2 on xenon rack server, discrete 10Gbit network card,
> > 40Gbit server farm link (low utilization on link), intended as fully
> > supported and run service.  Not possible to have newer kernel and
> > still get service agreement in my organization)
>
> Let me help out here.  The CentOS/RHEL8 kernels have a huge amount of
> backports.  I've attached a patch/diff of net/ipv4/tcp_bbr.c changes
> missing in RHEL8.
>
> It looks like these patches are missing in CentOS/RHEL8:
>  [1] https://git.kernel.org/torvalds/c/78dc70ebaa38aa3
>  [2] https://git.kernel.org/torvalds/c/a87c83d5ee25cf7
>
> Could missing patch [1] result in the issue Erik is seeing?
> (It explicitly mentions improvements for WiFi...)

Thanks, Erik, for the detailed information. This is super-useful.

And thanks, Jesper, for the patch analysis. Yes, I agree that missing
patch [1]  is likely the cause of the lower BBR throughout in the
"new_hw" case. Since the "new_hw" is running an older kernel that's
missing this important patch, it would be expected to have lower
throughput in a workload like this. It's unfortunate that it's not
possible to have a newer kernel on the newer hardware; it does seem in
this case that this would probably do the trick.

best,
neal