In the last two weeks I have found two dramatically underbuffered Gbit
fiber networks.
This one appears to have about a 400 full size packet uplink buffer (5ms)[1]
https://imgur.com/a/Bm9hdNf
It was pretty remarkable to see how well multiple tcp flows still
achieved close to the full rate with such a small fixed size queue,
eventually.
A single bbr flow can't crack 150mbits: https://imgur.com/a/DpydL5K.
Thanks, Dave. The single-flow BBR upload case is interesting.
I took a look at the packet traces in the later thread:
For the single-flow BBR case it seems that....
(1) the BBR(v1) flow is running into a 300 Mbps bottleneck rate (visible in the slope of the green ACK line in the zoomed-in trace, attached).
(2) the BBR(v1) flow is achieving an average rate a bit above 150 Mbps because it repeatedly runs into receive window limits (the yellow line in the zoomed-out trace, attached). The frequent receive window limits mean that the flow spends a lot of time unable to send anything, thus leading to lower average throughput.
cheers,
neal