Key takeaway: pacing is inevitable, because it saves large content providers money (more efficient use of the most expensive silicon in the data center, the switch buffer memory), however to use pacing we walk away from 30 years of experience with TCP self clock, which is the foundation of all of our CC research....

Thanks for the interesting read. I have a few comments:

• IMO, many of the mentioned problems are related to using packet loss as congestion signal rather than self-clocking.
  
  
• In principle, one can keep utilization high and queuing delay low with a congestion window based and ACK-clock
  driven approach (see TCP LoLa https://ieeexplore.ieee.org/document/8109356). However, it currently lacks
  heuristics to deal with stretch/aggregated ACKs, but I think one can extend this like already done in BBR.
  
  
• Pacing is really useful and I think it is important to keep sending in case the ACK-clock is distorted
  by the mentioned problems, but only for a limited time. If one's estimate for the correct sending rate
  is too high, the amount of inflight data increases over time, which leads to queuing delay and/or loss.
  So having the inflight cap as in BBRv1 is also an important safety measure.
  
  
• "The maximum receive rate is probed by sending at 125% of max_BW .
  If the network is already full and flows have reached their fair share,
  the observed max_BW won’t change."
  This assumption isn't true if several flows are present at the bottleneck.
  If a flow sends with 1.25*max_BW on the saturated link, the observed
  max_BW will change (unless all flows are probing at the same
  time) because the probing flow preempts other flows, thereby
  reducing their current share. Together with the applied max-filter
  this is the reason why BBRv1 is constantly overestimating the available
  capacity and thus persistently increasing the amount inflight data
  until the inflight cap is hit. The math is in [32] (section 3) of your
  references. Luckily BBRv2 has much more safeguards built-in.
  
  
• "The lower queue occupancy indicates that it is not generally taking
  capacity away from other transport protocols..."
  I think that this indication is not very robust, e.g., it may hold in case
  there isn't significant packet loss observed. Observing an overall
  lower buffer occupancy does not necessarily tell you something about
  the individual flow shares. In BBRv1 you could have starving Cubic
  flows, because they were backing-off due to loss, while BBR kept
  sending.
  
  
• Last but not least, even BBR requires an ACK stream as
  feedback in order to estimate the delivery rate. But it is actually
  not self-clocked and keeps sending "blindly" for a while. This is
  quite useful to deal with the mentioned stretch/aggregated ACKs,
  if done with care.
  

Regards,
 Roland