<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.gmailmsg
{mso-style-name:gmail_msg;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Yuchung,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">In regards to DOCSIS PIE as part of upcoming DOCSIS 3.1 equipment, single queue PIE will first be deployed on the D3.1 cable modem governing the upstream direction.
So the test of BBR & other CCs in the mixed environment would be run with the BBR & other CC sending sources on a server/compute node behind the cable modem sending data upstream to some receiver. I do not see DOCSIS PIE on the CMTS in the downstream direction
in the near term.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Carl Klatsky<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Bloat [<a href="mailto:bloat-bounces@lists.bufferbloat.net">mailto:bloat-bounces@lists.bufferbloat.net</a>]
<b>On Behalf Of </b>Yuchung Cheng<br>
<b>Sent:</b> Thursday, December 08, 2016 5:31 PM<br>
<b>To:</b> Neal Cardwell <<a href="mailto:ncardwell@google.com">ncardwell@google.com</a>>; Mikael Abrahamsson <<a href="mailto:swmike@swm.pp.se">swmike@swm.pp.se</a>><br>
<b>Cc:</b> bloat <<a href="mailto:bloat@lists.bufferbloat.net">bloat@lists.bufferbloat.net</a>><br>
<b>Subject:</b> Re: [Bloat] TCP BBR paper is now generally available<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p>Also we are aware docsis pie is going to be deployed and we'll specifically test that scenario. With fq this issue is a lot smaller but we understand it is not preferred setting in some aqm for other good reasons.<o:p></o:p></p>
<p>But to set the expectation right, we are not going to make bbr prefectly flow level fair with cubic or reno. I am happy to argue why that makes no sense. We do want to avoid starvation of either.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Thu, Dec 8, 2016, 1:29 PM Neal Cardwell <<a href="mailto:ncardwell@google.com">ncardwell@google.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">Hi Mikael,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks for your questions. Yes, we do care about how BBR behaves in mixed environments, and particularly in mixed environments with Reno and CUBIC. And we are actively working in this and related areas.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">For the ACM Queue article we faced very hard and tight word count constraints, so unfortunately we were not able to go into as much detail as we wanted for the "Competition with Loss-Based Congestion Control" section.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">In our recent talk at the ICCRG session at IETF 97 we were able to go into more detail on the question of sharing paths with loss-based CC like Reno and CUBIC (in particular please see slides 22-25):<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> <a href="https://www.ietf.org/proceedings/97/slides/slides-97-iccrg-bbr-congestion-control-02.pdf" target="_blank">
https://www.ietf.org/proceedings/97/slides/slides-97-iccrg-bbr-congestion-control-02.pdf</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">There is also a video; the BBR section starts around 57:35:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <a href="https://www.youtube.com/watch?v=qjWTULVbiVc" target="_blank">
https://www.youtube.com/watch?v=qjWTULVbiVc</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">In summary, with the initial BBR release:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">o BBR and CUBIC end up with roughly equal shares when there is around 1-2x BDP of FIFO buffer. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">o When a FIFO buffer is deeper than that, as everyone on this list well knows, CUBIC/Reno will dump excessive packets in the queue; in such bufferbloated cases BBR will get a slightly lower share of throughput than CUBIC (see slide 23-24).
I say "slightly" because BBR's throughput drops off only very gradually, as you can see. And that's because of the dynamic in the passage from the ACM Queue paper you cited: "Even as loss-based congestion control fills the available buffer, ProbeBW still robustly
moves the BtlBw estimate toward the flow's fair share, and ProbeRTT finds an RTProp estimate just high enough for tit-for-tat convergence to a fair share." (I guess that last "to" should probably have been "toward".)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">o When a buffer is shallower than 1-2x BDP, or has an AQM targeting less than 1-2 BDP of queue, then BBR will tend to end up with a higher share of bandwidth than CUBIC or Reno (I think the tests you were referencing fall into that category).
Sometimes that is because the buffer is so shallow that the multiplicative backoff of CUBIC/Reno cause the bottleneck to be underutilized; in such cases then BBR is merely using underutilized bandwidth, and its higher share is a good thing. In more moderately
sized buffers in the 0-2x BDP range (or AQM-managed buffers), our active work under way right now (see slide 22) should improve things, based on our experiments in the lab and on YouTube. Basically the two approaches we are currently experimenting with are
(1) explicitly trying to more fully drain the queue more often, to try to get much closer to inflight==BDP each gain cycle, and (2) estimate the buffer available to our flow and and modulate the probing magnitude/frequency.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">In summary, our #1 priority for BBR right now is reducing queue pressure, in order to reduce delay and packet loss, and improve fairness when sharing paths with loss-based congestion control like CUBIC/Reno.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">cheers,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">neal<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Thu, Dec 8, 2016 at 9:01 AM, Mikael Abrahamsson <span class="gmailmsg">
<</span><a href="mailto:swmike@swm.pp.se" target="_blank">swmike@swm.pp.se</a><span class="gmailmsg">></span> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-bottom:12.0pt"><span class="gmailmsg">On Thu, 8 Dec 2016, Dave Täht wrote:<o:p></o:p></span></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal">drop tail works better than any single queue aqm in this scenario.<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><br>
*confused*<br>
<br>
I see nothing in the BBR paper about how it interoperates with other TCP algorithms. Your text above didn't help me at all.<br>
<br>
How is BBR going to be deployed? Is nobody interested how it behaves in a mixed environment?<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"><br>
<br>
-- <br>
Mikael Abrahamsson email: <a href="mailto:swmike@swm.pp.se" target="_blank">swmike@swm.pp.se</a><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
Bloat mailing list<br>
<a href="mailto:Bloat@lists.bufferbloat.net" target="_blank">Bloat@lists.bufferbloat.net</a><br>
<a href="https://lists.bufferbloat.net/listinfo/bloat" target="_blank">https://lists.bufferbloat.net/listinfo/bloat</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">_______________________________________________<br>
Bloat mailing list<br>
<a href="mailto:Bloat@lists.bufferbloat.net" target="_blank">Bloat@lists.bufferbloat.net</a><br>
<a href="https://lists.bufferbloat.net/listinfo/bloat" target="_blank">https://lists.bufferbloat.net/listinfo/bloat</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</body>
</html>