<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:553276787;
mso-list-type:hybrid;
mso-list-template-ids:-838439190 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">That is ridiculous.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">You clearly haven’t read the drafts, and so are speaking from a position of ignorance. Please get informed before making statements like this.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">There is *<b>absolutely</b>* nothing cable-specific or “private” about L4S. It is being developed in an open forum, the IETF!! Yes, the cable industry is adopting L4S – because we recognized its potential. Others are too, and anyone
can. It is a totally open spec, and has been since the initial drafts came out of the RITE project. The cable industry was not involved in RITE (in fact the technology was first demonstrated on DSL equipment), and we learned about L4S when the rest of the
world did. We decided to become early adopters. Yes, we were quiet about the fact that we were planning on adopting it (until now).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">If individuals drop out of participating in the IETF, they shouldn’t be upset if the IETF continues to make progress on developing Internet technology in their absence. It seems pretty disingenuous for DT to form his own private working
group to come up with an incompatible, and limited, alternative to the ongoing work in IETF, then spring it on the IETF and start this FUD war.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The craziest thing about this whole thread is that there is a heck of a lot in common between L4S and SCE. More in common than different. My initial belief was that we all had similar goals (eliminating buffering latency in the internet)
and we’d be able to achieve a meeting of the minds on the best way to use ECT(1) to achieve it. Now, I’m not so sure what DT’s motivation is. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">If I can boil this down for the people who are jumping into this without reading the drafts:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">Both L4S and SCE are attempting to provide congestion-controlled senders with better congestion signals so that flows can achieve link capacity without buffering delay.
<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">Both are proposing to use ECT(1) as part of the mechanism, but to use it in different ways.<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">SCE’s usage of ECT(1) potentially allows an automatic fallback to traditional Cubic behavior if the bottleneck link is a single-queue classic-ECN AQM (do any of these exist?), whereas
L4S will need to detect such a condition via RTT measurement<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">L4S’s usage of ECT(1) allows links to identify new senders and take advantage of new sender features like reordering tolerance that can further drive down latency in many common link
technologies.<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">SCE will only work if the bottleneck link implements fq. Some bottleneck network gear will not be able to implement fq or will not implement it due to its undesirable side effects
(see section 6 of RFC 8290).<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">L4S will work if the bottleneck link implements *<b>either</b>* fq or dual queue.<o:p></o:p></li></ul>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Beyond that, they are *<b>very,very</b>* similar. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">But, L4S has been demonstrated in real equipment and in simulation, and leverages an existing congestion controller that is available in Linux and Windows (with some tweaks). SCE leverages a paragraph in a draft that describes a first
guess about how a congestion controller might work.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">L4S has defined a congestion feedback mechanism so that these congestion signals can get back to the sender. SCE offers that “we’ll propose something later”.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">BBR currently does not listen to explicit congestion signals, but it could be updated to do so (for either SCE or L4S).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">-Greg<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Bloat <bloat-bounces@lists.bufferbloat.net> on behalf of "David P. Reed" <dpreed@deepplum.com><br>
<b>Date: </b>Sunday, March 17, 2019 at 12:07 PM<br>
<b>To: </b>Vint Cerf <vint@google.com><br>
<b>Cc: </b>bloat <bloat@lists.bufferbloat.net>, "ecn-sane@lists.bufferbloat.net" <ecn-sane@lists.bufferbloat.net><br>
<b>Subject: </b>Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;overflow-wrap: break-word">
<span style="font-size:12.0pt;font-family:"Arial",sans-serif">Vint -<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;overflow-wrap: break-word">
<span style="font-size:12.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;overflow-wrap: break-word">
<span style="font-size:12.0pt;font-family:"Arial",sans-serif">BBR is the end-to-end control logic that adjusts the source rate to match the share of the bolttleneck link it should use.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;overflow-wrap: break-word">
<span style="font-size:12.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;overflow-wrap: break-word">
<span style="font-size:12.0pt;font-family:"Arial",sans-serif">It depends on getting reliable current congestion information via packet drops and/or ECN.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;overflow-wrap: break-word">
<span style="font-size:12.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;overflow-wrap: break-word">
<span style="font-size:12.0pt;font-family:"Arial",sans-serif">So the proposal by these guys (not the cable guys) is an attempt to improve the quality of the congestion signal inserted by the router with the bottleneck outbound link.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;overflow-wrap: break-word">
<span style="font-size:12.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;overflow-wrap: break-word">
<span style="font-size:12.0pt;font-family:"Arial",sans-serif">THe cable guys are trying to get a "private" field in the IP header for their own use.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;overflow-wrap: break-word">
<span style="font-size:12.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;overflow-wrap: break-word">
<span style="font-size:12.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:.5in;overflow-wrap: break-word">
<span style="font-size:12.0pt;font-family:"Arial",sans-serif">-----Original Message-----<br>
From: "Vint Cerf" <vint@google.com><br>
Sent: Saturday, March 16, 2019 5:57pm<br>
To: "Holland, Jake" <jholland@akamai.com><br>
Cc: "Mikael Abrahamsson" <swmike@swm.pp.se>, "David P. Reed" <dpreed@deepplum.com>, "ecn-sane@lists.bufferbloat.net" <ecn-sane@lists.bufferbloat.net>, "bloat" <bloat@lists.bufferbloat.net><br>
Subject: Re: [Ecn-sane] [Bloat] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104<o:p></o:p></span></p>
<div id="SafeStyles1552845686">
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Arial",sans-serif">where does BBR fit into all this?
<o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Arial",sans-serif">v<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Arial",sans-serif">On Sat, Mar 16, 2019 at 5:39 PM Holland, Jake <<a href="mailto:jholland@akamai.com">jholland@akamai.com</a>> wrote:<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Arial",sans-serif">On 2019-03-15, 11:37, "Mikael Abrahamsson" <<a href="mailto:swmike@swm.pp.se" target="_blank">swmike@swm.pp.se</a>> wrote:<br>
L4S has a much better possibility of actually getting deployment into the <br>
wider Internet packet-moving equipment than anything being talked about <br>
here. Same with PIE as opposed to FQ_CODEL. I know it's might not be as <br>
good, but it fits better into actual silicon and it's being proposed by <br>
people who actually have better channels into the people setting hard <br>
requirements.<br>
<br>
I suggest you consider joining them instead of opposing them.<br>
<br>
<br>
Hi Mikael,<br>
<br>
I agree it makes sense that fq_anything has issues when you're talking<br>
about the OLT/CMTS/BNG/etc., and I believe it when you tell me PIE<br>
makes better sense there.<br>
<br>
But fq_x makes great sense and provides real value for the uplink in a<br>
home, small office, coffee shop, etc. (if you run the final rate limit<br>
on the home side of the access link.) I'm thinking maybe there's a<br>
disconnect here driven by the different use cases for where AQMs can go.<br>
<br>
The thing is, each of these is the most likely congestion point at<br>
different times, and it's worthwhile for each of them to be able to<br>
AQM (and mark packets) under congestion.<br>
<br>
One of the several things that bothers me with L4S is that I've seen<br>
precious little concern over interfering with the ability for another<br>
different AQM in-path to mark packets, and because it changes the<br>
semantics of CE, you can't have both working at the same time unless<br>
they both do L4S.<br>
<br>
SCE needs a lot of details filled in, but it's so much cleaner that it<br>
seems to me there's reasonably obvious answers to all (or almost all) of<br>
those detail questions, and because the semantics are so much cleaner,<br>
it's much easier to tell it's non-harmful.<br>
<br>
<aside regarding="non-harmful"><br>
The point you raised in another thread about reordering is mostly<br>
well-taken, and a good counterpoint to the claim "non-harmful relative<br>
to L4S".<br>
<br>
To me it seems sad and dumb that switches ended up trying to make<br>
ordering guarantees at cost of switching performance, because if it's<br>
useful to put ordering in the switch, then it must be equally useful to<br>
put it in the receiver's NIC or OS.<br>
<br>
So why isn't it in all the receivers' NIC or OS (where it would render<br>
the switch's ordering efforts moot) instead of in all the switches?<br>
<br>
I'm guessing the answer is a competition trap for the switch vendors,<br>
plus "with ordering goes faster than without, when you benchmark the<br>
switch with typical load and current (non-RACK) receivers".<br>
<br>
If that's the case, it seems like the drive for a competitive advantage<br>
caused deployment of a packet ordering workaround in the wrong network<br>
location(s), out of a pure misalignment of incentives.<br>
<br>
RACK rates to fix that in the end, but a lot of damage is already done,<br>
and the L4S approach gives switches a flag that can double as proof that<br>
RACK is there on the receiver, so they can stop trying to order those<br>
packets.<br>
<br>
So point granted, I understand and agree there's a cost to abandoning<br>
that advantage.<br>
</aside><br>
<br>
But as you also said so well in another thread, this is important. ("The<br>
last unicorn", IIRC.) How much does it matter if there's a feature that<br>
has value today, but only until RACK is widely deployed? If you were<br>
convinced RACK would roll out everywhere within 3 years and SCE would<br>
produce better results than L4S over the following 15 years, would that<br>
change your mind?<br>
<br>
It would for me, and that's why I'd like to see SCE explored before<br>
making a call. I think at its core, it provides the same thing L4S does<br>
(a high-fidelity explicit congestion signal for the sender), but with<br>
much cleaner semantics that can be incrementally added to congestion<br>
controls that people are already using.<br>
<br>
Granted, it still remains to be seen whether SCE in practice can match<br>
the results of L4S, and L4S was here first. But it seems to me L4S comes<br>
with some problems that have not yet been examined, and that are nicely<br>
dodged by a SCE-based approach.<br>
<br>
If L4S really is as good as they seem to think, I could imagine getting<br>
behind it, but I don't think that's proven yet. I'm not certain, but<br>
all the comparative analyses I remember seeing have been from more or<br>
less the same team, and I'm not convinced they don't have some<br>
misaligned incentives of their own.<br>
<br>
I understand a lot of work has gone into L4S, but this move to jump it<br>
from interesting experiment to de-facto standard without a more critical<br>
review that digs deeper into some of the potential deployment problems<br>
has me concerned.<br>
<br>
If it really does turn out to be good enough to be permanent, I'm not<br>
opposed to it, but I'm just not convinced that it's non-harmful, and my<br>
default position is that the cleaner solution is going to be better in<br>
the long run, if they can do the same job.<br>
<br>
It's not that I want it to be a fight, but I do want to end up with the<br>
best solution we can get. We only have the one internet.<br>
<br>
Just my 2c. <br>
<br>
-Jake<br>
<br>
<br>
_______________________________________________<br>
Ecn-sane mailing list<br>
<a href="mailto:Ecn-sane@lists.bufferbloat.net" target="_blank">Ecn-sane@lists.bufferbloat.net</a><br>
<a href="https://lists.bufferbloat.net/listinfo/ecn-sane" target="_blank">https://lists.bufferbloat.net/listinfo/ecn-sane</a><o:p></o:p></span></p>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Arial",sans-serif"><br>
-- <o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Arial",sans-serif">New postal address:
<o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Arial",sans-serif">Google<o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Arial",sans-serif">1875 Explorer Street, 10th Floor<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Arial",sans-serif">Reston, VA 20190<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>