From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com [IPv6:2a00:1450:4864:20::12d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 3798C3B29E for ; Tue, 9 Jul 2019 11:32:35 -0400 (EDT) Received: by mail-lf1-x12d.google.com with SMTP id q26so13726170lfc.3 for ; Tue, 09 Jul 2019 08:32:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=IgapQB5KoXCyiE47XwYOfNz8yDExIed+DMkrfqhZw5w=; b=KQDoPgzmC9Xg6QbWYRTwMGVgauQOl87Zn9lQO0Nf5yi0hMSa07mYB1OzGHXlIRnmJY g82scT9vAS1hid8uXDKa4GmTDTdmSapHM/d1gQUddhmAisEE4zbWlPcEq0xE0ymFN7r7 hUFszxWrNcv4hBxoxMM4jgC9MCPWx/ZFrYNuudyoa7+jKJeQpkPdIxpNtIGm+5en3/Sv n6ozhJ8YpjN/uAgdUSxFulGJkyXR1v6/hv0QHb5dkMd0dqu+PeG2eKPhmqi5FmjNQPDw jl/UsLBWqTrSU/Kf0tT5ABmL5MLHYbYPU6tBbOIsVQvQMc+P2xh0z9xOecT6uzADVwiJ XueQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=IgapQB5KoXCyiE47XwYOfNz8yDExIed+DMkrfqhZw5w=; b=dnxcW9QOS0wFMWNLjBNfz52XN4bUwvq9Wjtk0QV5duLOfcFvy8S2rsQH8mEhr6weaL kxu0ZmGbvm2JmeISt/e+sRjoaXgDK0ubI+BBGpsVQ7zPP3LjMPnvrzbU3FY5M6+f/cb4 u32/hEbTzlM90VuteZVzwq9ooyLuTmhHJddAAsmnT7sRPihLXQarilSmN5f8mE3asPF2 dO6qzc5N8vxRvEsuRJso85cOeo6Mv1b9X9/dByn9zYE2DXTSmRKKhGWgunNdtUfD8pCy dL4+/960PGJmDbivfCGmrmk54QxJCc+3+2dI6nmgNvXG2I2IKF2BL3Xh15KDjy808DjF Pm/g== X-Gm-Message-State: APjAAAUvOHwnfBaRdWtCeOZ5YYWB3mk0wKHIEWcVIHWlhlXYx9Qlpe7B 313PuXPFeAY33t/3WHzEiGG3LhjijG1ofDKs5QrcFw== X-Google-Smtp-Source: APXvYqzfIQH64ELmXBrCYn5aIa0YlRlmKYP6UwhH5DG3hA50V4+Ot2i7a6btcXvLKsq0s4ob8PLOBCRlmaNuUjQkd0w= X-Received: by 2002:a19:8c56:: with SMTP id i22mr12042121lfj.105.1562686353727; Tue, 09 Jul 2019 08:32:33 -0700 (PDT) MIME-Version: 1.0 References: <24f7b15a-129f-ca44-60e0-32c7d23eadf4@bobbriscoe.net> In-Reply-To: From: Neal Cardwell Date: Tue, 9 Jul 2019 11:32:16 -0400 Message-ID: To: "Black, David" Cc: Bob Briscoe , "ecn-sane@lists.bufferbloat.net" , tcpm IETF list , tsvwg IETF list Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Ecn-sane] [tcpm] [tsvwg] ECN CE that was ECT(0) incorrectly classified as L4S X-BeenThere: ecn-sane@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of explicit congestion notification's impact on the Internet List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Jul 2019 15:32:35 -0000 On the particular question of Q#2 ("Do most of today's TCP implementations recover the reduction in congestion window when they discover later that a fast retransmit was spurious?"), one relevant data point: Linux TCP should generally be able to revert reductions in cwnd from spurious fast recovery episodes, using either TCP timestamps (along the lines of RFC 4015, starting from at least as far back as 2005 and working well since 7026b912f97d912 in 2013) or DSACKs (functioning well from 5628adf1a0ff3 in late 2011). thanks, neal On Tue, Jul 9, 2019 at 10:44 AM Black, David wrote: > > Bob, > > > > Commenting as an individual, not a WG chair. > > > > > Q#1: If this glosses over any concerns you have, please explain. > > > > It does gloss over, at least for me. The TL;DR summary is that items 1-3= aren=E2=80=99t relevant or helpful, IMHO, leaving items 4 and 5, whose eff= ectiveness depends on widespread deployment of RACK and FQ AQMs (e.g., FQ-C= oDel) respectively. > > > > Items 1 & 2: The general expectation for Internet transport protocols is = that they=E2=80=99re robust against =E2=80=9Cstupid network tricks=E2=80=9D= like reordering, but existing protocols transport wind up being designed/i= mplemented for the network we have, not the one we wish we had. I=E2=80=99= m generally skeptical of =E2=80=9Chighly unlikely=E2=80=9D arguments, as ho= rrendous results in a highly unlikely scenario are not acceptable if that s= cenario occurs repeatedly, even with long intervals in between occurrences.= In light of that, I view items 1 and 2 as defining the problem scenario t= hat needs to be addressed, particularly if L4S is to be widely deployed, an= d prefer to focus on items 3-5 about how the problem is dealt with. > > > > Item 3: This begins by correctly points out that 3DupACK is the criteria = for triggering conventional TCP retransmission, e.g., 2DupACK doesn=E2=80= =99t. An aspect that isn=E2=80=99t mentioned is that AQMs for classic (non= -L4S) traffic should be randomly marking (above a queue threshold, CE marki= ng probability depends on queue occupancy), not threshold marking (above a = queue threshold, mark all packets with CE). If threshold marking is used, = 3 CE marks in a row is a near certainty, as for non-mice flows, one can exp= ect to have at least that many packets in an RTT window; this is a =E2=80= =9CDoctor it hurts when I do .=E2=80=9D/=E2=80=9DDon=E2=80=99t do tha= t!=E2=80=9D scenario where the right answer is to fix the broken threshold = marking implementation. > > > > Assuming probabilistic marking, one then needs to look at 3-in-a-row CE m= arking probabilities based on the marking rate. These are not small - for = example, at a 10% marking probability, the likelihood of CE-marking 3 packe= ts in a row starting from a specific packet is 1 in 1,000 (1/10th of 1%), b= ut across 500 packets in a flow, that probability is about 50%. My initia= l take-away from this is that if the two bottlenecks (conventional followed= by L4S) persist, then the =E2=80=9Cunusual scenario=E2=80=9D of 3 CE-marke= d packets in a row is nearly certain to happen, which suggests that item 3 = is not particularly helpful, leaving items 4 (RACK) and 5 (FQ-CoDel). > > > > So, while I don=E2=80=99t have a conclusion to draw, it appears to me tha= t the countermeasures to this conventional TCP flow misbehavior with L4S ar= e deployment of RACK at endpoints and deployment of FQ AQMs such as FQ-CoDe= l at non-L4S potential bottleneck nodes. Items 4 and 5 below effectively a= ssert wide deployment of those algorithms =E2=80=93 additional information = and data on that would be of interest. > > > > Thanks, --David > > > > From: tsvwg On Behalf Of Bob Briscoe > Sent: Thursday, June 13, 2019 12:48 PM > To: ecn-sane@lists.bufferbloat.net; tcpm IETF list > Cc: tsvwg IETF list > Subject: [tsvwg] ECN CE that was ECT(0) incorrectly classified as L4S > > > > [EXTERNAL EMAIL] > > > [I'm sending this to ecn-sane 'cos that's where I detect that this concer= n is still rumbling. > I'm also sending to tcpm@ietf 'cos there's a question for TCP experts jus= t before the quoted text below. > And tsvwg@ietf is where it ought to be discussed.] > > Now that the IPR issue with L4S has been put to bed, one by one I am goin= g through the other concerns that have been raised about L4S. > > In the IETF draft that records all the pros and cons of different identif= iers to use for L4S, under the "ECT(1) and CE" choice (which is currently t= he one adopted at the IETF) there was already an explanation of why there w= ould be vanishingly low risk of any harmful consequences from CE that was o= riginally ECT(0) being classified into the L4S queue: > https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-06#page-32 > > Re-reading that, I have found some things unstated that I had thought wer= e obvious. So I've spelled it all out long-hand in the text below, which is= now in my local copy of the draft and will be in the next revision unless = people suggest improvements/corrections here. > > Q#1: If this glosses over any concerns you have, please explain. > Otherwise I will continue to consider that this is effectively a non-issu= e, which is the conclusion everyone in the TCP community came to at the tim= e the L4S identifier was chosen back in 2015. > > Q#2: The last couple of lines are the only part I am not sure of. Do most= of today's TCP implementations recover the reduction in congestion window = when they discover later that a fast retransmit was spurious? There's a not= e at the end of the intro to rfc4015 saying there was insufficient consensu= s to standardize this behaviour, but that most likely means it's done in di= fferent ways, rather than it isn't done at all. > > > Bob > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > Risk of reordering classic CE packets: Classifying all CE packets > > into the L4S queue risks any CE packets that were originally > > ECT(0) being incorrectly classified as L4S. If there were delay > > in the Classic queue, these incorrectly classified CE packets > > would arrive early, which is a form of reordering. Reordering can > > cause TCP senders (and senders of similar transports) to > > retransmit spuriously. However, the risk of spurious > > retransmissions would be extremely low for the following reasons: > > > > 1. It is quite unusual to experience queuing at more than one > > bottleneck on the same path (the available capacities have to > > be identical). > > > > 2. In only a subset of these unusual cases would the first > > bottleneck support classic ECN marking while the second > > supported L4S ECN marking, which would be the only scenario > > where some ECT(0) packets could be CE marked by a non-L4S AQM > > then the remainder experienced further delay through the > > Classic side of a subsequent L4S DualQ AQM. > > > > 3. Even then, when a few packets are delivered early, it takes > > very unusual conditions to cause a spurious retransmission, in > > contrast to when some packets are delivered late. The first > > bottleneck has to apply CE-marks to at least N contiguous > > packets and the second bottleneck has to inject an > > uninterrupted sequence of at least N of these packets between > > two packets earlier in the stream (where N is the reordering > > window that the transport protocol allows before it considers > > a packet is lost). > > > > For example consider N=3D3, and consider the sequence of > > packets 100, 101, 102, 103,... and imagine that packets > > 150,151,152 from later in the flow are injected as follows: > > 100, 150, 151, 101, 152, 102, 103... If this were late > > reordering, even one packet arriving 50 out of sequence > > would trigger a spurious retransmission, but there is no > > spurious retransmission here, because packet 101 moves the > > cumulative ACK counter forward before 3 packets have > > arrived out of order. Later, when packets 148, 149, 153... > > arrive, even though there is a 3-packet hole, there will be > > no problem, because the packets to fill the hole are > > already in the receive buffer. > > > > 4. Even with the current recommended TCP (N=3D3) spurious > > retransmissions will be unlikely for all the above reasons. > > As RACK [I-D.ietf-tcpm-rack] is becoming widely deployed, it > > tends to adapt its reordering window to a larger value of N, > > which will make the chance of a contiguous sequence of N early > > arrivals vanishingly small. > > > > 5. Even a run of 2 CE marks within a classic ECN flow is > > unlikely, given FQ-CoDel is the only known widely deployed AQM > > that supports classic ECN marking and it takes great care to > > separate out flows and to space any markings evenly along each > > flow. > > > > It is extremely unlikely that the above set of 5 eventualities > > that are each unusual in themselves would all happen > > simultaneously. But, even if they did, the consequences would > > hardly be dire: the odd spurious fast retransmission. Admittedly > > TCP reduces its congestion window when it deems there has been a > > loss, but even this can be recovered once the sender detects that > > the retransmission was spurious. > > > > > > -- > > ________________________________________________________________ > > Bob Briscoe http://bobbriscoe.net/ > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm