From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id AEEEB3CB35 for ; Wed, 3 Apr 2019 04:16:47 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1554279403; bh=mUzbLhTXc3qV0nmy/skKKfSgiUL0G6MxSXQJCEsue6E=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=AyqTpoLmWAMhgkQae8+34wTmB+8x+8AELiS+CTl2Nf9PsuXMqCpW48kjfqFFmA9Qb 8f6wi24eaN70VcDXyL9rHJ+omRuKraWf/eNzcu/OZvKitft7RVtIV8tkAWNjwBsW72 XDjOvaI2XrKwOfOq2DPsf60jV0Iah6bm7E35N6l4= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [172.16.12.10] ([134.76.241.253]) by mail.gmx.com (mrgmx103 [212.227.17.168]) with ESMTPSA (Nemesis) id 0McEDD-1hSqwW2pZH-00JZnZ; Wed, 03 Apr 2019 10:16:43 +0200 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Sebastian Moeller In-Reply-To: Date: Wed, 3 Apr 2019 10:16:42 +0200 Cc: bloat , Jonathan Foulkes Content-Transfer-Encoding: quoted-printable Message-Id: References: <5C360DB4-777C-466E-9FC1-0CA8F719F797@gmx.de> To: Ryan Mounce X-Mailer: Apple Mail (2.3445.9.1) X-Provags-ID: V03:K1:OUwEGXulPRYbfZ7idwKWuLvMUPH6ubxghEuhwC1v/AQHsXOnY+d PWq5aJQfV8pc6OGCrIurhhFsVSPnBNHqINAlad5+5TKqhpk7+YAgUNmjDHUdbinbb4IQ6MU nksCm1mwtat2HeNf3e+AS/Epemd3wa3xVCaQ1MBuRLCZd5RIvCvZBqLloRyhPKO/YY39SSv AtkGZTUzIZfZquHnrvihA== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:JJmgE4ykWnM=:/LD1WLhLzGN44c3ncVOaiR UOwOqOCtAieNvByLfqyMOldue0JrWi0GX/M5KfJCyBd7QMvc0wgD2YFYoWDhfAWCao+d37OjF LB23d9wS/umnVkfRledZLAAByHEnLbu8RpQZmgXVWSL//ILBhA+o+icx+Y4wfDB2YHp8MtOi7 GqC+PzNoR5swMwXhFCdkMI1IQCyyMnm5WM/SSqD6kFbqJ5nXD7sUgr8wevHAcrcd2QzXPfC3J gDqzTI8KJoLXs2kkeyE9UkAKiWavzAt7lJlElRC0FvH4ath259uNSybO+GpGYUAI9NKtoKRv9 +IisRk/TCSehfKyP33hvdLOToAfgBhkdryRsX65hYQTKRsw/FWvfK3fDGCMeWGHZ4GdRVTyez k1Ntfc65e4vL1EUQfwTugGo42+W1IzRKkC7p8JJlhZLCD27R5AgRM6NGED0MHQPg0Um5YI7LJ LKubpHtgQ9ombSknZrYgDUrzu/xDIk0oVej88xjs7utKBjnjvu4iT6UXlCQLBSP0yb1wrwpco SEdbmavQgkEq/+fju18ndBTFwgou/feSXGJ295pZx2MZck52hhaOjzm+jiOET2893dl5Cru0/ xWeKV8SSxUUxA//vvnrW3UMSRK5NmRyDPJJPWkF0RY976mbX/tvVTTXB0ldGvijY0CA0hx0jC 8yNf2qK6tesDOmPodyokjcq3xhHL2RBFaVxEyZQdWTEuQpcG3kLee46maf87AmdRFFvgDoUt6 TsC7px18hXx8Ui0tWAPjF0dLIW97iXOXkEwtNZIE61MbYCk6wlQtUDh5GIK4Jd6k8uh+dA1oP C9v9WfpbrlNjYOt5Butp1zo5FVGFMabcHReyZ1qr1Gl3fUaJ4aaOWBnvkHgOfWUL08HC8bbRW BRHKCryqVa9CzruNk2LGQuyE28F0pXFGrtze64wO21TdwsASBG88XBF2kCHX5XfidozIwMuhQ OPH0dFRb9Xg== Subject: Re: [Bloat] number of home routers with ingress AQM X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 08:16:48 -0000 Hi Ryan, thanks for your insights, more below in-line. > On Apr 3, 2019, at 01:23, Ryan Mounce wrote: >=20 > On Wed, 3 Apr 2019 at 00:05, Sebastian Moeller = wrote: >>=20 >>=20 >>=20 >>> On Apr 2, 2019, at 15:15, Ryan Mounce wrote: >>>=20 >>> On Tue, 2 Apr 2019 at 22:08, Sebastian Moeller = wrote: >>>>=20 >>>> I just wondered if anybody has any reasonable estimate how many = end-users actually employ fair-queueing AQMs with active ECN-marking for = ingress traffic @home? I am trying to understand whether L4S approach to = simply declare these as insignificant in number is justifiable? >>>=20 >>> L4S people are concerned by RFC 3168 / "classic" ECN bottlenecks >>> *without* fq. >>=20 >> I know, but I believe that they misunderstand the issues = resulting from post-bottleneck shaping, like ingress shaping on the = remote side of the true bottleneck. The idea seems that sending at too = high a rate is unproblematic if the AQM can simply queue up these = packets and delay them accordingly. But in the ingress shaper case these = packets already traversed the bottleneck and aone has payed the = bandwidth price to hoist them to the home, delaying or even dropping on = the AQM side will not magically get the time back the packets took = traversing the link. >=20 > My understanding is that with an AQM performing RFC 3168 style ECN > marking, the L4S flows will build a standing queue within their flow > queue before the AQM starts ECN marking. I agree, so far so normal. > At this point the L4S flow > will be less responsive to marks with a linear (?) decrease rather > than treating it like loss (multiplicative decrease), however the AQM > will just keep on marking to keep in in check. Again agreed, except an AQM expecting a quadratic response will = not mark rapidly enough to slow down an L4S flow as fast as other flows. > The fq shaper will > continue to isolate it from other flows at the bottleneck and enforce > fairness between flows (or in the case of an ingress shaper after the > true bottleneck, continue to selectively apply drops/marks that > effectively 'nudge' flows towards fairness). Again agreed, except the L4S flow will not respond as the AQM = predicts/expects and will keep sending at a higher rate (at least the CE = mark should stop the flow from increasing the tx bandwidth...) and hence = will keep occupying the true bottleneck. In other words the fq-isolation = will work considerably worse than the rash judgment "misunderstanding = the meaning of a CE-mark does not matter for fq-AQMs" assumes. >=20 > If there's no fq and there is RFC 3168 style ECN marking, the L4S > flows assume that they're receiving aggressive DCTCP-style marking, > respond less aggressively to each mark, and will starve conventional > Reno friendly flows. Yes, except in the above case effectively the same is going to = happen, the L4S flow is still not throttling back hard enough and will = hence occupy more bandwidth of the bottleneck than intended. > L4S people are relying on there being almost no > such bottlenecks on the internet, and to be honest I think this is a > fair assumption. I disagree, on whether they a) have a robust and reliable = estimate of the number of such beasts and b) I would like to see less = assumption and more measurements here ;) > The most deployed single queue AQM may be PIE as > mandated by DOCSIS 3.1, which had ECN marking disabled before the > whole DualQ/L4S thing. Bob Briscoe thinks the main concern is people > that have found old Cisco routers with RED and re-commissioned them... Again thinks and assumes does not fill me with warm and fuzzy = feelings, I want to see hard cold data. >=20 > As L4S flows would still be still be getting ECN marked in either = case, > there would be no dropping of packets that have already traversed the > bottleneck in order to signal congestion. So long as there is fq to > enforce fairness, I can't see any problem. The problem is that fairness is enforced too late here, as the = under-responsive L4S flow already hogged more bottleneck bandwidth than = intended. >=20 > Sure, signalling congestion without loss doesn't mean that the packets > haven't already spent more than their fair share of time at the > previous bottleneck. And that is my point. > This is a more general problem with shaping > ingress further downstream from the true bottleneck - and I don't see > that it's any worse here than with any other TCP overshooting during > slow start. Just because ingress shaping has challenges I see no reason to = make these worse by design... The point I am making is neither against = L4S in general or its quest for a different TCP-CC with better = congestion signaling, my concern is about the half-backed attempt to = press a single codepoint into service where actually two codepoints (or = rather an independent full bit) are required. I detest the fact that the = project intends to simply argue away the issues this incomplete = isolation causes instead of figuring out how to solve the issue = properly. IMHO that either means make it possible to share L4S and = non-L4S flows _fairly_ on TCP-friendly ECN-marking AQMs, or find a way = to properly identify the handiwork of an L4S-complient AQM from a = TCP-friendly one.=20 >=20 > There are certainly more real threats such as Akamai's FAST TCP. I typically do not accept the argument, that one rule-violation = justifies ore rule-violations ;) > It's > like BBR in that it attempts to detect and respond to congestion based > on queuing latency, Somebody should give CC designers the Codel paper and make them = understand that this approach will need to be sensitive to changes in = the 5-10ms range to account for the presence of competent AQM in the = path... At the current time I see no real justification for pushing this = same broken idea time and time again, it never worked well, see = TCP-Vegas, LEDBAT, ...=20 > and tries to ignore low levels of random packet > loss that don't occur due to congestion. Yes, that is just as unfortunate a misdesign, but it is = encouraging that BBRv2 actually tries to look harder at this issue and = come up with a "loss-model" to somehow disambiguate between different = losses. > Their implementation > definitely presents issues for ingress shaping: > https://forums.whirlpool.net.au/thread/2530363 This is why I call this mis-designed, the underlaying model of = the network in BBR obviously does not match reality, in my world that = means the model needs changing ;) >=20 > I saw that thread years ago, and then eventually saw the issue for > myself after changing ISP. Suddenly Windows Update was downloading > from my ISP's embedded Akamai cache using FAST TCP, and was > effectively unresponsive to cake's ingress shaping, instead filling > the queue in my ISP's BNG. I note that FAST TCP in all likelihood is not TCP-friendly. >=20 > Fact is... ingress shaping is a hack. That is harsh, but I give you sub-optimal and approximate, it = also works amazingly well given all theoretical reasons why it should = not work. > The real problem needs to be > solved in the BNG, in the CMTS, in the DSLAM, etc. But realistically it is not going to be solved in the way = end-users would like, simply due to the economic incentives not being = aligned for ISPs and end-users.=20 In addition the remote end might not even have enough information to = perform the kind of shaping a user might desire, e.g. I use cake's = isolation options to have approximate per-internal-IP-fairness, but for = IPv4 that requires having access to the internal IP addresses, which = simply are unkown at the upstream end. > L4S is just one > proposal and of course it deserves scrutiny before gobbling up the > precious ECT(1) codepoint, I could live with it eating ECT(1), as much as I prefer SCE's = conceptually purity and seeming simplicity, my issue is that it gobbles = ECT(1) even though ECT(1) does not solve the issue L4S is using it for; = and they know this. but try to just argue the issue away instead of at = least trying to empirically quantify it. > however it seems to have some traction with > vendors and a chance at wide deployment with a view to address address > exactly this problem *at* the bottleneck. Let's see how this plays out. > For that, it also deserves > consideration. >=20 >> Why do I care, because that ingress shaping setup is what I = use at home, and I have zero confidence that ISPs will come up with a = solution to my latency desires that I am going to be happy with... And = what I see from the L4S mixes light with a lot of shadows. >>=20 >>=20 >>> I don't think there would be any such ingress shapers >>> configured on home gateways. Certainly not by anyone on this list... >>> anyone running non-fq codel or flowblind cake for ingress shaping? >>=20 >> As stated above, I believe fq to not be a reliable safety = valve for the ingress shaping case. >=20 > -Ryan