From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id DBA2C3CB37 for ; Mon, 5 Aug 2019 07:00:33 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1565002829; bh=WK+5nuAvvUIPlf6JYWCBbP4ifWzdsu5X02hwSqegQDM=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=aEWCeGRVBFdE/7+BWikT9t1IwIMMsgda/3JqS9QKK6pLQ6BM4AqnyigAHvJO34Mti EXj72VwM0dg24mM15xRcUIUwyhtF8p1rGheHB0MpeP+Ad+pNc77YKZnDRfnX6WK+JY e0ARxkICAOSWzzXRaXxWUJC6HHeDW5wiydHKIKpw= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [10.11.12.32] ([134.76.241.253]) by mail.gmx.com (mrgmx103 [212.227.17.168]) with ESMTPSA (Nemesis) id 0LsTjw-1iJ1RF0hHU-011xDw; Mon, 05 Aug 2019 13:00:29 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) From: Sebastian Moeller In-Reply-To: Date: Mon, 5 Aug 2019 13:00:26 +0200 Cc: tcpm@ietf.org, ECN-Sane , tsvwg@ietf.org Content-Transfer-Encoding: quoted-printable Message-Id: <20C58AE1-CAC4-480F-8E80-1F5A09ED53EA@gmx.de> References: <24f7b15a-129f-ca44-60e0-32c7d23eadf4@bobbriscoe.net> <0E44351D-2520-45AD-A10A-8E2FFE186722@gmx.de> To: Ruediger.Geib@telekom.de X-Mailer: Apple Mail (2.3445.104.11) X-Provags-ID: V03:K1:xWMMx0W/HvsBTzzIiG4OGVaTsydN2TG24hNRMVQUDxFYJ2BeTVR 7fhAiEDivOQdev62PHYXpAEcPUD0J9KUMJY7HKQK8fZPywFDweT+re4IFaSCwMNCe5GlbA7 UoiUsR/ur5qNBJU8GDPhxowZoXZosLwDnW6PBbBOIkdhoe046Si5Z64LdgrsmNEJBAreoS4 B96mUYy2vMXgmSJ5ix1SQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:hAwhl4ZQ5UE=:JdM/WDNfGd1tRR4CDeilLk gy2lyhydztKSlKdUSTED15N3syUGGcQBzL4yjggeQCXU7PDvxFWUjy7ld2ZmyRRCLs9MVet7P OrwMIWhVEoRccA7kuVV0V7pwXyrmKPo7AY+NPSyoZN6ybcKknYBR0frqpmzekmL6pqhyVxu8k 6cQmUCVe0S+Q5eFrRtMB3weurYaba8JDFXv9lEQbhwHw023VT7RUHufdYeuPD6IgabBXqLyAl ACscMHWNuR0XmlrHisnIysB/ayHfCh7VGjOyPysa+LZk7qtaTAR3gcXteX2gsJcG3uHpm8a4D nWfF5ha7e44QpXYrtBOvggs2E3qlUUZ1LzBu34hHSw1tVtLvL5mv9qYudEJl6sCiA6unxvVox HCDQl3ZdseZvePSEehy+bUq2fKCj8LlKj0EsnIKHVxQ+qULdsFOH5tVAf4YRyq0hN2HzQCWbP TFe0yv06G0Ny2YIuy/SSjrzj/4xjj1sv/Ptda7rV/C879YPGjEw1KygYZWLzQqPLBDwI5qKUJ DjJJ3+Xsr2wgItyvAMv/R5QZOxA+Bv/LhRiXDuBhKj3t2tvGTsmW+toMWwvyZWU2pLY6iwYAr GouR1RbxjiQrJe0d02UBy1ZePilJ/5+uqZW3OiOe4HmijLd6ewxPhvX5rNG/0kTUxIArtlxuu ZUZiq+1xbcVfVBmI/k9mdOFLw5sBloAFYxEK3DQSPyDhzjSAbp7rbN7VaebW9uq6JthHeSE4z kGUxaxIKx+i9JIkyaHUUT3oGhxOaB2fgfNV+7KjSCtDac/D7RnulX2cKA8GY+mxqkgdx71oRR T2vWe6qC0+pzO3F5G15FTEXd2XUCvxGPnZgSpW+FYRLsBMynREkdVVTZpdrjOI/jLvZp3S0Lr S3Xbbi1jEYlPtdY3K3QYSaLY1fY4EVCnCQEr7C5zIXfqTWjav4qmXLsOKShoxflrN+e6XTmyF pLBZmrnYGCe4p1PZWC0vnsCdpWH6D3a5uy/OkrM16P10kmoKtCGYbNCdCwqh5a7J0aask5TLF y6gPGhIgS08t4yYj/x3P1sUjUgWZhYkCnwfCklqz8GeUrrMfLpfokx2Ip6PF6YGPkyF7RakKW cvRF2lasG7qPlaUBs7WmxyEKnAEWQlhM8G5jMoOTFExAlx4onlN2lJlH7tQhMZDfhfPxjCMVZ /v01Y= Subject: Re: [Ecn-sane] [tsvwg] ECN CE that was ECT(0) incorrectly classified as L4S X-BeenThere: ecn-sane@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of explicit congestion notification's impact on the Internet List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Aug 2019 11:00:34 -0000 Hi Ruediger, > On Aug 5, 2019, at 09:26, = wrote: >=20 > Hi Sebastian, >=20 > the access link is the bottleneck, that's what's to be expected. Mostly, then again there are situations with 1Gbps Plans where = it is not the actual access link, but rather the CPEs Gigabit ethernet = LAN ports that are the true bottleneck, but that does not substantially = change the issue, it is still the upstream shaper/policer that needs to = be worked around. > As far as I know, in the operator world shapers here by and large = removed policers. Good to know, shapers are somewhat nicer to user traffic than = hard policers, at least that is my interpretation. >=20 > A consecutive chain of narrower links results, if the Home Gateway = runs with an additional ingress or egress shaper operating below the = access bandwidth, if I get you right.=20 Yes, as you state below, this only is true for the ingress = direction, egress shaping works reliably and is typically not suffering = from this. That said, if the egress link bandwidth is larger than a = servers connection this issue can appear also for the egress direction. = For example overly hot peering/transit links can cause downstream = bottlenecks considerably narrower than the internet access link's upload = direction, but that, while unfortunate, is not at the core of my issue. >=20 > I understand that you aren't interested in having 300ms buffer delay = and may be some jitter for a phone conversation using best effort = transport. +1 > A main driver for changes in consumer IP access features in Germany = are publications of journals and regulators comparing IP access = performance of different providers. Good to know, > Should one provider have an advantage over the others by deploying a = solution as you (and Bob's team) work on, it likely will be generally = deployed. I do not believe that these mechanisms are actually in play in = the German market, as an example for roughly a decade the DOCSIS-ISPs = offer higher bandwidth for same or less money than the incumbent telco = and yet only managed to get 30% of the customers of their ~75% of = possible customers, so only 75*0.3 =3D 22.5 % market share, with the = incumbent only reaching 250/40 for the masses while the DOCSIS ISPs = offer 1000/50. And unlike latency, bandwidth (or rather rate) is the = number that consumers understand intuitively. If anything will expedite the roll-out of L4S style AQMs it is = the capability to use those to implement the "special services" that the = EU net neutrality regulation explicitly allows, as that is a product = that can be actually sold to customers, but I might be too pessimistic = here. >=20 > As far as I can see, latency aware consumers still are a minority and = gamers seem to be a big group belonging here. Interest in well = performing gaming seems to be growing, I guess (for me at least it's an = impression rather than a clear trend). Put that way, I see a way for ISPs to distinguish themselves = from the rest by being gaming friendly, but unless this results in = gamers paying more I fail to see the business case that management = probably needs before green-lighting the funds required to implement = this. This is where cablelabs approach to mandate this in the specs is = brilliant.=20 >=20 > I'd personally prefer an easy to deploy and operate standard solution = offering Best Effort based transport being TCP friendly and at the same = time congestion free for other flows at a BNG for traffic in access = direction (and for similar devices in other architectures of course).=20 >=20 > Fighting bufferbloat in the upstream direction the way you describe it = doesn't construct a chain of links which are consecutively narrower than = the bottleneck link, I think. Yes, fully agreed, that said, and ISPs CPE should implement an = AQM to really solve the latency issues for end-users. The initial L4S = paper side-stepped that requirement by making sure the uplinks were not = saturated during the test, and state that that needs a real solution for = proper roll-out. In theory the ISP could do the uplink shaping on its = end (and to constrain users to their contracted rates, ISPs do this = already) but as in the downstream case, running an AQM in front of a = bottleneck as opposed to behind it makes everything much easier. Also = with uplinks typically << downlinks, the typically weak CPE CPUs will = still be able to AQM the uplink, nicely distributing that computation = load away from the BNG/BRAS big iron .... Best Regards Sebastian >=20 > Regards, >=20 > Ruediger >=20 >=20 >=20 >=20 >=20 > -----Urspr=C3=BCngliche Nachricht----- > Von: Sebastian Moeller =20 > Gesendet: Freitag, 2. August 2019 15:15 > An: Geib, R=C3=BCdiger > Cc: Jonathan Morton ; tcpm@ietf.org; ECN-Sane = ; tsvwg@ietf.org > Betreff: Re: [Ecn-sane] [tsvwg] ECN CE that was ECT(0) incorrectly = classified as L4S >=20 > Hi Ruediger, >=20 >=20 >=20 >> On Aug 2, 2019, at 10:29, = wrote: >>=20 >> Hi Jonathan, >>=20 >> could you provide a real world example of links which are = consecutively narrower than sender access links? >=20 > Just an example from a network you might be comfortable with, in = DTAGs internet access network there typically are traffic limiting = elements at the BNGs (or at the BRAS for the legacy network), I am not = 100% sure whether these are implemented as policers or shapers, but they = tended to come with >=3D 300ms buffering. Since recently, the BNG/BRAS = traffic shaper's use the message field of the PPPoE Auth ACK to transfer = information about the TCP/IPv4 Goodput endusers can expect on their link = as a consequence of the BNG/BRAS"s traffic limiter. In DOCSIS and GPON = networks the traffic shaper seems mandated by the standards, in DSL = networks it seems optional (but there even without a shaper the limited = bandwidth of the access link would be a natural traffic choke point). > Fritzbox home router's now use this information to automatically set = egress (and I believe also) ingress traffic shaping on the CPE to reduce = the bufferbloat users experience. I have no insight in what Telekom's = own Speedport routers do, but I would not be surprised if they would do = the same (at least for egress).=20 > As Jonathan and Dave mentioned, quite a number of end-users, = especially the latency sensitive ones, employ their own ingress and = egress traffic shapers at their home routers as the 300ms buffers of the = BNG's are just not acceptable for any real-timish uses (VoIP, on-line = twitch gaming, even for interactive sessions like ssh 300ms delay are = undesirable). E.g. personally, I use an OpenWrt router with an FQ AQM = for both ingress and egress (based on Jonathan's excellent cake qdisc) = that allows a family of 5 to happily share a 50/10 connection between = video streaming and interactive use with very little interference = between the users, the same link with out the FQ-AQM active makes = interactive applications feel like submerged in molasses once the link = gets saturated... > As far as I can tell there is a number of different solutions = that offer home-router based bi-directional traffic shaping to solve = bufferbloat" from home (well, not fully solve it, but remedy its = consequences), including commercial options like evenroute's iqrouter, = and open-source options like OpenWrt (with sqm-scripts as shaper = packet).=20 > It is exactly this use case and the fact that latency-sensitive = users often opt for this solution, that causes me to ask the L4S crowd = to actually measure the effect of L4S on RFC3168-FQ-AQMs in the exact = configuration it is actually used today, to remedy the same issue L4S = wants to tackle. >=20 > Best Regards > Sebastian >=20 >=20 >>=20 >> I could figure out a small campus network which has a bottleneck at = the Internet access and a second one connecting the terminal equipment. = But in a small campus network, the individual terminal could very well = have a higher LAN access bandwidth, than the campus - Internet = connection (and then there's only one bottleneck again). >>=20 >> There may be a tradeoff between simplicity and general applicability. = Awareness of that tradeoff is important. To me, simplicity is the design = aim.=20 >>=20 >> Regards, >>=20 >> Ruediger=20 >>=20 >> -----Urspr=C3=BCngliche Nachricht----- >> Von: tsvwg Im Auftrag von Jonathan Morton >> Gesendet: Dienstag, 9. Juli 2019 17:41 >> An: Bob Briscoe >> Cc: tcpm IETF list ; ecn-sane@lists.bufferbloat.net; = tsvwg IETF list >> Betreff: Re: [tsvwg] [Ecn-sane] ECN CE that was ECT(0) incorrectly = classified as L4S >>=20 >>> On 13 Jun, 2019, at 7:48 pm, Bob Briscoe = wrote: >>>=20 >>> 1. It is quite unusual to experience queuing at more than one >>> bottleneck on the same path (the available capacities have = to >>> be identical). >>=20 >> Following up on David Black's comments, I'd just like to note that = the above is not the true criterion for multiple sequential queuing. >>=20 >> Many existing TCP senders are unpaced (aside from ack-clocking), = including FreeBSD, resulting in potentially large line-rate bursts at = the origin - especially during slow-start. Even in congestion = avoidance, each ack will trigger a closely spaced packet pair (or = sometimes a triplet). It is then easy to imagine, or to build a testbed = containing, an arbitrarily long sequence of consecutively narrower = links; upon entering each, the burst of packets will briefly collect in = a queue and then be paced out at the new rate. >>=20 >> TCP pacing does largely eliminate these bursts when implemented = correctly. However, Linux' pacing and IW is specifically (and = apparently deliberately) set up to issue a 10-packet line-rate burst on = startup. This effect has shown up in SCE tests to the point where we = had to patch this behaviour out of the sending kernel to prevent an = instant exit from slow-start. >>=20 >> - Jonathan Morton >>=20 >> _______________________________________________ >> Ecn-sane mailing list >> Ecn-sane@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/ecn-sane >=20