From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 903253B29E; Sun, 5 Jul 2020 08:02:00 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1593950511; bh=D9Dpv5HOtYp0Y8FiJVvch7F5You9SsOHsYZeOM/t+/8=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=id8vqKu+N+eCAmLx3r3QWcvdHqwgy04duCgdTlKsyEBaCiRZFGKvNKfgUb5PCkoqs m40GY2rud1GT5iHYjV6h9xweF3Gp9na0DFu2i6CA4hzqVg1y3IdvGdBaa1dRdGypcY NMKUHf52J+1EC+nJtOjf/pCiPrzuS1abyewwWL8k= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.42.229] ([77.0.164.216]) by mail.gmx.com (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MJE27-1kC4GK2IKx-00KgsL; Sun, 05 Jul 2020 14:01:51 +0200 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.14\)) From: Sebastian Moeller In-Reply-To: Date: Sun, 5 Jul 2020 14:01:50 +0200 Cc: Daniel Sterling , Make-Wifi-fast , Carlo Augusto Grazia , bloat , Jamshid Mahdavi Content-Transfer-Encoding: quoted-printable Message-Id: <9A7FBFF1-7F10-41DD-B5C3-5A45254CEB54@gmx.de> References: <5405F10B-F446-4B74-8894-33232145EB2E@gmx.de> To: Matt Mathis X-Mailer: Apple Mail (2.3445.104.14) X-Provags-ID: V03:K1:Eh2sWGXmcQTCzDt3hLkf7nmLfDq3ex+XPrvSPsHIBmqsZSdxxFI iWn4NF4ZYD2iSDdjqrxBdYzbHTGW769rej1RLFCHVqFbdr/fWZe45W0a7OTp2ewbHvBwzmG 4f+y631R7oyZ2en3LOxs2dZA8ZOFSTeKywRXMMZQT5j7305qa/RECjo0xAhWzB0yUzWUCkw OyP5y4uMLb/ym+6fZerwA== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:Mntf97vcotk=:xIH3Nk4FRxBTS7pxulYE7T q8xYiMwx5uDPYdNowrciHRFeOmb+efo+PnnWD8IqKcFIdNm97w3ADc2xWr+S1IUWtPki9o6Kd 9gXbz9PyNxXRqnf6PWK+SE347UhBCCeBLqeo3Ms6UkZDbL00w90iMgurUyqjG1q0bZkLjDWHO w3uB3Q1kp/NCTljxxsYyQ0RzF55QiipD/pb436m945Y06/UwOIAC7OGoAZmMUb+bNXhHliGeK qmAGHfSnX2nJ3GAb3haVy6cOeYQ/HZyasGl+lbeUlEFur7P3os9V0F5i7bSO3b137WPX+nF1q Y8V1RyHJJAP1qhYa7mQxDLF68cgg/eims6MgymzOuBGtvxLosm9RBL/8SX6zF/ocmZ3aENsEM c9Kvil31qkql5ZeMT1vFr+qRrtoU27/6TcYuLEW+BwT0Vg2t69b40ojlb33BufTrTIe2IKiME TQz3zcQg98KZV8GWvEoTevb6lfj96lngxtYpyEe2q9XpsrC5NERp2wIXC7H1SZJWtbAT/WyMh oBkuAeXMyPfvCdKc4LW1pNTBgnZDnZJugugXJ6UxUny5Ouizy3Mii7CAmvtNtCYEJnilao/dp ReNBvcsz4PuvCWcQ8egU2Mj8o6RoHxMTI4+QK6DPKf/A1sGVPb6+AB1A7es7v9vz3W1Mxw/Ue blxcahKROxh0ksn9O1AeQdHAQC44xaoOubIrsA938HaQ1cd57cgVdWBN7ZI1RCZASVbQmclaY zOkazwt+WYPwt37rt3fT1Y3p9dgAG+pv0uC5SJMDCDSIQcx3D/YblqWyEubcQdiUChj5pBATa uDgQdmo4T8VTqDpYC7VOdd/VjfAxhSz4noHq31oEfr66248Q4BMiNPq83VyBg3B58YOYLvHVs hNUY1oz+cdxXwcaECIVoixtaboEKdJKgxv+SWzOz0wCWlHnpNoDPd848zGWoaMnYSR3i7ZkLY lZhdiRDt9IPCTJpaLuVnrsLAdWNcvCeoFpFwfUzyqdZRtDApxkM50fBBF26ZDZrAZ05+yVuyt FSihhIDUUo/eAI/soeOqFryu4R/eLaDVF3Bln/0Pr1ex8ej1IJnshma1gVn6VyuGlP2WD54fA 1SRmypmHtuGZ+AKncdPcTPXVAHNAtLTbpiWf5v9TFGOYyGLUZacl6yU35wJ9M8wvAZBCXMe8e S+qCX4MX/Cdq4uMSSyMFQrU6iK8rpqpSu8zLakCon4hAF/HQkihJNzp4qFvWMwvXZTjMSyLkX PEqJ/IJIGOUs3gw6U Subject: Re: [Bloat] the future belongs to pacing X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Jul 2020 12:02:00 -0000 Hi Matt, > On Jul 5, 2020, at 08:10, Matt Mathis wrote: >=20 > I strongly suggest that people (re)read VJ88 - I do every couple of = years, and still discover things that I overlooked on previous readings. I promise to read it. And before I give the wrong impression and = for what it is worth*, I consider BBR (even v1) an interesting and = important evolutionary step and agree that "pacing" is a gentler = approach then bursting a full CWN into a link. >=20 > All of the negative comments about BBR and loss, ECN marks, As far as I can tell, BBRv2 aims for a decidedly non-rfc3168 = response to CE-marks. This IMHO is not a clear cut case of meaningfully = addressing my ECN comment. In the light of efficiently using TOR? switch = buffers efficiently, that kind of response might be defensible but it = does not really address my remark about it being unfortunate that BBR = ignores both immediate signals of congestion, (sparse) packet drops AND = explicit CE marks, the proposed (dctcp-like) CE-response seems rather = weak compared to the naive expectation of halving/80%-ing of the sending = rate, no? BBRv2 as I understand it will happily run roughshod over any = true rfc3168 AQM on the path, I do not have the numbers, but I am not = fully convinced that typically the most significant throttling on a CDN = to end-user path happens still inside the CDN's data center...=20 > or unfairness to cubic were correct for BBRv1 but have been addressed = in BBRv2. I am not sure that unfairness was brought up as an issue in this = thread. >=20 > My paper has a synopsis of BBR, which is intended to get people = started. See the references in the paper for more info: I will have a look at these as well... Thanks Best Regards Sebastian *) Being from outside the field, probably not much... >=20 > [12] Neal Cardwell, Yuchung Cheng, C. Stephen Gunn, Soheil Hassas = Yeganeh, and Van Jacobson. 2016. BBR: Congestion-Based Congestion = Control. Queue 14, 5, Pages 50 (October 2016). DOI: = https://doi.org/10.1145/3012426.3022184 > [13] Neal Cardwell, Yuchung Cheng, C. Stephen Gunn, Soheil Hassas = Yeganeh, and Van Jacobson. 2017. BBR: Congestion-Based Congestion = Control. Commun. ACM 60, 2 (January 2017), 58-66. DOI: = https://doi.org/10.1145/3009824 > [22] google/bbr. 2019. GitHub repository, retrieved = https://github.com/google/bbr >=20 > Key definitions: self clocked: data is triggered by ACKs. All screwy = packet and ACK scheduling in the network is reflected back into the = network on the next RTT. >=20 > Paced: data is transmitted on a timer, independent of ACK arrivals (as = long as the ACKs take less than twice the measured minRTT). Thus in = bulk transport there is little or no correlation between data = transmissions and events elsewhere in the network.=20 >=20 > Clarification about my earlier WiFi comment: The BBRv1 WiFi fix = missed 4.19 LTS, so bad results are "expected" for many distros. If you = want to do useful experiments, you must read = https://groups.google.com/g/bbr-dev/ and start from BBRv2 in [22]. >=20 > Thanks, > --MM-- > The best way to predict the future is to create it. - Alan Kay >=20 > We must not tolerate intolerance; > however our response must be carefully measured:=20 > too strong would be hypocritical and risks spiraling out = of control; > too weak risks being mistaken for tacit approval. >=20 >=20 > On Sat, Jul 4, 2020 at 11:29 AM Sebastian Moeller = wrote: >=20 >=20 > > On Jul 4, 2020, at 19:52, Daniel Sterling = wrote: > >=20 > > On Sat, Jul 4, 2020 at 1:29 PM Matt Mathis via Bloat > > wrote: > > "pacing is inevitable, because it saves large content providers = money > > (more efficient use of the most expensive silicon in the data = center, > > the switch buffer memory), however to use pacing we walk away from = 30 > > years of experience with TCP self clock" > >=20 > > at the risk of asking w/o doing any research, > >=20 > > could someone explain this to a lay person or point to a doc talking > > about this more? > >=20 > > What does BBR do that's different from other algorithms? >=20 > Well, it does not believe the network (blindly), that is = currently it ignores both ECN marks and (sparse) drops as signs of = congestion, instead it uses its own rate estimates to set its send rate = and cyclically will re-assess its rate estimate. Sufficiently severe = drops will be honored. IMHO a somewhat risky approach, that works = reasonably well, as often sparse drops are not real signs of congestion = but just random drops of say a wifi link (that said, these drops on wifi = typically also cause painful latency spikes as wifi often takes heroic = measures in attempting retransmitting for several 100s of milliseconds). >=20 >=20 > > Why does it > > break the clock? >=20 > One can argue that there is no real clock to break. TCP gates = the release on new packets on the reception of ACK signals from the = receiver, this is only a clock, if one does not really care for the = equi-temporal period property of a real clock. But for better or worse = that is the term that is used. IMHO (and I really am calling this from = way out in the left-field) gating would be a better term, but changing = the nomenclature probably is not an option at this point. >=20 > > Before BBR, was the clock the only way TCP did CC? >=20 > No, TCP also interpreted a drop (or rather 3 duplicated ACKs) = as signal of congestion and hit the brakes, by halving the congestion = window (the amount of data that could be in flight unacknowledged, which = roughly correlates with the send rate, if averaged over long enough time = windows). BBR explicitly does not do this unless it really is convinced = that someone dropped multiple packets purposefully to signal congestion. > In practice it works rather well, in theory it could do with = at least an rfc3168 compliant response to ECN marks (which an AQM uses = to explicitly signal congestion, unlike a drop an ECN mark is really = unambiguous, some hop on the way "told" the flow slow down). >=20 >=20 > >=20 > > Also, > >=20 > > I have UBNT "Amplifi" HD wifi units in my house. (HD units only; = none > > of the "mesh" units. Just HD units connected either via wifi or > > wired.) Empirically, I've found that in order to reduce latency, I > > need to set cake to about 1/4 of the total possible wifi speed; > > otherwise if a large download comes down from my internet link, that > > flow causes latency. > >=20 > > That is, if I'm using 5ghz at 20mhz channel width, I need to set > > cake's bandwidth argument to 40mbits to prevent video streams / > > downloads from impacting latency for any other stream. This is w/o = any > > categorization at all; no packet marking based on port or anything > > else; cake set to "best effort". > >=20 > > Anything higher and when a large amount of data comes thru, = something > > (assumedly the buffer in the Amplifi HD units) causes 100s of > > milliseconds of latency. > >=20 > > Can anyone speak to how BBR would react to this? My ISP is full > > gigabit; but cake is going to drop a lot of packets as it throttles > > that down to 40mbit before it sends the packets to the wifi AP. > >=20 > > Thanks, > > Dan > > _______________________________________________ > > Bloat mailing list > > Bloat@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/bloat >=20