From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from iramx2.ira.uni-karlsruhe.de (iramx2.ira.uni-karlsruhe.de [141.3.10.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id E17353B2A4 for ; Wed, 13 Jun 2018 03:56:07 -0400 (EDT) Received: from i72vorta.tm.uni-karlsruhe.de ([141.3.71.26] helo=i72vorta.tm.kit.edu) by iramx2.ira.uni-karlsruhe.de with esmtp port 25 iface 141.3.10.81 id 1fT0dS-0000sz-2l; Wed, 13 Jun 2018 09:56:06 +0200 Received: from [IPv6:::1] (ip6-localhost [IPv6:::1]) by i72vorta.tm.kit.edu (Postfix) with ESMTPS id 0C50E42049C; Wed, 13 Jun 2018 09:56:06 +0200 (CEST) To: Geoff Huston Cc: Dave Taht , bloat References: <345707C0-F15A-4CBB-A224-27C14BFEC0DF@apnic.net> <4E4AF97A-8B22-4355-AF63-B27529F22D94@apnic.net> From: "Bless, Roland (TM)" Openpgp: preference=signencrypt Autocrypt: addr=roland.bless@kit.edu; prefer-encrypt=mutual; keydata= xsFNBFi0OxABEACy2VohJ7VhSu/xPCt4/6qCrw4Pw2nSklWPfAYEk1QgrbiwgvLAP9WEhAIU w45cojBaDxytIGg8eaYeIKSmsXjHGbV/ZTfo8r11LX8yPYR0WHiMWZpl0SHUd/CZIkv2pChO 88vF/2FKN95HDcp24pwONF4VhxJoSFk6c0mDNf8Em/Glt9BcWX2AAvizTmpQDshaPje18WH3 4++KwPZDd/sJ/hHSXiPg1Gdhs/OG/C0CJguOAlqbgSVAe3qKOr1M4K5M+wVpsk373pXRfxd7 ZAmZ05iBTn+LfgVcz+AfaKKcsWri5CdTT+7JDL6QNQpox+b5FXZFSHnEIST+/qzfG7G2LqqY mml6TYY8XbaNyXZP0QKncfSpRx8uTRWReHUa1YbSuOxXYh6bXpcugD25mlC/Lu0g7tz4ijiK iIwq9+P2H1KfAAfYyYZh6nOoE6ET0TjOjUSa+mA8cqjPWX99kEEgf1Xo+P9fx9QLCLWIY7zc mSM+vjQKgdUFpMSCKcYEKOuwlPuOz8bVECafxaEtJJHjCOK8zowe2eC9OM+G+bmtAO3qYcYZ hQ/PV3sztt/PjgdtnFAYPFLc9189rHRxKsWSOb4xPkRw/YQAI9l15OlUEpsyOehxmAmTsesn tSViCz++PCdeXrQc1BCgl8nDytrxW+n5w1aaE8aL3hn8M0tonQARAQABzShSb2xhbmQgQmxl c3MgKFRNKSA8cm9sYW5kLmJsZXNzQGtpdC5lZHU+wsGABBMBCAAqAhsDBQkSzAMABQsJCAcC BhUICQoLAgQWAgMBAh4BAheABQJYtYdHAhkBAAoJEKON2tlkOJXuzWkP+wfjUnDNzRm4r34a AMWepcQziTgqf4I1crcL6VD44767HhyFsjcKH31E5G5gTDxbpsM4pmkghKeLrpPo30YK3qb7 E9ifIkpJTvMu0StSUmcXq0zPyHZ+HxHeMWkosljG3g/4YekCqgWwrB62T7NMYq0ATQe1MGCZ TAPwSPGCUZT3ioq50800FMI8okkGTXS3h2U922em7k8rv7E349uydv19YEcS7tI78pggMdap ASoP3QWB03tzPKwjqQqSevy64uKDEa0UgvAM3PRbJxOYZlX1c3q/CdWwpwgUiAhMtPWvavWW Tcw6Kkk6e0gw4oFlDQ+hZooLv5rlYR3egdV4DPZ1ugL51u0wQCQG9qKIMXslAdmKbRDkEcWG Oi2bWAdYyIHhhQF5LSuaaxC2P2vOYRHnE5yv5KTV3V7piFgPFjKDW+giCRd7VGfod6DY2b2y zwidCMve1Qsm8+NErH6U+hMpMLeCJDMu1OOvXYbFnTkqjeg5sKipUoSdgXsIo4kl+oArZlpK qComSTPhij7rMyeu/1iOwbNCjtiqgb55ZE7Ekd84mr9sbq4Jm/4QGnVI30q4U2vdGSeNbVjo d1nqjf3UNzP2ZC+H9xjsCFuKYbCX6Yy4SSuEcubtdmdBqm13pxua4ZqPSI0DQST2CHC7nxL1 AaRGRYYh5zo2vRg3ipkEzsFNBFi0OxABEAC2CJNp0/Ivkv4KOiXxitsMXZeK9fI0NU2JU1rW 04dMLF63JF8AFiJ6qeSL2mPHoMiL+fG5jlxy050xMdpMKxnhDVdMxwPtMiGxbByfvrXu18/M B7h+E1DHYVRdFFPaL2jiw+Bvn6wTT31MiuG9Wh0WAhoW8jY8IXxKQrUn7QUOKsWhzNlvVpOo SjMiW4WXksUA0EQVbmlskS/MnFOgCr8q/FqwC81KPy+VLHPB9K/B65uQdpaw78fjAgQVQqpx H7gUF1EYpdZWyojN+V8HtLJx+9yWAZjSFO593OF3/r0nDHEycuOjhefCrqr0DDgTYUNthOdU KO2CzT7MtweRtAf0n27zbwoYvkTviIbR+1lV1vNkxaUtZ6e1rtOxvonRM1O3ddFIzRp/Qufu HfPe0YqhEsrBIGW1aE/pZW8khNQlB6qt20snL9cFDrnB6+8kDG3e//OjK1ICQj9Y/yyrJVaX KfPbdHhLpsgh8TMDPoH+XXQlDJljMD0++/o7ckO3Sfa8Zsyh1WabyKQDYXDmDgi9lCoaQ7Lf uLUpoMvJV+EWo0jE4RW/wBGQbLJp5usy5i0fhBKuDwsKdLG3qOCf4depIcNuja6ZmZHRT+3R FFjvZ/dAhrCWpRTxZANlWlLZz6htToJulAZQJD6lcpVr7EVgDX/y4cNwKF79egWXPDPOvQAR AQABwsFlBBgBCAAPBQJYtDsQAhsMBQkSzAMAAAoJEKON2tlkOJXukMoP/jNeiglj8fenH2We 7SJuyBp8+5L3n8eNwfwY5C5G+etD0E6/lkt/Jj9UddTazxeB154rVFXRzmcN3+hGCOZgGAyV 1N7d8xM6dBqRtHmRMPu5fUxfSqrM9pmqAw2gmzAe0eztVvaM+x5x5xID2WZOiOq8dx9KOKrp Zorekjs3GEA3V1wlZ7Nksx/o8KZ04hLeKcR1r06zEDLN/yA+Fz8IPa0KqpuhrL010bQDgAhe 9o5TA0/cMJpxpLqHhX2As+5cQAhKDDsWJu3oBzZRkN7Hh/HTpWurmTQRRniLGSeiL0zdtilX fowyxGXH6QWi3MZYmpOq+etr7o4EGGbm2inxpVbM+NYmaJs+MAi/z5bsO/rABwdM5ysm8hwb CGt+1oEMORyMcUk/uRjclgTZM1NhGoXm1Un67+Rehu04i7DA6b8dd1H8AFgZSO2H4IKi+5yA Ldmo+ftCJS83Nf6Wi6hJnKG9aWQjKL+qmZqBEct/D2uRJGWAERU5+D0RwNV/i9lQFCYNjG9X Tew0BPYYnBtHFlz9rJTqGhDu4ubulSkbxAK3TIk8XzKdMvef3tV/7mJCmcaVbJ2YoNUtkdKJ goOigJTMBXMRu4Ibyq1Ei+d90lxhojKKlf9yguzpxk5KYFGUizp0dtvdNuXRBtYrwzykS6vB zTlLqHZ0pvGjNfTSvuuN Organization: Institute of Telematics, Karlsruhe Institute of Technology Message-ID: <7da2305e-04f5-6e49-9cb7-6441d086fa1c@kit.edu> Date: Wed, 13 Jun 2018 09:56:05 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <4E4AF97A-8B22-4355-AF63-B27529F22D94@apnic.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 8bit X-ATIS-AV: ClamAV (iramx2.ira.uni-karlsruhe.de) X-ATIS-Timestamp: iramx2.ira.uni-karlsruhe.de 1528876566.198506797 Subject: Re: [Bloat] geoff huston's take on BBR X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2018 07:56:08 -0000 Hi Geoff, Am 13.06.2018 um 01:02 schrieb Geoff Huston: >> On 13 Jun 2018, at 1:58 am, Bless, Roland (TM) wrote: >>> >>> no - I started the flows at 10, 20 and 30 seconds after the initial flow started. >> >> This is nevertheless advantageous for BBR, since it performs its >> ProbeRTT phase every 10s. So using 11, 23, and 34 seconds may >> make a difference in convergence speed (that was at least observed >> in our experiments). > > > I’m not sure that I follow this - It was my understanding that BBR used the RTT interval as its control timer, and maintained a constant send rate for 6 successive RTTY intervals, then inflated the sending r4ate by 25% for the next RTT interval and deflated it by the same amount for the next RTT interval. I don’t believe that there is any absolute timer in BBR along the lines of a 10 second timer that you appear to be suggesting here. I didn't write that BBR uses a 10s timer, but BBR actually uses a time window of 10s. What you describe is BBR's ProbeBW phase, where it probes for more bandwidth. However, your understanding is not quite correct. Since BBR uses a maximum filter (windowed by 10RTTs), it actually increases its sending rate _immediately_ if the 1.25 pacing gain achieved a higher delivery rate (which is nearly always the case if multiple flows are present). So depending on where the pacing_gain is raised in the gain cycle (it starts randomly), BBR will increase its sending rate even within this gain cycle of 8 RTTs and keep it for further 10 RTTs. BBR will decrease its sending rate only after 10 RTTs. That is why multiple BBR flows together are sending always faster than the bottleneck rate in total. I would recommend that you check https://queue.acm.org/detail.cfm?id=3022184 again. Now for the 10s interval. You are correct that it's not a 10s timer, but I wasn't suggesting that either. The RTprop estimate is defined on page 7 of the article and described as "Since path changes happen on time scales >>RTprop, an unbiased, efficient estimator at time T is ...= min(RTT_t) \forall t \in [T-W_R,T] i.e., a running min over time window W_R (which is typically tens of seconds to minutes)." and on p. 31: "BBR flows cooperate to periodically drain the bottleneck queue using a state called ProbeRTT. In any state other than ProbeRTT itself, if the RTProp estimate has not been updated (i.e., by getting a lower RTT measurement) for more than 10 seconds, then BBR enters ProbeRTT and reduces the cwnd to a very small value (four packets)." So it's basically a time window and also implemented like that. A concise BBR description can be also found in our paper in section II (http://doc.tm.kit.edu/2017-kit-icnp-bbr-authors-copy.pdf). Regards, Roland