From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from iramx2.ira.uni-karlsruhe.de (iramx2.ira.uni-karlsruhe.de [IPv6:2a00:1398:2::10:81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 405423B29E for ; Fri, 31 May 2019 08:39:10 -0400 (EDT) Received: from i72vorta.tm.uni-karlsruhe.de ([141.3.71.26] helo=i72vorta.tm.kit.edu) by iramx2.ira.uni-karlsruhe.de with esmtpsa port 25 iface 141.3.10.8 id 1hWgoO-00068u-6l; Fri, 31 May 2019 14:39:08 +0200 Received: from [IPv6:::1] (localhost [127.0.0.1]) by i72vorta.tm.kit.edu (Postfix) with ESMTPS id F133C420380; Fri, 31 May 2019 14:39:07 +0200 (CEST) To: Dave Taht Cc: BBR Development , bloat References: From: Roland Bless Openpgp: preference=signencrypt Autocrypt: addr=roland.bless@kit.edu; prefer-encrypt=mutual; keydata= mQINBFi0OxABEACy2VohJ7VhSu/xPCt4/6qCrw4Pw2nSklWPfAYEk1QgrbiwgvLAP9WEhAIU w45cojBaDxytIGg8eaYeIKSmsXjHGbV/ZTfo8r11LX8yPYR0WHiMWZpl0SHUd/CZIkv2pChO 88vF/2FKN95HDcp24pwONF4VhxJoSFk6c0mDNf8Em/Glt9BcWX2AAvizTmpQDshaPje18WH3 4++KwPZDd/sJ/hHSXiPg1Gdhs/OG/C0CJguOAlqbgSVAe3qKOr1M4K5M+wVpsk373pXRfxd7 ZAmZ05iBTn+LfgVcz+AfaKKcsWri5CdTT+7JDL6QNQpox+b5FXZFSHnEIST+/qzfG7G2LqqY mml6TYY8XbaNyXZP0QKncfSpRx8uTRWReHUa1YbSuOxXYh6bXpcugD25mlC/Lu0g7tz4ijiK iIwq9+P2H1KfAAfYyYZh6nOoE6ET0TjOjUSa+mA8cqjPWX99kEEgf1Xo+P9fx9QLCLWIY7zc mSM+vjQKgdUFpMSCKcYEKOuwlPuOz8bVECafxaEtJJHjCOK8zowe2eC9OM+G+bmtAO3qYcYZ hQ/PV3sztt/PjgdtnFAYPFLc9189rHRxKsWSOb4xPkRw/YQAI9l15OlUEpsyOehxmAmTsesn tSViCz++PCdeXrQc1BCgl8nDytrxW+n5w1aaE8aL3hn8M0tonQARAQABtChSb2xhbmQgQmxl c3MgKFRNKSA8cm9sYW5kLmJsZXNzQGtpdC5lZHU+iQJABBMBCAAqAhsDBQkSzAMABQsJCAcC BhUICQoLAgQWAgMBAh4BAheABQJYtYdHAhkBAAoJEKON2tlkOJXuzWkP+wfjUnDNzRm4r34a AMWepcQziTgqf4I1crcL6VD44767HhyFsjcKH31E5G5gTDxbpsM4pmkghKeLrpPo30YK3qb7 E9ifIkpJTvMu0StSUmcXq0zPyHZ+HxHeMWkosljG3g/4YekCqgWwrB62T7NMYq0ATQe1MGCZ TAPwSPGCUZT3ioq50800FMI8okkGTXS3h2U922em7k8rv7E349uydv19YEcS7tI78pggMdap ASoP3QWB03tzPKwjqQqSevy64uKDEa0UgvAM3PRbJxOYZlX1c3q/CdWwpwgUiAhMtPWvavWW Tcw6Kkk6e0gw4oFlDQ+hZooLv5rlYR3egdV4DPZ1ugL51u0wQCQG9qKIMXslAdmKbRDkEcWG Oi2bWAdYyIHhhQF5LSuaaxC2P2vOYRHnE5yv5KTV3V7piFgPFjKDW+giCRd7VGfod6DY2b2y zwidCMve1Qsm8+NErH6U+hMpMLeCJDMu1OOvXYbFnTkqjeg5sKipUoSdgXsIo4kl+oArZlpK qComSTPhij7rMyeu/1iOwbNCjtiqgb55ZE7Ekd84mr9sbq4Jm/4QGnVI30q4U2vdGSeNbVjo d1nqjf3UNzP2ZC+H9xjsCFuKYbCX6Yy4SSuEcubtdmdBqm13pxua4ZqPSI0DQST2CHC7nxL1 AaRGRYYh5zo2vRg3ipkEuQINBFi0OxABEAC2CJNp0/Ivkv4KOiXxitsMXZeK9fI0NU2JU1rW 04dMLF63JF8AFiJ6qeSL2mPHoMiL+fG5jlxy050xMdpMKxnhDVdMxwPtMiGxbByfvrXu18/M B7h+E1DHYVRdFFPaL2jiw+Bvn6wTT31MiuG9Wh0WAhoW8jY8IXxKQrUn7QUOKsWhzNlvVpOo SjMiW4WXksUA0EQVbmlskS/MnFOgCr8q/FqwC81KPy+VLHPB9K/B65uQdpaw78fjAgQVQqpx H7gUF1EYpdZWyojN+V8HtLJx+9yWAZjSFO593OF3/r0nDHEycuOjhefCrqr0DDgTYUNthOdU KO2CzT7MtweRtAf0n27zbwoYvkTviIbR+1lV1vNkxaUtZ6e1rtOxvonRM1O3ddFIzRp/Qufu HfPe0YqhEsrBIGW1aE/pZW8khNQlB6qt20snL9cFDrnB6+8kDG3e//OjK1ICQj9Y/yyrJVaX KfPbdHhLpsgh8TMDPoH+XXQlDJljMD0++/o7ckO3Sfa8Zsyh1WabyKQDYXDmDgi9lCoaQ7Lf uLUpoMvJV+EWo0jE4RW/wBGQbLJp5usy5i0fhBKuDwsKdLG3qOCf4depIcNuja6ZmZHRT+3R FFjvZ/dAhrCWpRTxZANlWlLZz6htToJulAZQJD6lcpVr7EVgDX/y4cNwKF79egWXPDPOvQAR AQABiQIlBBgBCAAPBQJYtDsQAhsMBQkSzAMAAAoJEKON2tlkOJXukMoP/jNeiglj8fenH2We 7SJuyBp8+5L3n8eNwfwY5C5G+etD0E6/lkt/Jj9UddTazxeB154rVFXRzmcN3+hGCOZgGAyV 1N7d8xM6dBqRtHmRMPu5fUxfSqrM9pmqAw2gmzAe0eztVvaM+x5x5xID2WZOiOq8dx9KOKrp Zorekjs3GEA3V1wlZ7Nksx/o8KZ04hLeKcR1r06zEDLN/yA+Fz8IPa0KqpuhrL010bQDgAhe 9o5TA0/cMJpxpLqHhX2As+5cQAhKDDsWJu3oBzZRkN7Hh/HTpWurmTQRRniLGSeiL0zdtilX fowyxGXH6QWi3MZYmpOq+etr7o4EGGbm2inxpVbM+NYmaJs+MAi/z5bsO/rABwdM5ysm8hwb CGt+1oEMORyMcUk/uRjclgTZM1NhGoXm1Un67+Rehu04i7DA6b8dd1H8AFgZSO2H4IKi+5yA Ldmo+ftCJS83Nf6Wi6hJnKG9aWQjKL+qmZqBEct/D2uRJGWAERU5+D0RwNV/i9lQFCYNjG9X Tew0BPYYnBtHFlz9rJTqGhDu4ubulSkbxAK3TIk8XzKdMvef3tV/7mJCmcaVbJ2YoNUtkdKJ goOigJTMBXMRu4Ibyq1Ei+d90lxhojKKlf9yguzpxk5KYFGUizp0dtvdNuXRBtYrwzykS6vB zTlLqHZ0pvGjNfTSvuuN Organization: Institute of Telematics, Karlsruhe Institute of Technology (KIT) Message-ID: Date: Fri, 31 May 2019 14:39:07 +0200 User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.1) Gecko/20060111 Thunderbird/1.5 Mnenhy/0.7.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 8bit X-ATIS-AV: ClamAV (iramx2.ira.uni-karlsruhe.de) X-ATIS-Timestamp: iramx2.ira.uni-karlsruhe.de esmtpsa 1559306348.281938769 Subject: Re: [Bloat] BBR high RTT unfairness: Fifty Shades of Congestion Control: A Performance and Interactions Evaluation X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 May 2019 12:39:10 -0000 Hi Dave, On 30.05.19 at 15:38 Dave Taht wrote: > On Thu, May 30, 2019 at 4:31 AM Roland Bless wrote: >> >> Hi Dave, >> >> On 29.05.19 at 17:05 Dave Taht wrote: >>> I have been trying to work through this paper: >>> https://arxiv.org/pdf/1903.03852.pdf >>> which is enormous and well worth reading. >>> >>> I have a theory, though, about TABLE XII, which contrasts four BBR >>> flows at different RTTs, in that BBRv1's probe phase makes a 200ms >>> assumption, thus >>> not seeing the real rtt at ong rtts, and thus the longest RTT flow >>> gets the most bandwidth on this test, and the second (testable) theory >>> is that were these rtts not exactly on the 100ms boundaries, we would >>> see more throughput fairness. >> >> Nope, the main reason for RTT unfairness in BBRv1 is its >> CWnd cap at 2*(RTT_min*est_bw) (2*estimated bottleneck BDP share). > > The striking thing about that table was that the 300ms result was the > ~same as the 100ms result for throughput, while the ones on the 200 > and 400ms ones were 2x and 4x respectively. Yes, correct. > My thought was that at extraordinary RTTs (anything > planet girdling > e.g. > 200ms) that trying a probe of 250ms (or some degree of variance > periodically - 220ms, 260ms, 180ms) or changing the period of the > probe itself, would desync things and get closer to the real RTT, > particularly when BBR was duking it out with itself. > > This would also make up for researchers (which includes myself until I > trained myself out of it) tending to always start a test with multiple > flows all at exactly the same time, which could be another flaw in > this dataset. You are right and we ran into this trap also at first and experienced similar effects in our BBR evaluations. Therefore, we used starting times that are not multiple of 10s (0s, 23s, 31s, 38s, 42s, 47s), because BBR probes every 10s for the min RTT, cf. Section V.D of our paper below. >> As we showed in http://doc.tm.kit.edu/2017-kit-icnp-bbr-authors-copy.pdf >> Section III: multiple BBR flows will always increase their CWnd up >> to this point (except when the buffer capacity is smaller than a BDP). >> Neal's explanation is in line with our findings. >> Consequently, each flow will converge towards a share of RTT_min*est_bw >> at the bottleneck queue, providing a larger bandwidth share for flows >> with a larger RTT_min. See also Section V.F of our paper that also >> evaluated RTT unfairness (moreover, the outcome depends also on the >> bottleneck buffer size). > > I get it, it's my point above about not seeing RTT_min properly with > synced flows... I'm not sure, my explanations hold even if all flows see the proper RTT_min. It seems that the 300ms RTT flow has difficulties in getting a larger share. >> Unfortunately, they didn't test TCP-LoLa in this context, since it is >> actually able to provide fairness among flows with different RTTs >> (while still limiting the overall queuing delay). > > I keep hoping people keep their labs setup, so that we could have 54 > shades of congestion control going forward (dctcp, bbrv2, lola, fu) > and a stable base of data to work from. > > Me being me I'd also like to vary the fq and aqm algorithms using the > same test setups. > > I'll ping the authors. > >> Moreover, Mario >> and Felix improved the convergence speed by introducing FFBquick, see: >> http://doc.tm.kit.edu/Poster/2019-FFBquick_Networking.pdf >> for a quick glance on the challenges and the solution. >> This was published as poster paper at Networking 2019: >> M. Hock, R. Bless, F. Neumeister, M. Zitterbart: FFBquick: Fast >> Convergence to Fairness for Delay-bounded Congestion Controls, >> Networking 2019, Warsaw, Poland, May 20-22. >> >> Regards >> Roland >> > > -- > > Dave Täht > CTO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-831-205-9740 >