From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from iramx2.ira.uni-karlsruhe.de (iramx2.ira.uni-karlsruhe.de [141.3.10.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id DD98A3CB35 for ; Tue, 27 Nov 2018 06:40:46 -0500 (EST) Received: from i72vorta.tm.uni-karlsruhe.de ([141.3.71.26] helo=i72vorta.tm.kit.edu) by iramx2.ira.uni-karlsruhe.de with esmtp port 25 iface 141.3.10.81 id 1gRbjR-0005Y5-Jq; Tue, 27 Nov 2018 12:40:45 +0100 Received: from [IPv6:::1] (ip6-localhost [IPv6:::1]) by i72vorta.tm.kit.edu (Postfix) with ESMTPS id 8D23B420DD3; Tue, 27 Nov 2018 12:40:45 +0100 (CET) To: Luca Muscariello Cc: Jonathan Morton , bloat References: <65EAC6C1-4688-46B6-A575-A6C7F2C066C5@heistp.net> <86b16a95-e47d-896b-9d43-69c65c52afc7@kit.edu> From: "Bless, Roland (TM)" Openpgp: preference=signencrypt Autocrypt: addr=roland.bless@kit.edu; prefer-encrypt=mutual; keydata= xsFNBFi0OxABEACy2VohJ7VhSu/xPCt4/6qCrw4Pw2nSklWPfAYEk1QgrbiwgvLAP9WEhAIU w45cojBaDxytIGg8eaYeIKSmsXjHGbV/ZTfo8r11LX8yPYR0WHiMWZpl0SHUd/CZIkv2pChO 88vF/2FKN95HDcp24pwONF4VhxJoSFk6c0mDNf8Em/Glt9BcWX2AAvizTmpQDshaPje18WH3 4++KwPZDd/sJ/hHSXiPg1Gdhs/OG/C0CJguOAlqbgSVAe3qKOr1M4K5M+wVpsk373pXRfxd7 ZAmZ05iBTn+LfgVcz+AfaKKcsWri5CdTT+7JDL6QNQpox+b5FXZFSHnEIST+/qzfG7G2LqqY mml6TYY8XbaNyXZP0QKncfSpRx8uTRWReHUa1YbSuOxXYh6bXpcugD25mlC/Lu0g7tz4ijiK iIwq9+P2H1KfAAfYyYZh6nOoE6ET0TjOjUSa+mA8cqjPWX99kEEgf1Xo+P9fx9QLCLWIY7zc mSM+vjQKgdUFpMSCKcYEKOuwlPuOz8bVECafxaEtJJHjCOK8zowe2eC9OM+G+bmtAO3qYcYZ hQ/PV3sztt/PjgdtnFAYPFLc9189rHRxKsWSOb4xPkRw/YQAI9l15OlUEpsyOehxmAmTsesn tSViCz++PCdeXrQc1BCgl8nDytrxW+n5w1aaE8aL3hn8M0tonQARAQABzShSb2xhbmQgQmxl c3MgKFRNKSA8cm9sYW5kLmJsZXNzQGtpdC5lZHU+wsGABBMBCAAqAhsDBQkSzAMABQsJCAcC BhUICQoLAgQWAgMBAh4BAheABQJYtYdHAhkBAAoJEKON2tlkOJXuzWkP+wfjUnDNzRm4r34a AMWepcQziTgqf4I1crcL6VD44767HhyFsjcKH31E5G5gTDxbpsM4pmkghKeLrpPo30YK3qb7 E9ifIkpJTvMu0StSUmcXq0zPyHZ+HxHeMWkosljG3g/4YekCqgWwrB62T7NMYq0ATQe1MGCZ TAPwSPGCUZT3ioq50800FMI8okkGTXS3h2U922em7k8rv7E349uydv19YEcS7tI78pggMdap ASoP3QWB03tzPKwjqQqSevy64uKDEa0UgvAM3PRbJxOYZlX1c3q/CdWwpwgUiAhMtPWvavWW Tcw6Kkk6e0gw4oFlDQ+hZooLv5rlYR3egdV4DPZ1ugL51u0wQCQG9qKIMXslAdmKbRDkEcWG Oi2bWAdYyIHhhQF5LSuaaxC2P2vOYRHnE5yv5KTV3V7piFgPFjKDW+giCRd7VGfod6DY2b2y zwidCMve1Qsm8+NErH6U+hMpMLeCJDMu1OOvXYbFnTkqjeg5sKipUoSdgXsIo4kl+oArZlpK qComSTPhij7rMyeu/1iOwbNCjtiqgb55ZE7Ekd84mr9sbq4Jm/4QGnVI30q4U2vdGSeNbVjo d1nqjf3UNzP2ZC+H9xjsCFuKYbCX6Yy4SSuEcubtdmdBqm13pxua4ZqPSI0DQST2CHC7nxL1 AaRGRYYh5zo2vRg3ipkEzsFNBFi0OxABEAC2CJNp0/Ivkv4KOiXxitsMXZeK9fI0NU2JU1rW 04dMLF63JF8AFiJ6qeSL2mPHoMiL+fG5jlxy050xMdpMKxnhDVdMxwPtMiGxbByfvrXu18/M B7h+E1DHYVRdFFPaL2jiw+Bvn6wTT31MiuG9Wh0WAhoW8jY8IXxKQrUn7QUOKsWhzNlvVpOo SjMiW4WXksUA0EQVbmlskS/MnFOgCr8q/FqwC81KPy+VLHPB9K/B65uQdpaw78fjAgQVQqpx H7gUF1EYpdZWyojN+V8HtLJx+9yWAZjSFO593OF3/r0nDHEycuOjhefCrqr0DDgTYUNthOdU KO2CzT7MtweRtAf0n27zbwoYvkTviIbR+1lV1vNkxaUtZ6e1rtOxvonRM1O3ddFIzRp/Qufu HfPe0YqhEsrBIGW1aE/pZW8khNQlB6qt20snL9cFDrnB6+8kDG3e//OjK1ICQj9Y/yyrJVaX KfPbdHhLpsgh8TMDPoH+XXQlDJljMD0++/o7ckO3Sfa8Zsyh1WabyKQDYXDmDgi9lCoaQ7Lf uLUpoMvJV+EWo0jE4RW/wBGQbLJp5usy5i0fhBKuDwsKdLG3qOCf4depIcNuja6ZmZHRT+3R FFjvZ/dAhrCWpRTxZANlWlLZz6htToJulAZQJD6lcpVr7EVgDX/y4cNwKF79egWXPDPOvQAR AQABwsFlBBgBCAAPBQJYtDsQAhsMBQkSzAMAAAoJEKON2tlkOJXukMoP/jNeiglj8fenH2We 7SJuyBp8+5L3n8eNwfwY5C5G+etD0E6/lkt/Jj9UddTazxeB154rVFXRzmcN3+hGCOZgGAyV 1N7d8xM6dBqRtHmRMPu5fUxfSqrM9pmqAw2gmzAe0eztVvaM+x5x5xID2WZOiOq8dx9KOKrp Zorekjs3GEA3V1wlZ7Nksx/o8KZ04hLeKcR1r06zEDLN/yA+Fz8IPa0KqpuhrL010bQDgAhe 9o5TA0/cMJpxpLqHhX2As+5cQAhKDDsWJu3oBzZRkN7Hh/HTpWurmTQRRniLGSeiL0zdtilX fowyxGXH6QWi3MZYmpOq+etr7o4EGGbm2inxpVbM+NYmaJs+MAi/z5bsO/rABwdM5ysm8hwb CGt+1oEMORyMcUk/uRjclgTZM1NhGoXm1Un67+Rehu04i7DA6b8dd1H8AFgZSO2H4IKi+5yA Ldmo+ftCJS83Nf6Wi6hJnKG9aWQjKL+qmZqBEct/D2uRJGWAERU5+D0RwNV/i9lQFCYNjG9X Tew0BPYYnBtHFlz9rJTqGhDu4ubulSkbxAK3TIk8XzKdMvef3tV/7mJCmcaVbJ2YoNUtkdKJ goOigJTMBXMRu4Ibyq1Ei+d90lxhojKKlf9yguzpxk5KYFGUizp0dtvdNuXRBtYrwzykS6vB zTlLqHZ0pvGjNfTSvuuN Organization: Institute of Telematics, Karlsruhe Institute of Technology Message-ID: <7e6cc6f4-bd2f-49b5-0f64-292f56a0592c@kit.edu> Date: Tue, 27 Nov 2018 12:40:45 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 8bit X-ATIS-AV: ClamAV (iramx2.ira.uni-karlsruhe.de) X-ATIS-Timestamp: iramx2.ira.uni-karlsruhe.de 1543318845.686798846 Subject: Re: [Bloat] when does the CoDel part of fq_codel help in the real world? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Nov 2018 11:40:47 -0000 Hi Luca, Am 27.11.18 um 11:40 schrieb Luca Muscariello: > OK. We agree. > That's correct, you need *at least* the BDP in flight so that the > bottleneck queue never empties out. No, that's not what I meant, but it's quite simple. You need: data min_inflight=2 * RTTmin * bottleneck_rate to filly utilize the bottleneck link. If this is true, the bottleneck queue will be empty. If your amount of inflight data is larger, the bottleneck queue buffer will store the excess packets. With just min_inflight there will be no bottleneck queue, the packets are "on the wire". > This can be easily proven using fluid models for any congestion > controlled source no matter if it is  > loss-based, delay-based, rate-based, formula-based etc. > > A highly paced source gives you the ability to get as close as > theoretically possible to the BDP+epsilon > as possible. Yep, but that BDP is "on the wire" and epsilon will be in the bottleneck buffer. > link fully utilized is defined as Q>0 unless you don't include the > packet currently being transmitted. I do, > so the TXtteer is never idle. But that's a detail. I wouldn't define link fully utilized as Q>0, but if Q>0 then the link is fully utilized (that's what I meant by the direction of implication). Rgards, Roland > > > On Tue, Nov 27, 2018 at 11:35 AM Bless, Roland (TM) > > wrote: > > Hi, > > Am 27.11.18 um 11:29 schrieb Luca Muscariello: > > I have never said that you need to fill the buffer to the max size to > > get full capacity, which is an absurdity. > > Yes, it's absurd, but that's what today's loss-based CC algorithms do. > > > I said you need at least the BDP so that the queue never empties out. > > The link is fully utilized IFF the queue is never emptied. > > I was also a bit imprecise: you'll need a BDP in flight, but > you don't need to fill the buffer at all. The latter sentence > is valid only in the direction: queue not empty -> link fully utilized. > > Regards, >  Roland > > > > > > > > > On Tue 27 Nov 2018 at 11:26, Bless, Roland (TM) > > > >> wrote: > > > >     Hi Luca, > > > >     Am 27.11.18 um 10:24 schrieb Luca Muscariello: > >     > A congestion controlled protocol such as TCP or others, > including > >     QUIC, > >     > LEDBAT and so on > >     > need at least the BDP in the transmission queue to get full link > >     > efficiency, i.e. the queue never empties out. > > > >     This is not true. There are congestion control algorithms > >     (e.g., TCP LoLa [1] or BBRv2) that can fully utilize the > bottleneck link > >     capacity without filling the buffer to its maximum capacity. > The BDP > >     rule of thumb basically stems from the older loss-based congestion > >     control variants that profit from the standing queue that they > built > >     over time when they detect a loss: > >     while they back-off and stop sending, the queue keeps the > bottleneck > >     output busy and you'll not see underutilization of the link. > Moreover, > >     once you get good loss de-synchronization, the buffer size > requirement > >     for multiple long-lived flows decreases. > > > >     > This gives rule of thumbs to size buffers which is also very > practical > >     > and thanks to flow isolation becomes very accurate. > > > >     The positive effect of buffers is merely their role to absorb > >     short-term bursts (i.e., mismatch in arrival and departure rates) > >     instead of dropping packets. One does not need a big buffer to > >     fully utilize a link (with perfect knowledge you can keep the link > >     saturated even without a single packet waiting in the buffer). > >     Furthermore, large buffers (e.g., using the BDP rule of thumb) > >     are not useful/practical anymore at very high speed such as > 100 Gbit/s: > >     memory is also quite costly at such high speeds... > > > >     Regards, > >      Roland > > > >     [1] M. Hock, F. Neumeister, M. Zitterbart, R. Bless. > >     TCP LoLa: Congestion Control for Low Latencies and High > Throughput. > >     Local Computer Networks (LCN), 2017 IEEE 42nd Conference on, pp. > >     215-218, Singapore, Singapore, October 2017 > >     http://doc.tm.kit.edu/2017-LCN-lola-paper-authors-copy.pdf > > > >     > Which is:  > >     > > >     > 1) find a way to keep the number of backlogged flows at a > >     reasonable value.  > >     > This largely depends on the minimum fair rate an application may > >     need in > >     > the long term. > >     > We discussed a little bit of available mechanisms to achieve > that > >     in the > >     > literature. > >     > > >     > 2) fix the largest RTT you want to serve at full utilization > and size > >     > the buffer using BDP * N_backlogged.   > >     > Or the other way round: check how much memory you can use  > >     > in the router/line card/device and for a fixed N, compute > the largest > >     > RTT you can serve at full utilization.  > >     > > >     > 3) there is still some memory to dimension for sparse flows in > >     addition > >     > to that, but this is not based on BDP.  > >     > It is just enough to compute the total utilization of sparse > flows and > >     > use the same simple model Toke has used  > >     > to compute the (de)prioritization probability. > >     > > >     > This procedure would allow to size FQ_codel but also SFQ. > >     > It would be interesting to compare the two under this buffer > sizing.  > >     > It would also be interesting to compare another mechanism > that we have > >     > mentioned during the defense > >     > which is AFD + a sparse flow queue. Which is, BTW, already > >     available in > >     > Cisco nexus switches for data centres. > >     > > >     > I think that the the codel part would still provide the ECN > feature, > >     > that all the others cannot have. > >     > However the others, the last one especially can be > implemented in > >     > silicon with reasonable cost. > > >