From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from iramx2.ira.uni-karlsruhe.de (iramx2.ira.uni-karlsruhe.de [141.3.10.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id B79973CB35 for ; Tue, 27 Nov 2018 06:43:50 -0500 (EST) Received: from i72vorta.tm.uni-karlsruhe.de ([141.3.71.26] helo=i72vorta.tm.kit.edu) by iramx2.ira.uni-karlsruhe.de with esmtp port 25 iface 141.3.10.81 id 1gRbmQ-0005p9-0e; Tue, 27 Nov 2018 12:43:50 +0100 Received: from [IPv6:::1] (ip6-localhost [IPv6:::1]) by i72vorta.tm.kit.edu (Postfix) with ESMTPS id EFB04420DD3; Tue, 27 Nov 2018 12:43:49 +0100 (CET) From: "Bless, Roland (TM)" To: Luca Muscariello Cc: Jonathan Morton , bloat References: <65EAC6C1-4688-46B6-A575-A6C7F2C066C5@heistp.net> <86b16a95-e47d-896b-9d43-69c65c52afc7@kit.edu> <7e6cc6f4-bd2f-49b5-0f64-292f56a0592c@kit.edu> Openpgp: preference=signencrypt Autocrypt: addr=roland.bless@kit.edu; prefer-encrypt=mutual; keydata= xsFNBFi0OxABEACy2VohJ7VhSu/xPCt4/6qCrw4Pw2nSklWPfAYEk1QgrbiwgvLAP9WEhAIU w45cojBaDxytIGg8eaYeIKSmsXjHGbV/ZTfo8r11LX8yPYR0WHiMWZpl0SHUd/CZIkv2pChO 88vF/2FKN95HDcp24pwONF4VhxJoSFk6c0mDNf8Em/Glt9BcWX2AAvizTmpQDshaPje18WH3 4++KwPZDd/sJ/hHSXiPg1Gdhs/OG/C0CJguOAlqbgSVAe3qKOr1M4K5M+wVpsk373pXRfxd7 ZAmZ05iBTn+LfgVcz+AfaKKcsWri5CdTT+7JDL6QNQpox+b5FXZFSHnEIST+/qzfG7G2LqqY mml6TYY8XbaNyXZP0QKncfSpRx8uTRWReHUa1YbSuOxXYh6bXpcugD25mlC/Lu0g7tz4ijiK iIwq9+P2H1KfAAfYyYZh6nOoE6ET0TjOjUSa+mA8cqjPWX99kEEgf1Xo+P9fx9QLCLWIY7zc mSM+vjQKgdUFpMSCKcYEKOuwlPuOz8bVECafxaEtJJHjCOK8zowe2eC9OM+G+bmtAO3qYcYZ hQ/PV3sztt/PjgdtnFAYPFLc9189rHRxKsWSOb4xPkRw/YQAI9l15OlUEpsyOehxmAmTsesn tSViCz++PCdeXrQc1BCgl8nDytrxW+n5w1aaE8aL3hn8M0tonQARAQABzShSb2xhbmQgQmxl c3MgKFRNKSA8cm9sYW5kLmJsZXNzQGtpdC5lZHU+wsGABBMBCAAqAhsDBQkSzAMABQsJCAcC BhUICQoLAgQWAgMBAh4BAheABQJYtYdHAhkBAAoJEKON2tlkOJXuzWkP+wfjUnDNzRm4r34a AMWepcQziTgqf4I1crcL6VD44767HhyFsjcKH31E5G5gTDxbpsM4pmkghKeLrpPo30YK3qb7 E9ifIkpJTvMu0StSUmcXq0zPyHZ+HxHeMWkosljG3g/4YekCqgWwrB62T7NMYq0ATQe1MGCZ TAPwSPGCUZT3ioq50800FMI8okkGTXS3h2U922em7k8rv7E349uydv19YEcS7tI78pggMdap ASoP3QWB03tzPKwjqQqSevy64uKDEa0UgvAM3PRbJxOYZlX1c3q/CdWwpwgUiAhMtPWvavWW Tcw6Kkk6e0gw4oFlDQ+hZooLv5rlYR3egdV4DPZ1ugL51u0wQCQG9qKIMXslAdmKbRDkEcWG Oi2bWAdYyIHhhQF5LSuaaxC2P2vOYRHnE5yv5KTV3V7piFgPFjKDW+giCRd7VGfod6DY2b2y zwidCMve1Qsm8+NErH6U+hMpMLeCJDMu1OOvXYbFnTkqjeg5sKipUoSdgXsIo4kl+oArZlpK qComSTPhij7rMyeu/1iOwbNCjtiqgb55ZE7Ekd84mr9sbq4Jm/4QGnVI30q4U2vdGSeNbVjo d1nqjf3UNzP2ZC+H9xjsCFuKYbCX6Yy4SSuEcubtdmdBqm13pxua4ZqPSI0DQST2CHC7nxL1 AaRGRYYh5zo2vRg3ipkEzsFNBFi0OxABEAC2CJNp0/Ivkv4KOiXxitsMXZeK9fI0NU2JU1rW 04dMLF63JF8AFiJ6qeSL2mPHoMiL+fG5jlxy050xMdpMKxnhDVdMxwPtMiGxbByfvrXu18/M B7h+E1DHYVRdFFPaL2jiw+Bvn6wTT31MiuG9Wh0WAhoW8jY8IXxKQrUn7QUOKsWhzNlvVpOo SjMiW4WXksUA0EQVbmlskS/MnFOgCr8q/FqwC81KPy+VLHPB9K/B65uQdpaw78fjAgQVQqpx H7gUF1EYpdZWyojN+V8HtLJx+9yWAZjSFO593OF3/r0nDHEycuOjhefCrqr0DDgTYUNthOdU KO2CzT7MtweRtAf0n27zbwoYvkTviIbR+1lV1vNkxaUtZ6e1rtOxvonRM1O3ddFIzRp/Qufu HfPe0YqhEsrBIGW1aE/pZW8khNQlB6qt20snL9cFDrnB6+8kDG3e//OjK1ICQj9Y/yyrJVaX KfPbdHhLpsgh8TMDPoH+XXQlDJljMD0++/o7ckO3Sfa8Zsyh1WabyKQDYXDmDgi9lCoaQ7Lf uLUpoMvJV+EWo0jE4RW/wBGQbLJp5usy5i0fhBKuDwsKdLG3qOCf4depIcNuja6ZmZHRT+3R FFjvZ/dAhrCWpRTxZANlWlLZz6htToJulAZQJD6lcpVr7EVgDX/y4cNwKF79egWXPDPOvQAR AQABwsFlBBgBCAAPBQJYtDsQAhsMBQkSzAMAAAoJEKON2tlkOJXukMoP/jNeiglj8fenH2We 7SJuyBp8+5L3n8eNwfwY5C5G+etD0E6/lkt/Jj9UddTazxeB154rVFXRzmcN3+hGCOZgGAyV 1N7d8xM6dBqRtHmRMPu5fUxfSqrM9pmqAw2gmzAe0eztVvaM+x5x5xID2WZOiOq8dx9KOKrp Zorekjs3GEA3V1wlZ7Nksx/o8KZ04hLeKcR1r06zEDLN/yA+Fz8IPa0KqpuhrL010bQDgAhe 9o5TA0/cMJpxpLqHhX2As+5cQAhKDDsWJu3oBzZRkN7Hh/HTpWurmTQRRniLGSeiL0zdtilX fowyxGXH6QWi3MZYmpOq+etr7o4EGGbm2inxpVbM+NYmaJs+MAi/z5bsO/rABwdM5ysm8hwb CGt+1oEMORyMcUk/uRjclgTZM1NhGoXm1Un67+Rehu04i7DA6b8dd1H8AFgZSO2H4IKi+5yA Ldmo+ftCJS83Nf6Wi6hJnKG9aWQjKL+qmZqBEct/D2uRJGWAERU5+D0RwNV/i9lQFCYNjG9X Tew0BPYYnBtHFlz9rJTqGhDu4ubulSkbxAK3TIk8XzKdMvef3tV/7mJCmcaVbJ2YoNUtkdKJ goOigJTMBXMRu4Ibyq1Ei+d90lxhojKKlf9yguzpxk5KYFGUizp0dtvdNuXRBtYrwzykS6vB zTlLqHZ0pvGjNfTSvuuN Organization: Institute of Telematics, Karlsruhe Institute of Technology Message-ID: Date: Tue, 27 Nov 2018 12:43:49 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <7e6cc6f4-bd2f-49b5-0f64-292f56a0592c@kit.edu> Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 8bit X-ATIS-AV: ClamAV (iramx2.ira.uni-karlsruhe.de) X-ATIS-Timestamp: iramx2.ira.uni-karlsruhe.de 1543319030.084984768 Subject: Re: [Bloat] when does the CoDel part of fq_codel help in the real world? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Nov 2018 11:43:50 -0000 Hi, Am 27.11.18 um 12:40 schrieb Bless, Roland (TM): > Hi Luca, > > Am 27.11.18 um 11:40 schrieb Luca Muscariello: >> OK. We agree. >> That's correct, you need *at least* the BDP in flight so that the >> bottleneck queue never empties out. > > No, that's not what I meant, but it's quite simple. > You need: data min_inflight=2 * RTTmin * bottleneck_rate to filly Sorry, it's meant to be: min_inflight= RTTmin * bottleneck_rate Regards, Roland > utilize the bottleneck link. > If this is true, the bottleneck queue will be empty. If your amount > of inflight data is larger, the bottleneck queue buffer will store > the excess packets. With just min_inflight there will be no > bottleneck queue, the packets are "on the wire". > >> This can be easily proven using fluid models for any congestion >> controlled source no matter if it is  >> loss-based, delay-based, rate-based, formula-based etc. >> >> A highly paced source gives you the ability to get as close as >> theoretically possible to the BDP+epsilon >> as possible. > > Yep, but that BDP is "on the wire" and epsilon will be in the bottleneck > buffer. > >> link fully utilized is defined as Q>0 unless you don't include the >> packet currently being transmitted. I do, >> so the TXtteer is never idle. But that's a detail. > > I wouldn't define link fully utilized as Q>0, but if Q>0 then > the link is fully utilized (that's what I meant by the direction > of implication). > > Rgards, > Roland > >> >> >> On Tue, Nov 27, 2018 at 11:35 AM Bless, Roland (TM) >> > wrote: >> >> Hi, >> >> Am 27.11.18 um 11:29 schrieb Luca Muscariello: >> > I have never said that you need to fill the buffer to the max size to >> > get full capacity, which is an absurdity. >> >> Yes, it's absurd, but that's what today's loss-based CC algorithms do. >> >> > I said you need at least the BDP so that the queue never empties out. >> > The link is fully utilized IFF the queue is never emptied. >> >> I was also a bit imprecise: you'll need a BDP in flight, but >> you don't need to fill the buffer at all. The latter sentence >> is valid only in the direction: queue not empty -> link fully utilized. >> >> Regards, >>  Roland >> >> > >> > >> > >> > On Tue 27 Nov 2018 at 11:26, Bless, Roland (TM) >> >> > >> wrote: >> > >> >     Hi Luca, >> > >> >     Am 27.11.18 um 10:24 schrieb Luca Muscariello: >> >     > A congestion controlled protocol such as TCP or others, >> including >> >     QUIC, >> >     > LEDBAT and so on >> >     > need at least the BDP in the transmission queue to get full link >> >     > efficiency, i.e. the queue never empties out. >> > >> >     This is not true. There are congestion control algorithms >> >     (e.g., TCP LoLa [1] or BBRv2) that can fully utilize the >> bottleneck link >> >     capacity without filling the buffer to its maximum capacity. >> The BDP >> >     rule of thumb basically stems from the older loss-based congestion >> >     control variants that profit from the standing queue that they >> built >> >     over time when they detect a loss: >> >     while they back-off and stop sending, the queue keeps the >> bottleneck >> >     output busy and you'll not see underutilization of the link. >> Moreover, >> >     once you get good loss de-synchronization, the buffer size >> requirement >> >     for multiple long-lived flows decreases. >> > >> >     > This gives rule of thumbs to size buffers which is also very >> practical >> >     > and thanks to flow isolation becomes very accurate. >> > >> >     The positive effect of buffers is merely their role to absorb >> >     short-term bursts (i.e., mismatch in arrival and departure rates) >> >     instead of dropping packets. One does not need a big buffer to >> >     fully utilize a link (with perfect knowledge you can keep the link >> >     saturated even without a single packet waiting in the buffer). >> >     Furthermore, large buffers (e.g., using the BDP rule of thumb) >> >     are not useful/practical anymore at very high speed such as >> 100 Gbit/s: >> >     memory is also quite costly at such high speeds... >> > >> >     Regards, >> >      Roland >> > >> >     [1] M. Hock, F. Neumeister, M. Zitterbart, R. Bless. >> >     TCP LoLa: Congestion Control for Low Latencies and High >> Throughput. >> >     Local Computer Networks (LCN), 2017 IEEE 42nd Conference on, pp. >> >     215-218, Singapore, Singapore, October 2017 >> >     http://doc.tm.kit.edu/2017-LCN-lola-paper-authors-copy.pdf >> > >> >     > Which is:  >> >     > >> >     > 1) find a way to keep the number of backlogged flows at a >> >     reasonable value.  >> >     > This largely depends on the minimum fair rate an application may >> >     need in >> >     > the long term. >> >     > We discussed a little bit of available mechanisms to achieve >> that >> >     in the >> >     > literature. >> >     > >> >     > 2) fix the largest RTT you want to serve at full utilization >> and size >> >     > the buffer using BDP * N_backlogged.   >> >     > Or the other way round: check how much memory you can use  >> >     > in the router/line card/device and for a fixed N, compute >> the largest >> >     > RTT you can serve at full utilization.  >> >     > >> >     > 3) there is still some memory to dimension for sparse flows in >> >     addition >> >     > to that, but this is not based on BDP.  >> >     > It is just enough to compute the total utilization of sparse >> flows and >> >     > use the same simple model Toke has used  >> >     > to compute the (de)prioritization probability. >> >     > >> >     > This procedure would allow to size FQ_codel but also SFQ. >> >     > It would be interesting to compare the two under this buffer >> sizing.  >> >     > It would also be interesting to compare another mechanism >> that we have >> >     > mentioned during the defense >> >     > which is AFD + a sparse flow queue. Which is, BTW, already >> >     available in >> >     > Cisco nexus switches for data centres. >> >     > >> >     > I think that the the codel part would still provide the ECN >> feature, >> >     > that all the others cannot have. >> >     > However the others, the last one especially can be >> implemented in >> >     > silicon with reasonable cost. >> > >> > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat >