General list for discussing Bufferbloat
 help / color / mirror / Atom feed
From: Dave Taht <dave@taht.net>
To: "Bless\, Roland \(TM\)" <roland.bless@kit.edu>
Cc: Luca Muscariello <luca.muscariello@gmail.com>,
	Jonathan Morton <chromatix99@gmail.com>,
	bloat <bloat@lists.bufferbloat.net>
Subject: Re: [Bloat] when does the CoDel part of fq_codel help in the real world?
Date: Wed, 28 Nov 2018 23:39:56 -0800	[thread overview]
Message-ID: <878t1cwcvn.fsf@taht.net> (raw)
In-Reply-To: <7e6cc6f4-bd2f-49b5-0f64-292f56a0592c@kit.edu> (Roland Bless's message of "Tue, 27 Nov 2018 12:40:45 +0100")

"Bless, Roland (TM)" <roland.bless@kit.edu> writes:

> Hi Luca,
>
> Am 27.11.18 um 11:40 schrieb Luca Muscariello:
>> OK. We agree.
>> That's correct, you need *at least* the BDP in flight so that the
>> bottleneck queue never empties out.
>
> No, that's not what I meant, but it's quite simple.
> You need: data min_inflight=2 * RTTmin * bottleneck_rate to filly
> utilize the bottleneck link.
> If this is true, the bottleneck queue will be empty. If your amount
> of inflight data is larger, the bottleneck queue buffer will store
> the excess packets. With just min_inflight there will be no
> bottleneck queue, the packets are "on the wire".
>
>> This can be easily proven using fluid models for any congestion
>> controlled source no matter if it is 
>> loss-based, delay-based, rate-based, formula-based etc.
>> 
>> A highly paced source gives you the ability to get as close as
>> theoretically possible to the BDP+epsilon
>> as possible.
>
> Yep, but that BDP is "on the wire" and epsilon will be in the bottleneck
> buffer.

I'm hoping I made my point effectively earlier, that

" data min_inflight=2 * RTTmin * bottleneck_rate "

when it is nearly certain that more than one flow exists, means aiming
for the BDP in a single flow is generally foolish. Liked the stanford
result, I think it's pretty general. I see hundreds of flows active
every minute. There was another paper that looked into some magic
200-ish number as simultaneous flows active, normally

>
>> link fully utilized is defined as Q>0 unless you don't include the
>> packet currently being transmitted. I do,
>> so the TXtteer is never idle. But that's a detail.
>
> I wouldn't define link fully utilized as Q>0, but if Q>0 then
> the link is fully utilized (that's what I meant by the direction
> of implication).
>
> Rgards,
>  Roland
>
>> 
>> 
>> On Tue, Nov 27, 2018 at 11:35 AM Bless, Roland (TM)
>> <roland.bless@kit.edu <mailto:roland.bless@kit.edu>> wrote:
>> 
>>     Hi,
>> 
>>     Am 27.11.18 um 11:29 schrieb Luca Muscariello:
>>     > I have never said that you need to fill the buffer to the max size to
>>     > get full capacity, which is an absurdity.
>> 
>>     Yes, it's absurd, but that's what today's loss-based CC algorithms do.
>> 
>>     > I said you need at least the BDP so that the queue never empties out.
>>     > The link is fully utilized IFF the queue is never emptied.
>> 
>>     I was also a bit imprecise: you'll need a BDP in flight, but
>>     you don't need to fill the buffer at all. The latter sentence
>>     is valid only in the direction: queue not empty -> link fully utilized.
>> 
>>     Regards,
>>      Roland
>> 
>>     >
>>     >
>>     >
>>     > On Tue 27 Nov 2018 at 11:26, Bless, Roland (TM)
>>     <roland.bless@kit.edu <mailto:roland.bless@kit.edu>
>>     > <mailto:roland.bless@kit.edu <mailto:roland.bless@kit.edu>>> wrote:
>>     >
>>     >     Hi Luca,
>>     >
>>     >     Am 27.11.18 um 10:24 schrieb Luca Muscariello:
>>     >     > A congestion controlled protocol such as TCP or others,
>>     including
>>     >     QUIC,
>>     >     > LEDBAT and so on
>>     >     > need at least the BDP in the transmission queue to get full link
>>     >     > efficiency, i.e. the queue never empties out.
>>     >
>>     >     This is not true. There are congestion control algorithms
>>     >     (e.g., TCP LoLa [1] or BBRv2) that can fully utilize the
>>     bottleneck link
>>     >     capacity without filling the buffer to its maximum capacity.
>>     The BDP
>>     >     rule of thumb basically stems from the older loss-based congestion
>>     >     control variants that profit from the standing queue that they
>>     built
>>     >     over time when they detect a loss:
>>     >     while they back-off and stop sending, the queue keeps the
>>     bottleneck
>>     >     output busy and you'll not see underutilization of the link.
>>     Moreover,
>>     >     once you get good loss de-synchronization, the buffer size
>>     requirement
>>     >     for multiple long-lived flows decreases.
>>     >
>>     >     > This gives rule of thumbs to size buffers which is also very
>>     practical
>>     >     > and thanks to flow isolation becomes very accurate.
>>     >
>>     >     The positive effect of buffers is merely their role to absorb
>>     >     short-term bursts (i.e., mismatch in arrival and departure rates)
>>     >     instead of dropping packets. One does not need a big buffer to
>>     >     fully utilize a link (with perfect knowledge you can keep the link
>>     >     saturated even without a single packet waiting in the buffer).
>>     >     Furthermore, large buffers (e.g., using the BDP rule of thumb)
>>     >     are not useful/practical anymore at very high speed such as
>>     100 Gbit/s:
>>     >     memory is also quite costly at such high speeds...
>>     >
>>     >     Regards,
>>     >      Roland
>>     >
>>     >     [1] M. Hock, F. Neumeister, M. Zitterbart, R. Bless.
>>     >     TCP LoLa: Congestion Control for Low Latencies and High
>>     Throughput.
>>     >     Local Computer Networks (LCN), 2017 IEEE 42nd Conference on, pp.
>>     >     215-218, Singapore, Singapore, October 2017
>>     >     http://doc.tm.kit.edu/2017-LCN-lola-paper-authors-copy.pdf
>>     >
>>     >     > Which is: 
>>     >     >
>>     >     > 1) find a way to keep the number of backlogged flows at a
>>     >     reasonable value. 
>>     >     > This largely depends on the minimum fair rate an application may
>>     >     need in
>>     >     > the long term.
>>     >     > We discussed a little bit of available mechanisms to achieve
>>     that
>>     >     in the
>>     >     > literature.
>>     >     >
>>     >     > 2) fix the largest RTT you want to serve at full utilization
>>     and size
>>     >     > the buffer using BDP * N_backlogged.  
>>     >     > Or the other way round: check how much memory you can use 
>>     >     > in the router/line card/device and for a fixed N, compute
>>     the largest
>>     >     > RTT you can serve at full utilization. 
>>     >     >
>>     >     > 3) there is still some memory to dimension for sparse flows in
>>     >     addition
>>     >     > to that, but this is not based on BDP. 
>>     >     > It is just enough to compute the total utilization of sparse
>>     flows and
>>     >     > use the same simple model Toke has used 
>>     >     > to compute the (de)prioritization probability.
>>     >     >
>>     >     > This procedure would allow to size FQ_codel but also SFQ.
>>     >     > It would be interesting to compare the two under this buffer
>>     sizing. 
>>     >     > It would also be interesting to compare another mechanism
>>     that we have
>>     >     > mentioned during the defense
>>     >     > which is AFD + a sparse flow queue. Which is, BTW, already
>>     >     available in
>>     >     > Cisco nexus switches for data centres.
>>     >     >
>>     >     > I think that the the codel part would still provide the ECN
>>     feature,
>>     >     > that all the others cannot have.
>>     >     > However the others, the last one especially can be
>>     implemented in
>>     >     > silicon with reasonable cost.
>>     >
>> 
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

  parent reply	other threads:[~2018-11-29  7:40 UTC|newest]

Thread overview: 102+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-26 19:08 Pete Heist
2018-11-26 19:28 ` Neal Cardwell
2018-11-27 20:42   ` Dave Taht
2018-11-27 20:54     ` Toke Høiland-Jørgensen
2018-11-27 21:00       ` Dave Taht
2018-11-27 21:05         ` Toke Høiland-Jørgensen
2018-11-26 21:29 ` Jonathan Morton
2018-11-27  9:24   ` Luca Muscariello
2018-11-27 10:26     ` Bless, Roland (TM)
2018-11-27 10:29       ` Luca Muscariello
2018-11-27 10:35         ` Bless, Roland (TM)
2018-11-27 10:40           ` Luca Muscariello
2018-11-27 10:50             ` Mikael Abrahamsson
2018-11-27 11:01               ` Luca Muscariello
2018-11-27 11:21                 ` Mikael Abrahamsson
2018-11-27 12:17                   ` Jonathan Morton
2018-11-27 13:37                     ` Luca Muscariello
2018-11-27 13:49                       ` Mikael Abrahamsson
2018-11-27 14:07                         ` Luca Muscariello
2018-11-27 14:18                           ` Mikael Abrahamsson
2018-11-27 18:44                           ` Kathleen Nichols
2018-11-27 19:25                             ` Dave Taht
2018-11-27 21:57                             ` Roland Bless
2018-11-27 11:53                 ` Bless, Roland (TM)
2018-11-27 11:58                   ` Luca Muscariello
2018-11-27 12:22                     ` Bless, Roland (TM)
2018-11-27 11:06               ` Jonathan Morton
2018-11-27 11:07               ` Michael Welzl
2018-11-29  7:35               ` Dave Taht
2018-11-29 18:43                 ` Stephen Hemminger
2018-11-29 19:08                   ` Dave Taht
2018-11-30  5:51                   ` Mikael Abrahamsson
2018-11-30 19:02                     ` Dave Taht
2018-11-30 19:58                     ` Stephen Hemminger
2018-11-27 11:04             ` Michael Welzl
2018-11-27 12:48               ` Bless, Roland (TM)
2018-11-27 11:40             ` Bless, Roland (TM)
2018-11-27 11:43               ` Bless, Roland (TM)
2018-11-29  7:39               ` Dave Taht [this message]
2018-11-29  7:45                 ` Jonathan Morton
2018-11-29  7:54                   ` Dave Taht
2018-11-29  8:09                     ` Luca Muscariello
2018-11-29 13:49                   ` Bless, Roland (TM)
2018-11-29  8:41                 ` Bless, Roland (TM)
2018-11-29  7:33       ` Dave Taht
2018-11-29  8:13         ` Bless, Roland (TM)
2018-11-29 10:00         ` Pete Heist
2018-11-29 16:09       ` Luca Muscariello
2018-11-29 17:07         ` Mario Hock
2018-11-29 22:30           ` Luca Muscariello
2018-11-30  9:55             ` Mario Hock
2018-11-30 10:32               ` Luca Muscariello
2018-11-30 11:04                 ` Jonathan Morton
2018-11-30 11:53                   ` jf
2018-12-03  9:42                   ` Mario Hock
2018-11-27 11:52     ` Toke Høiland-Jørgensen
2018-11-28  3:37       ` [Bloat] AFD Dave Taht
2018-11-27 20:58     ` [Bloat] when does the CoDel part of fq_codel help in the real world? Dave Taht
2018-11-27 22:19       ` Luca Muscariello
2018-11-27 22:30         ` Roland Bless
2018-11-27 23:17           ` Dave Taht
2018-11-28  3:47             ` Kathleen Nichols
2018-11-28  9:56       ` Luca Muscariello
2018-11-28 10:40         ` Dave Taht
2018-11-28 10:48           ` Luca Muscariello
2018-11-28 12:10             ` Bless, Roland (TM)
2018-11-29  7:22               ` Dave Taht
2018-11-29  7:20             ` Dave Taht
2018-11-27 20:50   ` Dave Taht
2018-11-26 21:56 ` Michael Welzl
2018-11-26 22:13   ` Toke Høiland-Jørgensen
2018-11-27  8:54     ` Pete Heist
2018-11-27  9:31       ` Jonathan Morton
2018-11-27 13:19       ` Michael Richardson
2018-11-27 18:59         ` Jonathan Morton
2018-11-27 20:10   ` Dave Taht
2018-11-27 21:17     ` Michael Welzl
2018-11-27 21:20       ` Michael Welzl
2018-11-29  7:11         ` Dave Taht
2018-11-29  7:28           ` [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?) Mikael Abrahamsson
2018-11-29  7:36             ` Jonathan Morton
2018-11-29  7:46               ` Mikael Abrahamsson
2018-11-29  8:08                 ` Michael Welzl
2018-11-29 10:30                   ` Jonathan Morton
2018-11-29 12:06                     ` Michael Welzl
2018-11-29 12:52                       ` Jonathan Morton
2018-11-30  7:54                         ` Michael Welzl
2019-02-03 18:20                           ` Dave Taht
2018-11-29 12:12                   ` Michael Welzl
2018-11-29 12:56                     ` Jonathan Morton
2018-11-29 13:30                       ` Mikael Abrahamsson
2018-11-29 23:27                         ` Jonathan Morton
2018-11-30  6:01                           ` Mikael Abrahamsson
2018-11-29 14:06                     ` Bless, Roland (TM)
2018-11-29  8:09                 ` Jonathan Morton
2018-11-29  8:19                   ` Mikael Abrahamsson
2018-11-29  8:34                     ` Jonathan Morton
2018-11-29 10:15                 ` Sebastian Moeller
2018-11-29 10:53                   ` Mikael Abrahamsson
2018-11-30  4:54               ` Dave Taht
2018-11-28  2:04     ` [Bloat] when does the CoDel part of fq_codel help in the real world? Pete Heist
2018-11-28  3:52       ` Dave Taht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.bufferbloat.net/postorius/lists/bloat.lists.bufferbloat.net/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=878t1cwcvn.fsf@taht.net \
    --to=dave@taht.net \
    --cc=bloat@lists.bufferbloat.net \
    --cc=chromatix99@gmail.com \
    --cc=luca.muscariello@gmail.com \
    --cc=roland.bless@kit.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox