From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vs1-xe32.google.com (mail-vs1-xe32.google.com [IPv6:2607:f8b0:4864:20::e32]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 6D2063B2A4 for ; Sun, 2 Apr 2023 09:45:30 -0400 (EDT) Received: by mail-vs1-xe32.google.com with SMTP id g17so23099123vst.10 for ; Sun, 02 Apr 2023 06:45:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680443130; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=G25BvI5xb+noCuK+C5y1EpO797MZyGFWKZeiV8UDMr4=; b=Ty/amaNlI7Bolu4V72gbu9OrONCqoDxyKJKz2ww57jzl2OHhXTozupL71chxtjGYpc 1lm03rwPRaH8KiZZWSkEOSzT5QP8Mtsn92WZrF4MgojiryRFkOYsidCFV/in9wlBvv8r iAeg4Dd8MIoJSrYaQUziKeCAS4K46ZU3sSS3LCvOVRKJINm5Nx9DO80TX20kaI7yUtoR Cun1G3ytI7NN5qREtwcERApwf79ZaBTbH5Xtf2o+BIrbkyT9Mex9Cta1WncGuELJuc0T 1UkY2y67v/sHk5f8ofVJTeLcBzZQEFWsFKfCyAMbx8eWZcPLxzKbsrj/imYSJOFJIgkW /Eeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680443130; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=G25BvI5xb+noCuK+C5y1EpO797MZyGFWKZeiV8UDMr4=; b=t+6FRaWCYGdyYOh5ZsjZLqdZK1BijXoWPO5ugnhHKs02j+w+bsd1Nzm0hVPeUOAti2 Qc55FVyxAw1OE+m/hbUD8dLmCbNRZjPqumm68/wU82Pfx1ix85HUWCHEhYKvcCZmAfG1 E2Fdt9XeVl9sZNhHl08yQAauV6sjYsgR2KK+V9I2lIfhpeGJAw67MZe4I8IbGoU67wlE Rfcn3gpvOMfdQ2ESx34giL7Nl5j3gBU7K3LBZDZexUFq9mD7KoiryayrBqQz037ZoxK8 nLkKwvyuOS0oGTKF+h4MHmZTsbhkJDGOViOF48PZRheZHxj589RYiv/SP5Etz/duUNNW Yfaw== X-Gm-Message-State: AAQBX9feh8L5A2HjnJVjXcot5cog6Rl1Z09yjTS0v+fq5XSteTyOFwp2 mmrNiwDgPhxWGAwSvl6PGyRgRTswVK9UABgpiKRsFg== X-Google-Smtp-Source: AKy350ZUWWMucUF6romIz9nuvxJMCzs3PhbgbsDWjXpJHMuBS/uJlCntEsXsTWLUpTCkDC7E9opePF2jPsqu78UINsM= X-Received: by 2002:a67:c31b:0:b0:426:392a:92bd with SMTP id r27-20020a67c31b000000b00426392a92bdmr19021747vsj.7.1680443129629; Sun, 02 Apr 2023 06:45:29 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Neal Cardwell Date: Sun, 2 Apr 2023 09:45:12 -0400 Message-ID: To: Ayush Mishra Cc: Dave Taht , bloat , BBR Development , ayush@comp.nus.edu.sg Content-Type: multipart/alternative; boundary="000000000000cefb0b05f85aa79f" Subject: Re: [Bloat] [bbr-dev] Re: Are we heading towards a BBR-dominant Internet? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Apr 2023 13:45:30 -0000 --000000000000cefb0b05f85aa79f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Mar 28, 2023 at 5:36=E2=80=AFAM Ayush Mishra wrote: > Hey Neal, > > I was revisiting this thread before presenting this paper in iccrg > tomorrow - and I was particularly intrigued by one of the motivations you > mentioned for BBR: > > "BBR is not trying to maintain a higher throughput than CUBIC in these > kinds of scenarios with steady-state bulk flows. BBR is trying to be > robust to the kinds of random packet loss that happen in the real world > when there are flows dynamically entering/leaving a bottleneck." > > BBRv1 essentially tried to deal with this problem by doing away with > packet loss as a congestion signal and having an entirely different > philosophy to congestion control. However, if we set aside the issue of > buffer bloat, I would imagine packet loss is a bad congestion signal in > this situation because most loss-based congestion control algorithms use = it > as a binary signal with a binary response (back-off or no back-off). In > other words, I feel the blame must be placed on not just the congestion > signal, but also on how most algorithms respond to this congestion signal= . > I would even go a little further, and say we don't need to "blame" loss as a congestion signal: usually it's telling us something useful and important= . AFAICT the problem is in the combination of: (a) only using loss as a signal (b) only reacting to whether there is packet loss in a round trip as a signal (c) only using a single multiplicative decrease as a response to loss detected in fast recovery AFAICT any algorithm that has those properties (like Reno and CUBIC) simply can't scale to large BDPs if there are typical levels of loss or the traffic or available bandwidth is dynamic. At large BDPs and typically achievable loss rates, there will be packet loss in every round trip and the connection will always be decreasing rather than increasing, so will starve. For example, with a BDP of 10 Gbps * 100ms and MTU of 1500 bytes and loss rate of 0.0012% we'd expect a packet loss every round trip, and so we would expect starvation. In particular, a single CUBIC flow over such a path needs >40 secs between experiencing any losses, or a loss rate less than 0.0000029% (2.9e-8) [ https://tools.ietf.org/html/rfc8312#section-5.2 ]. > On a per-packet basis, packet loss is a binary signal. But over a window, > the loss percentage and distribution, for example, can be a rich signal. > There is probably scope for differentiating between different kinds of > packet losses (and deciding how to react to them) when packet loss is > coupled with the most recent delay measurement too. Now that BBRv2 reacts > to packet loss, are you making any of these considerations too? > Yes, I agree there is useful information there, and BBRv2 does look explicitly and indirectly at the loss rate when making decisions. BBRv2 does not look at coupling the loss signal with the most recent delay measurement, but I agree that seems like a fruitful direction, and we have been considering that as a component of future CC algorithms. > This is not something I plan to present in iccrg tomorrow, just something > I was curious about :) > Thanks for posting! I agree these are interesting topics. :-) best regards, neal > Warmest regards, > Ayush > > On Fri, Aug 26, 2022 at 9:36=E2=80=AFPM 'Neal Cardwell' via BBR Developme= nt < > bbr-dev@googlegroups.com> wrote: > >> Yes, I agree the assumptions are key here. One key aspect of this paper >> is that it focuses on the steady-state behavior of bulk flows. >> >> Once you allow for short flows (like web pages, RPCs, etc) to dynamicall= y >> enter and leave a bottleneck, the considerations become different. As is >> well-known, Reno/CUBIC will starve themselves if new flows enter and cau= se >> loss too frequently. For CUBIC, for a somewhat typical 30ms broadband pa= th >> with a flow fair share of 25 Mbit/sec, if new flows enter and cause loss >> more frequently than roughly every 2 seconds then CUBIC will not be able= to >> utilize its fair share. For a high-speed WAN path, with 100ms RTT and fa= ir >> share of 10 Gbit/sec, if new flows enter and cause loss more frequently >> than roughly every 40 seconds then CUBIC will not be able to utilize its >> fair share. Basically, loss-based CC can starve itself in some >> very typical kinds of dynamic scenarios that happen in the real world. >> >> BBR is not trying to maintain a higher throughput than CUBIC in these >> kinds of scenarios with steady-state bulk flows. BBR is trying to be rob= ust >> to the kinds of random packet loss that happen in the real world when th= ere >> are flows dynamically entering/leaving a bottleneck. >> >> cheers, >> neal >> >> >> >> >> On Thu, Aug 25, 2022 at 8:01 PM Dave Taht via Bloat < >> bloat@lists.bufferbloat.net> wrote: >> >>> I rather enjoyed this one. I can't help but wonder what would happen >>> if we plugged some different assumptions into their model. >>> >>> https://www.comp.nus.edu.sg/~bleong/publications/imc2022-nash.pdf >>> >>> -- >>> FQ World Domination pending: >>> https://blog.cerowrt.org/post/state_of_fq_codel/ >>> Dave T=C3=A4ht CEO, TekLibre, LLC >>> _______________________________________________ >>> Bloat mailing list >>> Bloat@lists.bufferbloat.net >>> https://lists.bufferbloat.net/listinfo/bloat >>> >> -- >> You received this message because you are subscribed to the Google Group= s >> "BBR Development" group. >> To unsubscribe from this group and stop receiving emails from it, send a= n >> email to bbr-dev+unsubscribe@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/bbr-dev/CADVnQykKbnxpNcpuZATug_4VLhV1%= 3DaoTTQE2263o8HF9ye_TQg%40mail.gmail.com >> >> . >> > --000000000000cefb0b05f85aa79f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Tue, Mar 28, 2023 at 5:36=E2=80=AF= AM Ayush Mishra <ayumishra.95@= gmail.com> wrote:
Hey Neal,

I was r= evisiting this thread before presenting this paper in iccrg tomorrow - and = I was particularly intrigued by one of the motivations you mentioned for BB= R:

"BBR is not trying to maintai= n a higher throughput than CUBIC in these kinds of scenarios with steady-st= ate bulk flows. BBR is trying to be robust to the kinds of random packet loss that happen=20 in the real world when there are flows dynamically entering/leaving a=20 bottleneck."

BBRv1 essentially tried to deal = with this problem by doing away with packet loss as a congestion signal and= having an entirely different philosophy to congestion control. However, if= we set aside the issue of buffer bloat, I would imagine packet loss is a b= ad congestion signal in this situation because most loss-based congestion c= ontrol algorithms use it as a binary signal with a binary response (back-of= f or no back-off). In other words, I feel the blame must be placed on not j= ust the congestion signal, but also on how most algorithms respond to this = congestion signal.

I would even= go a little further, and say we don't need to "blame" loss a= s a congestion signal: usually it's telling us something useful and imp= ortant.

AFAICT the=C2=A0problem is in the combinat= ion of:
=C2=A0(a) only using loss as a signal
=C2=A0(b)= only reacting to whether there is packet loss in a round trip as a signal<= /div>
=C2=A0(c) only using a single multiplicative decrease as a respon= se to loss detected in fast recovery

AFAICT any al= gorithm that has those properties (like Reno and CUBIC) simply can't sc= ale to large BDPs if there are=C2=A0typical levels of loss or the traffic o= r available bandwidth is dynamic. At large BDPs and typically achievable lo= ss rates, there will be packet loss in every round trip and the connection = will always be decreasing rather than increasing, so will starve. For examp= le, with a BDP of 10 Gbps * 100ms and MTU of 1500 bytes and loss rate of=C2= =A0 0.0012% we'd expect a packet loss every round trip, and so we would= expect starvation. In particular, a single CUBIC flow over such a path nee= ds=C2=A0>40 secs between experiencing any losses, or a loss rate less th= an 0.0000029%=C2=A0 (2.9e-8) [ https://tools.ietf.org/html/rfc8312#section-5.2 ].
=C2=A0
On a per-packet basis, packet loss is a binary signal. But o= ver a window, the loss percentage and distribution, for example, can be a r= ich signal. There is probably scope for differentiating between different k= inds of packet losses (and deciding how to react to them) when packet loss = is coupled with the most recent delay measurement too. Now that BBRv2 react= s to packet loss, are you making any of these considerations too?

Yes, I agree there is=C2=A0useful inform= ation there, and BBRv2 does look explicitly and indirectly at the loss rate= when making decisions. BBRv2 does not look at coupling the loss signal wit= h the most recent delay measurement, but I agree that seems like a fruitful= direction, and we have been considering that as a component of future CC a= lgorithms.
=C2=A0
This is not something I plan to present in ic= crg tomorrow, just something I was curious about :)

Thanks for posting! I agree these are interesting topi= cs. :-)

best regards,
neal
=C2=A0
<= div dir=3D"ltr">
Warmest regards,
Ayush

On Fri, Aug = 26, 2022 at 9:36=E2=80=AFPM 'Neal Cardwell' via BBR Development <= ;bbr-dev@goog= legroups.com> wrote:
Yes, I agree the assumptions are key here. One= key aspect of this paper is that it focuses on the steady-state behavior o= f bulk flows.

Once you allow for short flows (like web p= ages, RPCs, etc) to dynamically enter and leave a bottleneck, the considera= tions become different. As is well-known, Reno/CUBIC will starve themselves= if new flows enter and cause loss too frequently. For CUBIC, for a somewha= t typical 30ms broadband path with a flow fair share of 25 Mbit/sec, if new= flows enter and cause loss more frequently than roughly every 2 seconds th= en CUBIC will not be able to utilize its fair share. For a high-speed WAN p= ath, with 100ms RTT and fair share of 10 Gbit/sec,=C2=A0 if new flows enter= and cause loss more frequently than roughly every 40 seconds then CUBIC wi= ll not be able to utilize its fair share. Basically, loss-based CC can star= ve itself in some very=C2=A0typical=C2=A0kinds of dynamic scenarios that ha= ppen in the real world.

BBR is not trying to maint= ain a higher throughput than CUBIC in these kinds of scenarios with steady-= state bulk flows. BBR is trying to be robust to the kinds of random packet = loss that happen in the real world when there are flows dynamically enterin= g/leaving a bottleneck.

cheers,
neal




On Thu, Aug 25, 2022 at 8:01= PM Dave Taht via Bloat <bloat@lists.bufferbloat.net> wrote:
I rather enjoyed this one. I c= an't help but wonder what would happen
if we plugged some different assumptions into their model.

https://www.comp.nus.edu.sg/~bleong= /publications/imc2022-nash.pdf

--
FQ World Domination pending: https://blog.cerowrt.or= g/post/state_of_fq_codel/
Dave T=C3=A4ht CEO, TekLibre, LLC
_______________________________________________
Bloat mailing list
Bloat@list= s.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

--
You received this message because you are subscribed to the Google Groups &= quot;BBR Development" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to bbr-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/CADVnQykKbnxpNcpuZATug_4VLhV1%3= DaoTTQE2263o8HF9ye_TQg%40mail.gmail.com.
--000000000000cefb0b05f85aa79f--