From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id D921B3B29E for ; Mon, 29 Aug 2022 12:47:49 -0400 (EDT) Received: by mail-ed1-x52b.google.com with SMTP id z41so3502682ede.0 for ; Mon, 29 Aug 2022 09:47:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=C/P8ibK98dF1A0NZ6Kg5iGtS/dS8Gy3K4RGpo6SLxic=; b=B3JKxGhuOAnemGWmYEMF+fICbdq2R1P6DFkGih0EaxzPR2d5HWc0/dAXRkvij1Bd/6 2lh7MyHpf+8YzKofymSwpgngp8kt1YWHK2lCwh6IZCJOCf2Xx89iFCeicDfXkqVtoVIz dAPxj51x6D3YvAbwV64xic2/l94L2rg47BdBk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=C/P8ibK98dF1A0NZ6Kg5iGtS/dS8Gy3K4RGpo6SLxic=; b=7enTmUaVO0FUQR6/NgMe+01r9ElBFtZnqz+RmbG9yDxXm3yNTAQpepPBqTCTzJYxaI Z3vfqzL4P4zitZ4y1rtkOl4LIy1ZN9LnFewf+b55vg88MtQ6Tbmm2Di7WmOraeHHtywO BW+Rk3QhtRnaT4SkggupRUXnM8C6Ct4nDX+bsR8g2K8svtkE3iNWXIuUPukb/g51YsoC Uh0sqiYmQtW4k1IphflfDAvIXHFlnre06LLFzulUf3VSq0cXAE1hANKAIhhkfHKvAPzQ RqPq8tS5ta+RCmhFVZhSOjtyaFz+WW6A87PCgu0JixBZMh4BAtmd6mtw5UM5gY0eh/DH owrA== X-Gm-Message-State: ACgBeo3sJbgp2TW/7zSk4SSw/8b5SlG5d6fLqArgjdVpQNbKS9gFZjgF Kz1q3X1rJ34bJxMoNajby3ugbeEWMGxdxUrhLk7IqLctbtYLB+WdGMOsQc1FctnmQuda1xObEPA inADEM9xImWqexd49xPaGn3VEHjWPsQ== X-Google-Smtp-Source: AA6agR7SCz2vWrhH/fZ7DQdlui5GQmtKWY+ilKCblp2RWqgYCUteZ5gMD5oBk6vtPi62qqpMZiWGajuRrYSzojqKT6g= X-Received: by 2002:a05:6402:22d0:b0:447:9c9a:f6b9 with SMTP id dm16-20020a05640222d000b004479c9af6b9mr17717739edb.296.1661791668555; Mon, 29 Aug 2022 09:47:48 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Bob McMahon Date: Mon, 29 Aug 2022 09:47:37 -0700 Message-ID: To: Neal Cardwell Cc: Dave Taht , bloat , BBR Development , ayush@comp.nus.edu.sg Content-Type: multipart/alternative; boundary="00000000000018d8e505e76406bf" Subject: Re: [Bloat] [bbr-dev] Re: Are we heading towards a BBR-dominant Internet? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Aug 2022 16:47:50 -0000 --00000000000018d8e505e76406bf Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks Neal. You might want to check out the flows released as iperf 2. Basically instantiate flows and run them. There typically is a controller running python3 (v 3.10 or better) that uses ssh pipes to DUTs. The design is event driven and utilizes python's asyncio which is quite powerful. The DUTs just need iperf2 and ssh. The code is at an alpha level and we're looking for broader industry support and contributions. Both in realtime plotting but also in things like multivariate regression detection using statistical process controls (SPC) e.g. Hoteling . There is some crude clustering code around latency too which uses Kolmogorov-Smirnov distance matrices per the histograms. A suggestion is that those in developer and control test roles synchronize their device clocks with PTP. Iperf 2 supports one way delay (OWD) calculations but these only work if the clocks are sync'd. These in turn can be used per Little's law to calculate effective average queue depth, though this typically assumes a steady state measurement. Bob On Sun, Aug 28, 2022 at 4:54 PM Neal Cardwell wrote: > If you are talking about the screenshot of the UI at > https://github.com/google/transperf, yes, that particular test is a > simple bulk flow test to show a simple case to give a sense of what the U= I > looks like. :-) > > We use a few different approaches that can examine dynamic flows causing > packet loss: > > (1) The test configuration language is Python, so you can construct > arbitrarily fancy dynamic flow scenarios with arbitrary numbers of flows > starting and stopping at arbitrary times. > > (2) The tests can also use netperf command line options to run periodic > short transfers. (And we welcome patches to integrate support for other > tools.) > > (3) We also run a fair number of tests for robustness to loss just using > randomly injected packet loss (using netem). > > These are just some of the approaches we have used, and I don't claim tha= t > these are the only or best approaches to look at this. :-) > > cheers, > neal > > > On Sun, Aug 28, 2022 at 6:39 PM Bob McMahon > wrote: > >> Hi Neal, >> >> These look like steady-state bulk flow tests unless I'm missing somethin= g. >> >> Bob >> >> On Sun, Aug 28, 2022 at 11:43 AM Neal Cardwell >> wrote: >> >>> Sure. For testing these kinds of properties of the BBR algorithm we use >>> various transperf test cases. The transperf tool is something Soheil Ha= ssas >>> Yeganeh and our team cooked up and open-sourced here: >>> >>> https://github.com/google/transperf >>> >>> Best regards, >>> neal >>> >>> On Sat, Aug 27, 2022, 4:43 PM Bob McMahon >>> wrote: >>> >>>> Curious to what you're doing during development, if you can share? >>>> >>>> Thanks, >>>> Bob >>>> >>>> On Sat, Aug 27, 2022 at 7:44 AM Neal Cardwell >>>> wrote: >>>> >>>>> Hi Bob, >>>>> >>>>> Good question. I can imagine a number of different techniques to >>>>> generate and measure the traffic flows for this kind of study, and do= n't >>>>> have any particular suggestions. >>>>> >>>>> neal >>>>> >>>>> >>>>> On Fri, Aug 26, 2022 at 4:54 PM Bob McMahon >>>>> wrote: >>>>> >>>>>> Hi Neal, >>>>>> >>>>>> Any thoughts on tooling to generate and measure the traffic flows BB= R >>>>>> is designed to optimize? I've been adding some low duty cycle suppor= t in iperf >>>>>> 2 with things like --boun= ceback >>>>>> and --burst-period and --burst-period >>>>>> . We could pull >>>>>> the size and period from a known distribution or distributions thoug= h not >>>>>> sure what to pick. >>>>>> >>>>>> Thanks, >>>>>> Bob >>>>>> >>>>>> Bob >>>>>> >>>>>> On Fri, Aug 26, 2022 at 6:36 AM 'Neal Cardwell' via BBR Development = < >>>>>> bbr-dev@googlegroups.com> wrote: >>>>>> >>>>>>> Yes, I agree the assumptions are key here. One key aspect of this >>>>>>> paper is that it focuses on the steady-state behavior of bulk flows= . >>>>>>> >>>>>>> Once you allow for short flows (like web pages, RPCs, etc) to >>>>>>> dynamically enter and leave a bottleneck, the considerations become >>>>>>> different. As is well-known, Reno/CUBIC will starve themselves if n= ew flows >>>>>>> enter and cause loss too frequently. For CUBIC, for a somewhat typi= cal 30ms >>>>>>> broadband path with a flow fair share of 25 Mbit/sec, if new flows = enter >>>>>>> and cause loss more frequently than roughly every 2 seconds then CU= BIC will >>>>>>> not be able to utilize its fair share. For a high-speed WAN path, w= ith >>>>>>> 100ms RTT and fair share of 10 Gbit/sec, if new flows enter and ca= use loss >>>>>>> more frequently than roughly every 40 seconds then CUBIC will not b= e able >>>>>>> to utilize its fair share. Basically, loss-based CC can starve itse= lf in >>>>>>> some very typical kinds of dynamic scenarios that happen in the rea= l world. >>>>>>> >>>>>>> BBR is not trying to maintain a higher throughput than CUBIC in >>>>>>> these kinds of scenarios with steady-state bulk flows. BBR is tryin= g to be >>>>>>> robust to the kinds of random packet loss that happen in the real w= orld >>>>>>> when there are flows dynamically entering/leaving a bottleneck. >>>>>>> >>>>>>> cheers, >>>>>>> neal >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Thu, Aug 25, 2022 at 8:01 PM Dave Taht via Bloat < >>>>>>> bloat@lists.bufferbloat.net> wrote: >>>>>>> >>>>>>>> I rather enjoyed this one. I can't help but wonder what would happ= en >>>>>>>> if we plugged some different assumptions into their model. >>>>>>>> >>>>>>>> https://www.comp.nus.edu.sg/~bleong/publications/imc2022-nash.pdf >>>>>>>> >>>>>>>> -- >>>>>>>> FQ World Domination pending: >>>>>>>> https://blog.cerowrt.org/post/state_of_fq_codel/ >>>>>>>> Dave T=C3=A4ht CEO, TekLibre, LLC >>>>>>>> _______________________________________________ >>>>>>>> Bloat mailing list >>>>>>>> Bloat@lists.bufferbloat.net >>>>>>>> https://lists.bufferbloat.net/listinfo/bloat >>>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "BBR Development" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to bbr-dev+unsubscribe@googlegroups.com. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/bbr-dev/CADVnQykKbnxpNcpuZATug_4V= LhV1%3DaoTTQE2263o8HF9ye_TQg%40mail.gmail.com >>>>>>> >>>>>>> . >>>>>>> >>>>>> >>>>>> This electronic communication and the information and any files >>>>>> transmitted with it, or attached to it, are confidential and are int= ended >>>>>> solely for the use of the individual or entity to whom it is address= ed and >>>>>> may contain information that is confidential, legally privileged, pr= otected >>>>>> by privacy laws, or otherwise restricted from disclosure to anyone e= lse. If >>>>>> you are not the intended recipient or the person responsible for del= ivering >>>>>> the e-mail to the intended recipient, you are hereby notified that a= ny use, >>>>>> copying, distributing, dissemination, forwarding, printing, or copyi= ng of >>>>>> this e-mail is strictly prohibited. If you received this e-mail in e= rror, >>>>>> please return the e-mail to the sender, delete it from your computer= , and >>>>>> destroy any printed copy of it. >>>>> >>>>> >>>> This electronic communication and the information and any files >>>> transmitted with it, or attached to it, are confidential and are inten= ded >>>> solely for the use of the individual or entity to whom it is addressed= and >>>> may contain information that is confidential, legally privileged, prot= ected >>>> by privacy laws, or otherwise restricted from disclosure to anyone els= e. If >>>> you are not the intended recipient or the person responsible for deliv= ering >>>> the e-mail to the intended recipient, you are hereby notified that any= use, >>>> copying, distributing, dissemination, forwarding, printing, or copying= of >>>> this e-mail is strictly prohibited. If you received this e-mail in err= or, >>>> please return the e-mail to the sender, delete it from your computer, = and >>>> destroy any printed copy of it. >>> >>> >> This electronic communication and the information and any files >> transmitted with it, or attached to it, are confidential and are intende= d >> solely for the use of the individual or entity to whom it is addressed a= nd >> may contain information that is confidential, legally privileged, protec= ted >> by privacy laws, or otherwise restricted from disclosure to anyone else.= If >> you are not the intended recipient or the person responsible for deliver= ing >> the e-mail to the intended recipient, you are hereby notified that any u= se, >> copying, distributing, dissemination, forwarding, printing, or copying o= f >> this e-mail is strictly prohibited. If you received this e-mail in error= , >> please return the e-mail to the sender, delete it from your computer, an= d >> destroy any printed copy of it. > > --=20 This electronic communication and the information and any files transmitted= =20 with it, or attached to it, are confidential and are intended solely for=20 the use of the individual or entity to whom it is addressed and may contain= =20 information that is confidential, legally privileged, protected by privacy= =20 laws, or otherwise restricted from disclosure to anyone else. If you are=20 not the intended recipient or the person responsible for delivering the=20 e-mail to the intended recipient, you are hereby notified that any use,=20 copying, distributing, dissemination, forwarding, printing, or copying of= =20 this e-mail is strictly prohibited. If you received this e-mail in error,= =20 please return the e-mail to the sender, delete it from your computer, and= =20 destroy any printed copy of it. --00000000000018d8e505e76406bf Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks Neal. You might want to check out the flows released a= s iperf 2. Basically instantiate flows and run them. There typically is= a controller running python3 (v 3.10 or better) that uses ssh pipes to DUT= s. The design is event driven and utilizes python's asyncio which is quite powerful. Th= e DUTs just need iperf2 and ssh.

The code is at an alpha level and w= e're looking for broader industry support and contributions. Both in re= altime=C2=A0plotting but also in things like multivariate regression detect= ion using statistical=C2=A0process controls (SPC) e.g. Hoteling. Ther= e is some crude clustering code around latency too which uses Kolmogorov-Smi= rnov=C2=A0distance matrices per the histograms.

A suggestion is = that those in developer and control test roles synchronize their device clo= cks with PTP. Iperf 2 supports one way delay (OWD) calculations but these o= nly work if the clocks are sync'd.=C2=A0 These in turn can be used per= =C2=A0Little's= law to calculate effective average queue depth, though this typically = assumes a steady state measurement.

Bob=C2=A0

On Sun, Aug 28, 2022 at= 4:54 PM Neal Cardwell <ncardwel= l@google.com> wrote:
If you are talking about the screenshot of the= UI at=C2=A0=C2=A0https://github.com/google/transperf,=C2=A0y= es, that particular test is a simple bulk flow test to show a simple case t= o give a sense of what the UI looks like. :-)

We use a f= ew different approaches that can examine dynamic flows causing packet loss:= =C2=A0

(1) The test configuration language is Python= , so you can construct arbitrarily fancy dynamic flow scenarios with arbitr= ary numbers of flows starting and stopping at arbitrary times.
(2) The tests can also use netperf command line options to run= periodic short transfers. (And we welcome patches to integrate support for= other tools.)

(3) We also run a fair number of te= sts for robustness to loss just using randomly injected packet loss (using = netem).

These are just some of the approaches we h= ave used, and I don't claim that these are the only or best approaches = to look at this. :-)

cheers,
neal
<= div>

On Sun, Aug 28, 2022 at 6:39 PM Bob McMahon <bob.mcmahon@broadcom= .com> wrote:
Hi Neal,

These look like steady-sta= te bulk flow tests unless I'm missing something.

Bob
=
On Sun= , Aug 28, 2022 at 11:43 AM Neal Cardwell <ncardwell@google.com> wrote:
Sure. For testing these kinds of properties of the BBR algorithm = we use various transperf test cases. The transperf tool is something Soheil= Hassas Yeganeh and our team cooked up and open-sourced here:


Best regards,
neal

On Sat, Aug 27, 2022, 4:43 PM Bob McMahon <b= ob.mcmahon@broadcom.com> wrote:
Curious to what you're doing du= ring development, if you can share?

Thanks,
Bob

On Sat, Aug 27, 20= 22 at 7:44 AM Neal Cardwell <ncardwell@google.com> w= rote:
Hi Bob,

Good question. I can imagine a number o= f different techniques to generate and measure the traffic flows for this k= ind of study, and don't have any particular suggestions.

=
neal


On Fri, Aug 26, 2022 at 4:54 PM Bob McM= ahon <bob.mcmahon@broadcom.com> wrote:
=
Hi Neal,=

Any thoughts on tooling to generate and measure the traffic flows B= BR is designed to optimize? I've been adding some low duty cycle suppor= t in iperf 2 with things like --bounceback and --burst-period and --burst-period. = We could pull the size and period from a known distribution or distribution= s though not sure what to pick.

Thanks,
Bob

Bob

<= div class=3D"gmail_quote">
On Fri, Aug= 26, 2022 at 6:36 AM 'Neal Cardwell' via BBR Development <bbr-dev@googlegroups.com> wrote:
Yes, I agree the assum= ptions are key here. One key aspect of this paper is that it focuses on the= steady-state behavior of bulk flows.

Once you allow for= short flows (like web pages, RPCs, etc) to dynamically enter and leave a b= ottleneck, the considerations become different. As is well-known, Reno/CUBI= C will starve themselves if new flows enter and cause loss too frequently. = For CUBIC, for a somewhat typical 30ms broadband path with a flow fair shar= e of 25 Mbit/sec, if new flows enter and cause loss more frequently than ro= ughly every 2 seconds then CUBIC will not be able to utilize its fair share= . For a high-speed WAN path, with 100ms RTT and fair share of 10 Gbit/sec,= =C2=A0 if new flows enter and cause loss more frequently than roughly every= 40 seconds then CUBIC will not be able to utilize its fair share. Basicall= y, loss-based CC can starve itself in some very=C2=A0typical=C2=A0kinds of = dynamic scenarios that happen in the real world.

B= BR is not trying to maintain a higher throughput than CUBIC in these kinds = of scenarios with steady-state bulk flows. BBR is trying to be robust to th= e kinds of random packet loss that happen in the real world when there are = flows dynamically entering/leaving a bottleneck.

c= heers,
neal




On = Thu, Aug 25, 2022 at 8:01 PM Dave Taht via Bloat <bl= oat@lists.bufferbloat.net> wrote:
I rather enjoyed this one. I can't help but wo= nder what would happen
if we plugged some different assumptions into their model.

https://www.c= omp.nus.edu.sg/~bleong/publications/imc2022-nash.pdf

--
FQ World Domination pending: h= ttps://blog.cerowrt.org/post/state_of_fq_codel/
Dave T=C3=A4ht CEO, TekLibre, LLC
_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/list= info/bloat

--
You received this message because you are subscribed to the Google Groups &= quot;BBR Development" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to bbr-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/C= ADVnQykKbnxpNcpuZATug_4VLhV1%3DaoTTQE2263o8HF9ye_TQg%40mail.gmail.com.<= br>

This ele= ctronic communication and the information and any files transmitted with it= , or attached to it, are confidential and are intended solely for the use o= f the individual or entity to whom it is addressed and may contain informat= ion that is confidential, legally privileged, protected by privacy laws, or= otherwise restricted from disclosure to anyone else. If you are not the in= tended recipient or the person responsible for delivering the e-mail to the= intended recipient, you are hereby notified that any use, copying, distrib= uting, dissemination, forwarding, printing, or copying of this e-mail is st= rictly prohibited. If you received this e-mail in error, please return the = e-mail to the sender, delete it from your computer, and destroy any printed= copy of it.

This ele= ctronic communication and the information and any files transmitted with it= , or attached to it, are confidential and are intended solely for the use o= f the individual or entity to whom it is addressed and may contain informat= ion that is confidential, legally privileged, protected by privacy laws, or= otherwise restricted from disclosure to anyone else. If you are not the in= tended recipient or the person responsible for delivering the e-mail to the= intended recipient, you are hereby notified that any use, copying, distrib= uting, dissemination, forwarding, printing, or copying of this e-mail is st= rictly prohibited. If you received this e-mail in error, please return the = e-mail to the sender, delete it from your computer, and destroy any printed= copy of it.
This ele= ctronic communication and the information and any files transmitted with it= , or attached to it, are confidential and are intended solely for the use o= f the individual or entity to whom it is addressed and may contain informat= ion that is confidential, legally privileged, protected by privacy laws, or= otherwise restricted from disclosure to anyone else. If you are not the in= tended recipient or the person responsible for delivering the e-mail to the= intended recipient, you are hereby notified that any use, copying, distrib= uting, dissemination, forwarding, printing, or copying of this e-mail is st= rictly prohibited. If you received this e-mail in error, please return the = e-mail to the sender, delete it from your computer, and destroy any printed= copy of it.
This ele= ctronic communication and the information and any files transmitted with it= , or attached to it, are confidential and are intended solely for the use o= f the individual or entity to whom it is addressed and may contain informat= ion that is confidential, legally privileged, protected by privacy laws, or= otherwise restricted from disclosure to anyone else. If you are not the in= tended recipient or the person responsible for delivering the e-mail to the= intended recipient, you are hereby notified that any use, copying, distrib= uting, dissemination, forwarding, printing, or copying of this e-mail is st= rictly prohibited. If you received this e-mail in error, please return the = e-mail to the sender, delete it from your computer, and destroy any printed= copy of it. --00000000000018d8e505e76406bf--