From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id DD2D63B29E for ; Mon, 29 Aug 2022 16:07:35 -0400 (EDT) Received: by mail-qk1-x735.google.com with SMTP id c9so6919028qkk.6 for ; Mon, 29 Aug 2022 13:07:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=aytGULNox65q3NV6oHhc1eWXWAZ1zng8h2jr2mVN0Nk=; b=WRWjELEV+ys6EKLbf1VBw6hDLvnaCW2rt38Rc/VK8gM8QvqueHx9dtI3/sN9wdUP+a XJblBDfky1XCDWmPBHj37XU3m2VDJ3QYnzzFg2vwKpBHjWWI76yx40/AJ9+hp8v1FK2b r3G76EX63gYu/MJPfKJZUDlRhxdYYKIAyWANl959/oweztiGJcI4qUd3pNJ4eYQPsf02 01uVK2l+Yh1LQ/Ahf/D4OmUl9vehM9ncYajMe4xJnwts7xzlhDddfzKmijOEAQ7zmohT YQCLGt3LHpNaRKg0ALaBspeQ6hAWf8yh1HabFiLau/osgYBR4TEIgjtYXsmXw14zqCnz P/fA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=aytGULNox65q3NV6oHhc1eWXWAZ1zng8h2jr2mVN0Nk=; b=GiWaKAQUZxH44p/pOsAA+fMrNB5QtiZ9ttScjaTOzW2lUUOkqztUgtkzngy/k//wjQ iAJ6pzfKCgJZ3xqq8BNQNYIjngWZr8neC9uGMYuEoqSKqE94IzHyko7o2wfPz5yZtNIn J7Dej/n2lHFjDbDyfhdoFHcZkK4NCeNxIHHG7VKLCp8rrfESPMKq9VOpUyGbC/RpSNhS 8p3XzzI7NsspO7J8+7sPEq34BxYYKH21pbsznQydXWE16+RYaiVeVCwQDM8aW+kUPMYP RNyY4Yb/FqlpvUxJ48HmoJLHQtkK3+Ls/ttKihIzzgEoPw5JtH47l8Q5JkhZSJ4PaExX LDwA== X-Gm-Message-State: ACgBeo2CdIbDX/8i/+LLp+B5wFS0yfBN/E4gHU2FxPgCUis7mFR7YFZN e75BBOLV+Tndn8pyMvvLq7EHD+OH5cNCy2m0lhJyOA== X-Google-Smtp-Source: AA6agR6Hltssp3vd7amTZerdtEhpbE+XuXeaP+W9zt+XTZSOmwW+onNDYojoDYcg6cIrAwJtjIWN3vcMawJlwAEsFMA= X-Received: by 2002:a05:620a:410c:b0:6b2:82d8:dcae with SMTP id j12-20020a05620a410c00b006b282d8dcaemr9864199qko.259.1661803655133; Mon, 29 Aug 2022 13:07:35 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Neal Cardwell Date: Mon, 29 Aug 2022 16:07:18 -0400 Message-ID: To: Bob McMahon Cc: Dave Taht , bloat , BBR Development , ayush@comp.nus.edu.sg Content-Type: multipart/alternative; boundary="0000000000008d9cc105e766d0ca" Subject: Re: [Bloat] [bbr-dev] Re: Are we heading towards a BBR-dominant Internet? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Aug 2022 20:07:36 -0000 --0000000000008d9cc105e766d0ca Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks for the pointers, Bob. best regards, neal On Mon, Aug 29, 2022 at 12:47 PM 'Bob McMahon' via BBR Development < bbr-dev@googlegroups.com> wrote: > Thanks Neal. You might want to check out the flows released as iperf 2. > Basically > instantiate flows and run them. There typically is a controller running > python3 (v 3.10 or better) that uses ssh pipes to DUTs. The design is eve= nt > driven and > utilizes python's asyncio > which is quite powerful. The DUTs just need iperf2 and ssh. > > The code is at an alpha level and we're looking for broader industry > support and contributions. Both in realtime plotting but also in things > like multivariate regression detection using statistical process controls > (SPC) e.g. Hoteling > . There > is some crude clustering code around latency too which uses > Kolmogorov-Smirnov > distance > matrices per the histograms. > > A suggestion is that those in developer and control test roles synchroniz= e > their device clocks with PTP. Iperf 2 supports one way delay (OWD) > calculations but these only work if the clocks are sync'd. These in turn > can be used per Little's law > to calculate effective > average queue depth, though this typically assumes a steady state > measurement. > > Bob > > On Sun, Aug 28, 2022 at 4:54 PM Neal Cardwell > wrote: > >> If you are talking about the screenshot of the UI at >> https://github.com/google/transperf, yes, that particular test is a >> simple bulk flow test to show a simple case to give a sense of what the = UI >> looks like. :-) >> >> We use a few different approaches that can examine dynamic flows causing >> packet loss: >> >> (1) The test configuration language is Python, so you can construct >> arbitrarily fancy dynamic flow scenarios with arbitrary numbers of flows >> starting and stopping at arbitrary times. >> >> (2) The tests can also use netperf command line options to run periodic >> short transfers. (And we welcome patches to integrate support for other >> tools.) >> >> (3) We also run a fair number of tests for robustness to loss just using >> randomly injected packet loss (using netem). >> >> These are just some of the approaches we have used, and I don't claim >> that these are the only or best approaches to look at this. :-) >> >> cheers, >> neal >> >> >> On Sun, Aug 28, 2022 at 6:39 PM Bob McMahon >> wrote: >> >>> Hi Neal, >>> >>> These look like steady-state bulk flow tests unless I'm missing >>> something. >>> >>> Bob >>> >>> On Sun, Aug 28, 2022 at 11:43 AM Neal Cardwell >>> wrote: >>> >>>> Sure. For testing these kinds of properties of the BBR algorithm we us= e >>>> various transperf test cases. The transperf tool is something Soheil H= assas >>>> Yeganeh and our team cooked up and open-sourced here: >>>> >>>> https://github.com/google/transperf >>>> >>>> Best regards, >>>> neal >>>> >>>> On Sat, Aug 27, 2022, 4:43 PM Bob McMahon >>>> wrote: >>>> >>>>> Curious to what you're doing during development, if you can share? >>>>> >>>>> Thanks, >>>>> Bob >>>>> >>>>> On Sat, Aug 27, 2022 at 7:44 AM Neal Cardwell >>>>> wrote: >>>>> >>>>>> Hi Bob, >>>>>> >>>>>> Good question. I can imagine a number of different techniques to >>>>>> generate and measure the traffic flows for this kind of study, and d= on't >>>>>> have any particular suggestions. >>>>>> >>>>>> neal >>>>>> >>>>>> >>>>>> On Fri, Aug 26, 2022 at 4:54 PM Bob McMahon >>>>>> wrote: >>>>>> >>>>>>> Hi Neal, >>>>>>> >>>>>>> Any thoughts on tooling to generate and measure the traffic flows >>>>>>> BBR is designed to optimize? I've been adding some low duty cycle s= upport >>>>>>> in iperf 2 with things >>>>>>> like --bounceback and --burst-period and --burst-period >>>>>>> . We could pull >>>>>>> the size and period from a known distribution or distributions thou= gh not >>>>>>> sure what to pick. >>>>>>> >>>>>>> Thanks, >>>>>>> Bob >>>>>>> >>>>>>> Bob >>>>>>> >>>>>>> On Fri, Aug 26, 2022 at 6:36 AM 'Neal Cardwell' via BBR Development= < >>>>>>> bbr-dev@googlegroups.com> wrote: >>>>>>> >>>>>>>> Yes, I agree the assumptions are key here. One key aspect of this >>>>>>>> paper is that it focuses on the steady-state behavior of bulk flow= s. >>>>>>>> >>>>>>>> Once you allow for short flows (like web pages, RPCs, etc) to >>>>>>>> dynamically enter and leave a bottleneck, the considerations becom= e >>>>>>>> different. As is well-known, Reno/CUBIC will starve themselves if = new flows >>>>>>>> enter and cause loss too frequently. For CUBIC, for a somewhat typ= ical 30ms >>>>>>>> broadband path with a flow fair share of 25 Mbit/sec, if new flows= enter >>>>>>>> and cause loss more frequently than roughly every 2 seconds then C= UBIC will >>>>>>>> not be able to utilize its fair share. For a high-speed WAN path, = with >>>>>>>> 100ms RTT and fair share of 10 Gbit/sec, if new flows enter and c= ause loss >>>>>>>> more frequently than roughly every 40 seconds then CUBIC will not = be able >>>>>>>> to utilize its fair share. Basically, loss-based CC can starve its= elf in >>>>>>>> some very typical kinds of dynamic scenarios that happen in the re= al world. >>>>>>>> >>>>>>>> BBR is not trying to maintain a higher throughput than CUBIC in >>>>>>>> these kinds of scenarios with steady-state bulk flows. BBR is tryi= ng to be >>>>>>>> robust to the kinds of random packet loss that happen in the real = world >>>>>>>> when there are flows dynamically entering/leaving a bottleneck. >>>>>>>> >>>>>>>> cheers, >>>>>>>> neal >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Aug 25, 2022 at 8:01 PM Dave Taht via Bloat < >>>>>>>> bloat@lists.bufferbloat.net> wrote: >>>>>>>> >>>>>>>>> I rather enjoyed this one. I can't help but wonder what would >>>>>>>>> happen >>>>>>>>> if we plugged some different assumptions into their model. >>>>>>>>> >>>>>>>>> https://www.comp.nus.edu.sg/~bleong/publications/imc2022-nash.pdf >>>>>>>>> >>>>>>>>> -- >>>>>>>>> FQ World Domination pending: >>>>>>>>> https://blog.cerowrt.org/post/state_of_fq_codel/ >>>>>>>>> Dave T=C3=A4ht CEO, TekLibre, LLC >>>>>>>>> _______________________________________________ >>>>>>>>> Bloat mailing list >>>>>>>>> Bloat@lists.bufferbloat.net >>>>>>>>> https://lists.bufferbloat.net/listinfo/bloat >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "BBR Development" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to bbr-dev+unsubscribe@googlegroups.com. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/bbr-dev/CADVnQykKbnxpNcpuZATug_4= VLhV1%3DaoTTQE2263o8HF9ye_TQg%40mail.gmail.com >>>>>>>> >>>>>>>> . >>>>>>>> >>>>>>> >>>>>>> This electronic communication and the information and any files >>>>>>> transmitted with it, or attached to it, are confidential and are in= tended >>>>>>> solely for the use of the individual or entity to whom it is addres= sed and >>>>>>> may contain information that is confidential, legally privileged, p= rotected >>>>>>> by privacy laws, or otherwise restricted from disclosure to anyone = else. If >>>>>>> you are not the intended recipient or the person responsible for de= livering >>>>>>> the e-mail to the intended recipient, you are hereby notified that = any use, >>>>>>> copying, distributing, dissemination, forwarding, printing, or copy= ing of >>>>>>> this e-mail is strictly prohibited. If you received this e-mail in = error, >>>>>>> please return the e-mail to the sender, delete it from your compute= r, and >>>>>>> destroy any printed copy of it. >>>>>> >>>>>> >>>>> This electronic communication and the information and any files >>>>> transmitted with it, or attached to it, are confidential and are inte= nded >>>>> solely for the use of the individual or entity to whom it is addresse= d and >>>>> may contain information that is confidential, legally privileged, pro= tected >>>>> by privacy laws, or otherwise restricted from disclosure to anyone el= se. If >>>>> you are not the intended recipient or the person responsible for deli= vering >>>>> the e-mail to the intended recipient, you are hereby notified that an= y use, >>>>> copying, distributing, dissemination, forwarding, printing, or copyin= g of >>>>> this e-mail is strictly prohibited. If you received this e-mail in er= ror, >>>>> please return the e-mail to the sender, delete it from your computer,= and >>>>> destroy any printed copy of it. >>>> >>>> >>> This electronic communication and the information and any files >>> transmitted with it, or attached to it, are confidential and are intend= ed >>> solely for the use of the individual or entity to whom it is addressed = and >>> may contain information that is confidential, legally privileged, prote= cted >>> by privacy laws, or otherwise restricted from disclosure to anyone else= . If >>> you are not the intended recipient or the person responsible for delive= ring >>> the e-mail to the intended recipient, you are hereby notified that any = use, >>> copying, distributing, dissemination, forwarding, printing, or copying = of >>> this e-mail is strictly prohibited. If you received this e-mail in erro= r, >>> please return the e-mail to the sender, delete it from your computer, a= nd >>> destroy any printed copy of it. >> >> > This electronic communication and the information and any files > transmitted with it, or attached to it, are confidential and are intended > solely for the use of the individual or entity to whom it is addressed an= d > may contain information that is confidential, legally privileged, protect= ed > by privacy laws, or otherwise restricted from disclosure to anyone else. = If > you are not the intended recipient or the person responsible for deliveri= ng > the e-mail to the intended recipient, you are hereby notified that any us= e, > copying, distributing, dissemination, forwarding, printing, or copying of > this e-mail is strictly prohibited. If you received this e-mail in error, > please return the e-mail to the sender, delete it from your computer, and > destroy any printed copy of it. > > -- > You received this message because you are subscribed to the Google Groups > "BBR Development" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to bbr-dev+unsubscribe@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/bbr-dev/CAHb6Lvqi9%3Dr0H6uNekoCs8tm450B= d-6%2BA9v7XMszX73h7yVMPg%40mail.gmail.com > > . > --0000000000008d9cc105e766d0ca Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks for the pointers,=C2=A0 Bob.

best regards,
neal


On Mon, Aug 29, 20= 22 at 12:47 PM 'Bob McMahon' via BBR Development <bbr-dev@googlegroups.com> wrote:
Than= ks Neal. You might want to check out the flows released as i= perf 2. Basically instantiate flows and run them. There typically is a = controller running python3 (v 3.10 or better) that uses ssh pipes to DUTs. = The design is event driven and utilizes python's async= io which is quite powerful. The DUTs just need iperf2 and ssh.

T= he code is at an alpha level and we're looking for broader industry sup= port and contributions. Both in realtime=C2=A0plotting but also in things l= ike multivariate regression detection using statistical=C2=A0process contro= ls (SPC) e.g. Hoteling. There is some crude cluster= ing code around latency too which uses Kolmogorov-Smirnov= =C2=A0distance matrices per the histograms.

A suggestion is that= those in developer and control test roles synchronize their device clocks = with PTP. Iperf 2 supports one way delay (OWD) calculations but these only = work if the clocks are sync'd.=C2=A0 These in turn can be used per=C2= =A0Little's law to calculate effective average queue depth, though = this typically assumes a steady state measurement.

Bob=C2=A0
On Sun, = Aug 28, 2022 at 4:54 PM Neal Cardwell <ncardwell@google.com> wrote:
If you are ta= lking about the screenshot of the UI at=C2=A0=C2=A0https://github= .com/google/transperf,=C2=A0yes, that particular test is a simple bulk = flow test to show a simple case to give a sense of what the UI looks like. = :-)

We use a few different approaches that can examine d= ynamic flows causing packet loss:=C2=A0

(1) The test= configuration language is Python, so you can construct arbitrarily fancy d= ynamic flow scenarios with arbitrary numbers of flows starting and stopping= at arbitrary times.

(2) The tests can also use ne= tperf command line options to run periodic short transfers. (And we welcome= patches to integrate support for other tools.)

(3= ) We also run a fair number of tests for robustness to loss just using rand= omly injected packet loss (using netem).

These are= just some of the approaches we have used, and I don't claim that these= are the only or best approaches to look at this. :-)

<= div>cheers,
neal


On Sun, Aug 28, 2022= at 6:39 PM Bob McMahon <bob.mcmahon@broadcom.com> wrote:
Hi Neal,

These look like steady-state bulk flow tests unless I'm missin= g something.

Bob

On Sun, Aug 28, 2022 at 11:43 AM Neal Cardwel= l <ncardwell@g= oogle.com> wrote:
Sure. For testing these kind= s of properties of the BBR algorithm we use various transperf test cases. T= he transperf tool is something Soheil Hassas Yeganeh and our team cooked up= and open-sourced here:

=

Best regards,
=
neal

On Sat, Aug 27, = 2022, 4:43 PM Bob McMahon <bob.mcmahon@broadcom.com> wrote:=
Curious to what you're doing during development, if you can share?
Thanks,
Bob

On Sat, Aug 27, 2022 at 7:44 AM Neal Cardwell <ncardwell@google.com> wrote:
Hi Bob,

Good= question. I can imagine a number of different techniques to generate and m= easure the traffic flows for this kind of study, and don't have any par= ticular suggestions.

neal


On F= ri, Aug 26, 2022 at 4:54 PM Bob McMahon <bob.mcmahon@br= oadcom.com> wrote:
Hi Neal,

Any thoughts on tooling to gene= rate and measure the traffic flows BBR is designed to optimize? I've be= en adding some low duty cycle support in iperf 2<= /a> with things like --bounceback and --b= urst-period and --burst-period. We could pull the size and period from = a known distribution or distributions though not sure what to pick.

= Thanks,
Bob

Bob

On Fri, Aug 26, 2022 at 6:36 AM 'Neal Cardwell= ' via BBR Development <bbr-dev@googlegroups.com= > wrote:
Yes, I agree the assumptions are key here. One key aspect of t= his paper is that it focuses on the steady-state behavior of bulk flows.
Once you allow for short flows (like web pages, RPCs, etc)= to dynamically enter and leave a bottleneck, the considerations become dif= ferent. As is well-known, Reno/CUBIC will starve themselves if new flows en= ter and cause loss too frequently. For CUBIC, for a somewhat typical 30ms b= roadband path with a flow fair share of 25 Mbit/sec, if new flows enter and= cause loss more frequently than roughly every 2 seconds then CUBIC will no= t be able to utilize its fair share. For a high-speed WAN path, with 100ms = RTT and fair share of 10 Gbit/sec,=C2=A0 if new flows enter and cause loss = more frequently than roughly every 40 seconds then CUBIC will not be able t= o utilize its fair share. Basically, loss-based CC can starve itself in som= e very=C2=A0typical=C2=A0kinds of dynamic scenarios that happen in the real= world.

BBR is not trying to maintain a higher thr= oughput than CUBIC in these kinds of scenarios with steady-state bulk flows= . BBR is trying to be robust to the kinds of random packet loss that happen= in the real world when there are flows dynamically entering/leaving a bott= leneck.

cheers,
neal




On Thu, Aug 25, 2022 at 8:01 PM Dave Taht vi= a Bloat <bloat@lists.bufferbloat.net> wrote:<= br>
I rather enjoyed= this one. I can't help but wonder what would happen
if we plugged some different assumptions into their model.

https://www.c= omp.nus.edu.sg/~bleong/publications/imc2022-nash.pdf

--
FQ World Domination pending: h= ttps://blog.cerowrt.org/post/state_of_fq_codel/
Dave T=C3=A4ht CEO, TekLibre, LLC
_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/list= info/bloat

--
You received this message because you are subscribed to the Google Groups &= quot;BBR Development" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to bbr-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/C= ADVnQykKbnxpNcpuZATug_4VLhV1%3DaoTTQE2263o8HF9ye_TQg%40mail.gmail.com.<= br>

This ele= ctronic communication and the information and any files transmitted with it= , or attached to it, are confidential and are intended solely for the use o= f the individual or entity to whom it is addressed and may contain informat= ion that is confidential, legally privileged, protected by privacy laws, or= otherwise restricted from disclosure to anyone else. If you are not the in= tended recipient or the person responsible for delivering the e-mail to the= intended recipient, you are hereby notified that any use, copying, distrib= uting, dissemination, forwarding, printing, or copying of this e-mail is st= rictly prohibited. If you received this e-mail in error, please return the = e-mail to the sender, delete it from your computer, and destroy any printed= copy of it.

This ele= ctronic communication and the information and any files transmitted with it= , or attached to it, are confidential and are intended solely for the use o= f the individual or entity to whom it is addressed and may contain informat= ion that is confidential, legally privileged, protected by privacy laws, or= otherwise restricted from disclosure to anyone else. If you are not the in= tended recipient or the person responsible for delivering the e-mail to the= intended recipient, you are hereby notified that any use, copying, distrib= uting, dissemination, forwarding, printing, or copying of this e-mail is st= rictly prohibited. If you received this e-mail in error, please return the = e-mail to the sender, delete it from your computer, and destroy any printed= copy of it.

This ele= ctronic communication and the information and any files transmitted with it= , or attached to it, are confidential and are intended solely for the use o= f the individual or entity to whom it is addressed and may contain informat= ion that is confidential, legally privileged, protected by privacy laws, or= otherwise restricted from disclosure to anyone else. If you are not the in= tended recipient or the person responsible for delivering the e-mail to the= intended recipient, you are hereby notified that any use, copying, distrib= uting, dissemination, forwarding, printing, or copying of this e-mail is st= rictly prohibited. If you received this e-mail in error, please return the = e-mail to the sender, delete it from your computer, and destroy any printed= copy of it.

This ele= ctronic communication and the information and any files transmitted with it= , or attached to it, are confidential and are intended solely for the use o= f the individual or entity to whom it is addressed and may contain informat= ion that is confidential, legally privileged, protected by privacy laws, or= otherwise restricted from disclosure to anyone else. If you are not the in= tended recipient or the person responsible for delivering the e-mail to the= intended recipient, you are hereby notified that any use, copying, distrib= uting, dissemination, forwarding, printing, or copying of this e-mail is st= rictly prohibited. If you received this e-mail in error, please return the = e-mail to the sender, delete it from your computer, and destroy any printed= copy of it.

--
You received this message because you are subscribed to the Google Groups &= quot;BBR Development" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to bbr-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/CAHb6Lvqi9%3Dr0H6uNekoCs8tm45= 0Bd-6%2BA9v7XMszX73h7yVMPg%40mail.gmail.com.
--0000000000008d9cc105e766d0ca--