Thanks for the pointers, Bob. best regards, neal On Mon, Aug 29, 2022 at 12:47 PM 'Bob McMahon' via BBR Development < bbr-dev@googlegroups.com> wrote: > Thanks Neal. You might want to check out the flows released as iperf 2. > Basically > instantiate flows and run them. There typically is a controller running > python3 (v 3.10 or better) that uses ssh pipes to DUTs. The design is event > driven and > utilizes python's asyncio > which is quite powerful. The DUTs just need iperf2 and ssh. > > The code is at an alpha level and we're looking for broader industry > support and contributions. Both in realtime plotting but also in things > like multivariate regression detection using statistical process controls > (SPC) e.g. Hoteling > . There > is some crude clustering code around latency too which uses > Kolmogorov-Smirnov > distance > matrices per the histograms. > > A suggestion is that those in developer and control test roles synchronize > their device clocks with PTP. Iperf 2 supports one way delay (OWD) > calculations but these only work if the clocks are sync'd. These in turn > can be used per Little's law > to calculate effective > average queue depth, though this typically assumes a steady state > measurement. > > Bob > > On Sun, Aug 28, 2022 at 4:54 PM Neal Cardwell > wrote: > >> If you are talking about the screenshot of the UI at >> https://github.com/google/transperf, yes, that particular test is a >> simple bulk flow test to show a simple case to give a sense of what the UI >> looks like. :-) >> >> We use a few different approaches that can examine dynamic flows causing >> packet loss: >> >> (1) The test configuration language is Python, so you can construct >> arbitrarily fancy dynamic flow scenarios with arbitrary numbers of flows >> starting and stopping at arbitrary times. >> >> (2) The tests can also use netperf command line options to run periodic >> short transfers. (And we welcome patches to integrate support for other >> tools.) >> >> (3) We also run a fair number of tests for robustness to loss just using >> randomly injected packet loss (using netem). >> >> These are just some of the approaches we have used, and I don't claim >> that these are the only or best approaches to look at this. :-) >> >> cheers, >> neal >> >> >> On Sun, Aug 28, 2022 at 6:39 PM Bob McMahon >> wrote: >> >>> Hi Neal, >>> >>> These look like steady-state bulk flow tests unless I'm missing >>> something. >>> >>> Bob >>> >>> On Sun, Aug 28, 2022 at 11:43 AM Neal Cardwell >>> wrote: >>> >>>> Sure. For testing these kinds of properties of the BBR algorithm we use >>>> various transperf test cases. The transperf tool is something Soheil Hassas >>>> Yeganeh and our team cooked up and open-sourced here: >>>> >>>> https://github.com/google/transperf >>>> >>>> Best regards, >>>> neal >>>> >>>> On Sat, Aug 27, 2022, 4:43 PM Bob McMahon >>>> wrote: >>>> >>>>> Curious to what you're doing during development, if you can share? >>>>> >>>>> Thanks, >>>>> Bob >>>>> >>>>> On Sat, Aug 27, 2022 at 7:44 AM Neal Cardwell >>>>> wrote: >>>>> >>>>>> Hi Bob, >>>>>> >>>>>> Good question. I can imagine a number of different techniques to >>>>>> generate and measure the traffic flows for this kind of study, and don't >>>>>> have any particular suggestions. >>>>>> >>>>>> neal >>>>>> >>>>>> >>>>>> On Fri, Aug 26, 2022 at 4:54 PM Bob McMahon >>>>>> wrote: >>>>>> >>>>>>> Hi Neal, >>>>>>> >>>>>>> Any thoughts on tooling to generate and measure the traffic flows >>>>>>> BBR is designed to optimize? I've been adding some low duty cycle support >>>>>>> in iperf 2 with things >>>>>>> like --bounceback and --burst-period and --burst-period >>>>>>> . We could pull >>>>>>> the size and period from a known distribution or distributions though not >>>>>>> sure what to pick. >>>>>>> >>>>>>> Thanks, >>>>>>> Bob >>>>>>> >>>>>>> Bob >>>>>>> >>>>>>> On Fri, Aug 26, 2022 at 6:36 AM 'Neal Cardwell' via BBR Development < >>>>>>> bbr-dev@googlegroups.com> wrote: >>>>>>> >>>>>>>> Yes, I agree the assumptions are key here. One key aspect of this >>>>>>>> paper is that it focuses on the steady-state behavior of bulk flows. >>>>>>>> >>>>>>>> Once you allow for short flows (like web pages, RPCs, etc) to >>>>>>>> dynamically enter and leave a bottleneck, the considerations become >>>>>>>> different. As is well-known, Reno/CUBIC will starve themselves if new flows >>>>>>>> enter and cause loss too frequently. For CUBIC, for a somewhat typical 30ms >>>>>>>> broadband path with a flow fair share of 25 Mbit/sec, if new flows enter >>>>>>>> and cause loss more frequently than roughly every 2 seconds then CUBIC will >>>>>>>> not be able to utilize its fair share. For a high-speed WAN path, with >>>>>>>> 100ms RTT and fair share of 10 Gbit/sec, if new flows enter and cause loss >>>>>>>> more frequently than roughly every 40 seconds then CUBIC will not be able >>>>>>>> to utilize its fair share. Basically, loss-based CC can starve itself in >>>>>>>> some very typical kinds of dynamic scenarios that happen in the real world. >>>>>>>> >>>>>>>> BBR is not trying to maintain a higher throughput than CUBIC in >>>>>>>> these kinds of scenarios with steady-state bulk flows. BBR is trying to be >>>>>>>> robust to the kinds of random packet loss that happen in the real world >>>>>>>> when there are flows dynamically entering/leaving a bottleneck. >>>>>>>> >>>>>>>> cheers, >>>>>>>> neal >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Aug 25, 2022 at 8:01 PM Dave Taht via Bloat < >>>>>>>> bloat@lists.bufferbloat.net> wrote: >>>>>>>> >>>>>>>>> I rather enjoyed this one. I can't help but wonder what would >>>>>>>>> happen >>>>>>>>> if we plugged some different assumptions into their model. >>>>>>>>> >>>>>>>>> https://www.comp.nus.edu.sg/~bleong/publications/imc2022-nash.pdf >>>>>>>>> >>>>>>>>> -- >>>>>>>>> FQ World Domination pending: >>>>>>>>> https://blog.cerowrt.org/post/state_of_fq_codel/ >>>>>>>>> Dave Täht CEO, TekLibre, LLC >>>>>>>>> _______________________________________________ >>>>>>>>> Bloat mailing list >>>>>>>>> Bloat@lists.bufferbloat.net >>>>>>>>> https://lists.bufferbloat.net/listinfo/bloat >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "BBR Development" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to bbr-dev+unsubscribe@googlegroups.com. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/bbr-dev/CADVnQykKbnxpNcpuZATug_4VLhV1%3DaoTTQE2263o8HF9ye_TQg%40mail.gmail.com >>>>>>>> >>>>>>>> . >>>>>>>> >>>>>>> >>>>>>> This electronic communication and the information and any files >>>>>>> transmitted with it, or attached to it, are confidential and are intended >>>>>>> solely for the use of the individual or entity to whom it is addressed and >>>>>>> may contain information that is confidential, legally privileged, protected >>>>>>> by privacy laws, or otherwise restricted from disclosure to anyone else. If >>>>>>> you are not the intended recipient or the person responsible for delivering >>>>>>> the e-mail to the intended recipient, you are hereby notified that any use, >>>>>>> copying, distributing, dissemination, forwarding, printing, or copying of >>>>>>> this e-mail is strictly prohibited. If you received this e-mail in error, >>>>>>> please return the e-mail to the sender, delete it from your computer, and >>>>>>> destroy any printed copy of it. >>>>>> >>>>>> >>>>> This electronic communication and the information and any files >>>>> transmitted with it, or attached to it, are confidential and are intended >>>>> solely for the use of the individual or entity to whom it is addressed and >>>>> may contain information that is confidential, legally privileged, protected >>>>> by privacy laws, or otherwise restricted from disclosure to anyone else. If >>>>> you are not the intended recipient or the person responsible for delivering >>>>> the e-mail to the intended recipient, you are hereby notified that any use, >>>>> copying, distributing, dissemination, forwarding, printing, or copying of >>>>> this e-mail is strictly prohibited. If you received this e-mail in error, >>>>> please return the e-mail to the sender, delete it from your computer, and >>>>> destroy any printed copy of it. >>>> >>>> >>> This electronic communication and the information and any files >>> transmitted with it, or attached to it, are confidential and are intended >>> solely for the use of the individual or entity to whom it is addressed and >>> may contain information that is confidential, legally privileged, protected >>> by privacy laws, or otherwise restricted from disclosure to anyone else. If >>> you are not the intended recipient or the person responsible for delivering >>> the e-mail to the intended recipient, you are hereby notified that any use, >>> copying, distributing, dissemination, forwarding, printing, or copying of >>> this e-mail is strictly prohibited. If you received this e-mail in error, >>> please return the e-mail to the sender, delete it from your computer, and >>> destroy any printed copy of it. >> >> > This electronic communication and the information and any files > transmitted with it, or attached to it, are confidential and are intended > solely for the use of the individual or entity to whom it is addressed and > may contain information that is confidential, legally privileged, protected > by privacy laws, or otherwise restricted from disclosure to anyone else. If > you are not the intended recipient or the person responsible for delivering > the e-mail to the intended recipient, you are hereby notified that any use, > copying, distributing, dissemination, forwarding, printing, or copying of > this e-mail is strictly prohibited. If you received this e-mail in error, > please return the e-mail to the sender, delete it from your computer, and > destroy any printed copy of it. > > -- > You received this message because you are subscribed to the Google Groups > "BBR Development" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to bbr-dev+unsubscribe@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/bbr-dev/CAHb6Lvqi9%3Dr0H6uNekoCs8tm450Bd-6%2BA9v7XMszX73h7yVMPg%40mail.gmail.com > > . >