On Tue, Jul 6, 2021 at 7:49 PM Bob McMahon wrote: > Thanks for this. Looks interesting. Does ABC (love the acronym) try to > optimize throughput, latency or some combination, e.g. network power > (throughput/delay)? > ABC tries to optimize both throughput and delay, though the paper does not mention power specifically. > Things like roams, seen energy over the last n milliseconds, and "I've > lost n TXOP arbitrations over time y" could also have influence on the > control feedback loop to TCP. Not sure how TCP gets any of these stats. > > None of this is end/end though. It's likely just the last/first hop which > I think is probably still useful to TCP designers - not sure. > The paper explains their scheme to convey the accelerate/break signals from bottlenecks using the ECN bits, and then have the TCP receiver echo them to the TCP sender using the historic "NS" bit. neal > Bob > > On Tue, Jul 6, 2021 at 1:39 PM Neal Cardwell wrote: > >> >> >> On Tue, Jul 6, 2021 at 4:34 PM 'Bob McMahon' via BBR Development < >> bbr-dev@googlegroups.com> wrote: >> >>> Curious to the idea of wifi providing the opposite of ECN, i.e. my rate >>> selection algorithm just went from one stream to two stream so TCP speed up >>> your feedback loop decisions with a bias to faster? >>> >> >> That could make sense. That sounds a lot lot like the Accel-Brake >> Control work: >> >> https://www.usenix.org/conference/nsdi20/presentation/goyal >> >> ABC: A Simple Explicit Congestion Controller for Wireless Networks >> >> Authors: >> Prateesh Goyal, MIT CSAIL; Anup Agarwal, CMU; Ravi Netravali, UCLA; >> Mohammad Alizadeh and Hari Balakrishnan, MIT CSAIL >> >> Abstract: >> We propose Accel-Brake Control (ABC), a simple and deployable explicit >> congestion control protocol for network paths with time-varying wireless >> links. ABC routers mark each packet with an “accelerate” or “brake”, which >> causes senders to slightly increase or decrease their congestion windows. >> Routers use this feedback to quickly guide senders towards a desired target >> rate. ABC requires no changes to header formats or user devices, but >> achieves better performance than XCP. ABC is also incrementally deployable; >> it operates correctly when the bottleneck is a non-ABC router, and can >> coexist with non-ABC traffic sharing the same bottleneck link. We evaluate >> ABC using a Wi-Fi implementation and trace-driven emulation of cellular >> links. ABC achieves 30-40% higher throughput than Cubic+Codel for similar >> delays, and 2.2× lower delays than BBR on a Wi-Fi path. On cellular network >> paths, ABC achieves 50% higher throughput than Cubic+Codel. >> >> neal >> >> Bob >>> >>> On Mon, Jul 5, 2021 at 10:20 AM Dave Taht wrote: >>> >>>> eric, I thought I'd forward this to one of the bufferbloat.net lists. >>>> Feel free to join. It would also be helpful to duplicate your test scripts >>>> here. >>>> >>>> I agree with you, bbr is way too slow here. We see it bloat massively >>>> in wifi also. I would like BBR to far more reactive than it is, and also >>>> for it to respond to RFC3168 ECN marks. >>>> >>>> You said: >>>> >>>> *" *In my opinion, in high bw jitter scenario, window based bw >>>> undaption method is too slow to converge to the actual bottleneck bw, when >>>> actual bw imcreases, BBR needs at lease 8RTT to catch it, when it >>>> decreases, BBR then needs at most 10 RTT to expire the previous bw maximum, >>>> anyway it is too slow, causing severe buffer-bloating. " >>>> >>>> You might find the "flent" tool very useful for getting a more >>>> microscopic view as to what is going on. >>>> >>>> A test we would use to get a better picture of what is really going on >>>> would be >>>> >>>> flent -H netperf_test_server -x --socket-stats --step-size=.05 -t >>>> the_test_conditions --te=cc_algo=bbr --te=upload_streams=1 tcp_nup >>>> >>>> This samples a ton of tcp related stats and lets you plot them with >>>> flent-gui, there are 110 other tests in the suite. >>>> >>>> A more complicated invocation can get qdisc stats from the router >>>> involved, or iterate over CCs, or run at longer intervals. In general I >>>> have found running tests for 5 minutes or longer reveal interesting >>>> behaviors. >>>> >>>> flent -H $S --socket-stats -x --step-size=.05 -t >>>> 1-flow-fq_starlink_noecn-${M}-c >>>> >>>> c=${CC} --te=cpu_stats_hosts=$R --te=netstat_hosts=$R -4 >>>> --te=qdisc_stats_hosts= >>>> >>>> $RS --te=qdisc_stats_interfaces=$RV --te=upload_streams=1 >>>> --te=cc_algo=$CC tcp_1 >>>> >>>> up >>>> >>>> >>>> pic below >>>> >>>> ---------- Forwarded message --------- >>>> From: Eric xu >>>> Date: Mon, Jul 5, 2021 at 12:27 AM >>>> Subject: Re: [bbr-dev] BBR performance and optimization in cellular >>>> wireless network with high delay jitter >>>> To: BBR Development >>>> >>>> >>>> Hi, Neal, thanks for your answer. I still have two questions to >>>> discuss with you after having read the two links you mentioned above. >>>> >>>> 1, As mentioned in the second link, BBR's throughput on high-jitter >>>> links (especially in cellular) usually matches CUBIC, as demonstrated >>>> by the following figure, >>>> [image: 截图.PNG] >>>> however, what we can infer from the figure is that, the mean >>>> throughput(~14Mbps) is far from the available bw that is greater than >>>> 20Mbps as showed in the cumulative distrubution figure. Since CUBIC back >>>> off frequently in the LTE network with high error rate, but BBR does not >>>> acquire much more throughput than CUBIC, the only reason I can imagine is >>>> that BBR is highly affected by high delay jitter although cwnd provision is >>>> applied. >>>> >>>> *Is my understanding right? and is there any optimization to further >>>> improve thoughput in high bw and delay jitter scenario? * >>>> >>>> *2, Is there any method to enhance BBR's adaptive ability to high bw >>>> and delay jitter? *In my opinion, in high bw jitter scenario, window >>>> based bw undaption method is too slow to converge to the actual bottleneck >>>> bw, when actual bw imcreases, BBR needs at lease 8RTT to catch it, when it >>>> decreases, BBR then needs at most 10 RTT to expire the previous bw maximum, >>>> anyway it is too slow, causing severe buffer-bloating. In high delay >>>> jitter scenario, BBR also will cause bw under utilizaion. *So, what >>>> is your latest optimizaion to improve the performance in such a high bw and >>>> delay jitter network?* >>>> >>>> Thx a lot, >>>> Eric >>>> 在2021年7月4日星期日 UTC+8 下午9:59:57 写道： >>>> >>>>> Yes, BBR includes mechanisms to deal with this kind of jitter. Please >>>>> see the March 2019 thread on this topic: >>>>> https://groups.google.com/g/bbr-dev/c/kBZaq98xCC4 >>>>> >>>>> Our experience is that BBR's throughput on high-jitter links >>>>> (cellular, wifi, DOCSIS, datacenter ethernet w/ GRO/LRO) is quite good at >>>>> this point (usually matches CUBIC), and there are others who have >>>>> documented this as well, e.g.: >>>>> http://web.cs.wpi.edu/~claypool/papers/driving-bbr/ >>>>> >>>>> best, >>>>> neal >>>>> >>>>> >>>>> On Sun, Jul 4, 2021 at 8:53 AM Eric xu wrote: >>>>> >>>>>> Hello, dear friends. I wonder there is any optimization to BBR , for >>>>>> the reason of BBR low bandwidth utilization in wireless network with high >>>>>> delay jitter. In my opinion, High delay jitter will cause BBR to >>>>>> underestimate the BDP, thus limit the send rate even if pipeline is >>>>>> not full. Right? >>>>>> So, I wonder, *if there is any optimization to improve bw >>>>>> utilization in wireless network with high delay jitter, even in 5G network? >>>>>> *Thx a lot. >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "BBR Development" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to bbr-dev+u...@googlegroups.com. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/bbr-dev/bc675976-45f5-4ef6-b337-e693e4d21c6en%40googlegroups.com >>>>>> >>>>>> . >>>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "BBR Development" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to bbr-dev+unsubscribe@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/bbr-dev/303da319-87dd-46c0-928f-cad7f93835f8n%40googlegroups.com >>>> >>>> . >>>> >>>> >>>> -- >>>> Latest Podcast: >>>> https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/ >>>> >>>> Dave Täht CTO, TekLibre, LLC >>>> _______________________________________________ >>>> Make-wifi-fast mailing list >>>> Make-wifi-fast@lists.bufferbloat.net >>>> https://lists.bufferbloat.net/listinfo/make-wifi-fast >>> >>> >>> This electronic communication and the information and any files >>> transmitted with it, or attached to it, are confidential and are intended >>> solely for the use of the individual or entity to whom it is addressed and >>> may contain information that is confidential, legally privileged, protected >>> by privacy laws, or otherwise restricted from disclosure to anyone else. If >>> you are not the intended recipient or the person responsible for delivering >>> the e-mail to the intended recipient, you are hereby notified that any use, >>> copying, distributing, dissemination, forwarding, printing, or copying of >>> this e-mail is strictly prohibited. If you received this e-mail in error, >>> please return the e-mail to the sender, delete it from your computer, and >>> destroy any printed copy of it. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "BBR Development" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to bbr-dev+unsubscribe@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/bbr-dev/CAHb6Lvroqyg22Ax6ChOPYicYTjm_GnFapm8iX_5ae5EbvVEBzQ%40mail.gmail.com >>> >>> . >>> >> > This electronic communication and the information and any files > transmitted with it, or attached to it, are confidential and are intended > solely for the use of the individual or entity to whom it is addressed and > may contain information that is confidential, legally privileged, protected > by privacy laws, or otherwise restricted from disclosure to anyone else. If > you are not the intended recipient or the person responsible for delivering > the e-mail to the intended recipient, you are hereby notified that any use, > copying, distributing, dissemination, forwarding, printing, or copying of > this e-mail is strictly prohibited. If you received this e-mail in error, > please return the e-mail to the sender, delete it from your computer, and > destroy any printed copy of it.