[Make-wifi-fast] Fwd: [bbr-dev] BBR performance and optimization in cellular wireless network with high delay jitter

[Dave Taht]

eric, I thought I'd forward this to one of the bufferbloat.net lists. Feel
free to join. It would also be helpful to duplicate your test scripts here.

I agree with you, bbr is way too slow here. We see it bloat massively in
wifi also. I would like BBR to far more reactive than it is, and also for
it to respond to RFC3168 ECN marks.

You said:

*" *In my opinion, in high bw jitter scenario, window based bw undaption
method is too slow to converge to the actual bottleneck bw,  when actual bw
imcreases, BBR needs at lease 8RTT to catch it, when it decreases, BBR then
needs at most 10 RTT to expire the previous bw maximum, anyway it is too
slow, causing severe buffer-bloating. "

You might find the "flent" tool very useful for getting a more microscopic
view as to what is going on.

A test we would use to get a better picture of what is really going on
would be

flent -H netperf_test_server -x --socket-stats --step-size=.05 -t
the_test_conditions --te=cc_algo=bbr --te=upload_streams=1 tcp_nup

This samples a ton of tcp related stats and lets you plot them with
flent-gui, there are 110 other tests in the suite.

A more complicated invocation can get qdisc stats from the router involved,
or iterate over CCs, or run at longer intervals. In general I have found
running tests for 5 minutes or longer reveal interesting behaviors.

flent -H $S --socket-stats -x --step-size=.05 -t
1-flow-fq_starlink_noecn-${M}-c

c=${CC} --te=cpu_stats_hosts=$R --te=netstat_hosts=$R -4
--te=qdisc_stats_hosts=

$RS --te=qdisc_stats_interfaces=$RV --te=upload_streams=1 --te=cc_algo=$CC
tcp_1

up


pic below

---------- Forwarded message ---------
From: Eric xu <ericbin1224 at gmail.com>
Date: Mon, Jul 5, 2021 at 12:27 AM
Subject: Re: [bbr-dev] BBR performance and optimization in cellular
wireless network with high delay jitter
To: BBR Development <bbr-dev at googlegroups.com>


Hi, Neal, thanks for your answer. I still  have two questions to discuss
with you after having read the two links you mentioned above.

1, As mentioned in the second link, BBR's throughput on high-jitter links
(especially in cellular) usually matches CUBIC,  as  demonstrated by the
following figure,
[image: 截图.PNG]
however, what we can infer from the figure is that, the mean
throughput(~14Mbps) is far from the available bw that is greater than
20Mbps as showed in the cumulative distrubution figure. Since CUBIC back
off frequently in the LTE network with high error rate, but BBR does not
acquire much more throughput than CUBIC, the only reason I can imagine is
that BBR is highly affected by high delay jitter although cwnd provision is
applied.

*Is my understanding right? and is there any optimization to further
improve thoughput in high bw and delay jitter scenario? *

*2,  Is there any method to enhance BBR's adaptive ability to high bw and
delay jitter?  *In my opinion, in high bw jitter scenario, window based bw
undaption method is too slow to converge to the actual bottleneck bw,  when
actual bw imcreases, BBR needs at lease 8RTT to catch it, when it
decreases, BBR then needs at most 10 RTT to expire the previous bw maximum,
anyway it is too slow, causing severe buffer-bloating.  In high delay
jitter scenario, BBR also will cause bw under utilizaion. *So,  what is
your latest optimizaion to improve the performance in such a high bw and
delay jitter network?*

Thx a lot,
Eric
在2021年7月4日星期日 UTC+8 下午9:59:57<Neal Cardwell> 写道：

> Yes, BBR includes mechanisms to deal with this kind of jitter. Please see
> the March 2019 thread on this topic:
>   https://groups.google.com/g/bbr-dev/c/kBZaq98xCC4
>
> Our experience is that BBR's throughput on high-jitter links (cellular,
> wifi, DOCSIS, datacenter ethernet w/ GRO/LRO) is quite good at this point
> (usually matches CUBIC), and there are others who have documented this as
> well, e.g.:
>   http://web.cs.wpi.edu/~claypool/papers/driving-bbr/
>
> best,
> neal
>
>
> On Sun, Jul 4, 2021 at 8:53 AM Eric xu <ericb... at gmail.com> wrote:
>
>> Hello, dear friends. I wonder there is any optimization to BBR ,  for the
>> reason of BBR low bandwidth utilization in wireless network with high delay
>> jitter.  In my opinion,  High delay jitter will cause BBR to underestimate
>> the BDP, thus limit the send rate even if  pipeline is not full. Right?
>> So,  I wonder, *if there is any optimization to improve bw utilization
>> in wireless network with high delay jitter, even in 5G network? *Thx a
>> lot.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "BBR Development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to bbr-dev+u... at googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/bbr-dev/bc675976-45f5-4ef6-b337-e693e4d21c6en%40googlegroups.com
>> <https://groups.google.com/d/msgid/bbr-dev/bc675976-45f5-4ef6-b337-e693e4d21c6en%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
> --
You received this message because you are subscribed to the Google Groups
"BBR Development" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to bbr-dev+unsubscribe at googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/bbr-dev/303da319-87dd-46c0-928f-cad7f93835f8n%40googlegroups.com
<https://groups.google.com/d/msgid/bbr-dev/303da319-87dd-46c0-928f-cad7f93835f8n%40googlegroups.com?utm_medium=email&utm_source=footer>
.


-- 
Latest Podcast:
https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/

Dave Täht CTO, TekLibre, LLC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/make-wifi-fast/attachments/20210705/332c5e21/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 截图.PNG
Type: image/png
Size: 52872 bytes
Desc: not available
URL: <https://lists.bufferbloat.net/pipermail/make-wifi-fast/attachments/20210705/332c5e21/attachment-0001.png>