BBR performance on LTE

BBR performance on LTE

[Azin Neishaboori]

[-- Attachment #1.1: Type: text/plain, Size: 1100 bytes --]

Dear all
I know I have already asked a question recently. But I hope you consider
reading this question:

Consider the following setup: I have a laptop connected on 100Mbps Ehternet
to a router equipped with an LTE SIM and antennas. I run flent tests on an
Ubuntu VM on my laptop.I am trying to assess/compare performance of
different algorithms on bufferbloat mitigation on LTE links. FQ_Codel when
applied, was applied on the router box itself directly.

The results for TCP BBR in any combination seem really underwhelming. I
turned BBR on on my VM manually, by going to /etc/sysctl.conf and adding
net.ipv4.tcp_congestion_control=bbr and net.core.default_qdisc=fq. Then I
issue a sysctl -p to refresh.

I ran flent rrul test, and also in some tests used netem to induce packet
loss. With or without induced loss, BBR still underperformed. Attached
please see the graphs. Is this expected? Has anyone seen something like
this? I would really appreciate any insight you might have into this.

Thanks a lot
Best
Azin
[image: image.png]

Here is with induced loss (1%, 2% and 4%):
[image: image.png]

[-- Attachment #1.2: Type: text/html, Size: 1497 bytes --]

[-- Attachment #2: image.png --]
[-- Type: image/png, Size: 121929 bytes --]

[-- Attachment #3: image.png --]
[-- Type: image/png, Size: 126409 bytes --]

Re: BBR performance on LTE

[Azin Neishaboori]

[-- Attachment #1.1: Type: text/plain, Size: 2285 bytes --]

Dear researchers,
I would greatly appreciate your insight into the following problem:

Consider the following setup: I have a laptop connected on 100Mbps Ehternet
to a router equipped with an LTE SIM and antennas. I run flent tests on an
Ubuntu VM on my laptop.I am trying to assess/compare performance of
different algorithms on bufferbloat mitigation on LTE links. FQ_Codel when
applied, was applied on the router box itself directly.

The results for TCP BBR in any combination seem really underwhelming. I
turned BBR on on my VM manually, by going to /etc/sysctl.conf and adding
net.ipv4.tcp_congestion_control=bbr and net.core.default_qdisc=fq. Then I
issue a sysctl -p to refresh.

I ran flent rrul test, and also in some tests used netem to induce packet
loss. With or without induced loss, BBR still underperformed. Attached
please see the graphs. Is this expected? Has anyone seen something like
this? I would really appreciate any insight you might have into this.

[image: image.png]
[image: image.png]


Thanks a lot
Best
Azin

On Fri, Jan 18, 2019 at 5:00 PM Azin Neishaboori <azin.neishaboori@gmail.com>
wrote:

> Dear all
> I know I have already asked a question recently. But I hope you consider
> reading this question:
>
> Consider the following setup: I have a laptop connected on 100Mbps
> Ehternet to a router equipped with an LTE SIM and antennas. I run flent
> tests on an Ubuntu VM on my laptop.I am trying to assess/compare
> performance of different algorithms on bufferbloat mitigation on LTE links.
> FQ_Codel when applied, was applied on the router box itself directly.
>
> The results for TCP BBR in any combination seem really underwhelming. I
> turned BBR on on my VM manually, by going to /etc/sysctl.conf and adding
> net.ipv4.tcp_congestion_control=bbr and net.core.default_qdisc=fq. Then I
> issue a sysctl -p to refresh.
>
> I ran flent rrul test, and also in some tests used netem to induce packet
> loss. With or without induced loss, BBR still underperformed. Attached
> please see the graphs. Is this expected? Has anyone seen something like
> this? I would really appreciate any insight you might have into this.
>
> Thanks a lot
> Best
> Azin
> [image: image.png]
>
> Here is with induced loss (1%, 2% and 4%):
> [image: image.png]
>
>
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 3228 bytes --]

[-- Attachment #2: image.png --]
[-- Type: image/png, Size: 121929 bytes --]

[-- Attachment #3: image.png --]
[-- Type: image/png, Size: 126409 bytes --]

[-- Attachment #4: image.png --]
[-- Type: image/png, Size: 114251 bytes --]

[-- Attachment #5: image.png --]
[-- Type: image/png, Size: 114251 bytes --]

Re: BBR performance on LTE

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 784 bytes --]

I am a huge believer in also seeing the baseline *.flent.gz files and other
plots you can get via the flent-gui... if you could post them somewhere or
send to me privately? Two examples are obtaining the baseline RTT,
and it would be my hope, over time, that BBR would have found the baseline
RTT later in the test (the up or down bandwidth plots) than is shown by
these summary plots. Also as much as I love the rrul tests for quickly
showing the characteristics of a link under load, BBR makes the assumption
that flows start up one at a time, and a better string of basic tests would
be to use tcp_nup or tcp_ndown.

From squinting... It looks like the *baseline* RTT in this test is ~200ms.
There really isn't much in this world that works particularly well with
RTTs this large.

[-- Attachment #2: Type: text/html, Size: 866 bytes --]

Re: BBR performance on LTE

[Azin Neishaboori]

[-- Attachment #1: Type: text/plain, Size: 1587 bytes --]

Dear Dave

Thank you for responding. I will email you the *.flent.gz files tomorrow
and share the other plots here. Thank you for the advice to run tcp_nup and
down tests. I can do those and report as well tomorrow.


Regarding your comment on the 200ms delay, well, the bbr paper published by
the google team does mention the wifi and cellular LTE links. And the LTE
links do have as documented higher delays, even higher under mobility. Yet
the bbr paper claims that bbr works well for them as well.  But the LTE
test results I have got so far do not seem very promising.

Thank you again for taking time and responding to my email. Tomorrow I will
send further info as mentioned above.

Best
Azin

On Tue, Jan 22, 2019 at 4:07 PM Dave Taht <dave.taht@gmail.com> wrote:

> I am a huge believer in also seeing the baseline *.flent.gz files and
> other plots you can get via the flent-gui... if you could post them
> somewhere or send to me privately? Two examples are obtaining the baseline
> RTT,
> and it would be my hope, over time, that BBR would have found the baseline
> RTT later in the test (the up or down bandwidth plots) than is shown by
> these summary plots. Also as much as I love the rrul tests for quickly
> showing the characteristics of a link under load, BBR makes the assumption
> that flows start up one at a time, and a better string of basic tests would
> be to use tcp_nup or tcp_ndown.
>
> From squinting... It looks like the *baseline* RTT in this test is
> ~200ms.  There really isn't much in this world that works particularly well
> with RTTs this large.
>

[-- Attachment #2: Type: text/html, Size: 1979 bytes --]

Re: BBR performance on LTE

[Dave Taht]

btw, I didn't even remember we had a bloat-devel mailing list. please
use bloat instead

On Tue, Jan 22, 2019 at 1:59 PM Azin Neishaboori
<azin.neishaboori@gmail.com> wrote:
>
> Dear Dave
>
> Thank you for responding. I will email you the *.flent.gz files tomorrow and share the other plots here. Thank you for the advice to run tcp_nup and down tests. I can do those and report as well tomorrow.
>
>
> Regarding your comment on the 200ms delay, well, the bbr paper published by the google team does mention the wifi and cellular LTE links. And the LTE links do have as documented higher delays, even higher under mobility. Yet the bbr paper claims that bbr works well for them as well.  But the LTE test results I have got so far do not seem very promising.
>
> Thank you again for taking time and responding to my email. Tomorrow I will send further info as mentioned above.
>
> Best
> Azin
>
> On Tue, Jan 22, 2019 at 4:07 PM Dave Taht <dave.taht@gmail.com> wrote:
>>
>> I am a huge believer in also seeing the baseline *.flent.gz files and other plots you can get via the flent-gui... if you could post them somewhere or send to me privately? Two examples are obtaining the baseline RTT,
>> and it would be my hope, over time, that BBR would have found the baseline RTT later in the test (the up or down bandwidth plots) than is shown by these summary plots. Also as much as I love the rrul tests for quickly showing the characteristics of a link under load, BBR makes the assumption that flows start up one at a time, and a better string of basic tests would be to use tcp_nup or tcp_ndown.
>>
>> From squinting... It looks like the *baseline* RTT in this test is ~200ms.  There really isn't much in this world that works particularly well with RTTs this large.



-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: [Bloat] BBR performance on LTE

[Sebastian Moeller]

> On Jan 22, 2019, at 23:00, Dave Taht <dave.taht@gmail.com> wrote:
> 
> btw, I didn't even remember we had a bloat-devel mailing list. please
> use bloat instead

And I thought we use the bloat list to talk about bloat and how to get rid of it and use bloat-devel exclusively to talk about developing bloat. I took the radio silence as indicator, that we agree that there is sufficient "high-quality" bloat around that further development was not required yet... ;)



> 
> On Tue, Jan 22, 2019 at 1:59 PM Azin Neishaboori
> <azin.neishaboori@gmail.com> wrote:
>> 
>> Dear Dave
>> 
>> Thank you for responding. I will email you the *.flent.gz files tomorrow and share the other plots here. Thank you for the advice to run tcp_nup and down tests. I can do those and report as well tomorrow.
>> 
>> 
>> Regarding your comment on the 200ms delay, well, the bbr paper published by the google team does mention the wifi and cellular LTE links. And the LTE links do have as documented higher delays, even higher under mobility. Yet the bbr paper claims that bbr works well for them as well.  But the LTE test results I have got so far do not seem very promising.
>> 
>> Thank you again for taking time and responding to my email. Tomorrow I will send further info as mentioned above.
>> 
>> Best
>> Azin
>> 
>> On Tue, Jan 22, 2019 at 4:07 PM Dave Taht <dave.taht@gmail.com> wrote:
>>> 
>>> I am a huge believer in also seeing the baseline *.flent.gz files and other plots you can get via the flent-gui... if you could post them somewhere or send to me privately? Two examples are obtaining the baseline RTT,
>>> and it would be my hope, over time, that BBR would have found the baseline RTT later in the test (the up or down bandwidth plots) than is shown by these summary plots. Also as much as I love the rrul tests for quickly showing the characteristics of a link under load, BBR makes the assumption that flows start up one at a time, and a better string of basic tests would be to use tcp_nup or tcp_ndown.
>>> 
>>> From squinting... It looks like the *baseline* RTT in this test is ~200ms.  There really isn't much in this world that works particularly well with RTTs this large.
> 
> 
> 
> -- 
> 
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-205-9740
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] BBR performance on LTE

[Azin Neishaboori]

[-- Attachment #1: Type: text/plain, Size: 3114 bytes --]

Pardon my ignorance. As a late comer, I did not know the rules of the two
mailing lists. In fact I did not even know the bloat mailing list existed.
Only knew of the bloat-devel. I will use the former in the future.

Thanks again
Best
Azin

On Tue, Jan 22, 2019 at 5:27 PM Sebastian Moeller <moeller0@gmx.de> wrote:

>
>
> > On Jan 22, 2019, at 23:00, Dave Taht <dave.taht@gmail.com> wrote:
> >
> > btw, I didn't even remember we had a bloat-devel mailing list. please
> > use bloat instead
>
> And I thought we use the bloat list to talk about bloat and how to get rid
> of it and use bloat-devel exclusively to talk about developing bloat. I
> took the radio silence as indicator, that we agree that there is sufficient
> "high-quality" bloat around that further development was not required
> yet... ;)
>
>
>
> >
> > On Tue, Jan 22, 2019 at 1:59 PM Azin Neishaboori
> > <azin.neishaboori@gmail.com> wrote:
> >>
> >> Dear Dave
> >>
> >> Thank you for responding. I will email you the *.flent.gz files
> tomorrow and share the other plots here. Thank you for the advice to run
> tcp_nup and down tests. I can do those and report as well tomorrow.
> >>
> >>
> >> Regarding your comment on the 200ms delay, well, the bbr paper
> published by the google team does mention the wifi and cellular LTE links.
> And the LTE links do have as documented higher delays, even higher under
> mobility. Yet the bbr paper claims that bbr works well for them as well.
> But the LTE test results I have got so far do not seem very promising.
> >>
> >> Thank you again for taking time and responding to my email. Tomorrow I
> will send further info as mentioned above.
> >>
> >> Best
> >> Azin
> >>
> >> On Tue, Jan 22, 2019 at 4:07 PM Dave Taht <dave.taht@gmail.com> wrote:
> >>>
> >>> I am a huge believer in also seeing the baseline *.flent.gz files and
> other plots you can get via the flent-gui... if you could post them
> somewhere or send to me privately? Two examples are obtaining the baseline
> RTT,
> >>> and it would be my hope, over time, that BBR would have found the
> baseline RTT later in the test (the up or down bandwidth plots) than is
> shown by these summary plots. Also as much as I love the rrul tests for
> quickly showing the characteristics of a link under load, BBR makes the
> assumption that flows start up one at a time, and a better string of basic
> tests would be to use tcp_nup or tcp_ndown.
> >>>
> >>> From squinting... It looks like the *baseline* RTT in this test is
> ~200ms.  There really isn't much in this world that works particularly well
> with RTTs this large.
> >
> >
> >
> > --
> >
> > Dave Täht
> > CTO, TekLibre, LLC
> > http://www.teklibre.com
> > Tel: 1-831-205-9740
> > _______________________________________________
> > Bloat mailing list
> > Bloat@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat
>
> _______________________________________________
> Bloat-devel mailing list
> Bloat-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat-devel
>

[-- Attachment #2: Type: text/html, Size: 4285 bytes --]