From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf0-x229.google.com (mail-lf0-x229.google.com [IPv6:2a00:1450:4010:c07::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 9523C3B2A4 for ; Wed, 3 May 2017 22:13:07 -0400 (EDT) Received: by mail-lf0-x229.google.com with SMTP id r17so335402lfg.0 for ; Wed, 03 May 2017 19:13:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=lb5Sw0Sd+j51s4SvFy3wb0MtBJCPsMlXT/TkYeMSaKM=; b=YWvx9SYLi6phMs/SeIIYz/ustDuev7FamD03km1cfocGkOvmZmPgepChT9A22qNGxP 8Jmj/Ek8uyaXOZT25yN6AdsO3yUk/J+LnvOsX+ohoa6vBNeNxjOhzSI0hI+M+xcg492m K3v2O/NnuneiWsTFM7o2KY/a0pC4xQ70RCaYg9GmfgrwoHuZ8ylKj+gUQNGyWEAkg/FJ ME7XS04b3kHQSqVFfuus92awrF/ji76stRmlzSA5cdE139LDGmVWtf34zgLrBxqgTm4i i85xhFvJ+FUucDHqb+lIoCgbVnK6XDKECjYa0WXn7t3+W0BTJaQk63iL7A/8TCHha9p8 5qnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=lb5Sw0Sd+j51s4SvFy3wb0MtBJCPsMlXT/TkYeMSaKM=; b=bhvoyDhNGlbZbuqgdZIQriTbugX5gIyyYujxC+FRatcXP4MNjbxyPdFOZ7qGg7fmRb NGkrW5rSBMw9bvKQ7WHVdymlhCJaWERBJap/uVY21a2WIBrNQ89ejWnPMW6bGRpxVVHU ycrtoYwOWJRmPn5IUOeZZQTVJpOYia5Bv+DmPaHjzL+YeDhj6t3Bry8ueWS6R5PNaSkL P4TUy1fCWQX4byscCXZ5ugChFiYmuOyHMkppGkZ5dODEmsWcyU6g2iWDZCWupdf5cTrF wwAVOsN7bOfhUKp7cbhOxYjPAL5Kid+7iSEIBF7LAXtxSKT2XJZ2ZjGNX3N0Mkk5Mcfj r0sQ== X-Gm-Message-State: AN3rC/6BTlGUD6qZ7ZjMitJvZo2xY6tK2PJ5hoUTYpNRiBwHgcYki9Ue oPoECyfoyZjsZrZvYHKP7HRMHKdYNg== X-Received: by 10.25.87.12 with SMTP id l12mr12835086lfb.140.1493863986427; Wed, 03 May 2017 19:13:06 -0700 (PDT) MIME-Version: 1.0 Sender: gettysjim@gmail.com Received: by 10.25.77.79 with HTTP; Wed, 3 May 2017 19:13:05 -0700 (PDT) In-Reply-To: References: <85386ca9-f0be-f60f-796a-e5aa3b8ee212@gmail.com> <5ca09d5c-e674-110a-72e4-b8fd434c7a5d@gmail.com> From: Jim Gettys Date: Wed, 3 May 2017 22:13:05 -0400 X-Google-Sender-Auth: rig6J8km39-co32Xcd9aChON0-A Message-ID: To: Andy Furniss Cc: xnor , David Lang , Cake@lists.bufferbloat.net Content-Type: multipart/alternative; boundary=001a1141d1d840ffb5054ea95055 Subject: Re: [Cake] cake default target is too low for bbr? X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 May 2017 02:13:07 -0000 --001a1141d1d840ffb5054ea95055 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Wed, May 3, 2017 at 5:50 AM, Andy Furniss wrote: > Andy Furniss wrote: > >> Andy Furniss wrote: >> >> b) it reacts to increase in RTT. An experiment with 10 Mbps bottleneck, >>>> 40 ms RTT and a typical 1000 packet buffer, increase >>>> in RTT with BBR is ~3 ms while with cubic it is over 1000 ms. >>>> >>> >>> That is a nice aspect (though at 60mbit hfsc + 80ms bfifo I tested >>> with 5 tcps it was IIRC 20ms vs 80 for cubic). I deliberately test >>> using ifb on my PC because I want to pretend to be a router - IME >>> (OK it was a while ago) testing on eth directly gives different >>> results - like the locally generated tcp is backing off and giving >>> different results. >>> >> >> I retested this with 40ms latency (netem) with hfsc + 1000 pfifo on >> ifb. >> > > So, as Jonathan pointed out to me in another thread bbr needs fq and it > seems fq only wotks on root of a real eth, which means thay are invalid > tests. > =E2=80=8BSpecifically, BBR needs packet pacing to work properly: the algori= thm depends on the packets being properly paced. Today, fq is the only qdisc supporting pacing. The right answer would be to add packet pacing to cake/fq_codel directly. Until that is done, we don't know how BBR will work in our world. - Jim=E2=80=8B > > I will soon (need to find a crossover cable!) be able to see using a > third sender how cake varies shaping bbr in simulated ingress. > > I can test now how bbr fills buffers - some slightly strange results, > one netperf ends up being "good" =3D buffer only a few ms. > > 5 netperfs started together are not so good but nothing like cubic. > > 5 netperfs started with a gap of a second or two are initially terrible, > filling the buffer for about 30 seconds, then eventually falling back to > lower occupancy. > > TODO - maybe this is a netperf artifact like bbr/fq thinks it is app > limited. > > The worse thing about bbr + longer RTT I see so far is that its design > seems to be to deliberately bork latency by 2x rtt during initial > bandwidth probe. It does drain afterwards, but for something like dash > generating a regular spike is not very game friendly and the spec > "boasts" that unlike cubic a loss in the exponential phase is ignored, > making ingress shaping somewhat less effective. > > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake > --001a1141d1d840ffb5054ea95055 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Wed, May 3, 2017 = at 5:50 AM, Andy Furniss <adf.lists@gmail.com> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l= eft:1px solid rgb(204,204,204);padding-left:1ex">And= y Furniss wrote:
Andy Furniss wrote:

b) it reacts to increase in RTT. An experiment with 10 Mbps bottleneck, 40 = ms RTT and a typical 1000 packet buffer, increase
in RTT with BBR is ~3 ms while with cubic it is over 1000 ms.

That is a nice aspect (though at 60mbit hfsc + 80ms bfifo I tested
with 5 tcps it was IIRC 20ms vs 80 for cubic). I deliberately test
using ifb on my PC because I want to pretend to be a router - IME
(OK it was a while ago) testing on eth directly gives different
results - like the locally generated tcp is backing off and giving
different results.

I retested this with 40ms latency (netem) with hfsc + 1000 pfifo on
ifb.

 So, as Jonathan pointed out to me in another thread bbr needs fq and it
seems fq only wotks on root of a real eth, which means thay are invalid
tests.

=E2=80=8BSpecifically, BBR needs packet pacing to work pr= operly: the algorithm depends on the packets being properly paced.

Today, fq = is the only qdisc supporting pacing. =C2=A0

The right answer would be to add = packet pacing to cake/fq_codel directly.=C2=A0 Until that is done, we don&#= 39;t know how BBR will work in our world.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - Jim=E2=80=8B

I will soon (need to find a crossover cable!) be able to see using a
third sender how cake varies shaping bbr in simulated ingress.

I can test now how bbr fills buffers - some slightly strange results,
one netperf ends up being "good" =3D buffer only a few ms.

5 netperfs started together are not so good but nothing like cubic.

5 netperfs started with a gap of a second or two are initially terrible, filling the buffer for about 30 seconds, then eventually falling back to lower occupancy.

TODO - maybe this is a netperf artifact like bbr/fq thinks it is app
limited.

The worse thing about bbr + longer RTT I see so far is that its design
seems to be to deliberately bork latency by 2x rtt during initial
bandwidth probe. It does drain afterwards, but for something like dash
generating a regular spike is not very game friendly and the spec
"boasts" that unlike cubic a loss in the exponential phase is ign= ored,
making ingress shaping somewhat less effective.
=

_______________________________________________
Cake mailing list
Cake@lists.= bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

--001a1141d1d840ffb5054ea95055--