From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-x23e.google.com (mail-pf0-x23e.google.com [IPv6:2607:f8b0:400e:c00::23e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 550533B2CF; Thu, 22 Sep 2016 08:09:17 -0400 (EDT) Received: by mail-pf0-x23e.google.com with SMTP id 21so36183054pfy.0; Thu, 22 Sep 2016 05:09:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version; bh=FwZlvB2/HyFHZT0WeovwZLeKSeXsD6E/v4HAfeVNHQw=; b=J6Yl0m1/sV0iSAoxVUg2fSNaynKgY9l+M/UtWPIIhkNlXduqMy1DxDvfxKVNFO69mj ILXv3EbrBrceqKoijzQvIyekuunW7gPLT+K55MuLjMqEj1iT5rVNNm62SerlHlmrGGkY uh4QqD8IrxZ3DlcWAZVVXr8ZnSZ2ZcyLG7WBsQuO5TjZoN6MT/jycLTqPn5guUJcjNVi ekOfvPOkPT312PA74Er/lI6qmqZlq+O943iVZPkDDVxEizJ/xfmVgnYBeBe12ifZPaKM CXINq3UcSqcDIl+2skk/nUeH6gRFRxOKePQia/JDhssTfeGA17bJSebv+ghJ4i10gD7w NADw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to :references:subject:mime-version; bh=FwZlvB2/HyFHZT0WeovwZLeKSeXsD6E/v4HAfeVNHQw=; b=BD0oe7K7Mqhs3YHfWF/pj/6puuAK8jW06hTAX9Up2hthxBHdGR3OgiOfnLQoEorYHV zGQSQ8mtRtO721Fi4mG+Pe17RFgBXX+TxQvV4YrHYewfuNSDzkG8E3gECRPZwWOZjX07 +/rn2TK/iYca6j3U+nbeFJakd6Agcz3XRSc2bll0nDutkOtf6TbeObjUfspdhksEMkQv v8D12BRt+uF8XbovHIH88XZjOWmL/yQ988yPH8u/rbfMcPJDLjBA5o+RXLrk93iHOxjl TXVSpM7QiKh/GPIn5T7GtuDEzTizmiY8iHEg+76YXwIhd1AbJUurHkWAqRcnKruXKoCy ME9A== X-Gm-Message-State: AE9vXwP23l/ih9eOww+bU1dvnv/LOCXfgW1/+wbFC22JiK+5JhnMldQ+W3Vv+O9dw5YHOm+UPIyG X-Received: by 10.36.237.193 with SMTP id r184mr80946ith.2.1474546156599; Thu, 22 Sep 2016 05:09:16 -0700 (PDT) X-Google-Already-Archived: Yes X-Google-Already-Archived-Group-Id: 8d948487e3 X-Google-Doc-Id: cf8227070af5 X-Google-Thread-Id: 8f097305fe61b918 X-Google-Message-Url: http://groups.google.com/group/bbr-dev/msg/cf8227070af5 X-Google-Thread-Url: http://groups.google.com/group/bbr-dev/t/8f097305fe61b918 X-Google-Web-Client: true Date: Thu, 22 Sep 2016 05:09:15 -0700 (PDT) From: Alan Jenkins To: BBR Development Cc: swmike@swm.pp.se, bloat@lists.bufferbloat.net, cerowrt-devel@lists.bufferbloat.net Message-Id: In-Reply-To: References: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_575_470465246.1474546155930" X-Google-Token: EOuTj78FaQv_v5sMm8Q0 X-Google-IP: 89.243.172.136 Subject: Re: [Cerowrt-devel] taking apart BBR's behaviors in flent X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Sep 2016 12:09:17 -0000 ------=_Part_575_470465246.1474546155930 Content-Type: multipart/alternative; boundary="----=_Part_576_417684238.1474546155931" ------=_Part_576_417684238.1474546155931 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On Wednesday, 21 September 2016 20:25:32 UTC+1, Dave Taht wrote: > > > Looking at cake_flowblind_noecn, BBR1 and BBR4 just kills both CUBIC > flows. > > Same with PIE. > > Yep. The single queue AQMs expect their induced drops to matter to all > flows. BBR disregards them as noise. I think there's hope though, if > BBR can treat ECN CE as a clear indication of of congestion and not > filter it as it does drops. > Extra credit assignment: get the next version of DOCSIS PIE to turn on ECN? https://tools.ietf.org/html/draft-ietf-aqm-docsis-pie-02#section-4.7 > But cake/fq_codel is just fine with different cc's in the mix, and I'm > dying to look at the captures for what happens there. > Very glad to see that, I can keep using fq_codel :). > So it seems my intuition was wrong, at least for these scenarios. It > wasn't > > CUBIC that would kill BBR, it's the other way around. So (from the other thread) BBR is designed to use the traditional recommendation of 1 BDP's worth of buffer. In the absence of other CC's, it would limit itself to that. Understandable for bottlenecks in end-site modems or wifi. Shallower buffers cause somewhat increased packet loss (given multiple competing BBR streams). BBR is designed to survive this without difficulty (incurring retransmit latency). Competing loss-based CCs will suffer badly. The patch says it's designed to improve throughput "on today's high-speed long-haul links using commodity switches with shallow buffers" by not "[over-reacting] to losses caused by transient traffic bursts". If there is systemic congestion at those switches[1]... [1] ex https://www.ncta.com/sites/prod/files/MIT-Congestion-DC.pdf http://groups.csail.mit.edu/ana/Measurement-and-Analysis-of-Internet-Interconnection-and-Congestion-September2014.pdf ...I wait with interest to see what the ACM article says. My intuition was that "delay based TCPs can't work on the internet!" - > and was wrong, also. > > > Great to have testing > > tools! Thanks Flent! > > Thx, toke! I try not to remember just how hard it was to do this sort > of analysis on complex network flows when we started. > And thanks for the matrix of test results! It shows how powerful a tool it is, to see points raised so quickly. If I'm reading the drops graph right, I can see multi-second periods >= 10% packet loss when the buffer is limited to 25ms (bfifo_64k, bw=20Mbit-rtt=48ms-flows=2-noecn-bbr). Clearly explains why normal CUBIC gets crushed :). Alan ------=_Part_576_417684238.1474546155931 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
On Wednesday, 21 September 2016 20:25:32 UTC+1, Dave Taht = wrote:
> Looking at cake_fl= owblind_noecn, BBR1 and BBR4 just kills both CUBIC flows.
> Same with PIE.

Yep. The single queue AQMs expect their induced drops to matter to all
flows. BBR disregards them as noise. I think there's hope though, i= f
BBR can treat ECN CE as a clear indication of of congestion and not
filter it as it does drops.

Extra credit assig= nment: get the next version of DOCSIS PIE to turn on ECN?

https://to= ols.ietf.org/html/draft-ietf-aqm-docsis-pie-02#section-4.7
=C2=A0
<= blockquote class=3D"gmail_quote" style=3D"margin: 0;margin-left: 0.8ex;bord= er-left: 1px #ccc solid;padding-left: 1ex;"> But cake/fq_codel is just fine with different cc's in the mix, and I= 9;m
dying to look at the captures for what happens there.
<= div>
Very glad to see that, I can keep using fq_codel :).

<= blockquote class=3D"gmail_quote" style=3D"margin: 0;margin-left: 0.8ex;bord= er-left: 1px #ccc solid;padding-left: 1ex;">> So it seems my intuition w= as wrong, at least for these scenarios. It wasn't
> CUBIC that would kill BBR, it's the other way around.

So (from the other thread) BBR is designed to use the traditio= nal=20 recommendation of 1 BDP's worth of buffer.=C2=A0 In the absence of othe= r CC's,=20 it would limit itself to that.=C2=A0 Understandable for bottlenecks in end-= site modems or wifi.

Shallower buffers cause somewhat increased pack= et loss (given multiple competing BBR streams).=C2=A0 BBR is designed to su= rvive this without difficulty (incurring retransmit latency).=C2=A0 Competi= ng loss-based CCs will suffer badly.

The patch says it's designe= d to improve throughput "on today's high-speed long-haul links usi= ng commodity switches with shallow buffers" by not "[over-reactin= g] to losses caused by transient traffic bursts".

If there is s= ystemic congestion at those switches[1]...

[1] ex
https://www.nct= a.com/sites/prod/files/MIT-Congestion-DC.pdf
http://groups.csail.mit.edu= /ana/Measurement-and-Analysis-of-Internet-Interconnection-and-Congestion-Se= ptember2014.pdf

...I wait with interest to see what the ACM article = says.

My intuition was that "delay based TCPs can't work on the internet= !" -
and was wrong, also.

> Great to have testing
> tools! Thanks Flent!

Thx, toke! I try not to remember just how hard it was to do this sort
of analysis on complex network flows when we started.
<= div>
And thanks for the matrix of test results!

It shows how powe= rful a tool it is, to see points raised so quickly.

If I'm readi= ng the drops graph right, I can see multi-second periods >=3D 10% packet= loss when the buffer is limited to 25ms (bfifo_64k, bw=3D20Mbit-rtt=3D48ms= -flows=3D2-noecn-bbr).=C2=A0 Clearly explains why normal CUBIC gets crushed= :).

Alan
------=_Part_576_417684238.1474546155931-- ------=_Part_575_470465246.1474546155930--