From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ua0-x23c.google.com (mail-ua0-x23c.google.com [IPv6:2607:f8b0:400c:c08::23c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 33A553B260 for ; Fri, 25 Nov 2016 07:51:06 -0500 (EST) Received: by mail-ua0-x23c.google.com with SMTP id 51so20355666uai.0 for ; Fri, 25 Nov 2016 04:51:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version; bh=nBR90hj1L4YpgFZ77Nqdv24JJIz27X3+RemNFQ10trw=; b=WCht3erHiAyu26NXz1wnLm8BsMfAkymjuN9bIXYFRNFQYjnmCJEVZp8lbhRturVESP xi9KuCnWGWG2S3MJtspg4pSHxqZgQhaRKJBDqzQn5yEY/hcrfRUTGsaRaVJKXP+lxVUJ hF1wzV8rqXe3mhhA/jb/p5rmPp3P0gQ9tGLP4CBemk3J6NEmOf+4ZGQqYTmyrzgzLkUQ dlJG5WRcUS+2M0EAqBn0Ac7+3sfyc+9ReO76efrgCjYmieJXahEPe1yLwtlpB+jHB6DV puwwRx8I4tPRru1YSL/47RX57l2tGWvG+xlTg5nw+X6XjDXo7aItMGlPDupqERZ3PumN cR6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to :references:subject:mime-version; bh=nBR90hj1L4YpgFZ77Nqdv24JJIz27X3+RemNFQ10trw=; b=MqqroCjFGSV7iuUBYKF+Ci3SYF91rJMljTh60dZs4BH89bn1zDfq/tlYg26SNLaag1 XaTV7xE4oDkNC494s8roaHWpV9CYxLJFZPMcvnk11g9gHR8GB37Wz3soRKdUpu09zsh4 jwKdh8M6qxT4qeGn87869eHPcZ7t4GxZPKvve6S9vVKUh/wgE/A0KJc2tkrT3CwsCDw8 d3AN40gN+nzFHSweF1kdEcVANRu7cif8ZB7DsclW/9n83MbvjCTo9C3AgWH822L9QOPM M7U3rxEJuAADPMd35P5D06rQ37kBHHsQue/HzRwn5J7wddSSAjR8N1o5J01vqEJnbJTC 2soA== X-Gm-Message-State: AKaTC01Ol2GiF6IOMotoAhffgFOiwYpcFMhy9chwT3klebe7ChzMkm2Tdq3Wm1n1dcx3NDU+GA53 X-Received: by 10.157.11.120 with SMTP id p53mr205531otd.19.1480078265771; Fri, 25 Nov 2016 04:51:05 -0800 (PST) X-Google-Already-Archived: Yes X-Google-Already-Archived-Group-Id: 8d948487e3 X-Google-Doc-Id: b53d918e02786 X-Google-Thread-Id: fbfdcee10f790e0e X-Google-Message-Url: http://groups.google.com/group/bbr-dev/msg/b53d918e02786 X-Google-Thread-Url: http://groups.google.com/group/bbr-dev/t/fbfdcee10f790e0e X-Google-Web-Client: true Date: Fri, 25 Nov 2016 04:51:05 -0800 (PST) From: Bernd Paysan To: BBR Development Cc: ycheng@google.com, sgunderson@bigfoot.com, bloat@lists.bufferbloat.net Message-Id: In-Reply-To: References: <20161021084726.GA1913@sesse.net> <20161027170447.GA28383@sesse.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_10310_1896661776.1480078265357" X-Google-Token: ELnn4MEFJPrjReoLPog0 X-Google-IP: 2003:86:2e18:c00:bce5:1872:57f4:61d6 X-Mailman-Approved-At: Wed, 07 Dec 2016 00:12:34 -0500 Subject: Re: [Bloat] "BBR" TCP patches submitted to linux kernel X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2016 12:51:06 -0000 ------=_Part_10310_1896661776.1480078265357 Content-Type: multipart/alternative; boundary="----=_Part_10311_759083588.1480078265357" ------=_Part_10311_759083588.1480078265357 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Am Donnerstag, 27. Oktober 2016 20:14:42 UTC+2 schrieb Dave Taht: > > On Thu, Oct 27, 2016 at 10:57 AM, Yuchung Cheng > wrote: > Well, against cubic on the same link in single queue mode, even > without ecn, life looks like this: > > > http://blog.cerowrt.org/flent/bbr-ecncaps/bandwidth-share-creaming-cubic-flowblind-aqm.svg > > but fq_codel is fine, so long as there is no ecn vs nonecn collission > > http://blog.cerowrt.org/flent/bbr-ecncaps/bandwidth-share-ecn-fq.png Looks like you are on the right track. I've been done some work on flow/congestion control on my net2o protocol, which is not limited by TCP's constraints, and I think you should look at what I did. See my 31c3 presentation. Flow/congestion control discussion starts at 46:00, page 79 on the slides. https://fossil.net2o.de/net2o/doc/trunk/wiki/31c3.md The primary algorithm is really simple and straight-forward: I send a short burst out, and measure the bandwidth achieved on the receiver side - this time is used to calculate the delays between two bursts, the average sending rate. In TCP, you would measure the ACKs back at the sender, and calculate bandwidth achieved based on the delays between the acks; that should be a bit less precise, but good enough. These bursts allow to adapt to changing loads quickly, and there is no need to fill the buffer at all (you don't need to create more increase in RTD than those few packets in the burst take). This primary algorithm is ignorant of the buffer fill state, so it works together with a buffer filled up by normal TCP congestion control. I took a great deal at providing fairness even without a fair queuing policy (much more complicated second order regulation; I'm still not happy with the results and trying to figure out something better), therefore, FQ is really necessary, everywhere, including on the last mile routers (see timings of 4 concurrent streams on page 108 w/o FQ and 109 w. net2o's own FQ). The spikes in the last diagrams result from the receivers regularly writing the data away to disk, and when they do so, temporarily stopping the transmission of their stream for a short period of time. That shows how fast the bursts can react on changed bandwidth. With just one client, I can hide that delay with a second thread, but with 4 clients on one CPU, it just shows up. ------=_Part_10311_759083588.1480078265357 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Am Donnerstag, 27. Oktober 2016 20:14:42 UTC+2 schrieb Dav= e Taht:
On Thu, Oct 27, 2016 at= 10:57 AM, Yuchung Cheng <ych...@google.com> wrote:
Well, against cubic on the same link in single queue mode, even
without ecn, life looks like this:

http://blog.cerowrt.org/flent= /bbr-ecncaps/bandwidth-share-creaming-cubic-flowblind-aqm.sv= g

but fq_codel is fine, so long as there is no ecn vs nonecn collission

http://blog.cerowrt.= org/flent/bbr-ecncaps/bandwidth-share-ecn-fq.png
=C2=A0
Looks like you are on the right track. I= 've been done some work on flow/congestion control on my net2o protocol= , which is not limited by TCP's constraints, and I think you should loo= k at what I did. See my 31c3 presentation.

Flow/co= ngestion control discussion starts at 46:00, page 79 on the slides.

https://fossil.net2o.de/net2o/doc/trunk/wiki/31c3.md
<= /div>

The primary algorithm is really simple and straigh= t-forward: I send a short burst out, and measure the bandwidth achieved on = the receiver side - this time is used to calculate the delays between two b= ursts, the average sending rate. =C2=A0In TCP, you would measure the ACKs b= ack at the sender, and calculate bandwidth achieved based on the delays bet= ween the acks; that should be a bit less precise, but good enough. =C2=A0Th= ese bursts allow to adapt to changing loads quickly, and there is no need t= o fill the buffer at all (you don't need to create more increase in RTD= than those few packets in the burst take). =C2=A0This primary algorithm is= ignorant of the buffer fill state, so it works together with a buffer fill= ed up by normal TCP congestion control.

I took a g= reat deal at providing fairness even without a fair queuing policy (much mo= re complicated second order regulation; I'm still not happy with the re= sults and trying to figure out something better), therefore, FQ is really n= ecessary, everywhere, including on the last mile routers (see timings of 4 = concurrent streams on page 108 w/o FQ and 109 w. net2o's own FQ). =C2= =A0The spikes in the last diagrams result from the receivers regularly writ= ing the data away to disk, and when they do so, temporarily stopping the tr= ansmission of their stream for a short period of time. =C2=A0That shows how= fast the bursts can react on changed bandwidth. =C2=A0With just one client= , I can hide that delay with a second thread, but with 4 clients on one CPU= , it just shows up.
------=_Part_10311_759083588.1480078265357-- ------=_Part_10310_1896661776.1480078265357--