From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-x233.google.com (mail-qt0-x233.google.com [IPv6:2607:f8b0:400d:c0d::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 811C33B2A4 for ; Tue, 12 Jun 2018 07:25:16 -0400 (EDT) Received: by mail-qt0-x233.google.com with SMTP id y89-v6so23269793qtd.10 for ; Tue, 12 Jun 2018 04:25:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=AT9UFQtkYnBnYH2iH9SmPzHQIuTVJR7OEJFEZEbweCc=; b=Ra+v1dDimn+Qk3bXo4BVyNtnJcfza9GyzsFiXdOa+lgObVii8FhkeA/eWVQLscoXvA VC3uP0/fxz9mCNmxf/zcG1n7V+pvLIL6e1gxA7btjLFekWdgdmF1MqQbkdO6K5hbE7YM VFpyJRNoKuEGgyB4ZLSTvaGW3u67gZSo55UUVBJOdWQrMu1+4MpC+M5FwUPAjKFoucG1 RNhwlLaANYWtMCRetd2ubIBRb9s3k+8SFLcJKUaH8LNTVY6xfTMfEWZ/0sLO0i7Btimh tt/xzXxejAs9HBZ6cpoHrUPYtsLfjU1drKtTVLfW2Z8yAUfeXkLaZc1HrB2ZMON7p3ZG hF0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=AT9UFQtkYnBnYH2iH9SmPzHQIuTVJR7OEJFEZEbweCc=; b=iN7U200Ed+vc25Wdk38pM54ul+3PeN7twIHiX7E7Am2jJ4ieHnGLoncK1E9RzvxwOL Vvyz8MqnX+8PDs1qp/yCoLlVKuKecWjDDmoLBdApnyaTgUia5GoD03B2m/M3xdh/HAgm GV9bBRSkiXksXu9TAsfMSzm1OPTe8oFwQZye0d4J1Xtp+wYA+ZrnM+V4iMD9Nho0+J6v 9/Q1h183gP3iYXF2Ir4CVmnc/r9qtV9cqmGz2DpdQ8wrEliH5aIItWsr/UAId2uDTgu2 J71jtZSxaz6baE/zNl9vXrj0hpmk4avKlEfwOzkRNDKjyjGa7jmdfaPJyBouI7gsBDcE 8g6g== X-Gm-Message-State: APt69E2ndxmxKHkCAyXrz6jwK51dteK8ODYBt2V2ncmat0jQJaXeDqDF lBu349KmEC4wAwARi4p9IXzUOog92c+5MkfzOVEYEw== X-Google-Smtp-Source: ADUXVKKStcgELQpoK9bT024P5F4SPpm0et4hHIgXpt2d1cMu273J1xE5q0bBlq0NzmB8fuNVneEPoU3KI7GnGSJSCDw= X-Received: by 2002:a0c:93a2:: with SMTP id f31-v6mr2875830qvf.151.1528802716010; Tue, 12 Jun 2018 04:25:16 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:aed:24f0:0:0:0:0:0 with HTTP; Tue, 12 Jun 2018 04:25:15 -0700 (PDT) In-Reply-To: <345707C0-F15A-4CBB-A224-27C14BFEC0DF@apnic.net> References: <345707C0-F15A-4CBB-A224-27C14BFEC0DF@apnic.net> From: Dave Taht Date: Tue, 12 Jun 2018 04:25:15 -0700 Message-ID: To: Geoff Huston Cc: Kevin Darbyshire-Bryant , bloat Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Bloat] geoff huston's take on BBR X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jun 2018 11:25:16 -0000 On Tue, Jun 12, 2018 at 12:49 AM, Geoff Huston wrote: > > >> On 12 Jun 2018, at 4:55 pm, Dave Taht wrote: >> >> On Mon, Jun 11, 2018 at 10:58 PM, Kevin Darbyshire-Bryant >> wrote: >>> >>> >>>> On 11 Jun 2018, at 22:27, Dave Taht wrote: >>>> >>>> https://ripe76.ripe.net/presentations/10-2018-05-15-bbr.pdf >>> >>> Fascinating! >>> >>> " =E2=80=A2 BBR changes all those assumptions, and could potentia= lly push many networks into sustained instability >>> >>> =E2=80=A2 =E2=80=93 We cannot use the conventional network cont= rol mechanisms to regulate BBR flows >>> >>> =E2=80=A2 Selective packet drop just won=E2=80=99t create back pressure= on the flow=E2=80=9D >>> >>> And I keep on seeing questions on whether BBR understands ECN - if not= =E2=80=A6. well I think we see the results. >> >> I think geoff goofed in his testing of BBR, starting all flows at the >> same time, thus syncing up their probing periods. Real traffic is not >> correlated this way. >> (I made the same mistake on my initial bbr testing) > > no - I started the flows at 10, 20 and 30 seconds after the initial flow = started. k. > >> >> I do agree that bbr treats aqm drops as "noise", not backpressure. And >> bbr scares me. >> >> I look forward very much to bbr one day soon doing some sort of sane, >> conservative, response to ecn marks. >> > > > I=E2=80=99m not sure that I understand this comment. > > Part of the pressure going on here is the issue of whether the endpoints = can and should trust the signals and.or manipulation that they get from the= network infrastructure. BBR is using a different form of feedback to contr= ol its send rate. Essentially BBR is taking a delay variance measurement 1 = / 8 of the time to adjust its internal model of the end-to-end delay bandwi= dth product (every 8th RTT). ECN provides a constant information flow, and = this certainly matches the requirements of loss-based TCP, where every ACK = contributes to the TCP flow dynamic, but it does not seem to me to be a goo= d match to BBR=E2=80=99s requirements. ECN CE is explicit. Someone on the path is saying "slow down". If BBR merely responded like cubic to an ECN mark, it would be a win. This of course doesn't handle the malicious middlebox or sender problem, but neither does anything else. I was pleased to see recently from https://www.ietf.org/proceedings/98/slides/slides-98-maprg-tcp-ecn-experien= ce-with-enabling-ecn-on-the-internet-padma-bhooma-00.pdf that france was showing 6% CE markings on uplnks. Since free.fr deployed fq_codel in 2011, I figure that's almost entirely from their deployment. I was boggled at the argentine result (30%???), although I know openwrt and derivatives with sqm is very popular there, that can't explain all of that. > The idea with BBR is to drive the network path such at the internal route= rs are sitting just at the initial onset of queuing. In theory ECN will not= trigger at the onset of queuing, but will trigger later in the cycle of qu= eue buildup. a few ms of queue is not a problem.... we (in the fq_codel world) are seeing 5ms queues and full throughput on ethernet, 15-20ms on wifi. (and BBR and cubic share almost equally, all the time, at least at the range of rates we've tested) Wifi: https://arxiv.org/pdf/1703.00064.pdf Cake shaper: https://arxiv.org/abs/1804.07617 There's a paper with the BBR vs cubic vs fq_codel result around here somewh= ere. I am painfully aware that upgrading edge routers and other bottleneck devices to have smart queue management is a long game... ... but I long ago gave up on pure end to end approaches. > > >> PS having fq on the link makes cubic and bbr cohabitate just fine. >> fq_codel vs bbr behavior was reasonable, though bbr lost a lot more >> packets before finding a decent state. > > I guess that "It Depends" - my long delay experiments in the presentation= referenced above showed cubic being completely crowded out by BBR. Please give sqm or cake a shot. > > Geoff > --=20 Dave T=C3=A4ht CEO, TekLibre, LLC http://www.teklibre.com Tel: 1-669-226-2619