From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-sg2apc01on0060.outbound.protection.outlook.com [104.47.125.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 40ED83BA8E for ; Tue, 12 Jun 2018 03:49:45 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apnic.onmicrosoft.com; s=selector1-apnic-net; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SemcAPBscbACyOiR5ssV3go8Hyknugf4zOpCBcdpAZQ=; b=NQzhk5RQrvCXi0NrDL1z0JBtz0zoM6UJ2WVDlVDznJJKVrNmwq8q7WaolQAQLedOSQKkthQZyvwkt+rKpFZDcxyKHjZ37Gi1cmznkGXQZz7RH44sHW36JnQyrnn172fvbrA3ylNgjIqrkTRy/pY3HpljGXZZuTpSL5raT/A3jNs= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=gih@apnic.net; Received: from 2001-44b8-1121-1a00-28c8-2ae5-406f-15dc.static.ipv6.internode.on.net (2001:44b8:1121:1a00:28c8:2ae5:406f:15dc) by SG2PR04MB1182.apcprd04.prod.outlook.com (2a01:111:e400:7810::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.841.17; Tue, 12 Jun 2018 07:49:40 +0000 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.4 \(3445.8.2\)) From: Geoff Huston In-Reply-To: Date: Tue, 12 Jun 2018 17:49:26 +1000 Cc: Kevin Darbyshire-Bryant , bloat Content-Transfer-Encoding: quoted-printable Message-Id: <345707C0-F15A-4CBB-A224-27C14BFEC0DF@apnic.net> References: To: Dave Taht X-Mailer: Apple Mail (2.3445.8.2) X-Originating-IP: [2001:44b8:1121:1a00:28c8:2ae5:406f:15dc] X-ClientProxiedBy: ME2PR01CA0171.ausprd01.prod.outlook.com (2603:10c6:220:20::15) To SG2PR04MB1182.apcprd04.prod.outlook.com (2a01:111:e400:7810::28) X-MS-PublicTrafficType: Email X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:SG2PR04MB1182; X-Microsoft-Exchange-Diagnostics: 1; SG2PR04MB1182; 3:+WVgXeuQJ8CZCtM/zbK78V5ZIA0cQQ8dzUaPReBbWcnhxb0q/9aTAcGi1AGZDy7ZWbgA/CHfdN7tPLEL+LSHflOWljZHo+cbVIB2i6BHqa+VNrS9628d59REQnpOdNLAmm9Om7ysb8vazdZMmEn+yffF/6Y+iyR6l3gpEZGKJ+mbHbsF+SYZL5BcXWUDd1H1olfB8zS3TnZlfeMOc/4z7TmjVhxG5tYEKu421vvgfajqlxFU48L2ZLdk3KFubznk; 25:Pocqxk/wFmIAkOjWC1q4Pkf4ggTRDjwBPTL6cfxxS/6Sg2fvhsHN/PZTp7UQ8/0KEBdiUGir8ECAMtDldwjXlyyPMTd7X4Gzd8xrZXG3O2DxCiYL1rjcD6cEuGAieakXZzzO6k9tAH/J5GY4DzwNfAAYAw+HwS/ax0h3owLk2A9t7u5YXAsbgGsE9zIpns0mKWT0d4QVgXPH7DfWP8Qqs36AORfxFszSOEsa6t2MrmvPWLDHGLi+LggxUI9vErgMW5XKUHNi6hEqHmZZ56K5WeGVz1d9zLOvuYyB/h0u9okQo/aIjznpQ7wcFT61ywEc4lpdPZNr0lCNfssf/Id4SA==; 31:smf/QwEJwdMj0wyATP5sQGPoOH0SioG78zOVKnu5h+gn4c4KHGoqAQxsyjXfqg4NNWDqCobRYzeTBmSY6qa7Rv3lQZyZS+MK8Iu+mmLTc/hH9jzN1GL+IqLa5PrJO4UB3Sh5A5i+CMDrpomxkxc+TopynvZLLEBGAm4W2FAJpAvt4z4bvqKcytyPHSZxPVaipOR0LlZAHBnX40fBL5V+9nqp6qYe6MJGkiqyXHSvcB8= X-MS-TrafficTypeDiagnostic: SG2PR04MB1182: X-Microsoft-Exchange-Diagnostics: 1; SG2PR04MB1182; 20:+tI6LowHzzlu585GcXKIsN04hpEI6iCyOCtwNQpCgvsXyz+yc+TRL5gM+R9MMjCDCvSz2dKqqvt+T53xrM7dk8G58Hofbr7EmzXguqx2he6gZ9fxRsrEWa0g9KV0y0Jozb3T3sYFZpCccLFMG5iXh7ojCeq6UsXPt6SG+TQVMzBcdL7aixxYshIduVt0/jFRRRrQ/9yoIfiwR6npedHch2A9LxNsI3hooAc0UqcEuRXbDmb7i7xcr31kqgD2TVOG; 4:ueKwDQ+bD+fXDAxuJCL4NtSxAbswqbyQisBGCD+dmuzFBQ6oMeOfJfMwWpxnlj2AJo4zGSuFeIq+GRScKIwtez9RRYSeikUbQyakv6zD3kGRZmqz2vsgzGobpwjEDgKxctcdJM1Mh+gcc/ZMWIgU/R5Nu0MGGL78zbqM8qleadxDZQmGGHCsN75+ockBY/7moORYcgMlryKrU3ysNlVhCb0XUakXc3s/glwMpNSDeY3J4lVy4X6W7ZoIzjhv/NlYsGu8FmSKB5CA3DUtNVobb+0m/zLUXeDw+xQBnLbxovo1Vkqt4ejNaCcw1ryIuJLJ X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(85827821059158); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(3231254)(944501410)(52105095)(93006095)(93001095)(10201501046)(3002001)(149027)(150027)(6041310)(20161123562045)(20161123564045)(201703131423095)(201703011903075)(201702281528075)(20161123555045)(201703061421075)(20161123558120)(20161123560045)(6072148)(201708071742011)(7699016); SRVR:SG2PR04MB1182; BCL:0; PCL:0; RULEID:; SRVR:SG2PR04MB1182; X-Forefront-PRVS: 07013D7479 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(7966004)(39380400002)(39850400004)(376002)(396003)(366004)(346002)(189003)(199004)(106356001)(57306001)(53546011)(966005)(508600001)(476003)(25786009)(68736007)(36756003)(50226002)(46003)(8936002)(50466002)(33656002)(81156014)(6116002)(81166006)(5660300001)(105586002)(316002)(83716003)(6666003)(486006)(53936002)(76176011)(6246003)(7736002)(6486002)(2906002)(52396003)(229853002)(2486003)(8676002)(6506007)(97736004)(4326008)(52146003)(23676004)(52116002)(186003)(305945005)(6916009)(6306002)(54906003)(386003)(8746002)(86362001)(39060400002)(2616005)(446003)(11346002)(16526019)(47776003)(82746002)(6512007)(42262002); DIR:OUT; SFP:1101; SCL:1; SRVR:SG2PR04MB1182; H:2001-44b8-1121-1a00-28c8-2ae5-406f-15dc.static.ipv6.internode.on.net; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: apnic.net does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtTRzJQUjA0TUIxMTgyOzIzOjh5MWpzR0hZbmVGczFjQXV6YXV5eHRZNDlV?= =?utf-8?B?UmdkWjJTZkl0NlBJZ3F4VE0yU2VOYkhlajVOWTJtajM5V2FEck1jT0dlY2ty?= =?utf-8?B?TUdNNXM0VXpkTHc1M0ZDd0RjY2pMTmlGNUx0R2lkU1RUQmpjZWhsQzZsb1dT?= =?utf-8?B?dmk1RGJOYlVLaDFuMkttSzFEUVV0SUN1K1pqbU5JQ05lTHRNRWFRdUppbVdQ?= =?utf-8?B?NFlQeTh3NERJQmJ3S2d3bzJmNHNhcFdudjFJODN2dSt5cGE2bElYaURqTktF?= =?utf-8?B?YjJya0pNMmJaOWFuZXN1VUdmNFcvTXJIUDdwTmIvZ2Y4Rkx3Vk1Yc3kwb2FX?= =?utf-8?B?b1h5L0o2VjZ1UGtsWkZiZ3p1KzRuQmlpempycXBKbmszRG42QW5vL0tobWFP?= =?utf-8?B?d0pPK0Y0WG4wa1hCMGtMOHA4Y1RpTW5uNXFnTUo3T2NlblJ3THk1VWdRMUFS?= =?utf-8?B?SWszdmdZbUo2dC9XVllKWjU3eEUzT0JmQ2RtNlEvOUJyR2dnNUxiMUxCMkI0?= =?utf-8?B?bW4rK0c0dUdzK25Ib2pFNVl6d1BKL1VBK1crT0t6YUhaMzZFbEtnclpKQUtr?= =?utf-8?B?TG5CdGFERTBTbDMzSXE3SjlLQkh3eVRQRWt6OXVkMEsxMjg3MDVwQ1JuV01q?= =?utf-8?B?bjBWM3ZkZDg5R0FQQjVRQXhZMVNvdks5RWZWTktqNUlBRHVZNDR4aDFWR2NP?= =?utf-8?B?NXlBR2lhVFBTbmc2czlpVy80RVZVS21haTJ3V0xTR3dKMUdHNHc0L0EyRlE2?= =?utf-8?B?K0xJMWpFV2dGWUhHNDFONEpqWi94MzNnRFpyN0lNeEw0MG9MM3NycENBemw2?= =?utf-8?B?a0poYTBGV1dNTWgzMGFjeFJsa1JTZnV5bHJ0N0FTaXg2d0prTDVDejB4cjIy?= =?utf-8?B?V3Q1WHV2aStXdEdLd21LRmVMRHVMNFR1UXVncENtakQvNTJGSjRpSVplQ1V6?= =?utf-8?B?S1VrWnlhL3pJTEFKSGdEMzBtNWZETm5QdThjUk1Ta2hET1d3L0JIZzA4eS9x?= =?utf-8?B?b1RUTXh2d1RPNys1RHU0VUNVVnd4RENTQTlPaksvQkdITnV0ZFdxVjRiRld0?= =?utf-8?B?bU9XZ3Q2NUg3U0NQMnQzMFN2cEUzQVlacUpKeUNRUTdEUjBnSlVrNlpiNEVY?= =?utf-8?B?bmpBTERaalJKdTQ0S0ZBZTZIQ29oMVpvWlNidGdmVDlJdTRjZ3NWZHZoOWFL?= =?utf-8?B?bmRubWNJS0JRdzJySGZvZWdvSkNOUkdrYjAxc3p1OGpxZ2RrY0VUNkt4emgv?= =?utf-8?B?eUJqeUJzQXZITnlXYktFbEhOM3F1My8rZ0hRU0JoOE5mMFV0QUZJMm9QWGdR?= =?utf-8?B?bCtlSXlXZzBCUGJndEtPNm4wMzdSVXZLUk9CQ2hmT09QOFlIQ1VFN05RcDFl?= =?utf-8?B?Mlk3WkE2QzRoUUpaSjIyUXVubElHWFUzaSs2TWxlakp6SVVFYURUWWszVzVN?= =?utf-8?B?Zk0yRjR5TlUySUYxejlIY0hWWHJIcFlCRURrYTdJN1dwaVVIWG5KblFhWWhi?= =?utf-8?B?OFlkSlVJeUxCTlRJT0dkVEhnczRUNTFWZjFYZExaZUdJMGNxUHJ3Q2NWZkxI?= =?utf-8?B?eW55Qkk5NTJLWm94OEUxNzF1WFlvVmNQTXV0K0tGdmVCOUlMSmV6TGJMSE15?= =?utf-8?B?KzN1TVJ4YkhFVUpVQlRTT3ZmS0p5bHU5aUNDeWt2amlxUTEzbWN2WUdrM0cw?= =?utf-8?B?a3N4bytFMlNCZHpLZlhVaUd5L0dZQmhsbS8wbzVudi9Bd0JZWTNlQTlJRXBP?= =?utf-8?B?VERqenlIY1FPczF3cVNxYTZ3SHhuU2oyRGtMeU9NWSt2VndLL0Q0eFh6cjk5?= =?utf-8?B?MFBsMzBLVVlRbmJ1WUFTdU5FUkVlY0tEcDBVd0o4VlZnZ1JDY3FXUzVMdUxS?= =?utf-8?B?MmRmMzJpTjNncXMrQ1ZQc0p4OTRYVktjZHNLVXFNYmRoTTFyV0ZLUUlDQm1G?= =?utf-8?Q?gaZkisGkVqQUsZbK6xNpzfugIHeGSc=3D?= X-Microsoft-Antispam-Message-Info: VNO+f07VRNdapb8Ev8I3MJDLy6zJVsEh/DkjrtVCVQKIzUaSXspJZA3xP65AfTfQ1UNVgV/qMA3urz0HBC5OzYuPv83RJCqmqys8NH511UGmz0Zacrs8oBSTYRVs7IJ1mG/n1gHEyVeLn72gs4c7Z9BgARbdqAbKw3XfajMLFczC6advCXzuoZwN7mqGQgRO X-Microsoft-Exchange-Diagnostics: 1; SG2PR04MB1182; 6:0M2ZPCfElNHFrISBiRN+OAeMxPLY5HIssXwH4aSCvi514amPdUWj+UgNSN389rntQ71IdtXLZRDwq/92JdaRj3FuiOIPi/WiTik8UP4BI+NCovNb2FwDx/CX+pN6mTlU1De6RNphuaR0oUu32i5LW7rZYcOzZTYzK4uuwWcckI0tC8cZ2RgvLbO0hejUkdBsaNX9HS00cToUcnWkb4YQN9FNYVd18gwxaKIL+hDL009q5Zr+bvMiRFp8f2spcgi/n/FCVDcOENUhEcRhW1xvWlZHOQGDkBCr501IFIkCTzNlX5ANn5DN8D5De9j9yBiYVLrSaag60FiAf+VlgHAcUJBHVJ1jb0k3xkupD6FgMH+QHQIarDPQ74j6RC+d2mMPJDv2mrF1PxD0qCCq559fVfyoLWRAkmNtpUcjy8QccqnlVL9xxdl9rfRKRCUXfzoq6Z+ML6BWPwV+E1bMdt7pYw==; 5:I0+I3SBBIMbgPV4TpXsp/fEHtlz+hzV0Fyz5JG/t8vpusHSoacBH3+/5bT3U0Q3u17RWQhHQ1YDYrFifVmKPA9XOi5D9D/jU9Kh6TYo+lDbPk+ms4eZUCCFbzla7GQUrZxRDoUi60qQtDWuADUthlgqWCgDj7qEYRbvHyP+Rv8U=; 24:sipjHHj7v7qcKElv4Hsz2Ge6WxvmcNMSo15Efd8PTx1ATR6S6OoJLAi2p3wrqVQf5ac1oCNvrrsOagClBp3WynOgiMm82uA1faK/cPKpN6o= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; SG2PR04MB1182; 7:3yRsvQUfAxm1oPr6xM6EpUnjk4BoB962y6q+CXE+7Nj8r0ipvArkvfphItiCfOvi+mQxbK1yKGlknldo2qUDTeYSa+Lpj5CDLH0CFdcqWkiYDnnyQjk/j6NmZFiRw1BWMLqvBsWx0sbkyLIBOtE/gCR5RzCW+93UVGLKf0/TlswoHoEdmZ14r/VR4AXbjCK6BJ8xkAsGwyJVUM0TaBa+E1WFlvTEiD4WO8Qikw7enwcDBUggmwktjwcjUaoPF2zM X-MS-Office365-Filtering-Correlation-Id: 55399f6c-0a4e-4ef5-6043-08d5d0390f01 X-OriginatorOrg: apnic.net X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jun 2018 07:49:40.7790 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 55399f6c-0a4e-4ef5-6043-08d5d0390f01 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 127d8d0d-7ccf-473d-ab09-6e44ad752ded X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2PR04MB1182 X-Mailman-Approved-At: Tue, 12 Jun 2018 10:38:57 -0400 Subject: Re: [Bloat] geoff huston's take on BBR X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jun 2018 07:49:46 -0000 > On 12 Jun 2018, at 4:55 pm, Dave Taht wrote: >=20 > On Mon, Jun 11, 2018 at 10:58 PM, Kevin Darbyshire-Bryant > wrote: >>=20 >>=20 >>> On 11 Jun 2018, at 22:27, Dave Taht wrote: >>>=20 >>> https://ripe76.ripe.net/presentations/10-2018-05-15-bbr.pdf >>=20 >> Fascinating! >>=20 >> " =E2=80=A2 BBR changes all those assumptions, and could = potentially push many networks into sustained instability >>=20 >> =E2=80=A2 =E2=80=93 We cannot use the conventional network = control mechanisms to regulate BBR flows >>=20 >> =E2=80=A2 Selective packet drop just won=E2=80=99t create back = pressure on the flow=E2=80=9D >>=20 >> And I keep on seeing questions on whether BBR understands ECN - if = not=E2=80=A6. well I think we see the results. >=20 > I think geoff goofed in his testing of BBR, starting all flows at the > same time, thus syncing up their probing periods. Real traffic is not > correlated this way. > (I made the same mistake on my initial bbr testing) no - I started the flows at 10, 20 and 30 seconds after the initial flow = started. >=20 > I do agree that bbr treats aqm drops as "noise", not backpressure. And > bbr scares me. >=20 > I look forward very much to bbr one day soon doing some sort of sane, > conservative, response to ecn marks. >=20 I=E2=80=99m not sure that I understand this comment. Part of the pressure going on here is the issue of whether the endpoints = can and should trust the signals and.or manipulation that they get from = the network infrastructure. BBR is using a different form of feedback to = control its send rate. Essentially BBR is taking a delay variance = measurement 1 / 8 of the time to adjust its internal model of the = end-to-end delay bandwidth product (every 8th RTT). ECN provides a = constant information flow, and this certainly matches the requirements = of loss-based TCP, where every ACK contributes to the TCP flow dynamic, = but it does not seem to me to be a good match to BBR=E2=80=99s = requirements.=20 The idea with BBR is to drive the network path such at the internal = routers are sitting just at the initial onset of queuing. In theory ECN = will not trigger at the onset of queuing, but will trigger later in the = cycle of queue buildup. > PS having fq on the link makes cubic and bbr cohabitate just fine. > fq_codel vs bbr behavior was reasonable, though bbr lost a lot more > packets before finding a decent state. I guess that "It Depends" - my long delay experiments in the = presentation referenced above showed cubic being completely crowded out = by BBR. Geoff