From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-BY2-obe.outbound.protection.outlook.com (mail-eopbgr810139.outbound.protection.outlook.com [40.107.81.139]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id A53373B2A4 for ; Thu, 14 Mar 2019 15:23:10 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cablelabs.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kzrSxq4vn75+ou51vdL+V+8KUnJpdg+N9sE8gek90gk=; b=oItE848EWjiSj6m3wnDJuJ7GebFAHArVk8rcoKyhCqzFyG4noYtyyjJxcmTurfW98tmtETw+F6ogSxpx4xdhvXNi5WTKTzIIsn0dtdUlZ+nmM1gBJG2DL8EsoDrN8OR6piTYJLJqagpyZEFq4Tw28R+v77mnR2+MtXeNBmOoLY0= Received: from SN6PR06MB4655.namprd06.prod.outlook.com (52.135.117.85) by SN6PR06MB3902.namprd06.prod.outlook.com (52.132.126.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1709.13; Thu, 14 Mar 2019 19:23:07 +0000 Received: from SN6PR06MB4655.namprd06.prod.outlook.com ([fe80::712c:21cc:3dcf:637b]) by SN6PR06MB4655.namprd06.prod.outlook.com ([fe80::712c:21cc:3dcf:637b%4]) with mapi id 15.20.1686.021; Thu, 14 Mar 2019 19:23:07 +0000 From: Greg White To: Sebastian Moeller , Ingemar Johansson S CC: "bloat@lists.bufferbloat.net" Thread-Topic: [Bloat] Packet reordering and RACK (was The "Some Congestion Experienced" ECN codepoint) Thread-Index: AdTaP3pwxpdEpVg7RnKI2fhrwnA4AQAAo5OAAAnBlgA= Date: Thu, 14 Mar 2019 19:23:07 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/10.14.0.181208 authentication-results: spf=none (sender IP is ) smtp.mailfrom=g.white@CableLabs.com; x-originating-ip: [2620:0:2b10:23:dc16:1370:20ac:1369] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 16730e7f-149e-4251-d89d-08d6a8b27d16 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(2017052603328)(7153060)(7193020); SRVR:SN6PR06MB3902; x-ms-traffictypediagnostic: SN6PR06MB3902: x-ms-exchange-purlcount: 1 x-microsoft-antispam-prvs: x-forefront-prvs: 09760A0505 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(366004)(39850400004)(396003)(136003)(376002)(346002)(189003)(199004)(66574012)(11346002)(6506007)(97736004)(2616005)(36756003)(486006)(2906002)(53546011)(82746002)(83716004)(316002)(446003)(110136005)(58126008)(966005)(478600001)(99286004)(71190400001)(71200400001)(8936002)(68736007)(6116002)(53936002)(4326008)(33656002)(6512007)(6306002)(186003)(7736002)(229853002)(14454004)(46003)(25786009)(5660300002)(305945005)(76176011)(81156014)(8676002)(81166006)(256004)(476003)(102836004)(14444005)(105586002)(6486002)(6246003)(86362001)(6436002)(106356001)(85282002); DIR:OUT; SFP:1102; SCL:1; SRVR:SN6PR06MB3902; H:SN6PR06MB4655.namprd06.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: CableLabs.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: n47grlfgix2D4B/f7wgqER3fbBKh0Ro3jkLLgzrhThgnjBatCo80hh85eD5b0H2aIYM40kMxbFPXwgUDch8RBmt8MGNbka4+x5JcAPkc+FmQ6JLwP+dU7ofrYwoMUZrvmDlAyEeOp9b9ZpPxSnAnLJSWt0SNjL2HFyDQxXhFHAlD4ttOIIs1mPiehKdKMz3G+h1fNRGsXOHzat6A45Mv8IUqNzPx23v1UpdqAUlF9altlU+CDUtnEdgSpdMXP/HBVyxsJExTWQXXhQjikwS4eG46K1gHif6dD1GucRTD00PBqBPySuFdEzeDHCNFThLdNi2GLqT3vpnwJ+ascekp0gHoBEGyYU2ug9S5RZsYNr4PZCpIn1ZUulErxh/YpBda/HCAhts/p/2WKuMmnrWI9Eyp9zAFCM5xf3DT7DSojF4= Content-Type: text/plain; charset="utf-8" Content-ID: Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: cablelabs.com X-MS-Exchange-CrossTenant-Network-Message-Id: 16730e7f-149e-4251-d89d-08d6a8b27d16 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Mar 2019 19:23:07.4217 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: ce4fbcd1-1d81-4af0-ad0b-2998c441e160 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR06MB3902 Subject: Re: [Bloat] Packet reordering and RACK (was The "Some Congestion Experienced" ECN codepoint) X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Mar 2019 19:23:10 -0000 U2ViYXN0aWFuLA0KDQpUaGUgbGF0ZW5jeSBiZW5lZml0IG9mIGVsaW1pbmF0aW5nIHRoZSBpbi1v cmRlciBkZWxpdmVyeSBhc3N1bXB0aW9uIGNvbWVzIGZyb20gdGhlIGZhY3QgdGhhdCBMMiBsaW5r cyBhcmVuJ3QgcmVvcmRlcmluZyB3aXRoaW4gZWFjaCBtaWNyb2Zsb3cuICBUaGV5IGFyZSByZW9y ZGVyaW5nIG9uIHRoZSBsaW5rIGFzIGEgd2hvbGUuICBTbywgKGUuZy4pIHBhY2tldHMgZnJvbSBt aWNyb2Zsb3dzIHgsIHkgYW5kIHogY2FuIGJlIGhlbGQgdXAgaW4gYSByZXNlcXVlbmNpbmcgYnVm ZmVyIHdhaXRpbmcgZm9yIGEgcGFja2V0IGZyb20gbWljcm9mbG93IHcuICBTbywgaXQgaXMgYSBk ZWZpbml0ZSBsYXRlbmN5IGJlbmVmaXQgZm9yIGZsb3dzIHgsIHkgJiB6IGlmIHRoaXMgY2FuIGJl IHNodXQgb2ZmLiAgDQoNClBoaWxvc29waGljYWxseSwgc2luY2UgcHJvdG9jb2xzIGFuZCBhcHBs aWNhdGlvbnMgY2FuIHZhcnkgaW4gdGhlaXIgbmVlZCBmb3IgaW4tb3JkZXIgZGVsaXZlcnksIHdo eSBkb2VzIGl0IG5vdCBtYWtlIHNlbnNlIHRvIHJlbHkgb24gdGhlIGFwcGxpY2F0aW9ucy9wcm90 b2NvbHMgdGhhdCBuZWVkIGluLW9yZGVyIGRlbGl2ZXJ5IHRvIGltcGxlbWVudCB0aGVpciBvd24g cmVzZXF1ZW5jaW5nPyANCg0KLUdyZWcNCg0KDQrvu79PbiAzLzE0LzE5LCAyOjQzIEFNLCAiQmxv YXQgb24gYmVoYWxmIG9mIFNlYmFzdGlhbiBNb2VsbGVyIiA8YmxvYXQtYm91bmNlc0BsaXN0cy5i dWZmZXJibG9hdC5uZXQgb24gYmVoYWxmIG9mIG1vZWxsZXIwQGdteC5kZT4gd3JvdGU6DQoNCiAg ICBIaSwNCiAgICANCiAgICANCiAgICA+IE9uIE1hciAxNCwgMjAxOSwgYXQgMDk6MjYsIEluZ2Vt YXIgSm9oYW5zc29uIFMgPGluZ2VtYXIucy5qb2hhbnNzb25AZXJpY3Nzb24uY29tPiB3cm90ZToN CiAgICA+IA0KICAgID4gSGkNCiAgICA+IA0KICAgID4gSW4gYWRkaXRpb24gdG8gdGhlIGJlbG93 LCBpbiBOUiAoPU5ldyBSYWRpbywgYSBwYXJ0IG9mIDVHKSB0aGUgUkxDIGxheWVyIHdpbGwgDQog ICAgPiBubyBsb25nZXIgZW5zdXJlDQogICAgPiBpbiBzZXF1ZW5jZSBkZWxpdmVyeSB0byBoaWdo ZXIgbGF5ZXJzLiBQYWNrZXQgcmVvcmRlcmluZyBjYW4gb2NjdXIgb24gdGhlIE1BQyANCiAgICA+ IGxheWVyIGFzIHNldmVyYWwgSEFSUSAoSHlicmlkIEFSUSkgcnVuIHNpbXVsdGFuZW91c2x5IHRv IHRyYW5zbWl0IHBhY2tldHMsIA0KICAgID4gc29tZSBwcm9jZXNzZXMgbmVlZCB0byByZXRyYW5z bWl0IGFuZCB0aGVyZSB5b3UgZ2V0IHBhY2tldCByZW9yZGVyaW5nLg0KICAgIA0KICAgIAlVbmZv cnR1bmF0ZS4uLg0KICAgIA0KICAgID4gVGhlIFBEQ1AgbGF5ZXIgY2FuIGhvd2V2ZXIgKG9wdGlv bmFsbHkpIGVuZm9yY2UgaW4gc2VxdWVuY2UgZGVsaXZlcnksIA0KICAgID4gcGVyc29uYWxseSBJ IGFtIHNjZXB0aWMgYWJvdXQgdGhlIGJlbmVmaXRzIG9mIHRoaXMgYXMgaXQgYWRkcyBleHRyYSBI b0wgDQogICAgPiBibG9ja2luZyB0byBzb2x2ZSBhIHByb2JsZW0gdGhhdCBSQUNLIGNhbiBzb2x2 ZS4NCiAgICANCiAgICAJSW4gdGhlIGNvbnRleHQgb2YgdGhlIEw0UyAob3Zlci0pIHByb21pc2Vz IGl0IGNhbiBub3QgSU1ITywgdW5sZXNzIHRoZSBsb3NzeSBsaW5rIGlzIHNsb3dlciB0aGFuIHRo ZSBpbnRlcm5ldCBhY2Nlc3MgbGluay4gTXkgcmF0aW9uYWxlIGlzIHRoYXQgZGlyZWN0IHJldHJh bnNtaXRzIG9mIHBhY2tldHMgdGhhdCBkaWQgbm90IHBhc3MgdGhlIGN1cnJlbnQgInBoeXNpY2Fs IiBjb25uZWN0aW9uIGFuZCBzb3J0aW5nIGF0IHRoZSByZW1vdGUgZW5kIG9mIHRoZSBsaW5rIGJl Zm9yZSBwYXNzaW5nIHRoZSBwYWNrZXRzIG9uLCBpcyBvbmx5IGdvaW5nIHRvIGludHJvZHVjZSBt b3JlIChzb3J0aW5nLSlkZWxheSB0aGVuIHNlbmRpbmcgb3V0IG9mIG9yZGVyIGFuZCBleHBlY3Qg dGhlIHJlY2VpdmVyIHRvIHB1dCB0aGluZ3MgaW4gc2VxdWVuY2UgYWdhaW4sIElGRiB0aGUgcmV0 cmFuc21pdCBwcm9jZXNzIHRha2VzIHRoZSBvcmRlciBvZiB0aW1lIGFzIHRoZSB0cmFuc2ZlciBv ZiB0aGUgdW4tb3JkZXJlZCBwYWNrZXRzIG92ZXIgdGhlIGJvdHRsZW5lY2sgYWNjZXNzIGxpbmsu IEFzIGZhciBhcyBJIHVuZGVyc3RhbmQgTCRTIGlzIGFsbCBhYm91dCByZWR1Y2luZyBhcHBsaWNh dGlvbiB2aXNpYmxlIGxhdGVuY3ksIGFuZCBSQUNLLCBpbiBteSBsYXltYW4ncyB1bmRlcnN0YW5k aW5nLCBpcyBhbHNvIG5vdCBnb2luZyB0byBoZWxwIG11Y2ggaGVyZSBhcyBpdCBiYXNpY2FsbHkg aW50cm9kdWNlcyBhbm90aGVyIHRpbWVvdXQgKGFrYSBwb3RlbnRpYWwgZGVsYXkpLiBJIGFtIG5v dCB0cnlpbmcgdG8gcGFzcyBhIGp1ZGdtZW50IG9uIFJBQ0sgaGVyZSAoYW5kIGFzIGZhciBhcyBJ IHVuZGVyc3RhbmQgaXQgc29sdmVzIGEgZGlmZmVyZW50IHByb2JsZW0sIGJ5IG1ha2luZyBUQ1Ag bW9yZSB0b2xlcmFudCBhZ2FpbnN0IG1pbGQgcmVvcmRlcmluZyBpdCB3aWxsIGluY3JlYXNlIGJh bmR3aWR0aCB1dGlsaXphdGlvbiBhbmQgcmVkdWNlIGRlbGF5cyBmcm9tIGhhdmluZyB0byBzbG93 IGRvd24sIGJ1dCBpdCB3aWxsIGRvIG5vdGhpbmcgZm9yIHRoZSBwZXJjZWl2ZWQgbGF0ZW5jeSBm cm9tIHJlLW9kZXJlZCBwYWNrZXRzKSwgYWxsIEkgd2FudCB0byB1bmRlcnN0YW5kIGhvdyB0aGlz IGlzIGdvaW5nIHRvIHdvcmsgaW4gdGhlIGNvbnRleHQgb2YgInVsdHJhLWxvdyBsYXRlbmN5IiAo YSB0ZXJtIGZyb20gdGhlIEw0UyBSRkNzIHRoYXQgSSBiZWxpZXZlIHRvIGJlIGEgdGFkIHRvbyBt dWNoKT8NCiAgICANCiAgICANCiAgICA+IEluIGFkZGl0aW9uIGl0IGNvc3RzIG1vcmUgDQogICAg PiBtZW1vcnkgaW4gbm9kZXMgdGhhdCBwb3RlbnRpYWxseSBuZWVkIHRvIHRyYW5zbWl0IDEwcyBv ZiBHQnl0ZSBvZiBkYXRhLg0KICAgIA0KICAgIAlTdXJlLCBidXQgc3VjaCBpcyBsaWZlLCB0aGlz IHJlbWluZHMgbWUgYSBiaXQgb24gZGViYXRlcyBhYm91dCBjYWNoZSBjb2hlcmVuY3kgYW5kIHVu LWFsaWduZWQgbWVtb3J5IGFjY2Vzc2VzLCB0aGUgaGFyZHdhcmUgcGVvcGxlIHNlZW0gdG8gYXJn dWUgbGlmZSB3b3VsZCBiZSBzaW1wbGVyL2Zhc3RlciBpZiB0aGVzZSBjb3VsZCBiZSB0cmFkZWQt aW4sIGJ1dCB0aGF0IHNpbXBseSBwdXNoZXMgY29zdCBhbmQgY29tcGxleGl0eSBvbiB0aGUgc29m dHdhcmUgc2lkZSA7KQ0KICAgIA0KICAgIEJlc3QgUmVnYXJkcw0KICAgICAgICAgICAgU2ViYXN0 aWFuDQogICAgDQogICAgUC5TLjogV2hhdCBpcyB0aGUgYmVzdCBwbGFjZSB0byBkaXNjdXNzIEw0 Uz8gQ2VydGFpbmx5IG5vdCB0aGlzIG1haWxpbmcgbGlzdCwgb3I/DQogICAgDQogICAgDQogICAg DQogICAgPiANCiAgICA+IC9JbmdlbWFyDQogICAgPiA9PT09PT0NCiAgICA+IA0KICAgID4gRGF0 ZTogVHVlLCAxMiBNYXIgMjAxOSAyMTozOTo0MiAtMDcwMCAoUERUKQ0KICAgID4gRnJvbTogRGF2 aWQgTGFuZyA8ZGF2aWRAbGFuZy5obT4NCiAgICA+IFRvOiBTZWJhc3RpYW4gTW9lbGxlciA8bW9l bGxlcjBAZ214LmRlPg0KICAgID4gQ2M6IE1pa2FlbCBBYnJhaGFtc3NvbiA8c3dtaWtlQHN3bS5w cC5zZT4sICAiSG9sbGFuZCwgSmFrZSINCiAgICA+IAk8amhvbGxhbmRAYWthbWFpLmNvbT4sICBD YWtlIExpc3QgPGNha2VAbGlzdHMuYnVmZmVyYmxvYXQubmV0PiwNCiAgICA+IAkiY29kZWxAbGlz dHMuYnVmZmVyYmxvYXQubmV0IiA8Y29kZWxAbGlzdHMuYnVmZmVyYmxvYXQubmV0PiwgIGJsb2F0 DQogICAgPiAJPGJsb2F0QGxpc3RzLmJ1ZmZlcmJsb2F0Lm5ldD4sICAiZWNuLXNhbmVAbGlzdHMu YnVmZmVyYmxvYXQubmV0Ig0KICAgID4gCTxlY24tc2FuZUBsaXN0cy5idWZmZXJibG9hdC5uZXQ+ DQogICAgPiBTdWJqZWN0OiBSZTogW0Jsb2F0XSBbQ2FrZV0gVGhlICJTb21lIENvbmdlc3Rpb24g RXhwZXJpZW5jZWQiIEVDTg0KICAgID4gCWNvZGVwb2ludCAtIGEgbmV3IGludGVybmV0IGRyYWZ0 IC0NCiAgICA+IE1lc3NhZ2UtSUQ6IDxueWN2YXIuUVJPLjcuNzYuNi4xOTAzMTIyMTM3NDMwLjYy NDJAcXluYXQteW5jZ2JjPg0KICAgID4gQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFyc2V0 PVVTLUFTQ0lJOyBmb3JtYXQ9Zmxvd2VkDQogICAgPiANCiAgICA+IE9uIE1vbiwgMTEgTWFyIDIw MTksIFNlYmFzdGlhbiBNb2VsbGVyIHdyb3RlOg0KICAgID4gDQogICAgPj4gSG93IGlzIHBhY2tl dCByZW9yZGVyaW5nIGZvciBhbnlib2R5IGJ1dCB0aGUgZm9sa3MgcmVzcG9uc2libGUgZm9yDQog ICAgPj4gb3BlcmF0aW5nIHRoZSAiY29uZHVpdHMiIGluIGFueSB3YXkgYXR0cmFjdGl2ZT8NCiAg ICA+IA0KICAgID4gSXQncyBtb3JlIHRoYXQgbm90IHdvcnJ5aW5nIGFib3V0IG1haW50YWluaW5n IHRoZSBvcmRlciwgYW5kIGp1c3QgbW92aW5nIHRoZQ0KICAgID4gcGFja2V0cyBhcyBmYXN0IGFz IHBvc3NpYmxlIHJlZHVjZXMgdGhlIG92ZXJoZWFkLg0KICAgID4gDQogICAgPiBUaGUgbWFqb3Jp dHkgb2YgdGhlIHRpbWUsIHBhY2tldHMgd2lsbCBiZSBpbiBvcmRlciwgYnV0IHJhY2UgY29uZGl0 aW9ucyBhbmQNCiAgICA+IGNvcm5lciBjYXNlcyBhcmUgYWxsb3dlZCB0byBmb3J3YXJkIHBhY2tl dHMgb3V0IG9mIG9yZGVyIHJhdGhlciB0aGFuIGhhdmluZw0KICAgID4gdGhlIGRlbGF5IHNvbWUg cGFja2V0cyB0byBtYWludGFpbiB0aGUgb3JkZXIuDQogICAgPiANCiAgICA+IERhdmlkIExhbmcN CiAgICA+IA0KICAgID4gDQogICAgPiANCiAgICA+IF9fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fDQogICAgPiBCbG9hdCBtYWlsaW5nIGxpc3QNCiAgICA+IEJs b2F0QGxpc3RzLmJ1ZmZlcmJsb2F0Lm5ldA0KICAgID4gaHR0cHM6Ly9saXN0cy5idWZmZXJibG9h dC5uZXQvbGlzdGluZm8vYmxvYXQNCiAgICANCiAgICBfX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fXw0KICAgIEJsb2F0IG1haWxpbmcgbGlzdA0KICAgIEJsb2F0 QGxpc3RzLmJ1ZmZlcmJsb2F0Lm5ldA0KICAgIGh0dHBzOi8vbGlzdHMuYnVmZmVyYmxvYXQubmV0 L2xpc3RpbmZvL2Jsb2F0DQogICAgDQoNCg==