From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50040.outbound.protection.outlook.com [40.107.5.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 903283B2A3 for ; Sun, 25 Sep 2016 23:20:43 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=darbyshire-bryant.me.uk; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=CCqccdJ1zDj54wYGo0PDoOfYGfQr+aEFOVOk5NGO5ow=; b=vJsmL962/2l3EWTjlfMgQRtBOLkGqQ1erueF1F9T9YVIcwLGbNWPjC4kV+yrcyv2kDeKaDQsL0sBEGan1Ro8BQ6AmLTlsYtFNRjQjyYFUeP36VagE2z9xE8LI41sWA3VDHxojEi888kkvSH8p4TkbJqLxmg8M2ldhmpZKfI68YU= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=kevin@darbyshire-bryant.me.uk; Received: from [10.102.24.2] (109.159.227.69) by AMXPR07MB118.eurprd07.prod.outlook.com (10.242.70.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.639.5; Mon, 26 Sep 2016 03:20:41 +0000 To: From: Kevin Darbyshire-Bryant Message-ID: <3a99770e-6350-471f-72b6-b209d7d77d75@darbyshire-bryant.me.uk> Date: Mon, 26 Sep 2016 04:20:38 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [109.159.227.69] X-ClientProxiedBy: DB5PR01CA0062.eurprd01.prod.exchangelabs.com (10.163.24.30) To AMXPR07MB118.eurprd07.prod.outlook.com (10.242.70.144) X-MS-Office365-Filtering-Correlation-Id: dde8509e-a533-45ab-59a4-08d3e5bc183c X-Microsoft-Exchange-Diagnostics: 1; AMXPR07MB118; 2:I55OdVCSLm1qP/HBUG8/AkED/8P4UKh36UakQMQo1Si/xP4WXjSqhhRAFWdH1Y7gcE272JGakUGxEAny8ud5UmUsjA6A7CS9ABv1f5NQyRluJiCIJpbJJOEtbuE+paN7PItQOgqfWdmuIRauw2mQkjw1NLN9z+96meU4/oysxK9NKxTC8GL3qfvde73i/HZA; 3:EPjCwBlsSdogaLtQON56krCqYKXuziE+/F3FpZ58Sqw5bRNTwAS8QOUaxP0BqEryTzIVDPwgUaT1dNtHnpYXhTmJsmjPcbRgfWDLvR9BJ9YVZE/H3zK5R3LnOnjJdNcO X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AMXPR07MB118; X-Microsoft-Exchange-Diagnostics: 1; AMXPR07MB118; 25:TBnIZO+omgjusC/+liYigfNhfg+06PqlKRUVi9KruW68mXJL51b10GtG9QapCDGV3C/I0LZW4EQiRiPF/0pKHf5q1YZG7lmmw20IEE6haExZIF03BKW5HC17iatzNxhdNpT5nw8sIXB/TP2mWLIYtRG7xzMr3cUpXsgSoVT5oLB61Jjqxv+aIV7vl+BQa1T/TsmFfrCnqBKYRXauMZIkINFNbI0xJR+2T2wPN9nAZvHtz//nwAMZwZMTCE6sc58H/ElgnO8WaJbQnK4cYsIAZiYMBkpcR5VN5fhyUhA6HEnKdRdwv6OCFPk/h8oB3sDOGL+gwaTLjpNcqyQUtZHqHwwOSRUgVWvyARmd1IlOyzfS9zD32Ti5sDNTtpoq4fdSoKGEPBz1pv59Ubm6CBxIYhUttoJqt5RJncSvALWsCZuUXKFHipCKe9niBJViS2dyYKbpKBIJGoKzJ0G+9vJ87Ynsr5E71MBu+mvRughbGa2yarZrtFCJWz8AoW9yQsa+K57r4W70MyabymXTYLP2oQB7Pm412p7NqbyUkTtl7mbKLNMX0fPl+juMSSvc5W7fl9Wz+NpHXXFq8qW62g1pnEIFMmphYhdg6pn5inuwDaagBhemJhVUBuqJ+kw19CmK9yY1MVqr80EAeEi+bthB+fn3qRO0kYiAm4yXjupCkNfB6jnR932oTX+ZP7TxcPQP6fvR0s6jKRHAEonjVdfBZa7F4FHQAB2Krb1VqYGuzzJD535uThSRgRg0c+6m4HMzp/jaNLVVFoiCHrgZZvPJkPXss4tZuACl5SUttdJPjAVpp0hPVLTfP6k+d+RUJe/H X-Microsoft-Exchange-Diagnostics: 1; AMXPR07MB118; 31:Ukf7PYRYr3v2T8BiehROi+9CHZexmSaIdmenRde2tE+bic4lmgbPP5BmSVikYG142g8HYnVoKSHUxRPZpRz/4hvhYtWowzZGCqMD9SE88ubJEI9sIYWz64s2AxKDKFHiUatXdbjW4FK0UDaf+M9rwaqawiMbEJIIH2CPnG7UcrUj/q8x1+cMIccCwPalrBOOPvHLEaREVngcVRBWCTxUbDH+QntllMVIGMSn9e3dlo4=; 4:Daj2QWu/FO145UsIpqyF6yTzErb7glERP9KPA7S4TXukamKY265/QS8pcNgogu7BHR/DXVImuF+T7tnm/X2b1azBdMfqdKW9pnSmJ4Q9rmWGzxnfOdO9v3s7ooM+N6QBLaYmk+aSX7c+DNd9sA1XDdwft9Bn8f7QNalqAx4K4exw32C175xtVVoVglQV9ZhdEewymaMwtlxHg61zdvghQkzNyznILL5vCscYe3WJlCH3NHQMznyvPNlhH5pf1TeB3eQYayIVCDsN8xr6xhm6HBPAJpsBlanDtrinbee5LX9evmrhfQ3mXtHWmpAtHsy29ZOhJoO0pKzh/N3A0Rl7P+4bDMOLm7CkvFKpUmBscNoM6SG/n8jSDgyTJzHQtNAuGcIG4D3NwLLowG3w5hf1eB/jfBHFN7yt+sBJdfyUXLTcFSS8OfjSK8XjDI6Q7b8RcHTZthzkzAjEmysRyVhDCA== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(166708455590820); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6043046)(6042046); SRVR:AMXPR07MB118; BCL:0; PCL:0; RULEID:; SRVR:AMXPR07MB118; X-Forefront-PRVS: 00770C4423 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6049001)(6009001)(7916002)(189002)(199003)(2906002)(8676002)(7846002)(65826007)(81166006)(81156014)(36756003)(83506001)(5660300001)(33646002)(65806001)(47776003)(66066001)(3846002)(86362001)(65956001)(6916009)(42882006)(106356001)(31696002)(7736002)(586003)(6116002)(110136003)(64126003)(92566002)(50466002)(305945005)(68736007)(4001350100001)(15975445007)(54356999)(77096005)(19580395003)(50986999)(230700001)(74482002)(229853001)(450100001)(101416001)(105586002)(42186005)(31686004)(23676002)(107886002)(2351001)(189998001)(97736004); DIR:OUT; SFP:1101; SCL:1; SRVR:AMXPR07MB118; H:[10.102.24.2]; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: darbyshire-bryant.me.uk does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtBTVhQUjA3TUIxMTg7MjM6OUttNkdUY0w4bHphTm5OQkx6c3FhaVVTMDla?= =?utf-8?B?a3dmcm5mMzNGNENVM1ZpZzJOYXB4NjNUamZiRHoyZ0R4UGFvaHVwK1VkcUZE?= =?utf-8?B?MW8vLzVRZUZYT0VmcTUvM0grTExQc2NtM0NudmQza0pyVXVGS21NRS9acE9P?= =?utf-8?B?OGxKZ1laN3NuVHY3Zy9aUFNnUUsrbnFLVlZkeUo2QVhSMnFxbDBlN3l0SHdu?= =?utf-8?B?ZGx3RDhyRXdsZXlDVUlnNHFkekdKQnBqdld1RGJTWE5MbGZnbTh1cndEcXlv?= =?utf-8?B?MGFkSVdmRC8xcHNyeGZmSXgwTnkvN2YyUnB1RmMraUZsclNiVk5NaURaRkw1?= =?utf-8?B?MWZjWGlIWW9PcUs4TlA2NEEvQlJud0hxVUFqNy9QbEJST2x5RlJVRGV0VXUx?= =?utf-8?B?ck12TEEyRHlBbVAwbVV2MkdlTU1GNGkrQjUyZ0JtMGV2eDdybWwzM1dQQThC?= =?utf-8?B?Q1lxVEU5aDV4Smc5SE9MZ3Znd0daaTlrUVJ5aEJSSmJZTXBWdVE4QmQxUDBK?= =?utf-8?B?SXdIWW1OcDFIMEhIb2x0c2lkeFU1dmFZSzJDaWZDeWpQTEgzeWdLbnR4VGs4?= =?utf-8?B?aHJJQTgzVERvQzFRWi8wcVRGUzNVNTBLWVlKVHA2SUFLcjFCSlRTVzNPYjNO?= =?utf-8?B?Q0NZQXkwRmU0NlFUNnQzOHhTQi9VMnF4MEtMUHFwd2o5N1JZK2xkcVZUU2U3?= =?utf-8?B?cEZZdWZiT1VQdGx0WTUzWjNRUlJXNDN0d1Z1RjVTVFR6VktGMHUzZVdrSzVV?= =?utf-8?B?ZStLNDJ4VHNLdU55TFU2a2lyc1Z0VzdSTHB1VFFTTG9BUy94UndMSnJSUkRq?= =?utf-8?B?akNEdm45N0h4cnR1eWlyV2p2RVBrWHNncjJESGhyS1k2Q2NyQlk2Q01KMUxS?= =?utf-8?B?aXYvNUVrZnBwQXZIa2NZRGFyK0szaEwva2lDR3dKSXpKdEZURWw0ZjdDWlNj?= =?utf-8?B?ZGdaSXllQzdGRWxPaHFISzVNcThZZUNvbGJUcXZrdXUvVm9WODFLN3pid016?= =?utf-8?B?bVdhS1R6ZkFXQXM0SDVNY29kKzRNRnEwRFBpa0JtZHd6SlFPYkE0TS8zQTEr?= =?utf-8?B?UndaQWRzTEU3TWh3UHkzNzgwZWpmRUlyL2tRWE5GVWptd1cyelN5RERnRGJj?= =?utf-8?B?S2xKMURWUlV0NWdxWXlzTGw0ZS9Jd0tFb0ZGRTV6UVNuaVJnWFIrR2NzSDl2?= =?utf-8?B?Y2h4WXo2MklpcTA4K3oxWXJSY2hBcFE0SjdmQWh5anNRM2lYVUtaSDlheU1y?= =?utf-8?B?SkYzdjNlYVRPN1BZSm0yS25CanFBajU4ZnprUzArK1VZZVIvS3M0V1hJUFgx?= =?utf-8?B?anIxWWJKRzduMWZiQXQ2eUtHOFoyaDZucFlzc085TmlSeW1QRTRWMElGL2ZK?= =?utf-8?B?R3ZUTU5XcjVJS2ZKd1hvTGZoSHJ4Q1ZZUmgzZHJ2aldYK2psaHVuUXM0SSsr?= =?utf-8?B?dXQ1MHRwMXZFN1BiN1FGMzVvT25YbVFNQzRSUnR2TUVERlZEZUsxVS9VWUpG?= =?utf-8?B?c29NQWM0VjVEVXVxRnF0L2ZrYktIWGdHbkxuNC9XK2o3S2JKei9WQmRYSUps?= =?utf-8?B?NXdNMGpWak0zQ0ZVTStQVURlMnBnQnYwMGl3MnZhc3ZFQmMwam93VWUydkND?= =?utf-8?B?K2dER3Uvdm9QQjlpYzVOZWRXS1VFMWxhYVNaUHFndDB4Ly9kKzVFK2QzT0VW?= =?utf-8?B?UUl0aE5mU2Vhc3R4clJnMllaSmsvTmtBNFI1UmhPMksrOWt5bEZCUDBKQ1hJ?= =?utf-8?B?dlF1eFowdFdIQ0hjUm03V09VVURhVFpJNmVvekUrcUF3RkFQUkppeG50cVJz?= =?utf-8?Q?iGGuJmSzLYRF?= X-Microsoft-Exchange-Diagnostics: 1; AMXPR07MB118; 6:T4mivibimdC96v5JVm5ZghtpPys9OQVaW742JIEDW5uB5Ce6Nfg1/fkLflURp9NpysW5EHFTd7Q4pLrR7sW6MC7iKlYt76BD2nAxDznMmexW1iPCEQV21oCs9gCNolD8nZxyHZoL6xuCP4K9HeJAwUH/wT9RIQ1soJ8bhxW5Z3E0/0KGyXLHRSGA0dKAmB2PQCFsWo022HknIExkOEGe4qqMRlmUvZZWqWi2NT9kRKCIK7o2Le0rHOnW1/t0LGolYZkaZCeQ2d9cL4rxu0mzIv9PGB2MXnwTS6SctF/RYGSj90Qt9U4WQ2aHca6QAbmX; 5:8SUQM69tvubeWLnghWi6A6eo8duhTy2znJA/kG+OWNKOx02xMFEUQWM93jdcpJcpeFUyVSmK70io5e/EiMtESEhE+SEsd90QTxchwfkWIM9V6T9Qex5pbn1o+PWfncO2SMZEcoUDXc45lWj/tXXV5A==; 24:kC3WtIlopmkXyRJfv+VVjhKxM5BV8DYUL1KfoeR6a18/MAJdARt8lAemM3/nOq8TtOQSjIYJl1b+eLuVp5/yTuw/a0CFK1d13fEFdBV5Oso=; 7:bCxWhcbXVOUsLVtjNUyQGDapF6GW6sAu4+GA1zoHRbDF/KZRJLZSX/TTcMVC2Ebjj7g32Dy1GOlRIabUalQJIYJfHAyQomuSPZe8muzDmI+c9aJf96gIMoVLHr3/f/qnGZHcO/jo0J5dVUFAC6SPkw2nzXHJLHkzKn2ytRIFShyPfc6ISgb1DFoxE14aIRwQD975Kg3H/wsKGLB904bWjLwOVIId99EpQKm8p2K6y5KalfOl9ZyaX8elAMt3kNN1AM51Z7pAgq3pFl8IC5hfnGuvXg3ZGLY+7NHr+lCLTWQWeeMaoJ+15c2EJ5Xb5Q/L SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: darbyshire-bryant.me.uk X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2016 03:20:41.1612 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: AMXPR07MB118 Subject: [Cake] de-natting & host fairness X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 03:20:43 -0000 Greetings! A while back I started on a quest to make cake 'nat' aware as the lack of host fairness in a typical home router environment was the only thing that prevented cake from being the ultimate qdisc in my opinion. This involves dealing with conntrack which on egress is easy (the kernel fills in a data structure for us), ingress is less clear. I hacked something together but wasn't really happy with it. Another github user 'tegularius' presented some beautifully crafted code that did the lookups in a much neater way. Originally it too had an 'ingress' lookup problem. This was worked on and I hacked some conditional 'denat' options into cake & tc. For your 'delight' a denat cake https://github.com/kdarbyshirebryant/sch_cake/tree/natoptions along with a matching tc https://github.com/kdarbyshirebryant/tc-adv/tree/denat Typically I use 'dual-srchost srcnat' options on the egress interface, with 'dual-dsthost dstnat' in the ingress ifb interface. In *brief* testing, bandwidth is shared fairly between hosts, and fairly by flow within each host. And it's not crashed yet. Kevin