From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 80AC63B2A4 for ; Thu, 23 Apr 2020 14:30:13 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1587666610; bh=74OVPK3o7bJAoXWLfGcAOe4y7JRMARFSAlB7OMBZqYA=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=Pne08Yl77iPBrpc1ofH/QxvTs3aa3Fmh3V3iGvmeQh50j5cebbS52Mpj/1UBOB9LQ ALAzefUf0D3U0UCRUaL/2s3lPIIoVoXW4ve8CYa34MIuvWisq8ORCAtfjmXokke+uL W4UFfPGspv/9g/OdyHVsECKjAKoxGZA6vJwpQc9I= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from hms-beagle2.lan ([77.10.219.232]) by mail.gmx.com (mrgmx104 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MwfWU-1jH0DQ2o0Y-00y9oS; Thu, 23 Apr 2020 20:30:10 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.14\)) From: Sebastian Moeller In-Reply-To: <20200423173111.GL28541@sakura> Date: Thu, 23 Apr 2020 20:30:09 +0200 Cc: =?utf-8?Q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Cake List Content-Transfer-Encoding: quoted-printable Message-Id: References: <603DFF79-D0C0-41BD-A2FB-E40B95A9CBB0@gmail.com> <20200423092909.GC28541@sakura> <87o8ri76u2.fsf@toke.dk> <20200423123329.GG28541@sakura> <877dy66tng.fsf@toke.dk> <20200423173111.GL28541@sakura> To: Maxime Bizon X-Mailer: Apple Mail (2.3445.104.14) X-Provags-ID: V03:K1:rMwvH9J7M0u6nGorR520fUKHjHMAv/PzI9i1BqckeCjKshYyoY2 S6/kTlyzracjoyZ91lvHCU52oP88RP4LybvRCbB61eW4KrsKNv7jr65zXlzdOuO1eE+hJKO ES+Gxp5wIv93EcC/4DeaXgjX0PzXh0jbXL0SqO69Rdcctd4VGTTZvmLklva+UqnJ0Jd5gDM dOwpYItf9zL3vn+VszOEg== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:YajTY2OAbDM=:IofwJMLMUFPQK7WAZuOezf GfAchz1yy9ttGvIVmfvSH4arXCNybnBa2WVsWXEeAjI1/+Cs1l6JK+vsIi5h7Qg0gxbkzO6dy DhbROVTS9XR0BCVaaC9VmPAuGAxIeg8/shz+nXQfCNRd7gzUpBvsAqMwdh79YZgMCRbL6jxNU hEMaI9P6UlkBsm8OfQWETFnqFOk5NasfPtf1Nsj6dAc7+Nj0Ba/7A13By0DhMb9lESzWFihFs HHdzUneKXq2L4b5XS4c2bQlc8NU/GKPWscXINUUZpE3GjC2tedkHuqj/VCybdI5ZcdSJF6IsN IJZ2f4dYer3KyGvNcKhIUAT6wgZc3M+sHsEcEBL3UsaConSDoIccAPoCcNN4akIVCe6NiQyye MM4/cPPnM9+izqEzkR/lsHKJ3sWB5nkt9jhtKwgWb9Zo7Zs8r+4r7FUFJNori7k32i6eZhHpd C+VtVwyuavZb80eNQR0csHsMaDzI8jtNU3zmrtUR2AEYDWtjhtTfkFvFbAm09u9pXxI1jrH7n bOxwDR9mtlajTqqRXct28wyhUK3oi4aBNW7EG4NsMYmO7DzstloyvlX31Ta2js/9MJ+J/18rZ 2Z+eIZzuBo6Qt0P1t5jjI9Jt/0E4I5L9N/kmdhB+HLk6ZPchEn9Kc4eVoQEH0PsSe71xLLlSv k3bgPfjuj/zi+Yjs9EfVeefLK9gydcM0BFqcXFZ9ljQj9sxD2JIdUcB/UkfxbsWQpMSrfimof neFWVaLidOz7VqCCgBIMxEzJMjbwb4bO/x4G9VVYZqCD6nFvi6NDuvXXSqsoYhKeomDJn0n5B LSpQ7Ptula9ra24lIKcGF9uHcoP8GPXl/11LF0kzDnixX+gESprgr58+c5ibBTtNVdcKt4FqD ITdEd8Dw/+mdkOZHO7hBbug6DIuXCwgJX9lAuKaYYavlDOglHjT8upBt0FXmzjA68BShSBIfL Y0/Do+gc/0zRUouNAMXaRzOIfowax6yC7f+qNUHvV+INZxA8y86ZBcM+S51jp1DcMwDoBRGKb zhdTrJ04KN+2XotWlUNUwx0GyzHBxj0/NlmvcODrIlhj5Tl28PUYZ8m2g9sDXsIyEphUms+N/ XoDFblqv42nkks92zcUAXunhVUEsh42HO6v7q7WSt5YZTjumN/aabCKe8I7viXN89F41tVJe4 IazS3OuDXUYMalnODJ+qoPncUDG+Gzzu2NO9K2l40w/a/C23IV56bjzkDfAa119DhmHdSLaAq MxrOiqGCEL9DFo36r Subject: Re: [Cake] Advantages to tightly tuning latency X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Apr 2020 18:30:13 -0000 Hi Maxime, if I might ask you a tangential question, did you also look at MAP-T = (translation) and if so, what made you choose MAP-E (encapsulation)? Best Regards Sebastian > On Apr 23, 2020, at 19:31, Maxime Bizon wrote: >=20 >=20 > On Thursday 23 Apr 2020 =C3=A0 18:42:11 (+0200), Toke = H=C3=B8iland-J=C3=B8rgensen wrote: >=20 >> Didn't make it in until 5.5, unfortunately... :( >>=20 >> I can try to produce a patch that you can manually apply on top of = 5.4 >> if you're interested? >=20 > I could do it, but the thing I'm more worried about is the lack of > test coverage from everyone else. >=20 >> Anyhow, my larger point was that we really do want to enable such use >> cases for XDP; but we are lacking the details of what exactly is = missing >> before we can get to something that's useful / deployable. So any >> details you could share about what feature set you are supporting in >> your own 'fast path' implementation would be really helpful. As would >> details about the hardware platform you are using. You can send them >> off-list if you don't want to make it public, of course :) >=20 > there is no hardware specific feature used, it's all software >=20 > imagine this "simple" setup, pretty much what anyone's home router is > doing: >=20 > with + inside, private IPv4 address > with IPv6, vlan interface over > with IPv4, MAP-E tunnel over >=20 > then: > - IPv6 routing between and > - IPv4 routing + NAT between and >=20 > iptables would be filled with usual rules, per interface ALLOW rules > in FORWARD chain, DNAT rules in PREROUTING to access LAN from WAN... >=20 > and then you want this to be fast :) >=20 > What we do is build a "flow" table on top of conntrack, so with a > single lookup we find the flow, the destination interface, and what > modifications to apply to the packet (L3 address to change, encap to > add/remove, etc etc) >=20 > Then we do this lookup more or less early in RX path, on our oldest > platform we even had to do this from the ethernet driver, and do TX > from there too, skipping qdisc layer and allowing cache maintenance > hacks (partial invalidation and wback) >=20 >=20 > nftable with flowtables seems to be have developped something that > could replace our flow cache, but I'm not sure if it can handle our > tunneling scheme yet. It even has a notion of offloaded flow for > hardware that can support it. >=20 > If you add an XDP offload to it, with an option to do the > lookup/modification/tx at the layer you want, depending on the > performance you need, whether you want qdisc.. that you'd give you > pretty much the same thing we use today, but with a cleaner design. >=20 >=20 >> Depends on the TCP stack (I think).=20 >=20 > I guess Linux deals with OFO better, but unfortunately that's not the > main OS used by our subscribers... >=20 >> Steam is perhaps a bad example as that is doing something very much = like >> bittorrent AFAIK; but point taken, people do occasionally run >> single-stream downloads and want them to be fast. I'm just annoyed = that >> this becomes the *one* benchmark people run, to the exclusion of >> everything else that has a much larger impact on the overall user >> experience :/ >=20 > that one is easy >=20 > convince ookla to add some kind of "latency under load" metric, and > have them report it as a big red flag when too high, and even better > add scary messages like "this connection is not suitable for online > gaming". >=20 > subscribers will bug telco, then telco will bug SOCs vendors >=20 > --=20 > Maxime > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake