From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id EF0023B2A4; Thu, 7 Dec 2017 03:49:56 -0500 (EST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 64B31745; Thu, 7 Dec 2017 08:49:56 +0000 (UTC) Received: from localhost (ovpn-200-32.brq.redhat.com [10.40.200.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5C78A5F93E; Thu, 7 Dec 2017 08:49:51 +0000 (UTC) Date: Thu, 7 Dec 2017 09:49:49 +0100 From: Jesper Dangaard Brouer To: Dave Taht Cc: Dave Taht , bloat , Christina Jacob , Joel =?UTF-8?B?V2ly?= =?UTF-8?B?xIFtdQ==?= Pauling , "cerowrt-devel@lists.bufferbloat.net" , David Ahern , Tariq Toukan , brouer@redhat.com Message-ID: <20171207094949.0459d5b8@redhat.com> In-Reply-To: References: <92906bd8-7bad-945d-83c8-a2f9598aac2c@lackof.org> <87bmjff7l6.fsf_-_@nemesis.taht.net> <20171204110923.3a213986@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Thu, 07 Dec 2017 08:49:56 +0000 (UTC) Subject: Re: [Cerowrt-devel] [Bloat] Linux network is damn fast, need more use XDP (Was: DC behaviors today) X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Dec 2017 08:49:57 -0000 (Removed netdev list) On Mon, 4 Dec 2017 09:00:41 -0800 Dave Taht wrote: > > If you have not heard, the netdev-community have worked on something > > called XDP (eXpress Data Path). This is a new layer in the network > > stack, that basically operates a the same "layer"/level as DPDK. > > Thus, surprise we get the same performance numbers as DPDK. E.g. I can > > do 13.4 Mpps forwarding with ixgbe on a single CPU (more CPUs=14.6Mps) > > > > We can actually use XDP for (software) offloading the Linux routing > > table. There are two methods we are experimenting with: > > > > (1) externally monitor route changes from userspace and update BPF-maps > > to reflect this. That approach is already accepted upstream[4][5]. I'm > > measuring 9,513,746 pps per CPU with that approach. > > > > (2) add a bpf helper to simply call fib_table_lookup() from the XDP hook. > > This is still experimental patches (credit to David Ahern), and I've > > measured 9,350,160 pps with this approach in a single CPU. Using more > > CPUs we hit 14.6Mpps (only used 3 CPUs in that test) > > Neat. Perhaps trying xdp on the itty bitty routers I usually work on > would be a win. Definitely. It will be a huge win for small routers. This is part of my grand scheme. We/I just need to implement XDP in one of these small router's driver. That said, XDP skip many layers and features of the network stack that you likely need on these small routers e.g. like NAT... > > [4] https://github.com/torvalds/linux/blob/master/samples/bpf/xdp_router_ipv4_user.c > > [5] https://github.com/torvalds/linux/blob/master/samples/bpf/xdp_router_ipv4_kern.c > > thx very much for the update. -- Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer