From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 5249A3B29E; Mon, 4 Dec 2017 12:19:24 -0500 (EST) Received: from [192.168.10.50] ([93.233.79.61]) by mail.gmx.com (mrgmx101 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MUDXS-1eVMLT12QM-00QzM0; Mon, 04 Dec 2017 18:19:13 +0100 To: Jesper Dangaard Brouer , Dave Taht Cc: "netdev@vger.kernel.org" , bloat@lists.bufferbloat.net, Christina Jacob , =?UTF-8?Q?Joel_Wir=c4=81mu_Pauling?= , "cerowrt-devel@lists.bufferbloat.net" , David Ahern , Tariq Toukan References: <92906bd8-7bad-945d-83c8-a2f9598aac2c@lackof.org> <87bmjff7l6.fsf_-_@nemesis.taht.net> <20171204110923.3a213986@redhat.com> From: Matthias Tafelmeier Message-ID: <77f6a9fe-6f95-f149-4cec-170d864f1c06@gmx.net> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <20171204110923.3a213986@redhat.com> Content-Type: multipart/signed; micalg=pgp-sha384; protocol="application/pgp-signature"; boundary="uXqgAS0lGRe8R0OkRICrEPPV1iiq1SXcu" X-Provags-ID: V03:K0:nNe5I74bGbD7UL+N/FJKbKhvMkAnF9gKR3dVm1xIch/tfLWWJJY hUbH4zUm7WpVMnco3yBC0QXiNeRHLHwLD5Iqdt26mRv8TiPAesq+Up3dXwudQzRSs+9te3K nLUlHszmtiDGSKkYu4+SJUlh/VKrK1Dr0BSOv7lswUXyiZ/VNbJA2POjtCf/AXMvJNH3jFa kKmOsilYLFqjcyL8QloCg== X-UI-Out-Filterresults: notjunk:1;V01:K0:cX70XP1Wfbg=:n+uiKmUteK59eAiIflUVTt k19gvlt5FYkKQcnALzIKHGyztcJCF1N+nx4hyYCUZQe7TsFvwg4rJ18BjO+p22aFm785tOvxy HXUIDIh0c5UTtnXUbkCNxdsWJxkXBoO8bp3S/fH196XZ9sb7wtP4c/RdeUa1tZTISj/e9rExz r4vYDPH9tLFFsPuVszwm6XPPJEuwZO27i9BGzfMgYpqUdoJ4a4dSZrGX+igUy8Wz6jE5WIcz8 Ja1GSHZaU5fTZKAYtd477mMeLH+6/bvVHMDgPsZe/zJDQ1JxbK3R5RupG+0nM30WfByRNUXuH KHMPzmX9UI6XkLYMhMLNTv5WjNtC12b4zcJpPumoXOrdaPaPNzieOuPr2znEXTZKsX4EOZFSk 8+ABZr0JbvS1nH9efsvXz2B/425+SivYSwcLBLUFs74f2s8opUJp9k5j+xBBycvxiJnwC2EFd I00KBHcmU4FUy2MONOrowObTJd0QNkHZgBkWVLAnX5hUgOacypR7BWPQZ8tBu4ClXOS5HWUlE zAQKVBaPyMJhqcT+Zhmb0b1bqWX9NV/i+bHZc8HbDt8dUFcIOV+Rj2SiQ1eVq58DtRBFz78cC PS0oOyayFgV+yLKwJWwjJygpAgypD2xUK00LZLaSd28zUmU2hrlzQquMmMWgl7ozqxcMN4MFe YVKN9AZ9Ta2xkdDHkswM9cUthnOWbDqVcVSxQd2zmZUf8Z6QqaQefpmlDpUlk19Kp9yBthP9D kcCUC1SBW7PRfPRFik2rHtwaeBwDvQoAcXi7nnO+LHSrGhavgpBH78PjWRJWhGgcGxGNVpKor UnYfaaRdIOvEHnNqCN93jXxKahnAQ== X-Mailman-Approved-At: Mon, 30 Mar 2020 07:21:14 -0400 Subject: Re: [Cerowrt-devel] [Bloat] Linux network is damn fast, need more use XDP (Was: DC behaviors today) X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Date: Mon, 04 Dec 2017 17:19:24 -0000 X-Original-Date: Mon, 4 Dec 2017 18:19:09 +0100 X-List-Received-Date: Mon, 04 Dec 2017 17:19:24 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --uXqgAS0lGRe8R0OkRICrEPPV1iiq1SXcu Content-Type: multipart/mixed; boundary="owmU1KDkR1OMOuTCNgqwaCMhi3E12m59p"; protected-headers="v1" From: Matthias Tafelmeier To: Jesper Dangaard Brouer , Dave Taht Cc: "netdev@vger.kernel.org" , bloat@lists.bufferbloat.net, Christina Jacob , =?UTF-8?Q?Joel_Wir=c4=81mu_Pauling?= , "cerowrt-devel@lists.bufferbloat.net" , David Ahern , Tariq Toukan Message-ID: <77f6a9fe-6f95-f149-4cec-170d864f1c06@gmx.net> Subject: Re: [Bloat] Linux network is damn fast, need more use XDP (Was: DC behaviors today) References: <92906bd8-7bad-945d-83c8-a2f9598aac2c@lackof.org> <87bmjff7l6.fsf_-_@nemesis.taht.net> <20171204110923.3a213986@redhat.com> In-Reply-To: <20171204110923.3a213986@redhat.com> --owmU1KDkR1OMOuTCNgqwaCMhi3E12m59p Content-Type: multipart/mixed; boundary="------------042A4838AD1AC4C1C0AB65F3" Content-Language: de-DE This is a multi-part message in MIME format. --------------042A4838AD1AC4C1C0AB65F3 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello, > Scaling up to more CPUs and TCP-stream, Tariq[1] and I have showed the > Linux kernel network stack scales to 94Gbit/s (linerate minus overhead)= =2E > But when the drivers page-recycler fails, we hit bottlenecks in the > page-allocator, that cause negative scaling to around 43Gbit/s. > > [1] http://lkml.kernel.org/r/cef85936-10b2-5d76-9f97-cb03b418fd94@mella= nox.com > > Linux have for a _long_ time been doing 10Gbit/s TCP-stream easily, on > a SINGLE CPU. This is mostly thanks to TSO/GRO aggregating packets, > but last couple of years the network stack have been optimized (with > UDP workloads), and as a result we can do 10G without TSO/GRO on a > single-CPU. This is "only" 812Kpps with MTU size frames. Cannot find the reference anymore, but there was once some workshop held by you during some netdev where you were stating that you're practially in rigorous exchange with NIC vendors as to having them tremendously increase the RX/TX rings(queues) numbers. Further, that there are hardly any limits to the number other than FPGA magic/physical HW - up to millions is viable was coined back then.=C2=A0 May I ask were this ended = up? Wouldn't that be key for massive parallelization either - With having a queue(producer), a CPU (consumer)=C2=A0 - vice versa - per flow at the extreme? Did this end up in this SMART-NIC thingummy? The latter is rather trageted at XDP, no? --=20 Besten Gru=C3=9F Matthias Tafelmeier --------------042A4838AD1AC4C1C0AB65F3 Content-Type: application/pgp-keys; name="0x8ADF343B.asc" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="0x8ADF343B.asc" -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v2 mQENBFJ0FzIBCADZ/hkwcprVGydMOqeqM+2k6v5e5kb4YDMKU7nMbCVmH4sn01T7 Yh9kDwG5LOMLD06BB2txjLBvTY+c0mpK+hE4pWr+i3qhU5CbVvx7jppJqCD6ZT/T A3I7NxsdixRvLIF4UXgKQOMKPIx+aw/sp86NqzCLAMse7F0vXUjAP5YANtJid2rf r/B37BGKhqDGhi4Appz4UZOzpRov/v8JD4XScuvJnl09/oi5cDj3Mn2uqOc/G6hA t7HXsbHh4dKxd3AftqPPzEkJAmm+9Z4ASG9hy8IXms8Czimr+BGL0CnfsJlX6DCU m6mVDqT1GJyzmP4zkWcPi+2fOI4KtpV+C7+bABEBAAG0Ok1hdHRoaWFzIFRhZmVs bWVpZXIgKHByaXZhdCkgPG1hdHRoaWFzLnRhZmVsbWVpZXJAZ214Lm5ldD6JAUEE EwECACsCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAhkBBQJaAMzNBQkLTxyN AAoJEOAWT1uK3zQ7u+4IAL+W82wbz1FwGfNHhOgOheCh/wlTLssgQ7XVGRduJ/m3 k22aodOKSV5aH3AUy9c9zkgkRHUU5XCG9FRujVeVYhvLP1JTG97oEjk8YGBAOqN7 D4hUHh0c3ZBpTqeE9cndXr504GMauh5mY74qdNl9nL+Gcv7CekENML1nLWBnoDWV NTpvZktPZpHozQHPBV6lk09ICxOocb7VHl+lyorStqiUcLciHTdOByC35ekJebv9 dDZ9oloI8tvLytyle1kuVQLJj0LrpkUjcjLSYoa7ZFVKCNK6FM1pWd1XRHmWqhAn i4+4l92+UHU2TASBFIUVO3kPEDOXn7kK4q6tQ2pRexuJARwEEwECAAYFAlP4qFwA CgkQc1YJs62PXiMJwggAgwa8bM1DVdB5wdWVbsEvjDWgoD4CZOH+3/nCAKFv+eKf d3GrJUtOh3T/QVpmbVgNwnyqqNLGlyIOHltVkrn9WqSC33kuXsIStR6KM+LXnA99 FjyAiTcVbzbfl/XNlIQrgV9+niSSUFCUge5242itPjBBCtlYHUkQ5Y9hsNwV9Hb7 dpVxUf01CJcNKlWscC7lTt2FqjJrIOw2NHxgHWxRlDqo9dFg4uwI+O90orKAyJ6N Sowu2Ca6DXB3jHgoG7WbAh1nEVus/JkyVsnTIMCsfOpwhJNd1fvy6JMVAe7+/p6/ JGZcMjUTadmsHeJHwqOJSVOoX3Y8CZkV8/PHIqmYvokBPgQTAQIAKAUCU/hT+gIb AwUJAeEzgAYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQ4BZPW4rfNDvHZAf7 BpqfTpLdR25q72DZ4T6SrkcZCOJ8jJQhQ1K9cAc0snpK+jcWg9iUgCpV8QJiXpGL dAkux/YCu7SRstOSbMv4G4Qb/g8y2bowFI/mAzK1o6s6CYt3URNBe7zRLK6sJbK0 f5fDpWoRufW9/Ppj1/S7dki5JpkUlyGa/y2O+X4C/P0Rh3HfL5HicRHamc7PVElh z/8nVA+KUkcA9ksVJqe50LahTbDyqOmd8cjSdUKlH2dsP/cAmZfU3IAa26UBKWVn rxnQ61VV5QLpcYvW3jTfzBy7xv1s7YSj2rFpIC5WKgPC/p+2gqVMSGI1hRlrNaSA qH9XgWNPCa5NlOxuN4mSN4kBQQQTAQIAKwIbAwYLCQgHAwIGFQgCCQoLBBYCAwEC HgECF4AFCQSrhqYFAlRXtDwCGQEACgkQ4BZPW4rfNDvqmQf/Wa2LXaOvzftegDI+ LAiFOB/Dq5yhFp6urk5+yC+YzCFin8HfP+LVXR8Xkei6fMmFMjfRU0MrNLBxFd3I UrIgRrtmJGaHB+vkIqNGgU8LcpHBdd6nprtIF53IhtOINwkmCgLzWi3sGYJ4yQyj 9OSNnh7j7ENFeZd8LgN/FgB5GjPisN3zJD19z065jlfeXvHIZOL90PaTqih90x6n oTr4dbKhk1t9zZYhY5W812gCMVn2g4wLLO+iijKOe8uNrOw22xDGckoL5UFRE8Vj Twup3eYyzb/2TVpAmM5GhnI+PodZ6GGcQRVKGMYwYyFVLDEcDRxAUpwfXfUzHwvD op2fQ7Q/TWF0dGhpYXMgVGFmZWxtZWllciAoSGFwbG9ycmhpbmkpIDxtYXR0aGlh cy50YWZlbG1laWVyQGdteC5uZXQ+iQEfBDABAgAJBQJWPZtPAh0AAAoJEOAWT1uK 3zQ7FrIH/3OFr/bZ2UQeJrn9n67G9o9neJv4ES9Lcq6xnCIc+ZRHqrBTwsYkfYC5 MEMTF7TMFNUJTr2Np3OG7iKcHVePpeMpHicXppJ4hUsIQ0kwXlynRAScrAqoQHBD IKzu5qHDME0UKIWr9iTASFHJgZGyH6OoPh7LIifV8cGdVPQ/5FF1kqM4YMZ3IygO C3CaYtEaOz0B1L00zJan8rbEnpsI1msZ3hjacGB2SD5kFAUMDbpoXVOOE07GSLF0 KKhMv02WdrKO8iedStubO9BON9Vf7IIq21RpDEhhjAzt4Ui2q0UEodTvnBX4ifFM UEU/+NdC3deuRwdxOq1ozSQlUTzVFASJATQEEwECAB4CGwMCHgECF4ACCwkCFQoC FgMFAlRXs+UFCQSrhqYACgkQ4BZPW4rfNDs/HQf+K5swcPreRRBXQbTBCgTQAoAI JtvG+TLlPPnpYMqkQoKIhw/USN2Je4Gqm3DhRcCteA5wbmhlHj9DbapbCOwE7vfK 3YC/hpntvnmgCl6atT2QbE4Ak7xeT2ljLiRYD1re7oE8fAUqkI2S+vePiK1+b8Cc OKPmuAJYmgAMmNVMKcknryNoFc7xseNEy58T+AoyCKcxV9ZJdyd+6Ye48LkRlmyf lfRnCvgfS74TEq7Gr5uCJPgqcjrl8SS3G6jgUrzPcV2mFROt9EH3d80T+GOIy4pB SeYGdfkeqUbflj0CeRIyazzAZurllCQWjpaeh009Y/wuzLm91zrVWwADP/5oRIkB HAQTAQIABgUCU/ioXAAKCRBzVgmzrY9eI23xB/461lM4c/08tEwmd0oC1jdwyidO ZRCj/vOqZ+Af6oB3FdpWseuKWdJ7zb8NR+BcEUQRqbaF/677cCrKIEnRoq7IzNsw KiqK9K5cFHLtm9TNZ0Mf5QP/PG47Jrex5l59LMMz+LW1Rv/uXJjTQDeQYsrYDAPK mJx2c0OyzZnr+CRrHJKUH0P+oVBvSQEnDTbCT2W9wuLrDIHF6H3YQLAuCS2sslq5 teAinTjTGnPmkP7hKcK3CC0BdiUgFFybIZOGthFm+bTG+V3qGUiam95dqDmQ2VTX QxCetGDTjUrvKdODh5qLFUM+StWLNEP+QtMcymZfrRIHayKS3GyjKBOPoCZpiQE0 BBMBAgAeAhsDBQkB4TOAAh4BAheABQJSdB8LAgsJAhUKAhYDAAoJEOAWT1uK3zQ7 5nMH/iscBMT7fEnIBeYZlFaxiJmFobRVWFP/A2IfzvKVIdY9vDqjN5M5chrRfsk9 HI0EPbiF3kmVwmRdl8J5fgN7O8QFbhW4ojda8UXBAsgF50kurqk2NrdgM+xLy2TI jVZdwQYK+R11SpO9xaf+nqLKV8r9VkQ5mzb+BEzDiaX7z9IBrm28v0BfelDZVRzW cOrskVnwX1PySt3xvCwwo3cwY39yno7H4AlgTXhAvhwI6DQMxZXm4MZcugSfB5b4 2uyslOOxMkgvLW1CmpJxzbWXYT+40vW0DbQAUC1VIr7hiPrunrRAWfbV/RWZl6lr 6gMJJXkLGN70MUX40FF0IhZw5Ja5AQ0EUnQXMgEIAPICr+5yNyuVcmsv5xpmRKnz KoTjJ7xt0EPiru895LEUPN25tJyi8PZmLciNJnEoQ231jjAloQvx1pb3cr35zzGX PTPJ5fEECZDxMWMjVvCMb4XK0YjqCF9i/uKic5zqjwRNAPEGTO/ZgS+e21lUJSmu KR6m5WQcKgBH+tqS3rodgjunnIN4UiNMxbq/VVGICWPjdgoTkqWE3r8QthKLg4Lj zILEH3HbG39l+vTwEKaP4q5xShFZjRUrZC2anBP+gQx/FbBff3ufpCL9LF5dkywZ P3NHVyaa/D8T66CTj7Rynd4NZy/qdqGMjMjEuf0RkgXF7Uar4GXmuOfrIcf/X6EA EQEAAYkBJQQYAQIADwIbDAUCVj2bwwUJB4vrkQAKCRDgFk9bit80O/0ZCACoWtov fl7vH3YNW0K3xWil9wj43X2OxwKiGBdfbI48bW+b6LJQwNwFePFQ/RQCBgg1eerU Oys0ymcmp0VeMdwpw27qWcMcbsDn3Pucqp1C2IUuXesbUcRo+QDqhl96KxAAWY5O JO16dfRrIxyX6Pb0uImqpDetT4Kbr1doTF8cIfRH2rszHKU9BEWag/us9V8H5S8h F7Ws2wH2JlWbQpP8E7/Z5kVM9psdqX9rwbrUAqpyNhtILoC0+zXkdnOxz1WZaBpr ckEYQS4/CQOliYJyd5nXYcwVXCNpdy1Vt53ArN9j/EIvfFxOPCPoBQj1b9nkHqNn qnlCR0LVNOnSE3Eb =3Droc5 -----END PGP PUBLIC KEY BLOCK----- --------------042A4838AD1AC4C1C0AB65F3-- --owmU1KDkR1OMOuTCNgqwaCMhi3E12m59p-- --uXqgAS0lGRe8R0OkRICrEPPV1iiq1SXcu Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCQAGBQJaJYONAAoJEOAWT1uK3zQ7C0EIANWjXi3KdBMZvQ4prXlU1IPt HTen/IyCaveInR7mE//qJEke3ahTFDF1XQtPuYc9m5XuvFZgQjPZ8vfTt0kconTx vlL5WnWqPBvZvbuNqnNIwYSKrp8w7RYBE261ywV4/xoYmBKWWLHy0fIyMKL2r2hJ N6RotldLi6+KsDduGYNJlxqUZY6Z1C0I3NReUmMFZwEp4lnZQWY9xJPlZiMvTbNH b8ETxgzUyJIYe5+8QSzeTflBSUC2CPiTOLAsIvUnHoTxvyaWIESIj4c+dvXfMazd OqkYzSFz3UdKmN3B+wSo+6KmWmmviM7yKZG+UTiBQJZOH5Q8nJxvGgaS5pLgBpI= =8z+s -----END PGP SIGNATURE----- --uXqgAS0lGRe8R0OkRICrEPPV1iiq1SXcu--