From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id A8C9E3CB37; Thu, 14 Mar 2024 12:35:38 -0400 (EDT) Received: by mail-wm1-x331.google.com with SMTP id 5b1f17b1804b1-413fea12b07so329215e9.1; Thu, 14 Mar 2024 09:35:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710434137; x=1711038937; darn=lists.bufferbloat.net; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=dvi4lU9wh/qN61Qlk1QkmL9OV1x3rs5aeYJaO9R+gL8=; b=I/TU+Vz9rQIM2AGeFP9pfToySfWfkaaY7gAw27HF4mwWEzHOyYTLdXe4Oh5FCwgS6W zf/fqfay6EE7AHuiRpRZuqOqjMM8khNDUEemNaDZxyMtIaAcxgxw1BniiHIb01V0c6Yk 0ftpxmjlhquV9enlm5kAGnogIymQwfX5nE1LfAi7zZQWnJhdSVWDiLB8dkTw69+Odj3y eVrbI+QJ+WQIV8qYBFmHzRhC1AOltI/leZKNPR9wuj2z4h0zQsnL03ABz9OoNn3YTycI nYhuHMapCVki/Zgi4HEFh4Hddesn4w5p+IHGXV1dV/Km1xApJiZ8Clx7nl7bCMDby0P6 4Ztg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710434137; x=1711038937; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dvi4lU9wh/qN61Qlk1QkmL9OV1x3rs5aeYJaO9R+gL8=; b=QDsnLbfH2EFkeDgsyE4bi+BacDg/sL/Lexpa/FpEuoGws4WGbWfcC/w/JNyrUs3XE3 fBFuP3EgkhDoEGrcPTe0vPqf9yu/TphusHezygkgxZWVyTnsld/QHNu+KqUzfkX5xARh KGVBIwBFth64OF4QzjdEngaid6Vw3OcstysbDhSFnxLwDBdD24v3pbxxiZ2q/VvpaTY4 Yg7KehbuJAnBbwTO9QZnTkh7rGHifNMhTpOCPaZdG+ikjQPZXksVEDCQx8OUcujVq849 L18uz5eakezV6S/kdb/qUWAjqfgcwW9Nve9EvCbRs21c5nBHsskOW5804WKfIEKG8oy2 C8ZA== X-Forwarded-Encrypted: i=1; AJvYcCXy3vVlSzq9HSAhVTimEua3RomvWkC/ywXFiA29+XZIqjEHKugWQJuFRZfGEP0TTOLmuJgBbwb2gwNRM/C6JINjRakvBPuKYucHhS8= X-Gm-Message-State: AOJu0YxRgP0hb096NhvkIQdARQIDbsHgKii4IEWoihoT4bgo0F2lzXu3 TcHOr9OvWx3uRYBej2NnX14OPF5jH56ghhYHP7FXJIY04GTx2ZVNci0IL9BTI8sxbp6iogAN0H7 DFKQ6ZhazbTBtx6o/NQe0hgLa25+9m+jp X-Google-Smtp-Source: AGHT+IFB6eXjlp+oFwv5BwizbXj+XXCk+m+RhALT/o/X2shWZe7PhhZ2dODYM49kFK0jf5vscv01iaNyjv9p2L2+Bgk= X-Received: by 2002:a05:600c:4f4f:b0:413:1dbd:b4a9 with SMTP id m15-20020a05600c4f4f00b004131dbdb4a9mr1761282wmq.10.1710434136968; Thu, 14 Mar 2024 09:35:36 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dave Taht Date: Thu, 14 Mar 2024 12:35:25 -0400 Message-ID: To: libreqos , bloat Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: [Bloat] Fwd: [RFC PATCH net-next 0/5] net: In-kernel QUIC implementation with Userspace handshake X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Mar 2024 16:35:38 -0000 quic takes over ---------- Forwarded message --------- From: Xin Long Date: Mon, Mar 11, 2024 at 12:23=E2=80=AFPM Subject: [RFC PATCH net-next 0/5] net: In-kernel QUIC implementation with Userspace handshake To: network dev Cc: , , Eric Dumazet , Paolo Abeni , Steve French , Namjae Jeon , Chuck Lever III , Jeff Layton , Sabrina Dubroca , Tyler Fanelli , Pengtao He Introduction =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D This is an implementation of the QUIC protocol as defined in RFC9000. QUIC is an UDP-Based Multiplexed and Secure Transport protocol, and it provides applications with flow-controlled streams for structured communication, low-latency connection establishment, and network path migration. QUIC includes security measures that ensure confidentiality, integrity, and availability in a range of deployment circumstances. This implementation of QUIC in the kernel space enables users to utilize the QUIC protocol through common socket APIs in user space. Additionally, kernel subsystems like SMB and NFS can seamlessly operate over the QUIC protocol after handshake using net/handshake APIs. Note that In-Kernel QUIC implementation does NOT target Crypto Offload support for existing Userland QUICs, and Crypto Offload intended for Userland QUICs can NOT be utilized for Kernel consumers, such as SMB. Therefore, there is no conflict between In-Kernel QUIC and Crypto Offload for Userland QUICs. This implementation offers fundamental support for the following RFCs: - RFC9000 - QUIC: A UDP-Based Multiplexed and Secure Transport - RFC9001 - Using TLS to Secure QUIC - RFC9002 - QUIC Loss Detection and Congestion Control - RFC9221 - An Unreliable Datagram Extension to QUIC - RFC9287 - Greasing the QUIC Bit - RFC9368 - Compatible Version Negotiation for QUIC - RFC9369 - QUIC Version 2 - Handshake APIs for tlshd Use - SMB/NFS over QUIC Implementation =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D The central idea is to implement QUIC within the kernel, incorporating an userspace handshake approach. Only the processing and creation of raw TLS Handshake Messages, facilitated by a tls library like gnutls, take place in userspace. These messages are exchanged through sendmsg/recvmsg() mechanisms, with cryptographic details carried in the control message (cmsg). The entirety of QUIC protocol, excluding TLS Handshake Messages processing and creation, resides in the kernel. Instead of utilizing a User Level Protocol (ULP) layer, it establishes a socket of IPPROTO_QUIC type (similar to IPPROTO_MPTCP) operating over UDP tunnels. Kernel consumers can initiate a handshake request from kernel to userspace via the existing net/handshake netlink. The userspace component, tlshd from ktls-utils, manages the QUIC handshake request processing. - Handshake Architecture: +------+ +------+ | APP1 | | APP2 | ... +------+ +------+ +-------------------------------------------------+ | libquic (ktls-utils) |<--------------+ | {quic_handshake_server/client/param()} | | +-------------------------------------------------+ +---------------------+ {send/recvmsg()} {set/getsockopt()} | tlshd (ktls-utils) | [CMSG handshake_info] [SOCKOPT_CRYPTO_SECRET] +---------------------+ [SOCKOPT_TRANSPORT_PARAM_EXT] | ^ | ^ | ^ Userspace | | | | | | ------------|-|----------------------------|-|------------------------|-|= -------------- Kernel | | | | | | v | v | v | +--------------------------------------------------+ +-------------+ | socket (IPPRTOTO_QUIC) | protocol |<----+ | handshake | +--------------------------------------------------+ | | netlink APIs| | inqueue | outqueue | cong | path | connection_id | | +-------------+ +--------------------------------------------------+ | | = | | packet | frame | crypto | pnmap | | +-----+ +-----+ +--------------------------------------------------+ | | | | | | input | output | |---| SMB | | NFS | ... +--------------------------------------------------+ | | | | | | UDP tunnels | | +-----+ +--+--+ +--------------------------------------------------+ +-----------= ---| - Post Handshake Architecture: +------+ +------+ | APP1 | | APP2 | ... +------+ +------+ {send/recvmsg()} {set/getsockopt()} [CMSG stream_info] [SOCKOPT_KEY_UPDATE] [SOCKOPT_CONNECTION_MIGRATION] [SOCKOPT_STREAM_OPEN/RESET/STOP_SENDING] [...] | ^ | ^ Userspace | | | | ------------|-|----------------------------|-|---------------- Kernel | | | | v | v | +--------------------------------------------------+ | socket (IPPRTOTO_QUIC) | protocol |<----+ {kernel_send/recvmsg()} +--------------------------------------------------+ | {kernel_set/getsockopt()} | inqueue | outqueue | cong | path | connection_id | | +--------------------------------------------------+ | | packet | frame | crypto | pnmap | | +-----+ +-----+ +--------------------------------------------------+ | | | | | | input | output | |---| SMB | | NFS | ... +--------------------------------------------------+ | | | | | | UDP tunnels | | +-----+ +--+--+ +--------------------------------------------------+ +-----------= ---| Usage =3D=3D=3D=3D=3D This implementation supports a mapping of QUIC into sockets APIs. Similar to TCP and SCTP, a typical Server and Client use the following system call sequence to communicate: Client Server ------------------------------------------------------------------ sockfd =3D socket(IPPROTO_QUIC) listenfd =3D socket(IPPROTO_QUIC) bind(sockfd) bind(listenfd) listen(listenfd) connect(sockfd) quic_client_handshake(sockfd) sockfd =3D accecpt(listenfd) quic_server_handshake(sockfd, cert) sendmsg(sockfd) recvmsg(sockfd) close(sockfd) close(sockfd) close(listenfd) Please note that quic_client_handshake() and quic_server_handshake() functi= ons are currently sourced from libquic in the github lxin/quic repository, and = might be integrated into ktls-utils in the future. These functions are responsibl= e for receiving and processing the raw TLS handshake messages until the completio= n of the handshake process. For utilization by kernel consumers, it is essential to have the tlshd serv= ice (from ktls-utils) installed and running in userspace. This service receives and manages kernel handshake requests for kernel sockets. In kernel, the AP= Is closely resemble those used in userspace: Client Server -----------------------------------------------------------------------= - __sock_create(IPPROTO_QUIC, &sock) __sock_create(IPPROTO_QUIC, &sock) kernel_bind(sock) kernel_bind(sock) kernel_listen(sock) kernel_connect(sock) tls_client_hello_x509(args:{sock}) kernel_accept(sock, &newsock) tls_server_hello_x509(args:{newsock= }) kernel_sendmsg(sock) kernel_recvmsg(newsock) sock_release(sock) sock_release(newsock) sock_release(sock) Please be aware that tls_client_hello_x509() and tls_server_hello_x509() ar= e APIs from net/handshake/. They are employed to dispatch the handshake reque= st to the userspace tlshd service and subsequently block until the handshake process is completed. For advanced usage, see man doc: https://github.com/lxin/quic/wiki/man and examples: https://github.com/lxin/quic/tree/main/tests The QUIC module is currently labeled as "EXPERIMENTAL". Xin Long (5): net: define IPPROTO_QUIC and SOL_QUIC constants for QUIC protocol net: include quic.h in include/uapi/linux for QUIC protocol net: implement QUIC protocol code in net/quic directory net: integrate QUIC build configuration into Kconfig and Makefile Documentation: introduce quic.rst to provide description of QUIC protocol Documentation/networking/quic.rst | 160 +++ include/linux/socket.h | 1 + include/uapi/linux/in.h | 2 + include/uapi/linux/quic.h | 189 +++ net/Kconfig | 1 + net/Makefile | 1 + net/quic/Kconfig | 34 + net/quic/Makefile | 20 + net/quic/cong.c | 229 ++++ net/quic/cong.h | 84 ++ net/quic/connection.c | 172 +++ net/quic/connection.h | 117 ++ net/quic/crypto.c | 979 ++++++++++++++++ net/quic/crypto.h | 140 +++ net/quic/frame.c | 1803 ++++++++++++++++++++++++++++ net/quic/frame.h | 162 +++ net/quic/hashtable.h | 125 ++ net/quic/input.c | 693 +++++++++++ net/quic/input.h | 169 +++ net/quic/number.h | 174 +++ net/quic/output.c | 638 ++++++++++ net/quic/output.h | 194 +++ net/quic/packet.c | 1179 +++++++++++++++++++ net/quic/packet.h | 99 ++ net/quic/path.c | 434 +++++++ net/quic/path.h | 131 +++ net/quic/pnmap.c | 217 ++++ net/quic/pnmap.h | 134 +++ net/quic/protocol.c | 711 +++++++++++ net/quic/protocol.h | 56 + net/quic/sample_test.c | 339 ++++++ net/quic/socket.c | 1823 +++++++++++++++++++++++++++++ net/quic/socket.h | 293 +++++ net/quic/stream.c | 248 ++++ net/quic/stream.h | 147 +++ net/quic/timer.c | 241 ++++ net/quic/timer.h | 29 + net/quic/unit_test.c | 1024 ++++++++++++++++ 38 files changed, 13192 insertions(+) create mode 100644 Documentation/networking/quic.rst create mode 100644 include/uapi/linux/quic.h create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/cong.c create mode 100644 net/quic/cong.h create mode 100644 net/quic/connection.c create mode 100644 net/quic/connection.h create mode 100644 net/quic/crypto.c create mode 100644 net/quic/crypto.h create mode 100644 net/quic/frame.c create mode 100644 net/quic/frame.h create mode 100644 net/quic/hashtable.h create mode 100644 net/quic/input.c create mode 100644 net/quic/input.h create mode 100644 net/quic/number.h create mode 100644 net/quic/output.c create mode 100644 net/quic/output.h create mode 100644 net/quic/packet.c create mode 100644 net/quic/packet.h create mode 100644 net/quic/path.c create mode 100644 net/quic/path.h create mode 100644 net/quic/pnmap.c create mode 100644 net/quic/pnmap.h create mode 100644 net/quic/protocol.c create mode 100644 net/quic/protocol.h create mode 100644 net/quic/sample_test.c create mode 100644 net/quic/socket.c create mode 100644 net/quic/socket.h create mode 100644 net/quic/stream.c create mode 100644 net/quic/stream.h create mode 100644 net/quic/timer.c create mode 100644 net/quic/timer.h create mode 100644 net/quic/unit_test.c -- 2.43.0 --=20 https://www.youtube.com/watch?v=3DN0Tmvv5jJKs Epik Mellon Podcast Dave T=C3=A4ht CSO, LibreQos