From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 4E1E43CB37 for ; Fri, 27 Jan 2023 09:04:15 -0500 (EST) Received: by mail-wr1-x42c.google.com with SMTP id h16so5033201wrz.12 for ; Fri, 27 Jan 2023 06:04:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=TcCHuTTbitH6WkrDYTpAXC9izBCYY6OP13DiTUg7Z1k=; b=CxpDMreQjSxvL0XCIeMvItPc27XwqzzPuilG8UR7oqo0IYAv1Jd04a+v5Q3zxlz+XL II8pTB8z+hN3w3y7k/teebZ2T+CRfcfTqG2jhPQt9kL52XM8qimEFwhTDmEaSDugqU/x kPZD6LjC8EcZseNp8GQCLlcQfT1WmL4FqD398bY4LmARJKhTp4TMZNRZnM490YbM508n H6f8Zd3wC+RBnscM03nigUoYMic/FuMKavyyIYlHJxjAf+wyA3Bz8dK0vLuSPSjOcFFg eC1Lspp8zb9jIUNL3NSUf0eEsx3USPABii0x85eCMejK7XTx3orKNOdIf0eiLWvfZ88Q PhoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TcCHuTTbitH6WkrDYTpAXC9izBCYY6OP13DiTUg7Z1k=; b=HoV/gndxgVTAO8a91c6tA+qGslG8VvGa1d7KVnOw8PXPMlg6jKkH+jzleYk5BDuBgR XesRRW29tdqI/NtX9H187TvxywF2Vn4wB6uHjRLEyTEMPP4OJzcXipXRC20M9DbZmceb /tVB5TzyYYHHCulV0vMaP3Jm8Y7kzYBreuh093chWP98i4LRf6C+yEAm9n7ZiVFjKQSt oGkHTQ6yq4Wyu0alY9bmInjyEY5preu+RNgYbOZor6+DJ7k3++nxich4tP+TCGkCWmxK bX94BLR/cM3ZPex8DozW2UZ53Ea5GJyI8biaVODEizntss/dGsdFOKutgTfaaKLtyOJJ 9ATg== X-Gm-Message-State: AFqh2kpJJmYKAcC7Z44Cw1htl33obVPFzc1Y64wm6YdxthyW3Cr/oj7W sUgKdQgAS5XAGJzl0vxD01oH9f6WM0gx0lWqYjBxeAUC X-Google-Smtp-Source: AMrXdXsdKFpR1UMyb1BaC1Cpn7ZSp2jC/L3hM+IFFeW4rTCayemPOa8afadrFsijBkmnEgbvDpkg9tfpHVQpPMVIG2M= X-Received: by 2002:a5d:5221:0:b0:2bd:c7e2:e972 with SMTP id i1-20020a5d5221000000b002bdc7e2e972mr1113437wra.383.1674828253367; Fri, 27 Jan 2023 06:04:13 -0800 (PST) MIME-Version: 1.0 References: <20230124170346.316866-1-jhs@mojatatu.com> In-Reply-To: <20230124170346.316866-1-jhs@mojatatu.com> From: Dave Taht Date: Fri, 27 Jan 2023 06:04:02 -0800 Message-ID: To: libreqos Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: [LibreQoS] Fwd: [PATCH net-next RFC 00/20] Introducing P4TC X-BeenThere: libreqos@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Many ISPs need the kinds of quality shaping cake can do List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jan 2023 14:04:15 -0000 ---------- Forwarded message --------- From: Jamal Hadi Salim Date: Tue, Jan 24, 2023 at 9:19 AM Subject: [PATCH net-next RFC 00/20] Introducing P4TC To: Cc: , , , , , , , , , , , , , , , , , , , We are seeking community feedback on P4TC patches. Apologies, I know this is a large number of patches but it is the best we c= ould do so as not to miss the essence of the work. We have a few more patches bu= t took them out for brevity of review. P4TC is an implementation of the Programming Protocol-independent Packet Processors (P4) that is kernel based, building on top of many years of Linu= x TC experiences: * P4TC is scriptable - building on and extending the implementation/deploy= ment concepts of the TC u32 classifier, pedit action, etc. * P4TC is designed to allow hardware offload based on experiences derived = from TC classifiers flower, u32, matchall, etc. By "scriptable" we mean: these patches enable kernel and user space code ch= ange independency for any P4 program that describes a new datapath. The workflow= is as follows: 1) A developer writes a P4 program, "myprog" 2) Compiles it using the P4C compiler. The compiler generates output in t= he form of shell scripts which form template definitions for the differen= t P4 objects "myprog" utilizes (objects described below in the patch list). 3) The developer (or operator) executes the shell scripts to manifest the functional equivalent of "myprog" into the kernel. 4) The developer (or operator) instantiates "myprog" via the tc P4 filter to ingress/egress of one or more netdevs/ports. Example: "tc filter add block 22 ingress protocol ip prio 6 p4 pname myprog" Once "myprog" is instantiated one can start updating table entries that are associated with "myprog". Example: tc p4runtime create myprog/mytable dstAddr 10.0.1.2/32 prio 10 \ action send param port type dev port1 Of course one can be more explicit and specify "skip_sw" or "skip_hw" to ei= ther offload the entry (if a NIC or switch driver is capable) or make it purely = run entirely in the kernel or in a cooperative mode between kernel and user spa= ce. Note: You do not need a compiler to create the template scripts used in step #3. You can hand code them - however, there will be cases where you ha= ve complex programs that would require the compiler. Note2: There are no binary blobs being loaded into the kernel, rather a bun= ch of "policies" to activate mechanisms in the kernel. There have been many discussions and meetings since about 2015 in regards t= o P4 over TC and now that the market has chosen P4 as the datapath specificat= ion lingua franca we are finally proving the naysayers that we do get stuff don= e! P4TC is designed to have very little impact on the core code for other user= s of TC. We make one change to the core - to be specific we change the implementation of action lists to use IDR instead of a linked list (see pat= ch #1); however, that change can be considered to be a control plane performan= ce improvement since IDR is faster in most cases. The rest of the core changes(patches 2-9) are to enable P4TC and are minima= list in nature. IOW, P4TC is self-contained and reuses the tc infrastructure wit= hout affecting other consumers of the TC infra. The core P4TC code implements several P4 objects. 1) Patch #10 implements the parser, kparser, which is based on Panda to all= ow for a scriptable approach for describing the equivalence to a P4 parser. 2) Patch #11 introduces P4 data types which are consumed by the rest of the= code 3) Patch #12 introduces the concept of templating Pipelines. i.e CRUD comma= nds for P4 pipelines. 4) Patch #13 introduces the concept of P4 user metadata and associated CRUD template commands. 5) Patch #14 introduces the concept of P4 header fields and associated CRUD template commands. Note header fields tie into the parser from patch #10= . 6) Patch #15 introduces the concept of action templates and associated CRUD commands. 7) Patch #16 introduces the concept of P4 table templates and associated CRUD commands for tables 8) Patch #17 introduces the concept of table _runtime control_ and associat= ed CRUD commands. 9) Patch #18 introduces the concept of P4 register templates and associated CRUD commands for registers. 9) Patch #19 introduces the concept of dynamic actions commands that are used by actions (see patch #15). 11) Patch #20 introduces the TC P4 classifier used at runtime. Speaking of testing - we have about 400 tdc test cases (which are left out from this patch series). This number is growing. These tests are run on our CICD system after commits are approved. The CICD= does a lot of other tests including: checkpatch, sparse, 32 bit and 64 bit builds tested on both X86, ARM 64 and emulated BE via qemu s390. We trigger performance testing in the CICD to catch performance regressions (currently only on the control path, but i= n the future for the datapath). Syzkaller runs 24/7 on dedicated hardware, and before main releases we put the code via coverity. All of this has helped find bugs and ensure stabilit= y. In addition we are working on a tool that will take a p4 program, run it th= rough the compiler, and generate permutations of traffic patterns that will test = both positive and negative code paths. The test generator tool is still work in progress and will be generated by the P4 compiler. There's a lot more info for the curious that we are leaving out for the sak= e of brevity. A good starting point is to checkout recent material on the sub= ject. There is a presentation on P4TC as well as a workshop that took place in Netdevconf 0x16), see: https://netdevconf.info/0x16/session.html?Your-Network-Datapath-Will-Be-P4-= Scripted https://netdevconf.info/0x16/session.html?P4TC-Workshop Jamal Hadi Salim (26): net/sched: act_api: change act_base into an IDR net/sched: act_api: increase action kind string length net/sched: act_api: increase TCA_ID_MAX net/sched: act_api: add init_ops to struct tc_action_op net/sched: act_api: introduce tc_lookup_action_byid() net/sched: act_api: export generic tc action searcher net/sched: act_api: create and export __tcf_register_action net/sched: act_api: add struct p4tc_action_ops as a parameter to lookup callback net: introduce rcu_replace_pointer_rtnl p4tc: add P4 data types p4tc: add pipeline create, get, update, delete p4tc: add metadata create, update, delete, get, flush and dump p4tc: add header field create, get, delete, flush and dump p4tc: add action template create, update, delete, get, flush and dump p4tc: add table create, update, delete, get, flush and dump p4tc: add table entry create, update, get, delete, flush and dump p4tc: add register create, update, delete, get, flush and dump p4tc: add dynamic action commands p4tc: add P4 classifier selftests: tc-testing: add P4TC pipeline control path tdc tests selftests: tc-testing: add P4TC metadata control path tdc tests selftests: tc-testing: add P4TC action templates tdc tests selftests: tc-testing: add P4TC table control path tdc tests selftests: tc-testing: add P4TC table entries control path tdc tests selftests: tc-testing: add P4TC register tdc tests MAINTAINERS: add p4tc entry Pratyush Khan (2): net/kparser: add kParser net/kparser: add kParser documentation Documentation/networking/kParser.rst | 327 + .../networking/parse_graph_example.svg | 2039 +++ MAINTAINERS | 14 + include/linux/rtnetlink.h | 12 + include/linux/skbuff.h | 17 + include/net/act_api.h | 17 +- include/net/kparser.h | 110 + include/net/p4tc.h | 665 + include/net/p4tc_types.h | 61 + include/net/sch_generic.h | 5 + include/net/tc_act/p4tc.h | 25 + include/uapi/linux/kparser.h | 674 + include/uapi/linux/p4tc.h | 510 + include/uapi/linux/pkt_cls.h | 17 +- include/uapi/linux/rtnetlink.h | 14 + net/Kconfig | 9 + net/Makefile | 1 + net/core/skbuff.c | 17 + net/kparser/Makefile | 17 + net/kparser/kparser.h | 418 + net/kparser/kparser_cmds.c | 917 ++ net/kparser/kparser_cmds_dump_ops.c | 586 + net/kparser/kparser_cmds_ops.c | 3778 +++++ net/kparser/kparser_condexpr.h | 52 + net/kparser/kparser_datapath.c | 1266 ++ net/kparser/kparser_main.c | 329 + net/kparser/kparser_metaextract.h | 891 ++ net/kparser/kparser_types.h | 586 + net/sched/Kconfig | 20 + net/sched/Makefile | 3 + net/sched/act_api.c | 156 +- net/sched/cls_p4.c | 339 + net/sched/p4tc/Makefile | 7 + net/sched/p4tc/p4tc_action.c | 1907 +++ net/sched/p4tc/p4tc_cmds.c | 3492 +++++ net/sched/p4tc/p4tc_hdrfield.c | 625 + net/sched/p4tc/p4tc_meta.c | 884 ++ net/sched/p4tc/p4tc_parser_api.c | 229 + net/sched/p4tc/p4tc_pipeline.c | 996 ++ net/sched/p4tc/p4tc_register.c | 749 + net/sched/p4tc/p4tc_table.c | 1636 ++ net/sched/p4tc/p4tc_tbl_api.c | 1895 +++ net/sched/p4tc/p4tc_tmpl_api.c | 609 + net/sched/p4tc/p4tc_types.c | 1294 ++ net/sched/p4tc/trace.c | 10 + net/sched/p4tc/trace.h | 45 + security/selinux/nlmsgtab.c | 8 +- .../tc-tests/p4tc/action_templates.json | 12378 ++++++++++++++++ .../tc-testing/tc-tests/p4tc/metadata.json | 2652 ++++ .../tc-testing/tc-tests/p4tc/pipeline.json | 3212 ++++ .../tc-testing/tc-tests/p4tc/register.json | 2752 ++++ .../tc-testing/tc-tests/p4tc/table.json | 8956 +++++++++++ .../tc-tests/p4tc/table_entries.json | 3818 +++++ 53 files changed, 62001 insertions(+), 45 deletions(-) create mode 100644 Documentation/networking/kParser.rst create mode 100644 Documentation/networking/parse_graph_example.svg create mode 100644 include/net/kparser.h create mode 100644 include/net/p4tc.h create mode 100644 include/net/p4tc_types.h create mode 100644 include/net/tc_act/p4tc.h create mode 100644 include/uapi/linux/kparser.h create mode 100644 include/uapi/linux/p4tc.h create mode 100644 net/kparser/Makefile create mode 100644 net/kparser/kparser.h create mode 100644 net/kparser/kparser_cmds.c create mode 100644 net/kparser/kparser_cmds_dump_ops.c create mode 100644 net/kparser/kparser_cmds_ops.c create mode 100644 net/kparser/kparser_condexpr.h create mode 100644 net/kparser/kparser_datapath.c create mode 100644 net/kparser/kparser_main.c create mode 100644 net/kparser/kparser_metaextract.h create mode 100644 net/kparser/kparser_types.h create mode 100644 net/sched/cls_p4.c create mode 100644 net/sched/p4tc/Makefile create mode 100644 net/sched/p4tc/p4tc_action.c create mode 100644 net/sched/p4tc/p4tc_cmds.c create mode 100644 net/sched/p4tc/p4tc_hdrfield.c create mode 100644 net/sched/p4tc/p4tc_meta.c create mode 100644 net/sched/p4tc/p4tc_parser_api.c create mode 100644 net/sched/p4tc/p4tc_pipeline.c create mode 100644 net/sched/p4tc/p4tc_register.c create mode 100644 net/sched/p4tc/p4tc_table.c create mode 100644 net/sched/p4tc/p4tc_tbl_api.c create mode 100644 net/sched/p4tc/p4tc_tmpl_api.c create mode 100644 net/sched/p4tc/p4tc_types.c create mode 100644 net/sched/p4tc/trace.c create mode 100644 net/sched/p4tc/trace.h create mode 100644 tools/testing/selftests/tc-testing/tc-tests/p4tc/action_templates.json create mode 100644 tools/testing/selftests/tc-testing/tc-tests/p4tc/metadata.json create mode 100644 tools/testing/selftests/tc-testing/tc-tests/p4tc/pipeline.json create mode 100644 tools/testing/selftests/tc-testing/tc-tests/p4tc/register.json create mode 100644 tools/testing/selftests/tc-testing/tc-tests/p4tc/table.= json create mode 100644 tools/testing/selftests/tc-testing/tc-tests/p4tc/table_entries.json -- 2.34.1 Jamal Hadi Salim (19): net/sched: act_api: change act_base into an IDR net/sched: act_api: increase action kind string length net/sched: act_api: increase TCA_ID_MAX net/sched: act_api: add init_ops to struct tc_action_op net/sched: act_api: introduce tc_lookup_action_byid() net/sched: act_api: export generic tc action searcher net/sched: act_api: create and export __tcf_register_action net/sched: act_api: add struct p4tc_action_ops as a parameter to lookup callback net: introduce rcu_replace_pointer_rtnl p4tc: add P4 data types p4tc: add pipeline create, get, update, delete p4tc: add metadata create, update, delete, get, flush and dump p4tc: add header field create, get, delete, flush and dump p4tc: add action template create, update, delete, get, flush and dump p4tc: add table create, update, delete, get, flush and dump p4tc: add table entry create, update, get, delete, flush and dump p4tc: add register create, update, delete, get, flush and dump p4tc: add dynamic action commands p4tc: add P4 classifier Pratyush Khan (1): net/kparser: add kParser include/linux/rtnetlink.h | 12 + include/linux/skbuff.h | 17 + include/net/act_api.h | 17 +- include/net/kparser.h | 110 + include/net/p4tc.h | 665 +++++ include/net/p4tc_types.h | 61 + include/net/sch_generic.h | 5 + include/net/tc_act/p4tc.h | 25 + include/uapi/linux/kparser.h | 674 +++++ include/uapi/linux/p4tc.h | 510 ++++ include/uapi/linux/pkt_cls.h | 17 +- include/uapi/linux/rtnetlink.h | 14 + net/Kconfig | 9 + net/Makefile | 1 + net/core/skbuff.c | 17 + net/kparser/Makefile | 17 + net/kparser/kparser.h | 418 +++ net/kparser/kparser_cmds.c | 917 +++++++ net/kparser/kparser_cmds_dump_ops.c | 586 +++++ net/kparser/kparser_cmds_ops.c | 3778 +++++++++++++++++++++++++++ net/kparser/kparser_condexpr.h | 52 + net/kparser/kparser_datapath.c | 1266 +++++++++ net/kparser/kparser_main.c | 329 +++ net/kparser/kparser_metaextract.h | 891 +++++++ net/kparser/kparser_types.h | 586 +++++ net/sched/Kconfig | 20 + net/sched/Makefile | 3 + net/sched/act_api.c | 156 +- net/sched/cls_p4.c | 339 +++ net/sched/p4tc/Makefile | 7 + net/sched/p4tc/p4tc_action.c | 1907 ++++++++++++++ net/sched/p4tc/p4tc_cmds.c | 3492 +++++++++++++++++++++++++ net/sched/p4tc/p4tc_hdrfield.c | 625 +++++ net/sched/p4tc/p4tc_meta.c | 884 +++++++ net/sched/p4tc/p4tc_parser_api.c | 229 ++ net/sched/p4tc/p4tc_pipeline.c | 1024 ++++++++ net/sched/p4tc/p4tc_register.c | 749 ++++++ net/sched/p4tc/p4tc_table.c | 1636 ++++++++++++ net/sched/p4tc/p4tc_tbl_api.c | 1898 ++++++++++++++ net/sched/p4tc/p4tc_tmpl_api.c | 609 +++++ net/sched/p4tc/p4tc_types.c | 1294 +++++++++ net/sched/p4tc/trace.c | 10 + net/sched/p4tc/trace.h | 45 + security/selinux/nlmsgtab.c | 8 +- 44 files changed, 25884 insertions(+), 45 deletions(-) create mode 100644 include/net/kparser.h create mode 100644 include/net/p4tc.h create mode 100644 include/net/p4tc_types.h create mode 100644 include/net/tc_act/p4tc.h create mode 100644 include/uapi/linux/kparser.h create mode 100644 include/uapi/linux/p4tc.h create mode 100644 net/kparser/Makefile create mode 100644 net/kparser/kparser.h create mode 100644 net/kparser/kparser_cmds.c create mode 100644 net/kparser/kparser_cmds_dump_ops.c create mode 100644 net/kparser/kparser_cmds_ops.c create mode 100644 net/kparser/kparser_condexpr.h create mode 100644 net/kparser/kparser_datapath.c create mode 100644 net/kparser/kparser_main.c create mode 100644 net/kparser/kparser_metaextract.h create mode 100644 net/kparser/kparser_types.h create mode 100644 net/sched/cls_p4.c create mode 100644 net/sched/p4tc/Makefile create mode 100644 net/sched/p4tc/p4tc_action.c create mode 100644 net/sched/p4tc/p4tc_cmds.c create mode 100644 net/sched/p4tc/p4tc_hdrfield.c create mode 100644 net/sched/p4tc/p4tc_meta.c create mode 100644 net/sched/p4tc/p4tc_parser_api.c create mode 100644 net/sched/p4tc/p4tc_pipeline.c create mode 100644 net/sched/p4tc/p4tc_register.c create mode 100644 net/sched/p4tc/p4tc_table.c create mode 100644 net/sched/p4tc/p4tc_tbl_api.c create mode 100644 net/sched/p4tc/p4tc_tmpl_api.c create mode 100644 net/sched/p4tc/p4tc_types.c create mode 100644 net/sched/p4tc/trace.c create mode 100644 net/sched/p4tc/trace.h -- 2.34.1 --=20 This song goes out to all the folk that thought Stadia would work: https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-69813666656= 07352320-FXtz Dave T=C3=A4ht CEO, TekLibre, LLC