* [LibreQoS] Fwd: [PATCH net-next RFC 00/20] Introducing P4TC
[not found] <20230124170346.316866-1-jhs@mojatatu.com>
@ 2023-01-27 14:04 ` Dave Taht
[not found] ` <20230126153022.23bea5f2@kernel.org>
1 sibling, 0 replies; 2+ messages in thread
From: Dave Taht @ 2023-01-27 14:04 UTC (permalink / raw)
To: libreqos
---------- Forwarded message ---------
From: Jamal Hadi Salim <jhs@mojatatu.com>
Date: Tue, Jan 24, 2023 at 9:19 AM
Subject: [PATCH net-next RFC 00/20] Introducing P4TC
To: <netdev@vger.kernel.org>
Cc: <kernel@mojatatu.com>, <deb.chatterjee@intel.com>,
<anjali.singhai@intel.com>, <namrata.limaye@intel.com>,
<khalidm@nvidia.com>, <tom@sipanda.io>, <pratyush@sipanda.io>,
<jiri@resnulli.us>, <xiyou.wangcong@gmail.com>, <davem@davemloft.net>,
<edumazet@google.com>, <kuba@kernel.org>, <pabeni@redhat.com>,
<vladbu@nvidia.com>, <simon.horman@corigine.com>,
<stefanc@marvell.com>, <seong.kim@amd.com>, <mattyk@nvidia.com>,
<dan.daly@intel.com>, <john.andy.fingerhut@intel.com>
We are seeking community feedback on P4TC patches.
Apologies, I know this is a large number of patches but it is the best we could
do so as not to miss the essence of the work. We have a few more patches but
took them out for brevity of review.
P4TC is an implementation of the Programming Protocol-independent Packet
Processors (P4) that is kernel based, building on top of many years of Linux TC
experiences:
* P4TC is scriptable - building on and extending the implementation/deployment
concepts of the TC u32 classifier, pedit action, etc.
* P4TC is designed to allow hardware offload based on experiences derived from
TC classifiers flower, u32, matchall, etc.
By "scriptable" we mean: these patches enable kernel and user space code change
independency for any P4 program that describes a new datapath. The workflow is
as follows:
1) A developer writes a P4 program, "myprog"
2) Compiles it using the P4C compiler. The compiler generates output in the
form of shell scripts which form template definitions for the different P4
objects "myprog" utilizes (objects described below in the patch list).
3) The developer (or operator) executes the shell scripts to manifest
the functional equivalent of "myprog" into the kernel.
4) The developer (or operator) instantiates "myprog" via the tc P4 filter
to ingress/egress of one or more netdevs/ports. Example:
"tc filter add block 22 ingress protocol ip prio 6 p4 pname myprog"
Once "myprog" is instantiated one can start updating table entries that are
associated with "myprog". Example:
tc p4runtime create myprog/mytable dstAddr 10.0.1.2/32 prio 10 \
action send param port type dev port1
Of course one can be more explicit and specify "skip_sw" or "skip_hw" to either
offload the entry (if a NIC or switch driver is capable) or make it purely run
entirely in the kernel or in a cooperative mode between kernel and user space.
Note: You do not need a compiler to create the template scripts used in
step #3. You can hand code them - however, there will be cases where you have
complex programs that would require the compiler.
Note2: There are no binary blobs being loaded into the kernel, rather a bunch
of "policies" to activate mechanisms in the kernel.
There have been many discussions and meetings since about 2015 in regards to
P4 over TC and now that the market has chosen P4 as the datapath specification
lingua franca we are finally proving the naysayers that we do get stuff done!
P4TC is designed to have very little impact on the core code for other users
of TC. We make one change to the core - to be specific we change the
implementation of action lists to use IDR instead of a linked list (see patch
#1); however, that change can be considered to be a control plane performance
improvement since IDR is faster in most cases.
The rest of the core changes(patches 2-9) are to enable P4TC and are minimalist
in nature. IOW, P4TC is self-contained and reuses the tc infrastructure without
affecting other consumers of the TC infra.
The core P4TC code implements several P4 objects.
1) Patch #10 implements the parser, kparser, which is based on Panda to allow
for a scriptable approach for describing the equivalence to a P4 parser.
2) Patch #11 introduces P4 data types which are consumed by the rest of the code
3) Patch #12 introduces the concept of templating Pipelines. i.e CRUD commands
for P4 pipelines.
4) Patch #13 introduces the concept of P4 user metadata and associated CRUD
template commands.
5) Patch #14 introduces the concept of P4 header fields and associated CRUD
template commands. Note header fields tie into the parser from patch #10.
6) Patch #15 introduces the concept of action templates and associated
CRUD commands.
7) Patch #16 introduces the concept of P4 table templates and associated
CRUD commands for tables
8) Patch #17 introduces the concept of table _runtime control_ and associated
CRUD commands.
9) Patch #18 introduces the concept of P4 register templates and associated
CRUD commands for registers.
9) Patch #19 introduces the concept of dynamic actions commands that are
used by actions (see patch #15).
11) Patch #20 introduces the TC P4 classifier used at runtime.
Speaking of testing - we have about 400 tdc test cases (which are left out
from this patch series). This number is growing.
These tests are run on our CICD system after commits are approved. The CICD does
a lot of other tests including:
checkpatch, sparse, 32 bit and 64 bit builds tested on both X86, ARM 64
and emulated BE via qemu s390. We trigger performance testing in the CICD
to catch performance regressions (currently only on the control path, but in
the future for the datapath).
Syzkaller runs 24/7 on dedicated hardware, and before main releases we put
the code via coverity. All of this has helped find bugs and ensure stability.
In addition we are working on a tool that will take a p4 program, run it through
the compiler, and generate permutations of traffic patterns that will test both
positive and negative code paths. The test generator tool is still work in
progress and will be generated by the P4 compiler.
There's a lot more info for the curious that we are leaving out for the sake
of brevity. A good starting point is to checkout recent material on the subject.
There is a presentation on P4TC as well as a workshop that took place in
Netdevconf 0x16), see:
https://netdevconf.info/0x16/session.html?Your-Network-Datapath-Will-Be-P4-Scripted
https://netdevconf.info/0x16/session.html?P4TC-Workshop
Jamal Hadi Salim (26):
net/sched: act_api: change act_base into an IDR
net/sched: act_api: increase action kind string length
net/sched: act_api: increase TCA_ID_MAX
net/sched: act_api: add init_ops to struct tc_action_op
net/sched: act_api: introduce tc_lookup_action_byid()
net/sched: act_api: export generic tc action searcher
net/sched: act_api: create and export __tcf_register_action
net/sched: act_api: add struct p4tc_action_ops as a parameter to
lookup callback
net: introduce rcu_replace_pointer_rtnl
p4tc: add P4 data types
p4tc: add pipeline create, get, update, delete
p4tc: add metadata create, update, delete, get, flush and dump
p4tc: add header field create, get, delete, flush and dump
p4tc: add action template create, update, delete, get, flush and dump
p4tc: add table create, update, delete, get, flush and dump
p4tc: add table entry create, update, get, delete, flush and dump
p4tc: add register create, update, delete, get, flush and dump
p4tc: add dynamic action commands
p4tc: add P4 classifier
selftests: tc-testing: add P4TC pipeline control path tdc tests
selftests: tc-testing: add P4TC metadata control path tdc tests
selftests: tc-testing: add P4TC action templates tdc tests
selftests: tc-testing: add P4TC table control path tdc tests
selftests: tc-testing: add P4TC table entries control path tdc tests
selftests: tc-testing: add P4TC register tdc tests
MAINTAINERS: add p4tc entry
Pratyush Khan (2):
net/kparser: add kParser
net/kparser: add kParser documentation
Documentation/networking/kParser.rst | 327 +
.../networking/parse_graph_example.svg | 2039 +++
MAINTAINERS | 14 +
include/linux/rtnetlink.h | 12 +
include/linux/skbuff.h | 17 +
include/net/act_api.h | 17 +-
include/net/kparser.h | 110 +
include/net/p4tc.h | 665 +
include/net/p4tc_types.h | 61 +
include/net/sch_generic.h | 5 +
include/net/tc_act/p4tc.h | 25 +
include/uapi/linux/kparser.h | 674 +
include/uapi/linux/p4tc.h | 510 +
include/uapi/linux/pkt_cls.h | 17 +-
include/uapi/linux/rtnetlink.h | 14 +
net/Kconfig | 9 +
net/Makefile | 1 +
net/core/skbuff.c | 17 +
net/kparser/Makefile | 17 +
net/kparser/kparser.h | 418 +
net/kparser/kparser_cmds.c | 917 ++
net/kparser/kparser_cmds_dump_ops.c | 586 +
net/kparser/kparser_cmds_ops.c | 3778 +++++
net/kparser/kparser_condexpr.h | 52 +
net/kparser/kparser_datapath.c | 1266 ++
net/kparser/kparser_main.c | 329 +
net/kparser/kparser_metaextract.h | 891 ++
net/kparser/kparser_types.h | 586 +
net/sched/Kconfig | 20 +
net/sched/Makefile | 3 +
net/sched/act_api.c | 156 +-
net/sched/cls_p4.c | 339 +
net/sched/p4tc/Makefile | 7 +
net/sched/p4tc/p4tc_action.c | 1907 +++
net/sched/p4tc/p4tc_cmds.c | 3492 +++++
net/sched/p4tc/p4tc_hdrfield.c | 625 +
net/sched/p4tc/p4tc_meta.c | 884 ++
net/sched/p4tc/p4tc_parser_api.c | 229 +
net/sched/p4tc/p4tc_pipeline.c | 996 ++
net/sched/p4tc/p4tc_register.c | 749 +
net/sched/p4tc/p4tc_table.c | 1636 ++
net/sched/p4tc/p4tc_tbl_api.c | 1895 +++
net/sched/p4tc/p4tc_tmpl_api.c | 609 +
net/sched/p4tc/p4tc_types.c | 1294 ++
net/sched/p4tc/trace.c | 10 +
net/sched/p4tc/trace.h | 45 +
security/selinux/nlmsgtab.c | 8 +-
.../tc-tests/p4tc/action_templates.json | 12378 ++++++++++++++++
.../tc-testing/tc-tests/p4tc/metadata.json | 2652 ++++
.../tc-testing/tc-tests/p4tc/pipeline.json | 3212 ++++
.../tc-testing/tc-tests/p4tc/register.json | 2752 ++++
.../tc-testing/tc-tests/p4tc/table.json | 8956 +++++++++++
.../tc-tests/p4tc/table_entries.json | 3818 +++++
53 files changed, 62001 insertions(+), 45 deletions(-)
create mode 100644 Documentation/networking/kParser.rst
create mode 100644 Documentation/networking/parse_graph_example.svg
create mode 100644 include/net/kparser.h
create mode 100644 include/net/p4tc.h
create mode 100644 include/net/p4tc_types.h
create mode 100644 include/net/tc_act/p4tc.h
create mode 100644 include/uapi/linux/kparser.h
create mode 100644 include/uapi/linux/p4tc.h
create mode 100644 net/kparser/Makefile
create mode 100644 net/kparser/kparser.h
create mode 100644 net/kparser/kparser_cmds.c
create mode 100644 net/kparser/kparser_cmds_dump_ops.c
create mode 100644 net/kparser/kparser_cmds_ops.c
create mode 100644 net/kparser/kparser_condexpr.h
create mode 100644 net/kparser/kparser_datapath.c
create mode 100644 net/kparser/kparser_main.c
create mode 100644 net/kparser/kparser_metaextract.h
create mode 100644 net/kparser/kparser_types.h
create mode 100644 net/sched/cls_p4.c
create mode 100644 net/sched/p4tc/Makefile
create mode 100644 net/sched/p4tc/p4tc_action.c
create mode 100644 net/sched/p4tc/p4tc_cmds.c
create mode 100644 net/sched/p4tc/p4tc_hdrfield.c
create mode 100644 net/sched/p4tc/p4tc_meta.c
create mode 100644 net/sched/p4tc/p4tc_parser_api.c
create mode 100644 net/sched/p4tc/p4tc_pipeline.c
create mode 100644 net/sched/p4tc/p4tc_register.c
create mode 100644 net/sched/p4tc/p4tc_table.c
create mode 100644 net/sched/p4tc/p4tc_tbl_api.c
create mode 100644 net/sched/p4tc/p4tc_tmpl_api.c
create mode 100644 net/sched/p4tc/p4tc_types.c
create mode 100644 net/sched/p4tc/trace.c
create mode 100644 net/sched/p4tc/trace.h
create mode 100644
tools/testing/selftests/tc-testing/tc-tests/p4tc/action_templates.json
create mode 100644
tools/testing/selftests/tc-testing/tc-tests/p4tc/metadata.json
create mode 100644
tools/testing/selftests/tc-testing/tc-tests/p4tc/pipeline.json
create mode 100644
tools/testing/selftests/tc-testing/tc-tests/p4tc/register.json
create mode 100644 tools/testing/selftests/tc-testing/tc-tests/p4tc/table.json
create mode 100644
tools/testing/selftests/tc-testing/tc-tests/p4tc/table_entries.json
--
2.34.1
Jamal Hadi Salim (19):
net/sched: act_api: change act_base into an IDR
net/sched: act_api: increase action kind string length
net/sched: act_api: increase TCA_ID_MAX
net/sched: act_api: add init_ops to struct tc_action_op
net/sched: act_api: introduce tc_lookup_action_byid()
net/sched: act_api: export generic tc action searcher
net/sched: act_api: create and export __tcf_register_action
net/sched: act_api: add struct p4tc_action_ops as a parameter to
lookup callback
net: introduce rcu_replace_pointer_rtnl
p4tc: add P4 data types
p4tc: add pipeline create, get, update, delete
p4tc: add metadata create, update, delete, get, flush and dump
p4tc: add header field create, get, delete, flush and dump
p4tc: add action template create, update, delete, get, flush and dump
p4tc: add table create, update, delete, get, flush and dump
p4tc: add table entry create, update, get, delete, flush and dump
p4tc: add register create, update, delete, get, flush and dump
p4tc: add dynamic action commands
p4tc: add P4 classifier
Pratyush Khan (1):
net/kparser: add kParser
include/linux/rtnetlink.h | 12 +
include/linux/skbuff.h | 17 +
include/net/act_api.h | 17 +-
include/net/kparser.h | 110 +
include/net/p4tc.h | 665 +++++
include/net/p4tc_types.h | 61 +
include/net/sch_generic.h | 5 +
include/net/tc_act/p4tc.h | 25 +
include/uapi/linux/kparser.h | 674 +++++
include/uapi/linux/p4tc.h | 510 ++++
include/uapi/linux/pkt_cls.h | 17 +-
include/uapi/linux/rtnetlink.h | 14 +
net/Kconfig | 9 +
net/Makefile | 1 +
net/core/skbuff.c | 17 +
net/kparser/Makefile | 17 +
net/kparser/kparser.h | 418 +++
net/kparser/kparser_cmds.c | 917 +++++++
net/kparser/kparser_cmds_dump_ops.c | 586 +++++
net/kparser/kparser_cmds_ops.c | 3778 +++++++++++++++++++++++++++
net/kparser/kparser_condexpr.h | 52 +
net/kparser/kparser_datapath.c | 1266 +++++++++
net/kparser/kparser_main.c | 329 +++
net/kparser/kparser_metaextract.h | 891 +++++++
net/kparser/kparser_types.h | 586 +++++
net/sched/Kconfig | 20 +
net/sched/Makefile | 3 +
net/sched/act_api.c | 156 +-
net/sched/cls_p4.c | 339 +++
net/sched/p4tc/Makefile | 7 +
net/sched/p4tc/p4tc_action.c | 1907 ++++++++++++++
net/sched/p4tc/p4tc_cmds.c | 3492 +++++++++++++++++++++++++
net/sched/p4tc/p4tc_hdrfield.c | 625 +++++
net/sched/p4tc/p4tc_meta.c | 884 +++++++
net/sched/p4tc/p4tc_parser_api.c | 229 ++
net/sched/p4tc/p4tc_pipeline.c | 1024 ++++++++
net/sched/p4tc/p4tc_register.c | 749 ++++++
net/sched/p4tc/p4tc_table.c | 1636 ++++++++++++
net/sched/p4tc/p4tc_tbl_api.c | 1898 ++++++++++++++
net/sched/p4tc/p4tc_tmpl_api.c | 609 +++++
net/sched/p4tc/p4tc_types.c | 1294 +++++++++
net/sched/p4tc/trace.c | 10 +
net/sched/p4tc/trace.h | 45 +
security/selinux/nlmsgtab.c | 8 +-
44 files changed, 25884 insertions(+), 45 deletions(-)
create mode 100644 include/net/kparser.h
create mode 100644 include/net/p4tc.h
create mode 100644 include/net/p4tc_types.h
create mode 100644 include/net/tc_act/p4tc.h
create mode 100644 include/uapi/linux/kparser.h
create mode 100644 include/uapi/linux/p4tc.h
create mode 100644 net/kparser/Makefile
create mode 100644 net/kparser/kparser.h
create mode 100644 net/kparser/kparser_cmds.c
create mode 100644 net/kparser/kparser_cmds_dump_ops.c
create mode 100644 net/kparser/kparser_cmds_ops.c
create mode 100644 net/kparser/kparser_condexpr.h
create mode 100644 net/kparser/kparser_datapath.c
create mode 100644 net/kparser/kparser_main.c
create mode 100644 net/kparser/kparser_metaextract.h
create mode 100644 net/kparser/kparser_types.h
create mode 100644 net/sched/cls_p4.c
create mode 100644 net/sched/p4tc/Makefile
create mode 100644 net/sched/p4tc/p4tc_action.c
create mode 100644 net/sched/p4tc/p4tc_cmds.c
create mode 100644 net/sched/p4tc/p4tc_hdrfield.c
create mode 100644 net/sched/p4tc/p4tc_meta.c
create mode 100644 net/sched/p4tc/p4tc_parser_api.c
create mode 100644 net/sched/p4tc/p4tc_pipeline.c
create mode 100644 net/sched/p4tc/p4tc_register.c
create mode 100644 net/sched/p4tc/p4tc_table.c
create mode 100644 net/sched/p4tc/p4tc_tbl_api.c
create mode 100644 net/sched/p4tc/p4tc_tmpl_api.c
create mode 100644 net/sched/p4tc/p4tc_types.c
create mode 100644 net/sched/p4tc/trace.c
create mode 100644 net/sched/p4tc/trace.h
--
2.34.1
--
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC
^ permalink raw reply [flat|nested] 2+ messages in thread
* [LibreQoS] Fwd: [PATCH net-next RFC 00/20] Introducing P4TC
[not found] ` <CAM0EoMnHcR9jFVtLt+L1FPRY5BK7_NgH3gsOZxQrXzEkaR1HYQ@mail.gmail.com>
@ 2023-01-27 14:05 ` Dave Taht
0 siblings, 0 replies; 2+ messages in thread
From: Dave Taht @ 2023-01-27 14:05 UTC (permalink / raw)
To: libreqos
---------- Forwarded message ---------
From: Jamal Hadi Salim <jhs@mojatatu.com>
Date: Fri, Jan 27, 2023 at 6:00 AM
Subject: Re: [PATCH net-next RFC 00/20] Introducing P4TC
To: Jakub Kicinski <kuba@kernel.org>
Cc: <netdev@vger.kernel.org>, <kernel@mojatatu.com>,
<deb.chatterjee@intel.com>, <anjali.singhai@intel.com>,
<namrata.limaye@intel.com>, <khalidm@nvidia.com>, <tom@sipanda.io>,
<pratyush@sipanda.io>, <jiri@resnulli.us>, <xiyou.wangcong@gmail.com>,
<davem@davemloft.net>, <edumazet@google.com>, <pabeni@redhat.com>,
<vladbu@nvidia.com>, <simon.horman@corigine.com>,
<stefanc@marvell.com>, <seong.kim@amd.com>, <mattyk@nvidia.com>,
<dan.daly@intel.com>, <john.andy.fingerhut@intel.com>
On Thu, Jan 26, 2023 at 6:30 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Tue, 24 Jan 2023 12:03:46 -0500 Jamal Hadi Salim wrote:
> > There have been many discussions and meetings since about 2015 in regards to
> > P4 over TC and now that the market has chosen P4 as the datapath specification
> > lingua franca
>
> Which market?
Network programmability involving hardware - where at minimal the
specification of the datapath is in P4 and
often the implementation is. For samples of specification using P4
(that are public) see for example MS Azure:
https://github.com/sonic-net/DASH/tree/main/dash-pipeline
If you are a vendor and want to sell a NIC in that space, the spec you
get is in P4. Your underlying hardware
doesnt have to be P4 native, but at minimal the abstraction (as we are
trying to provide with P4TC) has to be
able to consume the P4 specification.
For implementations where P4 is in use, there are many - some public
others not, sample space:
https://cloud.google.com/blog/products/gcp/google-cloud-using-p4runtime-to-build-smart-networks
There are NICs and switches which are P4 native in the market. IOW,
there is beacoup $ investment
in this space that makes it worth pursuing. TC is the kernel offload
mechanism that has gathered deployment
experience over many years - hence P4TC.
> Barely anyone understands the existing TC offloads.
Hyperboles like these are never helpful in a discussion.
TC offloads are deployed today, they work and many folks are actively
working on them.
Are there challenges? yes. For one (and this applies to all kernel
offloads) the process gets
in the way of exposing new features. So there are learnings that we
try to resolve in P4TC.
I'd be curious to hear about your suffering with TC offloads and see
if we can take that
experience and make things better.
>We'd need strong,
> and practical reasons to merge this. Speaking with my "have suffered
> thru the TC offloads working for a vendor" hat on, not the "junior
> maintainer" hat.
P4TC is "standalone" in that it does not affect other TC consumers or
any other subsystems on performance; it is also
sufficiently isolated in that you can choose to compile it out
altogether and more importantly it comes with committed
support.
And i should emphasize this discussion on getting P4 on TC has been
going on for a few years in the community
culminating with this.
cheers,
jamal
--
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2023-01-27 14:05 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20230124170346.316866-1-jhs@mojatatu.com>
2023-01-27 14:04 ` [LibreQoS] Fwd: [PATCH net-next RFC 00/20] Introducing P4TC Dave Taht
[not found] ` <20230126153022.23bea5f2@kernel.org>
[not found] ` <CAM0EoMnHcR9jFVtLt+L1FPRY5BK7_NgH3gsOZxQrXzEkaR1HYQ@mail.gmail.com>
2023-01-27 14:05 ` Dave Taht
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox