* Re: [Make-wifi-fast] [PATCH] ath10k: snoc: enable threaded napi on WCN3990
[not found] <20221220075215.1.Ic12e347e0d61a618124b742614e82bbd5d770173@changeid>
@ 2022-12-20 15:10 ` Dave Taht
2022-12-28 23:53 ` Abhishek Kumar
0 siblings, 1 reply; 6+ messages in thread
From: Dave Taht @ 2022-12-20 15:10 UTC (permalink / raw)
To: Abhishek Kumar
Cc: kvalo, ath10k, linux-wireless, linux-kernel, netdev,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Make-Wifi-fast
I am always interested in flent.org tcp_nup, tcp_ndown, and rrul_be
tests on wifi hardware. In AP mode, especially, against a few clients
in rtt_fair on the "ending the anomaly" test suite at the bottom of
this link: https://www.cs.kau.se/tohojo/airtime-fairness/ . Of these,
it's trying to optimize bandwidth more fairly and keep latencies low
when 4 or more stations are trying to transmit (in a world with 16 or
more stations online), that increasingly bothers me the most. I'm
seeing 5+ seconds on some rtt_fair-like tests nowadays.
I was also seeing huge simultaneous upload vs download disparities on
the latest kernels, on various threads over here:
https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002 and
more recently here:
https://forum.openwrt.org/t/reducing-multiplexing-latencies-still-further-in-wifi/133605
I don't understand why napi with the default budget (64) is even
needed on the ath10k, as a single txop takes a minimum of ~200us, but
perhaps your patch will help. Still, measuring the TCP statistics
in-band would be nice to see. Some new tools are appearing that can do
this, Apple's goresponsiveness, crusader... that are simpler to use
than flent.
On Tue, Dec 20, 2022 at 12:17 AM Abhishek Kumar <kuabhs@chromium.org> wrote:
>
> NAPI poll can be done in threaded context along with soft irq
> context. Threaded context can be scheduled efficiently, thus
> creating less of bottleneck during Rx processing. This patch is
> to enable threaded NAPI on ath10k driver.
>
> Based on testing, it was observed that on WCN3990, the CPU0 reaches
> 100% utilization when napi runs in softirq context. At the same
> time the other CPUs are at low consumption percentage. This
> does not allow device to reach its maximum throughput potential.
> After enabling threaded napi, CPU load is balanced across all CPUs
> and following improvments were observed:
> - UDP_RX increase by ~22-25%
> - TCP_RX increase by ~15%
>
> Tested-on: WCN3990 hw1.0 SNOC WLAN.HL.3.2.2-00696-QCAHLSWMTPL-1
> Signed-off-by: Abhishek Kumar <kuabhs@chromium.org>
> ---
>
> drivers/net/wireless/ath/ath10k/core.c | 16 ++++++++++++++++
> drivers/net/wireless/ath/ath10k/hw.h | 2 ++
> drivers/net/wireless/ath/ath10k/snoc.c | 3 +++
> 3 files changed, 21 insertions(+)
>
> diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c
> index 5eb131ab916fd..ee4b6ba508c81 100644
> --- a/drivers/net/wireless/ath/ath10k/core.c
> +++ b/drivers/net/wireless/ath/ath10k/core.c
> @@ -100,6 +100,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = QCA988X_HW_2_0_VERSION,
> @@ -140,6 +141,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = QCA9887_HW_1_0_VERSION,
> @@ -181,6 +183,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = QCA6174_HW_3_2_VERSION,
> @@ -217,6 +220,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = QCA6174_HW_2_1_VERSION,
> @@ -257,6 +261,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = QCA6174_HW_2_1_VERSION,
> @@ -297,6 +302,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = QCA6174_HW_3_0_VERSION,
> @@ -337,6 +343,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = QCA6174_HW_3_2_VERSION,
> @@ -381,6 +388,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = QCA99X0_HW_2_0_DEV_VERSION,
> @@ -427,6 +435,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = QCA9984_HW_1_0_DEV_VERSION,
> @@ -480,6 +489,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = QCA9888_HW_2_0_DEV_VERSION,
> @@ -530,6 +540,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = QCA9377_HW_1_0_DEV_VERSION,
> @@ -570,6 +581,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = QCA9377_HW_1_1_DEV_VERSION,
> @@ -612,6 +624,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = QCA9377_HW_1_1_DEV_VERSION,
> @@ -645,6 +658,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = QCA4019_HW_1_0_DEV_VERSION,
> @@ -692,6 +706,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = false,
> .use_fw_tx_credits = true,
> .delay_unmap_buffer = false,
> + .enable_threaded_napi = false,
> },
> {
> .id = WCN3990_HW_1_0_DEV_VERSION,
> @@ -725,6 +740,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> .hw_restart_disconnect = true,
> .use_fw_tx_credits = false,
> .delay_unmap_buffer = true,
> + .enable_threaded_napi = true,
> },
> };
>
> diff --git a/drivers/net/wireless/ath/ath10k/hw.h b/drivers/net/wireless/ath/ath10k/hw.h
> index 9643031a4427a..adf3076b96503 100644
> --- a/drivers/net/wireless/ath/ath10k/hw.h
> +++ b/drivers/net/wireless/ath/ath10k/hw.h
> @@ -639,6 +639,8 @@ struct ath10k_hw_params {
> bool use_fw_tx_credits;
>
> bool delay_unmap_buffer;
> +
> + bool enable_threaded_napi;
> };
>
> struct htt_resp;
> diff --git a/drivers/net/wireless/ath/ath10k/snoc.c b/drivers/net/wireless/ath/ath10k/snoc.c
> index cfcb759a87dea..b94150fb6ef06 100644
> --- a/drivers/net/wireless/ath/ath10k/snoc.c
> +++ b/drivers/net/wireless/ath/ath10k/snoc.c
> @@ -927,6 +927,9 @@ static int ath10k_snoc_hif_start(struct ath10k *ar)
>
> bitmap_clear(ar_snoc->pending_ce_irqs, 0, CE_COUNT_MAX);
>
> + if (ar->hw_params.enable_threaded_napi)
> + dev_set_threaded(&ar->napi_dev, true);
> +
> ath10k_core_napi_enable(ar);
> ath10k_snoc_irq_enable(ar);
> ath10k_snoc_rx_post(ar);
> --
> 2.39.0.314.g84b9a713c41-goog
>
--
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Make-wifi-fast] [PATCH] ath10k: snoc: enable threaded napi on WCN3990
2022-12-20 15:10 ` [Make-wifi-fast] [PATCH] ath10k: snoc: enable threaded napi on WCN3990 Dave Taht
@ 2022-12-28 23:53 ` Abhishek Kumar
2022-12-29 0:49 ` Dave Taht
0 siblings, 1 reply; 6+ messages in thread
From: Abhishek Kumar @ 2022-12-28 23:53 UTC (permalink / raw)
To: Dave Taht
Cc: kvalo, ath10k, linux-wireless, linux-kernel, netdev,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Make-Wifi-fast
Apologies for the late reply, Thanks Dave for your comment. My answer is inline.
On Tue, Dec 20, 2022 at 7:10 AM Dave Taht <dave.taht@gmail.com> wrote:
>
> I am always interested in flent.org tcp_nup, tcp_ndown, and rrul_be
> tests on wifi hardware. In AP mode, especially, against a few clients
> in rtt_fair on the "ending the anomaly" test suite at the bottom of
> this link: https://www.cs.kau.se/tohojo/airtime-fairness/ . Of these,
> it's trying to optimize bandwidth more fairly and keep latencies low
> when 4 or more stations are trying to transmit (in a world with 16 or
> more stations online), that increasingly bothers me the most. I'm
> seeing 5+ seconds on some rtt_fair-like tests nowadays.
I used testing using iperf and conductive setup and fetched the
throughput data(mentioned below).
>
> I was also seeing huge simultaneous upload vs download disparities on
> the latest kernels, on various threads over here:
> https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002 and
> more recently here:
> https://forum.openwrt.org/t/reducing-multiplexing-latencies-still-further-in-wifi/133605
Interesting, thanks for the pointer and probably the Qualcomm team is
aware of it.
>
> I don't understand why napi with the default budget (64) is even
> needed on the ath10k, as a single txop takes a minimum of ~200us, but
> perhaps your patch will help. Still, measuring the TCP statistics
> in-band would be nice to see. Some new tools are appearing that can do
> this, Apple's goresponsiveness, crusader... that are simpler to use
> than flent.
Here are some of the additional raw data with and without threaded napi:
==================================================
udp_rx(Without threaded NAPI)
435.98+-5.16 : Channel 44
439.06+-0.66 : Channel 157
udp_rx(With threaded NAPI)
509.73+-41.03 : Channel 44
549.97+-7.62 : Channel 157
===================================================
udp_tx(Without threaded NAPI)
461.31+-0.69 : Channel 44
461.46+-0.78 : Channel 157
udp_tx(With threaded NAPI)
459.20+-0.77 : Channel 44
459.78+-1.08 : Channel 157
===================================================
tcp_rx(Without threaded NAPI)
472.63+-2.35 : Channel 44
469.29+-6.31 : Channel 157
tcp_rx(With threaded NAPI)
498.49+-2.44 : Channel 44
541.14+-40.65 : Channel 157
===================================================
tcp_tx(Without threaded NAPI)
317.34+-2.37 : Channel 44
317.01+-2.56 : Channel 157
tcp_tx(With threaded NAPI)
371.34+-2.36 : Channel 44
376.95+-9.40 : Channel 157
====================================================
>
> On Tue, Dec 20, 2022 at 12:17 AM Abhishek Kumar <kuabhs@chromium.org> wrote:
> >
> > NAPI poll can be done in threaded context along with soft irq
> > context. Threaded context can be scheduled efficiently, thus
> > creating less of bottleneck during Rx processing. This patch is
> > to enable threaded NAPI on ath10k driver.
> >
> > Based on testing, it was observed that on WCN3990, the CPU0 reaches
> > 100% utilization when napi runs in softirq context. At the same
> > time the other CPUs are at low consumption percentage. This
> > does not allow device to reach its maximum throughput potential.
> > After enabling threaded napi, CPU load is balanced across all CPUs
> > and following improvments were observed:
> > - UDP_RX increase by ~22-25%
> > - TCP_RX increase by ~15%
> >
> > Tested-on: WCN3990 hw1.0 SNOC WLAN.HL.3.2.2-00696-QCAHLSWMTPL-1
> > Signed-off-by: Abhishek Kumar <kuabhs@chromium.org>
> > ---
> >
> > drivers/net/wireless/ath/ath10k/core.c | 16 ++++++++++++++++
> > drivers/net/wireless/ath/ath10k/hw.h | 2 ++
> > drivers/net/wireless/ath/ath10k/snoc.c | 3 +++
> > 3 files changed, 21 insertions(+)
> >
> > diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c
> > index 5eb131ab916fd..ee4b6ba508c81 100644
> > --- a/drivers/net/wireless/ath/ath10k/core.c
> > +++ b/drivers/net/wireless/ath/ath10k/core.c
> > @@ -100,6 +100,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = QCA988X_HW_2_0_VERSION,
> > @@ -140,6 +141,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = QCA9887_HW_1_0_VERSION,
> > @@ -181,6 +183,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = QCA6174_HW_3_2_VERSION,
> > @@ -217,6 +220,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = QCA6174_HW_2_1_VERSION,
> > @@ -257,6 +261,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = QCA6174_HW_2_1_VERSION,
> > @@ -297,6 +302,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = QCA6174_HW_3_0_VERSION,
> > @@ -337,6 +343,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = QCA6174_HW_3_2_VERSION,
> > @@ -381,6 +388,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = QCA99X0_HW_2_0_DEV_VERSION,
> > @@ -427,6 +435,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = QCA9984_HW_1_0_DEV_VERSION,
> > @@ -480,6 +489,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = QCA9888_HW_2_0_DEV_VERSION,
> > @@ -530,6 +540,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = QCA9377_HW_1_0_DEV_VERSION,
> > @@ -570,6 +581,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = QCA9377_HW_1_1_DEV_VERSION,
> > @@ -612,6 +624,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = QCA9377_HW_1_1_DEV_VERSION,
> > @@ -645,6 +658,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = QCA4019_HW_1_0_DEV_VERSION,
> > @@ -692,6 +706,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = false,
> > .use_fw_tx_credits = true,
> > .delay_unmap_buffer = false,
> > + .enable_threaded_napi = false,
> > },
> > {
> > .id = WCN3990_HW_1_0_DEV_VERSION,
> > @@ -725,6 +740,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > .hw_restart_disconnect = true,
> > .use_fw_tx_credits = false,
> > .delay_unmap_buffer = true,
> > + .enable_threaded_napi = true,
> > },
> > };
> >
> > diff --git a/drivers/net/wireless/ath/ath10k/hw.h b/drivers/net/wireless/ath/ath10k/hw.h
> > index 9643031a4427a..adf3076b96503 100644
> > --- a/drivers/net/wireless/ath/ath10k/hw.h
> > +++ b/drivers/net/wireless/ath/ath10k/hw.h
> > @@ -639,6 +639,8 @@ struct ath10k_hw_params {
> > bool use_fw_tx_credits;
> >
> > bool delay_unmap_buffer;
> > +
> > + bool enable_threaded_napi;
> > };
> >
> > struct htt_resp;
> > diff --git a/drivers/net/wireless/ath/ath10k/snoc.c b/drivers/net/wireless/ath/ath10k/snoc.c
> > index cfcb759a87dea..b94150fb6ef06 100644
> > --- a/drivers/net/wireless/ath/ath10k/snoc.c
> > +++ b/drivers/net/wireless/ath/ath10k/snoc.c
> > @@ -927,6 +927,9 @@ static int ath10k_snoc_hif_start(struct ath10k *ar)
> >
> > bitmap_clear(ar_snoc->pending_ce_irqs, 0, CE_COUNT_MAX);
> >
> > + if (ar->hw_params.enable_threaded_napi)
> > + dev_set_threaded(&ar->napi_dev, true);
> > +
> > ath10k_core_napi_enable(ar);
> > ath10k_snoc_irq_enable(ar);
> > ath10k_snoc_rx_post(ar);
> > --
> > 2.39.0.314.g84b9a713c41-goog
> >
>
>
> --
> This song goes out to all the folk that thought Stadia would work:
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> Dave Täht CEO, TekLibre, LLC
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Make-wifi-fast] [PATCH] ath10k: snoc: enable threaded napi on WCN3990
2022-12-28 23:53 ` Abhishek Kumar
@ 2022-12-29 0:49 ` Dave Taht
2022-12-29 5:54 ` Bob McMahon
0 siblings, 1 reply; 6+ messages in thread
From: Dave Taht @ 2022-12-29 0:49 UTC (permalink / raw)
To: Abhishek Kumar
Cc: kvalo, ath10k, linux-wireless, linux-kernel, netdev,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Make-Wifi-fast
On Wed, Dec 28, 2022 at 3:53 PM Abhishek Kumar <kuabhs@chromium.org> wrote:
>
> Apologies for the late reply, Thanks Dave for your comment. My answer is inline.
>
> On Tue, Dec 20, 2022 at 7:10 AM Dave Taht <dave.taht@gmail.com> wrote:
> >
> > I am always interested in flent.org tcp_nup, tcp_ndown, and rrul_be
> > tests on wifi hardware. In AP mode, especially, against a few clients
> > in rtt_fair on the "ending the anomaly" test suite at the bottom of
> > this link: https://www.cs.kau.se/tohojo/airtime-fairness/ . Of these,
> > it's trying to optimize bandwidth more fairly and keep latencies low
> > when 4 or more stations are trying to transmit (in a world with 16 or
> > more stations online), that increasingly bothers me the most. I'm
> > seeing 5+ seconds on some rtt_fair-like tests nowadays.
> I used testing using iperf and conductive setup and fetched the
> throughput data(mentioned below).
> >
> > I was also seeing huge simultaneous upload vs download disparities on
> > the latest kernels, on various threads over here:
> > https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002 and
> > more recently here:
> > https://forum.openwrt.org/t/reducing-multiplexing-latencies-still-further-in-wifi/133605
> Interesting, thanks for the pointer and probably the Qualcomm team is
> aware of it.
> >
> > I don't understand why napi with the default budget (64) is even
> > needed on the ath10k, as a single txop takes a minimum of ~200us, but
> > perhaps your patch will help. Still, measuring the TCP statistics
> > in-band would be nice to see. Some new tools are appearing that can do
> > this, Apple's goresponsiveness, crusader... that are simpler to use
> > than flent.
> Here are some of the additional raw data with and without threaded napi:
> ==================================================
> udp_rx(Without threaded NAPI)
> 435.98+-5.16 : Channel 44
> 439.06+-0.66 : Channel 157
>
> udp_rx(With threaded NAPI)
> 509.73+-41.03 : Channel 44
> 549.97+-7.62 : Channel 157
> ===================================================
> udp_tx(Without threaded NAPI)
> 461.31+-0.69 : Channel 44
> 461.46+-0.78 : Channel 157
>
> udp_tx(With threaded NAPI)
> 459.20+-0.77 : Channel 44
> 459.78+-1.08 : Channel 157
> ===================================================
> tcp_rx(Without threaded NAPI)
> 472.63+-2.35 : Channel 44
> 469.29+-6.31 : Channel 157
>
> tcp_rx(With threaded NAPI)
> 498.49+-2.44 : Channel 44
> 541.14+-40.65 : Channel 157
> ===================================================
> tcp_tx(Without threaded NAPI)
> 317.34+-2.37 : Channel 44
> 317.01+-2.56 : Channel 157
>
> tcp_tx(With threaded NAPI)
> 371.34+-2.36 : Channel 44
> 376.95+-9.40 : Channel 157
My concern is primarily with the induced tcp latency on this test. A
way to check that is to run wireshark on your test client driving the
test, capture the iperf traffic, and then plot the "Statistics->TCP
stream statistics for both throughput and rtt. Would it be possible
for you to do that and put up those plots somewhere?
The worst case test is a tcp bidirectional test which I don't know if
older iperfs can do. (iperf2 has new bounceback and bidir tests)
Ideally stuff going in either direction, would not look as horrible,
as it did, back in 2016, documented in this linuxplumbers presentation
here: https://blog.linuxplumbersconf.org/2016/ocw/system/presentations/3963/original/linuxplumbers_wifi_latency-3Nov.pdf
and discussed on lwn, here: https://lwn.net/Articles/705884/
I worry about folk achieving slightly better tcp throughput at the
expense of clobbering in-tcp-stream latency. Back then we were
shooting for no more than 40ms extra latency under load on this chip,
down from (unusable) seconds. Presently elsewhere, on other chips,
we're getting 8ms with stuff that's not in tree for the ath10k, there
is a slight cost in single stream throughput but when multiple streams
are in use, on multiple stations, things like web pages fly,
irrespective of load.
> ====================================================
>
> >
> > On Tue, Dec 20, 2022 at 12:17 AM Abhishek Kumar <kuabhs@chromium.org> wrote:
> > >
> > > NAPI poll can be done in threaded context along with soft irq
> > > context. Threaded context can be scheduled efficiently, thus
> > > creating less of bottleneck during Rx processing. This patch is
> > > to enable threaded NAPI on ath10k driver.
> > >
> > > Based on testing, it was observed that on WCN3990, the CPU0 reaches
> > > 100% utilization when napi runs in softirq context. At the same
> > > time the other CPUs are at low consumption percentage. This
> > > does not allow device to reach its maximum throughput potential.
> > > After enabling threaded napi, CPU load is balanced across all CPUs
> > > and following improvments were observed:
> > > - UDP_RX increase by ~22-25%
> > > - TCP_RX increase by ~15%
> > >
> > > Tested-on: WCN3990 hw1.0 SNOC WLAN.HL.3.2.2-00696-QCAHLSWMTPL-1
> > > Signed-off-by: Abhishek Kumar <kuabhs@chromium.org>
> > > ---
> > >
> > > drivers/net/wireless/ath/ath10k/core.c | 16 ++++++++++++++++
> > > drivers/net/wireless/ath/ath10k/hw.h | 2 ++
> > > drivers/net/wireless/ath/ath10k/snoc.c | 3 +++
> > > 3 files changed, 21 insertions(+)
> > >
> > > diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c
> > > index 5eb131ab916fd..ee4b6ba508c81 100644
> > > --- a/drivers/net/wireless/ath/ath10k/core.c
> > > +++ b/drivers/net/wireless/ath/ath10k/core.c
> > > @@ -100,6 +100,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = QCA988X_HW_2_0_VERSION,
> > > @@ -140,6 +141,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = QCA9887_HW_1_0_VERSION,
> > > @@ -181,6 +183,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = QCA6174_HW_3_2_VERSION,
> > > @@ -217,6 +220,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = QCA6174_HW_2_1_VERSION,
> > > @@ -257,6 +261,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = QCA6174_HW_2_1_VERSION,
> > > @@ -297,6 +302,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = QCA6174_HW_3_0_VERSION,
> > > @@ -337,6 +343,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = QCA6174_HW_3_2_VERSION,
> > > @@ -381,6 +388,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = QCA99X0_HW_2_0_DEV_VERSION,
> > > @@ -427,6 +435,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = QCA9984_HW_1_0_DEV_VERSION,
> > > @@ -480,6 +489,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = QCA9888_HW_2_0_DEV_VERSION,
> > > @@ -530,6 +540,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = QCA9377_HW_1_0_DEV_VERSION,
> > > @@ -570,6 +581,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = QCA9377_HW_1_1_DEV_VERSION,
> > > @@ -612,6 +624,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = QCA9377_HW_1_1_DEV_VERSION,
> > > @@ -645,6 +658,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = QCA4019_HW_1_0_DEV_VERSION,
> > > @@ -692,6 +706,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = false,
> > > .use_fw_tx_credits = true,
> > > .delay_unmap_buffer = false,
> > > + .enable_threaded_napi = false,
> > > },
> > > {
> > > .id = WCN3990_HW_1_0_DEV_VERSION,
> > > @@ -725,6 +740,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
> > > .hw_restart_disconnect = true,
> > > .use_fw_tx_credits = false,
> > > .delay_unmap_buffer = true,
> > > + .enable_threaded_napi = true,
> > > },
> > > };
> > >
> > > diff --git a/drivers/net/wireless/ath/ath10k/hw.h b/drivers/net/wireless/ath/ath10k/hw.h
> > > index 9643031a4427a..adf3076b96503 100644
> > > --- a/drivers/net/wireless/ath/ath10k/hw.h
> > > +++ b/drivers/net/wireless/ath/ath10k/hw.h
> > > @@ -639,6 +639,8 @@ struct ath10k_hw_params {
> > > bool use_fw_tx_credits;
> > >
> > > bool delay_unmap_buffer;
> > > +
> > > + bool enable_threaded_napi;
> > > };
> > >
> > > struct htt_resp;
> > > diff --git a/drivers/net/wireless/ath/ath10k/snoc.c b/drivers/net/wireless/ath/ath10k/snoc.c
> > > index cfcb759a87dea..b94150fb6ef06 100644
> > > --- a/drivers/net/wireless/ath/ath10k/snoc.c
> > > +++ b/drivers/net/wireless/ath/ath10k/snoc.c
> > > @@ -927,6 +927,9 @@ static int ath10k_snoc_hif_start(struct ath10k *ar)
> > >
> > > bitmap_clear(ar_snoc->pending_ce_irqs, 0, CE_COUNT_MAX);
> > >
> > > + if (ar->hw_params.enable_threaded_napi)
> > > + dev_set_threaded(&ar->napi_dev, true);
> > > +
> > > ath10k_core_napi_enable(ar);
> > > ath10k_snoc_irq_enable(ar);
> > > ath10k_snoc_rx_post(ar);
> > > --
> > > 2.39.0.314.g84b9a713c41-goog
> > >
> >
> >
> > --
> > This song goes out to all the folk that thought Stadia would work:
> > https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> > Dave Täht CEO, TekLibre, LLC
--
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Make-wifi-fast] [PATCH] ath10k: snoc: enable threaded napi on WCN3990
2022-12-29 0:49 ` Dave Taht
@ 2022-12-29 5:54 ` Bob McMahon
2022-12-30 3:44 ` Dave Taht
0 siblings, 1 reply; 6+ messages in thread
From: Bob McMahon @ 2022-12-29 5:54 UTC (permalink / raw)
To: Dave Taht
Cc: Abhishek Kumar, netdev, kvalo, linux-wireless, linux-kernel,
ath10k, Eric Dumazet, Make-Wifi-fast, Jakub Kicinski,
Paolo Abeni, David S. Miller
[-- Attachment #1: Type: text/plain, Size: 17056 bytes --]
A bit of a tangent but iperf 2's enhanced option provides sampled RTT and
CWND in the interval output. I requires -e. It also provide the connect
time, the icwnd, irtt and the mss per the connect.
[rjmcmahon@rjm-nas ~]$ iperf -c 192.168.1.85 -i 1 -e
------------------------------------------------------------
Client connecting to 192.168.1.85, TCP port 5001 with pid 20041 (1 flows)
Write buffer size: 131072 Byte
TOS set to 0x0 (Nagle on)
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 1] local 192.168.1.73%eno1 port 38822 connected with 192.168.1.85 port
5001 (sock=3) (icwnd/mss/irtt=14/1448/246) (ct=0.32 ms) on 2022-12-28
21:51:13 (PST)
[ ID] Interval Transfer Bandwidth Write/Err Rtry
Cwnd/RTT(var) NetPwr
[ 1] 0.00-1.00 sec 114 MBytes 952 Mbits/sec 908/0 3
353K/1901(59) us 62606
[ 1] 1.00-2.00 sec 111 MBytes 932 Mbits/sec 889/0 0
354K/2029(115) us 57429
[ 1] 2.00-3.00 sec 112 MBytes 937 Mbits/sec 894/0 3
354K/2061(84) us 56855
[ 1] 3.00-4.00 sec 112 MBytes 938 Mbits/sec 895/0 3
354K/2027(115) us 57873
[ 1] 4.00-5.00 sec 112 MBytes 938 Mbits/sec 895/0 6
354K/2046(68) us 57336
[ 1] 5.00-6.00 sec 112 MBytes 937 Mbits/sec 894/0 0
374K/2160(109) us 54249
[ 1] 6.00-7.00 sec 111 MBytes 933 Mbits/sec 890/0 2
374K/2155(115) us 54132
[ 1] 7.00-8.00 sec 111 MBytes 934 Mbits/sec 891/0 2
374K/2139(120) us 54598
[ 1] 8.00-9.00 sec 112 MBytes 938 Mbits/sec 895/0 0
374K/2129(126) us 55101
[ 1] 9.00-10.00 sec 112 MBytes 935 Mbits/sec 892/0 1
374K/2200(102) us 53144
[ 1] 0.00-10.03 sec 1.09 GBytes 935 Mbits/sec 8944/0 20
374K/4082(4010) us 28632
Bob
On Wed, Dec 28, 2022 at 4:49 PM Dave Taht via Make-wifi-fast <
make-wifi-fast@lists.bufferbloat.net> wrote:
> On Wed, Dec 28, 2022 at 3:53 PM Abhishek Kumar <kuabhs@chromium.org>
> wrote:
> >
> > Apologies for the late reply, Thanks Dave for your comment. My answer is
> inline.
> >
> > On Tue, Dec 20, 2022 at 7:10 AM Dave Taht <dave.taht@gmail.com> wrote:
> > >
> > > I am always interested in flent.org tcp_nup, tcp_ndown, and rrul_be
> > > tests on wifi hardware. In AP mode, especially, against a few clients
> > > in rtt_fair on the "ending the anomaly" test suite at the bottom of
> > > this link: https://www.cs.kau.se/tohojo/airtime-fairness/ . Of these,
> > > it's trying to optimize bandwidth more fairly and keep latencies low
> > > when 4 or more stations are trying to transmit (in a world with 16 or
> > > more stations online), that increasingly bothers me the most. I'm
> > > seeing 5+ seconds on some rtt_fair-like tests nowadays.
> > I used testing using iperf and conductive setup and fetched the
> > throughput data(mentioned below).
> > >
> > > I was also seeing huge simultaneous upload vs download disparities on
> > > the latest kernels, on various threads over here:
> > > https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002 and
> > > more recently here:
> > >
> https://forum.openwrt.org/t/reducing-multiplexing-latencies-still-further-in-wifi/133605
> > Interesting, thanks for the pointer and probably the Qualcomm team is
> > aware of it.
> > >
> > > I don't understand why napi with the default budget (64) is even
> > > needed on the ath10k, as a single txop takes a minimum of ~200us, but
> > > perhaps your patch will help. Still, measuring the TCP statistics
> > > in-band would be nice to see. Some new tools are appearing that can do
> > > this, Apple's goresponsiveness, crusader... that are simpler to use
> > > than flent.
> > Here are some of the additional raw data with and without threaded napi:
> > ==================================================
> > udp_rx(Without threaded NAPI)
> > 435.98+-5.16 : Channel 44
> > 439.06+-0.66 : Channel 157
> >
> > udp_rx(With threaded NAPI)
> > 509.73+-41.03 : Channel 44
> > 549.97+-7.62 : Channel 157
> > ===================================================
> > udp_tx(Without threaded NAPI)
> > 461.31+-0.69 : Channel 44
> > 461.46+-0.78 : Channel 157
> >
> > udp_tx(With threaded NAPI)
> > 459.20+-0.77 : Channel 44
> > 459.78+-1.08 : Channel 157
> > ===================================================
> > tcp_rx(Without threaded NAPI)
> > 472.63+-2.35 : Channel 44
> > 469.29+-6.31 : Channel 157
> >
> > tcp_rx(With threaded NAPI)
> > 498.49+-2.44 : Channel 44
> > 541.14+-40.65 : Channel 157
> > ===================================================
> > tcp_tx(Without threaded NAPI)
> > 317.34+-2.37 : Channel 44
> > 317.01+-2.56 : Channel 157
> >
> > tcp_tx(With threaded NAPI)
> > 371.34+-2.36 : Channel 44
> > 376.95+-9.40 : Channel 157
>
> My concern is primarily with the induced tcp latency on this test. A
> way to check that is to run wireshark on your test client driving the
> test, capture the iperf traffic, and then plot the "Statistics->TCP
> stream statistics for both throughput and rtt. Would it be possible
> for you to do that and put up those plots somewhere?
>
> The worst case test is a tcp bidirectional test which I don't know if
> older iperfs can do. (iperf2 has new bounceback and bidir tests)
>
> Ideally stuff going in either direction, would not look as horrible,
> as it did, back in 2016, documented in this linuxplumbers presentation
> here:
> https://blog.linuxplumbersconf.org/2016/ocw/system/presentations/3963/original/linuxplumbers_wifi_latency-3Nov.pdf
> and discussed on lwn, here: https://lwn.net/Articles/705884/
>
> I worry about folk achieving slightly better tcp throughput at the
> expense of clobbering in-tcp-stream latency. Back then we were
> shooting for no more than 40ms extra latency under load on this chip,
> down from (unusable) seconds. Presently elsewhere, on other chips,
> we're getting 8ms with stuff that's not in tree for the ath10k, there
> is a slight cost in single stream throughput but when multiple streams
> are in use, on multiple stations, things like web pages fly,
> irrespective of load.
>
>
> > ====================================================
> >
> > >
> > > On Tue, Dec 20, 2022 at 12:17 AM Abhishek Kumar <kuabhs@chromium.org>
> wrote:
> > > >
> > > > NAPI poll can be done in threaded context along with soft irq
> > > > context. Threaded context can be scheduled efficiently, thus
> > > > creating less of bottleneck during Rx processing. This patch is
> > > > to enable threaded NAPI on ath10k driver.
> > > >
> > > > Based on testing, it was observed that on WCN3990, the CPU0 reaches
> > > > 100% utilization when napi runs in softirq context. At the same
> > > > time the other CPUs are at low consumption percentage. This
> > > > does not allow device to reach its maximum throughput potential.
> > > > After enabling threaded napi, CPU load is balanced across all CPUs
> > > > and following improvments were observed:
> > > > - UDP_RX increase by ~22-25%
> > > > - TCP_RX increase by ~15%
> > > >
> > > > Tested-on: WCN3990 hw1.0 SNOC WLAN.HL.3.2.2-00696-QCAHLSWMTPL-1
> > > > Signed-off-by: Abhishek Kumar <kuabhs@chromium.org>
> > > > ---
> > > >
> > > > drivers/net/wireless/ath/ath10k/core.c | 16 ++++++++++++++++
> > > > drivers/net/wireless/ath/ath10k/hw.h | 2 ++
> > > > drivers/net/wireless/ath/ath10k/snoc.c | 3 +++
> > > > 3 files changed, 21 insertions(+)
> > > >
> > > > diff --git a/drivers/net/wireless/ath/ath10k/core.c
> b/drivers/net/wireless/ath/ath10k/core.c
> > > > index 5eb131ab916fd..ee4b6ba508c81 100644
> > > > --- a/drivers/net/wireless/ath/ath10k/core.c
> > > > +++ b/drivers/net/wireless/ath/ath10k/core.c
> > > > @@ -100,6 +100,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = QCA988X_HW_2_0_VERSION,
> > > > @@ -140,6 +141,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = QCA9887_HW_1_0_VERSION,
> > > > @@ -181,6 +183,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = QCA6174_HW_3_2_VERSION,
> > > > @@ -217,6 +220,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = QCA6174_HW_2_1_VERSION,
> > > > @@ -257,6 +261,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = QCA6174_HW_2_1_VERSION,
> > > > @@ -297,6 +302,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = QCA6174_HW_3_0_VERSION,
> > > > @@ -337,6 +343,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = QCA6174_HW_3_2_VERSION,
> > > > @@ -381,6 +388,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = QCA99X0_HW_2_0_DEV_VERSION,
> > > > @@ -427,6 +435,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = QCA9984_HW_1_0_DEV_VERSION,
> > > > @@ -480,6 +489,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = QCA9888_HW_2_0_DEV_VERSION,
> > > > @@ -530,6 +540,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = QCA9377_HW_1_0_DEV_VERSION,
> > > > @@ -570,6 +581,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = QCA9377_HW_1_1_DEV_VERSION,
> > > > @@ -612,6 +624,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = QCA9377_HW_1_1_DEV_VERSION,
> > > > @@ -645,6 +658,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = QCA4019_HW_1_0_DEV_VERSION,
> > > > @@ -692,6 +706,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = false,
> > > > .use_fw_tx_credits = true,
> > > > .delay_unmap_buffer = false,
> > > > + .enable_threaded_napi = false,
> > > > },
> > > > {
> > > > .id = WCN3990_HW_1_0_DEV_VERSION,
> > > > @@ -725,6 +740,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> > > > .hw_restart_disconnect = true,
> > > > .use_fw_tx_credits = false,
> > > > .delay_unmap_buffer = true,
> > > > + .enable_threaded_napi = true,
> > > > },
> > > > };
> > > >
> > > > diff --git a/drivers/net/wireless/ath/ath10k/hw.h
> b/drivers/net/wireless/ath/ath10k/hw.h
> > > > index 9643031a4427a..adf3076b96503 100644
> > > > --- a/drivers/net/wireless/ath/ath10k/hw.h
> > > > +++ b/drivers/net/wireless/ath/ath10k/hw.h
> > > > @@ -639,6 +639,8 @@ struct ath10k_hw_params {
> > > > bool use_fw_tx_credits;
> > > >
> > > > bool delay_unmap_buffer;
> > > > +
> > > > + bool enable_threaded_napi;
> > > > };
> > > >
> > > > struct htt_resp;
> > > > diff --git a/drivers/net/wireless/ath/ath10k/snoc.c
> b/drivers/net/wireless/ath/ath10k/snoc.c
> > > > index cfcb759a87dea..b94150fb6ef06 100644
> > > > --- a/drivers/net/wireless/ath/ath10k/snoc.c
> > > > +++ b/drivers/net/wireless/ath/ath10k/snoc.c
> > > > @@ -927,6 +927,9 @@ static int ath10k_snoc_hif_start(struct ath10k
> *ar)
> > > >
> > > > bitmap_clear(ar_snoc->pending_ce_irqs, 0, CE_COUNT_MAX);
> > > >
> > > > + if (ar->hw_params.enable_threaded_napi)
> > > > + dev_set_threaded(&ar->napi_dev, true);
> > > > +
> > > > ath10k_core_napi_enable(ar);
> > > > ath10k_snoc_irq_enable(ar);
> > > > ath10k_snoc_rx_post(ar);
> > > > --
> > > > 2.39.0.314.g84b9a713c41-goog
> > > >
> > >
> > >
> > > --
> > > This song goes out to all the folk that thought Stadia would work:
> > >
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> > > Dave Täht CEO, TekLibre, LLC
>
>
>
> --
> This song goes out to all the folk that thought Stadia would work:
>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> Dave Täht CEO, TekLibre, LLC
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast
--
This electronic communication and the information and any files transmitted
with it, or attached to it, are confidential and are intended solely for
the use of the individual or entity to whom it is addressed and may contain
information that is confidential, legally privileged, protected by privacy
laws, or otherwise restricted from disclosure to anyone else. If you are
not the intended recipient or the person responsible for delivering the
e-mail to the intended recipient, you are hereby notified that any use,
copying, distributing, dissemination, forwarding, printing, or copying of
this e-mail is strictly prohibited. If you received this e-mail in error,
please return the e-mail to the sender, delete it from your computer, and
destroy any printed copy of it.
[-- Attachment #2: Type: text/html, Size: 22552 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Make-wifi-fast] [PATCH] ath10k: snoc: enable threaded napi on WCN3990
2022-12-29 5:54 ` Bob McMahon
@ 2022-12-30 3:44 ` Dave Taht
2022-12-30 20:41 ` Bob McMahon
0 siblings, 1 reply; 6+ messages in thread
From: Dave Taht @ 2022-12-30 3:44 UTC (permalink / raw)
To: Bob McMahon
Cc: Abhishek Kumar, netdev, kvalo, linux-wireless, linux-kernel,
ath10k, Eric Dumazet, Make-Wifi-fast, Jakub Kicinski,
Paolo Abeni, David S. Miller
thx for leaning in bob! (for those of you that don't know, he's the
iperf2 maintainer, and if you haven't updated your distro/os/test
environment to 2.1.8 that came out in august, yet there's all kinds of
nifty new stuff in there: https://sourceforge.net/projects/iperf2 -
too much to list, and the -i 1 -e option is just the start of it....
Iperf2's principal flaw is that people think iperf3 is the successor
to it, and no, they are very different codebases, with very different
features. iperf = iperf2, for most, but not all.
On Wed, Dec 28, 2022 at 9:54 PM Bob McMahon <bob.mcmahon@broadcom.com> wrote:
>
> A bit of a tangent but iperf 2's enhanced option provides sampled RTT and CWND in the interval output. I requires -e. It also provide the connect time, the icwnd, irtt and the mss per the connect.
>
> [rjmcmahon@rjm-nas ~]$ iperf -c 192.168.1.85 -i 1 -e
> ------------------------------------------------------------
> Client connecting to 192.168.1.85, TCP port 5001 with pid 20041 (1 flows)
> Write buffer size: 131072 Byte
> TOS set to 0x0 (Nagle on)
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [ 1] local 192.168.1.73%eno1 port 38822 connected with 192.168.1.85 port 5001 (sock=3) (icwnd/mss/irtt=14/1448/246) (ct=0.32 ms) on 2022-12-28 21:51:13 (PST)
> [ ID] Interval Transfer Bandwidth Write/Err Rtry Cwnd/RTT(var) NetPwr
> [ 1] 0.00-1.00 sec 114 MBytes 952 Mbits/sec 908/0 3 353K/1901(59) us 62606
> [ 1] 1.00-2.00 sec 111 MBytes 932 Mbits/sec 889/0 0 354K/2029(115) us 57429
> [ 1] 2.00-3.00 sec 112 MBytes 937 Mbits/sec 894/0 3 354K/2061(84) us 56855
> [ 1] 3.00-4.00 sec 112 MBytes 938 Mbits/sec 895/0 3 354K/2027(115) us 57873
> [ 1] 4.00-5.00 sec 112 MBytes 938 Mbits/sec 895/0 6 354K/2046(68) us 57336
> [ 1] 5.00-6.00 sec 112 MBytes 937 Mbits/sec 894/0 0 374K/2160(109) us 54249
> [ 1] 6.00-7.00 sec 111 MBytes 933 Mbits/sec 890/0 2 374K/2155(115) us 54132
> [ 1] 7.00-8.00 sec 111 MBytes 934 Mbits/sec 891/0 2 374K/2139(120) us 54598
> [ 1] 8.00-9.00 sec 112 MBytes 938 Mbits/sec 895/0 0 374K/2129(126) us 55101
> [ 1] 9.00-10.00 sec 112 MBytes 935 Mbits/sec 892/0 1 374K/2200(102) us 53144
> [ 1] 0.00-10.03 sec 1.09 GBytes 935 Mbits/sec 8944/0 20 374K/4082(4010) us 28632
This is a wonderful new feature, much simpler than relying on packet
captures. However! this test result above is on a decent, ethernet,
network, where on wifi, or anything bufferbloated, you would see the
cwnd and rtt inflate and/or bounce around horribly.
>
>
> Bob
>
> On Wed, Dec 28, 2022 at 4:49 PM Dave Taht via Make-wifi-fast <make-wifi-fast@lists.bufferbloat.net> wrote:
>>
>> On Wed, Dec 28, 2022 at 3:53 PM Abhishek Kumar <kuabhs@chromium.org> wrote:
>> >
>> > Apologies for the late reply, Thanks Dave for your comment. My answer is inline.
>> >
>> > On Tue, Dec 20, 2022 at 7:10 AM Dave Taht <dave.taht@gmail.com> wrote:
>> > >
>> > > I am always interested in flent.org tcp_nup, tcp_ndown, and rrul_be
>> > > tests on wifi hardware. In AP mode, especially, against a few clients
>> > > in rtt_fair on the "ending the anomaly" test suite at the bottom of
>> > > this link: https://www.cs.kau.se/tohojo/airtime-fairness/ . Of these,
>> > > it's trying to optimize bandwidth more fairly and keep latencies low
>> > > when 4 or more stations are trying to transmit (in a world with 16 or
>> > > more stations online), that increasingly bothers me the most. I'm
>> > > seeing 5+ seconds on some rtt_fair-like tests nowadays.
>> > I used testing using iperf and conductive setup and fetched the
>> > throughput data(mentioned below).
>> > >
>> > > I was also seeing huge simultaneous upload vs download disparities on
>> > > the latest kernels, on various threads over here:
>> > > https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002 and
>> > > more recently here:
>> > > https://forum.openwrt.org/t/reducing-multiplexing-latencies-still-further-in-wifi/133605
>> > Interesting, thanks for the pointer and probably the Qualcomm team is
>> > aware of it.
>> > >
>> > > I don't understand why napi with the default budget (64) is even
>> > > needed on the ath10k, as a single txop takes a minimum of ~200us, but
>> > > perhaps your patch will help. Still, measuring the TCP statistics
>> > > in-band would be nice to see. Some new tools are appearing that can do
>> > > this, Apple's goresponsiveness, crusader... that are simpler to use
>> > > than flent.
>> > Here are some of the additional raw data with and without threaded napi:
>> > ==================================================
>> > udp_rx(Without threaded NAPI)
>> > 435.98+-5.16 : Channel 44
>> > 439.06+-0.66 : Channel 157
>> >
>> > udp_rx(With threaded NAPI)
>> > 509.73+-41.03 : Channel 44
>> > 549.97+-7.62 : Channel 157
>> > ===================================================
>> > udp_tx(Without threaded NAPI)
>> > 461.31+-0.69 : Channel 44
>> > 461.46+-0.78 : Channel 157
>> >
>> > udp_tx(With threaded NAPI)
>> > 459.20+-0.77 : Channel 44
>> > 459.78+-1.08 : Channel 157
>> > ===================================================
>> > tcp_rx(Without threaded NAPI)
>> > 472.63+-2.35 : Channel 44
>> > 469.29+-6.31 : Channel 157
>> >
>> > tcp_rx(With threaded NAPI)
>> > 498.49+-2.44 : Channel 44
>> > 541.14+-40.65 : Channel 157
>> > ===================================================
>> > tcp_tx(Without threaded NAPI)
>> > 317.34+-2.37 : Channel 44
>> > 317.01+-2.56 : Channel 157
>> >
>> > tcp_tx(With threaded NAPI)
>> > 371.34+-2.36 : Channel 44
>> > 376.95+-9.40 : Channel 157
>>
>> My concern is primarily with the induced tcp latency on this test. A
>> way to check that is to run wireshark on your test client driving the
>> test, capture the iperf traffic, and then plot the "Statistics->TCP
>> stream statistics for both throughput and rtt. Would it be possible
>> for you to do that and put up those plots somewhere?
>>
>> The worst case test is a tcp bidirectional test which I don't know if
>> older iperfs can do. (iperf2 has new bounceback and bidir tests)
>>
>> Ideally stuff going in either direction, would not look as horrible,
>> as it did, back in 2016, documented in this linuxplumbers presentation
>> here: https://blog.linuxplumbersconf.org/2016/ocw/system/presentations/3963/original/linuxplumbers_wifi_latency-3Nov.pdf
>> and discussed on lwn, here: https://lwn.net/Articles/705884/
>>
>> I worry about folk achieving slightly better tcp throughput at the
>> expense of clobbering in-tcp-stream latency. Back then we were
>> shooting for no more than 40ms extra latency under load on this chip,
>> down from (unusable) seconds. Presently elsewhere, on other chips,
>> we're getting 8ms with stuff that's not in tree for the ath10k, there
>> is a slight cost in single stream throughput but when multiple streams
>> are in use, on multiple stations, things like web pages fly,
>> irrespective of load.
>>
>>
>> > ====================================================
>> >
>> > >
>> > > On Tue, Dec 20, 2022 at 12:17 AM Abhishek Kumar <kuabhs@chromium.org> wrote:
>> > > >
>> > > > NAPI poll can be done in threaded context along with soft irq
>> > > > context. Threaded context can be scheduled efficiently, thus
>> > > > creating less of bottleneck during Rx processing. This patch is
>> > > > to enable threaded NAPI on ath10k driver.
>> > > >
>> > > > Based on testing, it was observed that on WCN3990, the CPU0 reaches
>> > > > 100% utilization when napi runs in softirq context. At the same
>> > > > time the other CPUs are at low consumption percentage. This
>> > > > does not allow device to reach its maximum throughput potential.
>> > > > After enabling threaded napi, CPU load is balanced across all CPUs
>> > > > and following improvments were observed:
>> > > > - UDP_RX increase by ~22-25%
>> > > > - TCP_RX increase by ~15%
>> > > >
>> > > > Tested-on: WCN3990 hw1.0 SNOC WLAN.HL.3.2.2-00696-QCAHLSWMTPL-1
>> > > > Signed-off-by: Abhishek Kumar <kuabhs@chromium.org>
>> > > > ---
>> > > >
>> > > > drivers/net/wireless/ath/ath10k/core.c | 16 ++++++++++++++++
>> > > > drivers/net/wireless/ath/ath10k/hw.h | 2 ++
>> > > > drivers/net/wireless/ath/ath10k/snoc.c | 3 +++
>> > > > 3 files changed, 21 insertions(+)
>> > > >
>> > > > diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c
>> > > > index 5eb131ab916fd..ee4b6ba508c81 100644
>> > > > --- a/drivers/net/wireless/ath/ath10k/core.c
>> > > > +++ b/drivers/net/wireless/ath/ath10k/core.c
>> > > > @@ -100,6 +100,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = QCA988X_HW_2_0_VERSION,
>> > > > @@ -140,6 +141,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = QCA9887_HW_1_0_VERSION,
>> > > > @@ -181,6 +183,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = QCA6174_HW_3_2_VERSION,
>> > > > @@ -217,6 +220,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = QCA6174_HW_2_1_VERSION,
>> > > > @@ -257,6 +261,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = QCA6174_HW_2_1_VERSION,
>> > > > @@ -297,6 +302,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = QCA6174_HW_3_0_VERSION,
>> > > > @@ -337,6 +343,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = QCA6174_HW_3_2_VERSION,
>> > > > @@ -381,6 +388,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = QCA99X0_HW_2_0_DEV_VERSION,
>> > > > @@ -427,6 +435,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = QCA9984_HW_1_0_DEV_VERSION,
>> > > > @@ -480,6 +489,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = QCA9888_HW_2_0_DEV_VERSION,
>> > > > @@ -530,6 +540,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = QCA9377_HW_1_0_DEV_VERSION,
>> > > > @@ -570,6 +581,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = QCA9377_HW_1_1_DEV_VERSION,
>> > > > @@ -612,6 +624,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = QCA9377_HW_1_1_DEV_VERSION,
>> > > > @@ -645,6 +658,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = QCA4019_HW_1_0_DEV_VERSION,
>> > > > @@ -692,6 +706,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = false,
>> > > > .use_fw_tx_credits = true,
>> > > > .delay_unmap_buffer = false,
>> > > > + .enable_threaded_napi = false,
>> > > > },
>> > > > {
>> > > > .id = WCN3990_HW_1_0_DEV_VERSION,
>> > > > @@ -725,6 +740,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
>> > > > .hw_restart_disconnect = true,
>> > > > .use_fw_tx_credits = false,
>> > > > .delay_unmap_buffer = true,
>> > > > + .enable_threaded_napi = true,
>> > > > },
>> > > > };
>> > > >
>> > > > diff --git a/drivers/net/wireless/ath/ath10k/hw.h b/drivers/net/wireless/ath/ath10k/hw.h
>> > > > index 9643031a4427a..adf3076b96503 100644
>> > > > --- a/drivers/net/wireless/ath/ath10k/hw.h
>> > > > +++ b/drivers/net/wireless/ath/ath10k/hw.h
>> > > > @@ -639,6 +639,8 @@ struct ath10k_hw_params {
>> > > > bool use_fw_tx_credits;
>> > > >
>> > > > bool delay_unmap_buffer;
>> > > > +
>> > > > + bool enable_threaded_napi;
>> > > > };
>> > > >
>> > > > struct htt_resp;
>> > > > diff --git a/drivers/net/wireless/ath/ath10k/snoc.c b/drivers/net/wireless/ath/ath10k/snoc.c
>> > > > index cfcb759a87dea..b94150fb6ef06 100644
>> > > > --- a/drivers/net/wireless/ath/ath10k/snoc.c
>> > > > +++ b/drivers/net/wireless/ath/ath10k/snoc.c
>> > > > @@ -927,6 +927,9 @@ static int ath10k_snoc_hif_start(struct ath10k *ar)
>> > > >
>> > > > bitmap_clear(ar_snoc->pending_ce_irqs, 0, CE_COUNT_MAX);
>> > > >
>> > > > + if (ar->hw_params.enable_threaded_napi)
>> > > > + dev_set_threaded(&ar->napi_dev, true);
>> > > > +
>> > > > ath10k_core_napi_enable(ar);
>> > > > ath10k_snoc_irq_enable(ar);
>> > > > ath10k_snoc_rx_post(ar);
>> > > > --
>> > > > 2.39.0.314.g84b9a713c41-goog
>> > > >
>> > >
>> > >
>> > > --
>> > > This song goes out to all the folk that thought Stadia would work:
>> > > https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> > > Dave Täht CEO, TekLibre, LLC
>>
>>
>>
>> --
>> This song goes out to all the folk that thought Stadia would work:
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> Dave Täht CEO, TekLibre, LLC
>> _______________________________________________
>> Make-wifi-fast mailing list
>> Make-wifi-fast@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/make-wifi-fast
>
>
> This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it.
--
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Make-wifi-fast] [PATCH] ath10k: snoc: enable threaded napi on WCN3990
2022-12-30 3:44 ` Dave Taht
@ 2022-12-30 20:41 ` Bob McMahon
0 siblings, 0 replies; 6+ messages in thread
From: Bob McMahon @ 2022-12-30 20:41 UTC (permalink / raw)
To: Dave Taht
Cc: Abhishek Kumar, netdev, kvalo, linux-wireless, linux-kernel,
Eric Dumazet, Make-Wifi-fast, Jakub Kicinski, Paolo Abeni,
David S. Miller
[-- Attachment #1.1: Type: text/plain, Size: 20982 bytes --]
FYI, we're trying to add "simple" responsiveness features to the 2.1.9
release. Industry input is appreciated. We've done a lot of one way delay
work as well as different traffic profiles.
I'm at a stage of trying to show Broadcom mgmt that releasing iperf 2 as
open source to skilled network engineers will enable the networking
industry (& the best network silicon providers) in the entire e2e path.
Some questions I get are:
- Who really uses it?
- Why can't other open source or commercially provided tools be used?
- Why don't others help out much with the open source effort?
- What are the key performance indicators that will influence engineers
in the industry in an actionable way?
- What are the futures per the networking community, e.g. test support
for UDP QUIC?
Thanks,
Bob
Bob
On Thu, Dec 29, 2022 at 7:44 PM Dave Taht <dave.taht@gmail.com> wrote:
> thx for leaning in bob! (for those of you that don't know, he's the
> iperf2 maintainer, and if you haven't updated your distro/os/test
> environment to 2.1.8 that came out in august, yet there's all kinds of
> nifty new stuff in there: https://sourceforge.net/projects/iperf2 -
> too much to list, and the -i 1 -e option is just the start of it....
>
> Iperf2's principal flaw is that people think iperf3 is the successor
> to it, and no, they are very different codebases, with very different
> features. iperf = iperf2, for most, but not all.
>
> On Wed, Dec 28, 2022 at 9:54 PM Bob McMahon <bob.mcmahon@broadcom.com>
> wrote:
> >
> > A bit of a tangent but iperf 2's enhanced option provides sampled RTT
> and CWND in the interval output. I requires -e. It also provide the connect
> time, the icwnd, irtt and the mss per the connect.
> >
> > [rjmcmahon@rjm-nas ~]$ iperf -c 192.168.1.85 -i 1 -e
> > ------------------------------------------------------------
> > Client connecting to 192.168.1.85, TCP port 5001 with pid 20041 (1 flows)
> > Write buffer size: 131072 Byte
> > TOS set to 0x0 (Nagle on)
> > TCP window size: 16.0 KByte (default)
> > ------------------------------------------------------------
> > [ 1] local 192.168.1.73%eno1 port 38822 connected with 192.168.1.85
> port 5001 (sock=3) (icwnd/mss/irtt=14/1448/246) (ct=0.32 ms) on 2022-12-28
> 21:51:13 (PST)
> > [ ID] Interval Transfer Bandwidth Write/Err Rtry
> Cwnd/RTT(var) NetPwr
> > [ 1] 0.00-1.00 sec 114 MBytes 952 Mbits/sec 908/0 3
> 353K/1901(59) us 62606
> > [ 1] 1.00-2.00 sec 111 MBytes 932 Mbits/sec 889/0 0
> 354K/2029(115) us 57429
> > [ 1] 2.00-3.00 sec 112 MBytes 937 Mbits/sec 894/0 3
> 354K/2061(84) us 56855
> > [ 1] 3.00-4.00 sec 112 MBytes 938 Mbits/sec 895/0 3
> 354K/2027(115) us 57873
> > [ 1] 4.00-5.00 sec 112 MBytes 938 Mbits/sec 895/0 6
> 354K/2046(68) us 57336
> > [ 1] 5.00-6.00 sec 112 MBytes 937 Mbits/sec 894/0 0
> 374K/2160(109) us 54249
> > [ 1] 6.00-7.00 sec 111 MBytes 933 Mbits/sec 890/0 2
> 374K/2155(115) us 54132
> > [ 1] 7.00-8.00 sec 111 MBytes 934 Mbits/sec 891/0 2
> 374K/2139(120) us 54598
> > [ 1] 8.00-9.00 sec 112 MBytes 938 Mbits/sec 895/0 0
> 374K/2129(126) us 55101
> > [ 1] 9.00-10.00 sec 112 MBytes 935 Mbits/sec 892/0 1
> 374K/2200(102) us 53144
> > [ 1] 0.00-10.03 sec 1.09 GBytes 935 Mbits/sec 8944/0 20
> 374K/4082(4010) us 28632
>
> This is a wonderful new feature, much simpler than relying on packet
> captures. However! this test result above is on a decent, ethernet,
> network, where on wifi, or anything bufferbloated, you would see the
> cwnd and rtt inflate and/or bounce around horribly.
>
>
> >
> >
> > Bob
> >
> > On Wed, Dec 28, 2022 at 4:49 PM Dave Taht via Make-wifi-fast <
> make-wifi-fast@lists.bufferbloat.net> wrote:
> >>
> >> On Wed, Dec 28, 2022 at 3:53 PM Abhishek Kumar <kuabhs@chromium.org>
> wrote:
> >> >
> >> > Apologies for the late reply, Thanks Dave for your comment. My answer
> is inline.
> >> >
> >> > On Tue, Dec 20, 2022 at 7:10 AM Dave Taht <dave.taht@gmail.com>
> wrote:
> >> > >
> >> > > I am always interested in flent.org tcp_nup, tcp_ndown, and rrul_be
> >> > > tests on wifi hardware. In AP mode, especially, against a few
> clients
> >> > > in rtt_fair on the "ending the anomaly" test suite at the bottom of
> >> > > this link: https://www.cs.kau.se/tohojo/airtime-fairness/ . Of
> these,
> >> > > it's trying to optimize bandwidth more fairly and keep latencies low
> >> > > when 4 or more stations are trying to transmit (in a world with 16
> or
> >> > > more stations online), that increasingly bothers me the most. I'm
> >> > > seeing 5+ seconds on some rtt_fair-like tests nowadays.
> >> > I used testing using iperf and conductive setup and fetched the
> >> > throughput data(mentioned below).
> >> > >
> >> > > I was also seeing huge simultaneous upload vs download disparities
> on
> >> > > the latest kernels, on various threads over here:
> >> > > https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002 and
> >> > > more recently here:
> >> > >
> https://forum.openwrt.org/t/reducing-multiplexing-latencies-still-further-in-wifi/133605
> >> > Interesting, thanks for the pointer and probably the Qualcomm team is
> >> > aware of it.
> >> > >
> >> > > I don't understand why napi with the default budget (64) is even
> >> > > needed on the ath10k, as a single txop takes a minimum of ~200us,
> but
> >> > > perhaps your patch will help. Still, measuring the TCP statistics
> >> > > in-band would be nice to see. Some new tools are appearing that can
> do
> >> > > this, Apple's goresponsiveness, crusader... that are simpler to use
> >> > > than flent.
> >> > Here are some of the additional raw data with and without threaded
> napi:
> >> > ==================================================
> >> > udp_rx(Without threaded NAPI)
> >> > 435.98+-5.16 : Channel 44
> >> > 439.06+-0.66 : Channel 157
> >> >
> >> > udp_rx(With threaded NAPI)
> >> > 509.73+-41.03 : Channel 44
> >> > 549.97+-7.62 : Channel 157
> >> > ===================================================
> >> > udp_tx(Without threaded NAPI)
> >> > 461.31+-0.69 : Channel 44
> >> > 461.46+-0.78 : Channel 157
> >> >
> >> > udp_tx(With threaded NAPI)
> >> > 459.20+-0.77 : Channel 44
> >> > 459.78+-1.08 : Channel 157
> >> > ===================================================
> >> > tcp_rx(Without threaded NAPI)
> >> > 472.63+-2.35 : Channel 44
> >> > 469.29+-6.31 : Channel 157
> >> >
> >> > tcp_rx(With threaded NAPI)
> >> > 498.49+-2.44 : Channel 44
> >> > 541.14+-40.65 : Channel 157
> >> > ===================================================
> >> > tcp_tx(Without threaded NAPI)
> >> > 317.34+-2.37 : Channel 44
> >> > 317.01+-2.56 : Channel 157
> >> >
> >> > tcp_tx(With threaded NAPI)
> >> > 371.34+-2.36 : Channel 44
> >> > 376.95+-9.40 : Channel 157
> >>
> >> My concern is primarily with the induced tcp latency on this test. A
> >> way to check that is to run wireshark on your test client driving the
> >> test, capture the iperf traffic, and then plot the "Statistics->TCP
> >> stream statistics for both throughput and rtt. Would it be possible
> >> for you to do that and put up those plots somewhere?
> >>
> >> The worst case test is a tcp bidirectional test which I don't know if
> >> older iperfs can do. (iperf2 has new bounceback and bidir tests)
> >>
> >> Ideally stuff going in either direction, would not look as horrible,
> >> as it did, back in 2016, documented in this linuxplumbers presentation
> >> here:
> https://blog.linuxplumbersconf.org/2016/ocw/system/presentations/3963/original/linuxplumbers_wifi_latency-3Nov.pdf
> >> and discussed on lwn, here: https://lwn.net/Articles/705884/
> >>
> >> I worry about folk achieving slightly better tcp throughput at the
> >> expense of clobbering in-tcp-stream latency. Back then we were
> >> shooting for no more than 40ms extra latency under load on this chip,
> >> down from (unusable) seconds. Presently elsewhere, on other chips,
> >> we're getting 8ms with stuff that's not in tree for the ath10k, there
> >> is a slight cost in single stream throughput but when multiple streams
> >> are in use, on multiple stations, things like web pages fly,
> >> irrespective of load.
> >>
> >>
> >> > ====================================================
> >> >
> >> > >
> >> > > On Tue, Dec 20, 2022 at 12:17 AM Abhishek Kumar <
> kuabhs@chromium.org> wrote:
> >> > > >
> >> > > > NAPI poll can be done in threaded context along with soft irq
> >> > > > context. Threaded context can be scheduled efficiently, thus
> >> > > > creating less of bottleneck during Rx processing. This patch is
> >> > > > to enable threaded NAPI on ath10k driver.
> >> > > >
> >> > > > Based on testing, it was observed that on WCN3990, the CPU0
> reaches
> >> > > > 100% utilization when napi runs in softirq context. At the same
> >> > > > time the other CPUs are at low consumption percentage. This
> >> > > > does not allow device to reach its maximum throughput potential.
> >> > > > After enabling threaded napi, CPU load is balanced across all CPUs
> >> > > > and following improvments were observed:
> >> > > > - UDP_RX increase by ~22-25%
> >> > > > - TCP_RX increase by ~15%
> >> > > >
> >> > > > Tested-on: WCN3990 hw1.0 SNOC WLAN.HL.3.2.2-00696-QCAHLSWMTPL-1
> >> > > > Signed-off-by: Abhishek Kumar <kuabhs@chromium.org>
> >> > > > ---
> >> > > >
> >> > > > drivers/net/wireless/ath/ath10k/core.c | 16 ++++++++++++++++
> >> > > > drivers/net/wireless/ath/ath10k/hw.h | 2 ++
> >> > > > drivers/net/wireless/ath/ath10k/snoc.c | 3 +++
> >> > > > 3 files changed, 21 insertions(+)
> >> > > >
> >> > > > diff --git a/drivers/net/wireless/ath/ath10k/core.c
> b/drivers/net/wireless/ath/ath10k/core.c
> >> > > > index 5eb131ab916fd..ee4b6ba508c81 100644
> >> > > > --- a/drivers/net/wireless/ath/ath10k/core.c
> >> > > > +++ b/drivers/net/wireless/ath/ath10k/core.c
> >> > > > @@ -100,6 +100,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = QCA988X_HW_2_0_VERSION,
> >> > > > @@ -140,6 +141,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = QCA9887_HW_1_0_VERSION,
> >> > > > @@ -181,6 +183,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = QCA6174_HW_3_2_VERSION,
> >> > > > @@ -217,6 +220,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = QCA6174_HW_2_1_VERSION,
> >> > > > @@ -257,6 +261,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = QCA6174_HW_2_1_VERSION,
> >> > > > @@ -297,6 +302,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = QCA6174_HW_3_0_VERSION,
> >> > > > @@ -337,6 +343,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = QCA6174_HW_3_2_VERSION,
> >> > > > @@ -381,6 +388,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = QCA99X0_HW_2_0_DEV_VERSION,
> >> > > > @@ -427,6 +435,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = QCA9984_HW_1_0_DEV_VERSION,
> >> > > > @@ -480,6 +489,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = QCA9888_HW_2_0_DEV_VERSION,
> >> > > > @@ -530,6 +540,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = QCA9377_HW_1_0_DEV_VERSION,
> >> > > > @@ -570,6 +581,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = QCA9377_HW_1_1_DEV_VERSION,
> >> > > > @@ -612,6 +624,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = QCA9377_HW_1_1_DEV_VERSION,
> >> > > > @@ -645,6 +658,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = QCA4019_HW_1_0_DEV_VERSION,
> >> > > > @@ -692,6 +706,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = false,
> >> > > > .use_fw_tx_credits = true,
> >> > > > .delay_unmap_buffer = false,
> >> > > > + .enable_threaded_napi = false,
> >> > > > },
> >> > > > {
> >> > > > .id = WCN3990_HW_1_0_DEV_VERSION,
> >> > > > @@ -725,6 +740,7 @@ static const struct ath10k_hw_params
> ath10k_hw_params_list[] = {
> >> > > > .hw_restart_disconnect = true,
> >> > > > .use_fw_tx_credits = false,
> >> > > > .delay_unmap_buffer = true,
> >> > > > + .enable_threaded_napi = true,
> >> > > > },
> >> > > > };
> >> > > >
> >> > > > diff --git a/drivers/net/wireless/ath/ath10k/hw.h
> b/drivers/net/wireless/ath/ath10k/hw.h
> >> > > > index 9643031a4427a..adf3076b96503 100644
> >> > > > --- a/drivers/net/wireless/ath/ath10k/hw.h
> >> > > > +++ b/drivers/net/wireless/ath/ath10k/hw.h
> >> > > > @@ -639,6 +639,8 @@ struct ath10k_hw_params {
> >> > > > bool use_fw_tx_credits;
> >> > > >
> >> > > > bool delay_unmap_buffer;
> >> > > > +
> >> > > > + bool enable_threaded_napi;
> >> > > > };
> >> > > >
> >> > > > struct htt_resp;
> >> > > > diff --git a/drivers/net/wireless/ath/ath10k/snoc.c
> b/drivers/net/wireless/ath/ath10k/snoc.c
> >> > > > index cfcb759a87dea..b94150fb6ef06 100644
> >> > > > --- a/drivers/net/wireless/ath/ath10k/snoc.c
> >> > > > +++ b/drivers/net/wireless/ath/ath10k/snoc.c
> >> > > > @@ -927,6 +927,9 @@ static int ath10k_snoc_hif_start(struct
> ath10k *ar)
> >> > > >
> >> > > > bitmap_clear(ar_snoc->pending_ce_irqs, 0, CE_COUNT_MAX);
> >> > > >
> >> > > > + if (ar->hw_params.enable_threaded_napi)
> >> > > > + dev_set_threaded(&ar->napi_dev, true);
> >> > > > +
> >> > > > ath10k_core_napi_enable(ar);
> >> > > > ath10k_snoc_irq_enable(ar);
> >> > > > ath10k_snoc_rx_post(ar);
> >> > > > --
> >> > > > 2.39.0.314.g84b9a713c41-goog
> >> > > >
> >> > >
> >> > >
> >> > > --
> >> > > This song goes out to all the folk that thought Stadia would work:
> >> > >
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >> > > Dave Täht CEO, TekLibre, LLC
> >>
> >>
> >>
> >> --
> >> This song goes out to all the folk that thought Stadia would work:
> >>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >> Dave Täht CEO, TekLibre, LLC
> >> _______________________________________________
> >> Make-wifi-fast mailing list
> >> Make-wifi-fast@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/make-wifi-fast
> >
> >
> > This electronic communication and the information and any files
> transmitted with it, or attached to it, are confidential and are intended
> solely for the use of the individual or entity to whom it is addressed and
> may contain information that is confidential, legally privileged, protected
> by privacy laws, or otherwise restricted from disclosure to anyone else. If
> you are not the intended recipient or the person responsible for delivering
> the e-mail to the intended recipient, you are hereby notified that any use,
> copying, distributing, dissemination, forwarding, printing, or copying of
> this e-mail is strictly prohibited. If you received this e-mail in error,
> please return the e-mail to the sender, delete it from your computer, and
> destroy any printed copy of it.
>
>
>
> --
> This song goes out to all the folk that thought Stadia would work:
>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> Dave Täht CEO, TekLibre, LLC
>
--
This electronic communication and the information and any files transmitted
with it, or attached to it, are confidential and are intended solely for
the use of the individual or entity to whom it is addressed and may contain
information that is confidential, legally privileged, protected by privacy
laws, or otherwise restricted from disclosure to anyone else. If you are
not the intended recipient or the person responsible for delivering the
e-mail to the intended recipient, you are hereby notified that any use,
copying, distributing, dissemination, forwarding, printing, or copying of
this e-mail is strictly prohibited. If you received this e-mail in error,
please return the e-mail to the sender, delete it from your computer, and
destroy any printed copy of it.
[-- Attachment #1.2: Type: text/html, Size: 28713 bytes --]
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4206 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-12-30 20:41 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20221220075215.1.Ic12e347e0d61a618124b742614e82bbd5d770173@changeid>
2022-12-20 15:10 ` [Make-wifi-fast] [PATCH] ath10k: snoc: enable threaded napi on WCN3990 Dave Taht
2022-12-28 23:53 ` Abhishek Kumar
2022-12-29 0:49 ` Dave Taht
2022-12-29 5:54 ` Bob McMahon
2022-12-30 3:44 ` Dave Taht
2022-12-30 20:41 ` Bob McMahon
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox