[Make-wifi-fast] [bbr-dev] Aggregating without bloating - hard times for tcp and wifi

Thu Nov 24 11:23:08 EST 2022

On Wed, Nov 23, 2022 at 9:25 PM Muhammad Ahsan <muhammadahsan at umt.edu.pk> wrote:
>
> Hi dev group guys,
>
>
>
> I need to use sysctl to set ms value for net.ipv4.tcp_limit_output_ms .
>
>
>
> The TSQ patch attached is not working or letting me do that . I am on linux 5.13.12

Why would you test using a 2 year old kernel? In general I try to
compile a net-next or a rc3+ candidate when doing network research. By
the time your paper is published, your
work is 3 or more years obsolete. Working within a modern kernel you
can get in general get help from the netdev mailing list.

Admittedly, embedded OSes like openwrt tend to lag a few years behind
also (presently at 5.15 for most chipsets).

> Manually changing tx_sk_pacing_shift = 7;  variable in main.c  needs to recompile kernel everytime…
>
>
>
> I need to have sysctl to control the ms value , to set  2TSQ,4TSQ,8TSQ  etc for my wifi chipset.

One of the reasons why I was reluctant to provide this knob was that
it seemed inevitable folk would optimize for bandwidth at all other
costs, in a lab environment. We picked numbers for the initial
implementations of this that made a good compromise between latency
and throughput. Certainly tuning it up would be good, but *first* I
try to encourage researchers to emulate real-world conditions, where
interference and other stations can cause txop scheduling delays
measured in the 100s or 1000s of ms. The flent rtt_fair test, to 4 or
more stations, in particular, can be quite revealing and our test
results and test benchmarks are here:

https://www.cs.kau.se/tohojo/airtime-fairness/

There is MUCH other low hanging fruit in wifi. As one example, if you
look at aircaps, you will find a lot of single packet, followed by an
aggregate, because the driver accepts and schedules that first packet
(and txop) and doesn't wait a little bit  to assemble a larger
aggregate, essentially wasting a txop. There is still, 7+ years since
this preso of mine:
https://www.youtube.com/watch?v=Rb-UnHDw02o&t=1591s not a lot of
understanding of how aggregation behaves. Furthermore ack-filtering is
proving helpful in many
scenarios.

I am sad to report that the initial mt79 firmware design does not
expose per station queues, which are IMHO utterly necessary for low
latency in wifi6 and wifi7. I have high hopes for the new qualcomm
chips...

Anyway, some more links:

https://forum.openwrt.org/t/reducing-multiplexing-latencies-still-further-in-wifi/133605/

https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002

While I think that most of the patches I care about are in your 5.13
version, there's stuff scheduled for 6.2 or 6.3, and the AQL mechanism
used by the ath10k, mt76, iwl, and now mt79, is proving to be a real
barrier to good throughput and latency over wifi.

>
>
>
> I will be thankful if anyone can help me in it.
>
>
>
>
>
> Rgds,
>
> Ahsan
>
>
>
>
>
>
>
> From: 'Neal Cardwell' via BBR Development <bbr-dev at googlegroups.com>
> Sent: Wednesday, November 23, 2022 1:10 AM
> To: Bob McMahon <bob.mcmahon at broadcom.com>
> Cc: Dave Taht <dave.taht at gmail.com>; Make-Wifi-fast <make-wifi-fast at lists.bufferbloat.net>; bloat <bloat at lists.bufferbloat.net>; BBR Development <bbr-dev at googlegroups.com>
> Subject: Re: [bbr-dev] Aggregating without bloating - hard times for tcp and wifi
>
>
>
> On Tue, Nov 22, 2022 at 2:43 PM 'Bob McMahon' via BBR Development <bbr-dev at googlegroups.com> wrote:
>
> Thanks for sharing this. Curious about how the xTSQ value can be set? Can it be done with sysctl?
>
> We continue our analysis by using the ms-version of TSQ patch, which enables the tune of the TSQ size allowing each TCP variant to enqueue more than 1 ms of data at the current TCP rate. In particular, we allow to enqueue the equivalent of x ms of data, naming each test xTSQ, with x being an integer value. It is important to notice that this patch has been included in the Linux kernel mainline, and each Wi-Fi driver can now set the desired xTSQ value.
>
>
>
> I believe they are setting the xTSQ value using the sk_pacing_shift field, which was added here:
>
>    https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3a9b76fd0db9f0d426533f96a68a62a58753a51e
>
>
>
> AFAIK the intent is only for drivers to set that, and there's no sysctl for that, but of course you could add a sysctl for testing if you wanted. :-)
>
>
>
> cheers,
>
> neal
>
>
>
>
>
>
>
> Another thing that could be interesting is the WiFi aggregation stop reasons, e.g. how many times agg stopped per hitting the max mpdu per ampdu vs the software fifo going empty (i.e. no more packets available to the driver from the TCP stack) per that TXOP.
>
> Finally, many (most?) APs are forwarding and feeding packets at at the hardware level so not sure that the linux stack matters as much for an AP based analysis, particularly when considering multi user transmissions, i.e. multiple WiFi clients are active and sharing TXOPs.
>
> Bob
>
> On Mon, Nov 21, 2022 at 10:04 PM Dave Taht <dave.taht at gmail.com> wrote:
>
> This paper came out last month. Good work, really exhaustive look at
> two chipsets, multiple congestion controls and the interactions with
> TSQ, with
> lots and lots of flent.
>
> https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9772053
>
> as for wifi6... don't make me start talking about wifi6... but some of
> these tests look like a good baseline to start comparing the ath11k,
> mt79, etc..
>
> Paper kind of misses the negative impact of AQL in the ath10k (and
> most likely also the mt76 and mt79 chips)
>
> --
> This song goes out to all the folk that thought Stadia would work:
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> Dave Täht CEO, TekLibre, LLC
>
> --
> You received this message because you are subscribed to the Google Groups "BBR Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bbr-dev+unsubscribe at googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/CAA93jw6yJU10wh9ajqX4yW2AvnJPStyWAAcV4%2BoQ8wSGsJgKZA%40mail.gmail.com.
>
>
> This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it.
>
> --
> You received this message because you are subscribed to the Google Groups "BBR Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bbr-dev+unsubscribe at googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/CAHb6LvrK_exkt_rbukYkF4_hwPXyy9T%2BZNGzdK79x91RrK49Mg%40mail.gmail.com.
>
> --
> You received this message because you are subscribed to the Google Groups "BBR Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bbr-dev+unsubscribe at googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/CADVnQyn9OSxPcF2JsUHdiWLTvAAdN5vs1uyiRBL0AS8yH8niYw%40mail.gmail.com.

-- 
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC