[Make-wifi-fast] a cheer up tweet for y'all

[Dave Taht]

You have wifi6 now!? Congrats! I have often thought about getting
involved in your project, but lacking time and funding was always a
stopper. Did ardc ever lean in on your behalf?
https://github.com/open-sdr/openwifi is it, yes? What FPGA are you
recommending for wifi6?

Did you ever manage to get the fq-codel apis working? Just from
reading your code I could not figure out where to put it...

A bit more below.

On Wed, Jan 10, 2024 at 8:55 AM XianJun Jiao via Make-wifi-fast
<make-wifi-fast at lists.bufferbloat.net> wrote:
>
> Regarding the forwarding, I feel that you two are argue different thing?
>
> If I remember correctly, the mac80211 framework also re-use the ethernet data-path above some level, so the Linux kernel stack can handle both Wi-Fi data and Ethernet data. I think we all agree on this level of forwarding.
>
> I think Bob raise an interesting observation about the NIC on a server: Ethernet VS Wi-Fi.
>
> To my understanding about why Ethernet NIC works fine (or pretty good) on a server is that now a days the Ethernet NIC actually faces a non-shared media (medias are isolated by a switch). So the queue/packet in the Ethernet NIC till on the media could achieve so called 'line rate forwarding' easily?
>
> On the contrary, the Wi-Fi always faces a shared media (ISM band, CSMA/CA/etc.), and this bring a big difference compared to the Ethernet NIC. I can elaborate further on this: (because I am doing Wi-Fi chip/FPGA implementation since 2017 – the openwifi project, and now we have implemented our Wi-Fi6.)
>
> More and more complications are packed into the chip level instead of the driver level, Especially since Wi-Fi6. – it doesn't matter there is "binary firmware blobs" or not. Even there isn't "binary firmware blobs", the complications will still be there. This is somehow the requirement from the real-time behaviour/aspect defined in the standard.
> The Wi-Fi LMAC has to operate precisely in terms of the time (normally order of microsecond, or sub-microsecond). – this has to be in chip.
> The Wi-Fi LMAC in the chip will decide: when (depends on CSMA/CA, queue status/priority, packet priority/etc ) the packet should be transmitted on which sub-band (called RU in Wi-Fi6/7, OFDMA) through which spatial stream (MIMO, MU-MIMO). In summary: time, frequency and spatial for a packet.
> Again reminding: the time, frequency, spatial are controlled by chip not driver due to their real-time aspects. After the driver handles the packet to Wi-Fi chip (doesn't matter how good/up-to-date/open the driver is), then the only thing driver can do is waiting for the packet transmission result from the chip.
> I haven't mentioned that in the chip there could also be multiple re-transmission processes, if the first attempts fail. The final packet delivery report only happens after all re-transmission attempts end.

Ath9k anecdote - it has 5 queues, of 4 txops each. That is a lot of
data outstanding at even the highest rates, and worse, I have seen 30
or more retries in the field programmed in, leading to seconds of HoL
blocking.

I am really unsure as to whether anyone really got the "minstrel"
lesson related to rate control that we leveraged in that chip, either,
in successive standard implementations.
https://blog.cerowrt.org/post/minstrel/

> In summary, in the case of Wi-Fi there are more and more complicated low-level behaviors out of the driver control, and this is not the case (most probably, or less the case) for Ethernet NIC (I guess).

I agree that the hard realtime aspects of any network interface have
to be done on board, but it should punt to a more central cpu and OS
for anything larger than a txop. In this presentation (
http://www.taht.net/~d/broadcom_aug9_2018.pdf ) I said a ms, these
days, if it takes over 120us, punting the functionality to software is
beginning to make more sense with the ready availability of
multi-cores.

We had to settle for double-buffering our fq-codel-for-wifi code
because of a few missing features, and the ath10k "AQL"debacle ended
up triple buffering because we couldn´t get past some stupid design
decisions in the firmware, and dang it, the same mistake was carried
forward into the mt76 and now mt79 codebases.

Still this was orders of magnitude less queuing latency than what we
had had before, but 3x worse than what we could have achieved with
software/hw co-design.

Getting essentially to a zero copy interface where the software folk
moving packets intelligently have the right interfaces to the hardware
is really, really hard, and takes multiple iterations of the design,
and involvement considerably earlier than what generally happens today
elsewhere, where finished firmware is thrown over the wall, and that
team moves on, while the os folk patch around foolish or broken
features for another decade.

Or you can try to bundle all the needed functionality into a combined
ethernet/wifi path for some definition of need that is usually
inadaquate for some markets.

> Best regards,
> --
> Xianjun Jiao
>
> ________________________________
> From: Make-wifi-fast <make-wifi-fast-bounces at lists.bufferbloat.net> on behalf of Toke Høiland-Jørgensen via Make-wifi-fast <make-wifi-fast at lists.bufferbloat.net>
> Sent: Wednesday, January 10, 2024 12:23 PM
> To: Bob McMahon <bob.mcmahon at broadcom.com>
> Cc: Make-Wifi-fast <make-wifi-fast at lists.bufferbloat.net>
> Subject: Re: [Make-wifi-fast] a cheer up tweet for y'all
>
> Bob McMahon <bob.mcmahon at broadcom.com> writes:
>
> > This approach is not going to work. Sun workstations as the forwarding
> > planes for WiFi doesn't work nor scale and is cost & power inefficient. The
> > WiFi forwarding plane needs to be all hardware and not based off of BSD. It
> > has to be like a port asic in an ethernet switch. No SoC.
> >
> > Ethernet NICs are targeting servers where the workstation/NIC model does
> > work. WiFi is never going to be the basis for cloud servers.
>
> Well, the original context of the question was "Linux WiFi drivers are
> terrible, what can we do about that", and, well, providing proper
> upstream drivers at HW launch is the way to solve that.
>
> And even so, every Linux-based CPE in existence is a contradiction of
> you assertion that software-based WiFi forwarding is "not going to
> work". On the contrary, the SOCs with proper open source drivers and
> support are the ones that work the best, because that means we can run
> OpenWrt on them instead of the vendor crapware that they ship with.
>
> -Toke
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast



-- 
40 years of net history, a couple songs:
https://www.youtube.com/watch?v=D9RGX6QFm5E
Dave Täht CSO, LibreQos