[Make-wifi-fast] a cheer up tweet for y'all

[Make-wifi-fast] a cheer up tweet for y'all

[Dave Taht]

https://twitter.com/RubenKelevra/status/1744406747953959140

Thread started here:

https://twitter.com/mtaht/status/1744400465238818929

In retrospect, a lot of this project was fun, and seeing the results
like this, very satisfying, and I suppose, increasingly satisfying in
the years to come. But I am still burnt to a crisp about it, and
intensely frustrated with the behavior of the chipset makers.

-- 
40 years of net history, a couple songs:
https://www.youtube.com/watch?v=D9RGX6QFm5E
Dave Täht CSO, LibreQos

Re: [Make-wifi-fast] a cheer up tweet for y'all

[Bob McMahon]

[-- Attachment #1.1: Type: text/plain, Size: 2442 bytes --]

What are you expecting from chip makers?  HW ASIC guys are rarely good at
writing sw and not so good at systems either. SW tends to be small, high
velocity teams that make a lot of mistakes through iterations. Some get
addicted to the velocity. ASICs tend to be low thousands of engineers
paying attention to a plethora of details like signal integrity to get a
$30M-50M mask right before spending another $100M+ for a run of millions.
Completely  different operating models. It's like comparing a container
ship to a NASA X-43.

Bob

PS. I think we're missing EDCA analysis which is a BSS manager and AP
thing.  The default values really aren't optimized for anything and just
are example settings. We also need active redundancy. Lots to be done still
from what I can tell.

On Mon, Jan 8, 2024 at 9:19 AM Dave Taht via Make-wifi-fast <
make-wifi-fast@lists.bufferbloat.net> wrote:

> https://twitter.com/RubenKelevra/status/1744406747953959140
>
> Thread started here:
>
> https://twitter.com/mtaht/status/1744400465238818929
>
> In retrospect, a lot of this project was fun, and seeing the results
> like this, very satisfying, and I suppose, increasingly satisfying in
> the years to come. But I am still burnt to a crisp about it, and
> intensely frustrated with the behavior of the chipset makers.
>
> --
> 40 years of net history, a couple songs:
> https://www.youtube.com/watch?v=D9RGX6QFm5E
> Dave Täht CSO, LibreQos
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast

-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.

[-- Attachment #1.2: Type: text/html, Size: 3326 bytes --]

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4206 bytes --]

Re: [Make-wifi-fast] a cheer up tweet for y'all

[Toke Høiland-Jørgensen]

Bob McMahon via Make-wifi-fast <make-wifi-fast@lists.bufferbloat.net>
writes:

> What are you expecting from chip makers?

High-quality open source drivers, upstreamed into the Linux mainline by
the time the hardware ships, would be a good start :)

Vendors seem to manage this for Ethernet NICs just fine, so it can't
really be a technical barrier that is keeping this from happening.

If we're going further down the wish list, "no binary firmware blobs"
would be next as far as I'm concerned. Not holding my breath on that
one, though :/

-Toke

Re: [Make-wifi-fast] a cheer up tweet for y'all

[Dave Taht]

On Tue, Jan 9, 2024 at 5:56 AM Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>
> Bob McMahon via Make-wifi-fast <make-wifi-fast@lists.bufferbloat.net>
> writes:
>
> > What are you expecting from chip makers?
>
> High-quality open source drivers, upstreamed into the Linux mainline by
> the time the hardware ships, would be a good start :)
>
> Vendors seem to manage this for Ethernet NICs just fine, so it can't
> really be a technical barrier that is keeping this from happening.
>
> If we're going further down the wish list, "no binary firmware blobs"
> would be next as far as I'm concerned. Not holding my breath on that
> one, though :/

Data Sheets with register layouts
Sending internal developers to Linux technical conference like netdev
and linux plumbers
Participation in interop events (esp for wifi and 5g)
An Open Source programs office
Ponies

>
> -Toke



-- 
40 years of net history, a couple songs:
https://www.youtube.com/watch?v=D9RGX6QFm5E
Dave Täht CSO, LibreQos

Re: [Make-wifi-fast] a cheer up tweet for y'all

[Bob McMahon]

[-- Attachment #1.1: Type: text/plain, Size: 1908 bytes --]

This approach is not going to work. Sun workstations as the forwarding
planes for WiFi doesn't work nor scale and is cost & power inefficient. The
WiFi forwarding plane needs to be all hardware and not based off of BSD. It
has to be like a port asic in an ethernet switch. No SoC.

Ethernet NICs are targeting servers where the workstation/NIC model does
work. WiFi is never going to be the basis for cloud servers.

Bob

On Tue, Jan 9, 2024 at 2:56 AM Toke Høiland-Jørgensen <toke@toke.dk> wrote:

> Bob McMahon via Make-wifi-fast <make-wifi-fast@lists.bufferbloat.net>
> writes:
>
> > What are you expecting from chip makers?
>
> High-quality open source drivers, upstreamed into the Linux mainline by
> the time the hardware ships, would be a good start :)
>
> Vendors seem to manage this for Ethernet NICs just fine, so it can't
> really be a technical barrier that is keeping this from happening.
>
> If we're going further down the wish list, "no binary firmware blobs"
> would be next as far as I'm concerned. Not holding my breath on that
> one, though :/
>
> -Toke
>

-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.

[-- Attachment #1.2: Type: text/html, Size: 2386 bytes --]

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4206 bytes --]

Re: [Make-wifi-fast] a cheer up tweet for y'all

[Toke Høiland-Jørgensen]

Bob McMahon <bob.mcmahon@broadcom.com> writes:

> This approach is not going to work. Sun workstations as the forwarding
> planes for WiFi doesn't work nor scale and is cost & power inefficient. The
> WiFi forwarding plane needs to be all hardware and not based off of BSD. It
> has to be like a port asic in an ethernet switch. No SoC.
>
> Ethernet NICs are targeting servers where the workstation/NIC model does
> work. WiFi is never going to be the basis for cloud servers.

Well, the original context of the question was "Linux WiFi drivers are
terrible, what can we do about that", and, well, providing proper
upstream drivers at HW launch is the way to solve that.

And even so, every Linux-based CPE in existence is a contradiction of
you assertion that software-based WiFi forwarding is "not going to
work". On the contrary, the SOCs with proper open source drivers and
support are the ones that work the best, because that means we can run
OpenWrt on them instead of the vendor crapware that they ship with.

-Toke

Re: [Make-wifi-fast] a cheer up tweet for y'all

[XianJun Jiao]

[-- Attachment #1: Type: text/plain, Size: 4207 bytes --]

Regarding the forwarding, I feel that you two are argue different thing?

If I remember correctly, the mac80211 framework also re-use the ethernet data-path above some level, so the Linux kernel stack can handle both Wi-Fi data and Ethernet data. I think we all agree on this level of forwarding.

I think Bob raise an interesting observation about the NIC on a server: Ethernet VS Wi-Fi.

To my understanding about why Ethernet NIC works fine (or pretty good) on a server is that now a days the Ethernet NIC actually faces a non-shared media (medias are isolated by a switch). So the queue/packet in the Ethernet NIC till on the media could achieve so called 'line rate forwarding' easily?

On the contrary, the Wi-Fi always faces a shared media (ISM band, CSMA/CA/etc.), and this bring a big difference compared to the Ethernet NIC. I can elaborate further on this: (because I am doing Wi-Fi chip/FPGA implementation since 2017 – the openwifi project, and now we have implemented our Wi-Fi6.)


  *
More and more complications are packed into the chip level instead of the driver level, Especially since Wi-Fi6. – it doesn't matter there is "binary firmware blobs" or not. Even there isn't "binary firmware blobs", the complications will still be there. This is somehow the requirement from the real-time behaviour/aspect defined in the standard.
  *   The Wi-Fi LMAC has to operate precisely in terms of the time (normally order of microsecond, or sub-microsecond). – this has to be in chip.
  *   The Wi-Fi LMAC in the chip will decide: when (depends on CSMA/CA, queue status/priority, packet priority/etc ) the packet should be transmitted on which sub-band (called RU in Wi-Fi6/7, OFDMA) through which spatial stream (MIMO, MU-MIMO). In summary: time, frequency and spatial for a packet.
  *   Again reminding: the time, frequency, spatial are controlled by chip not driver due to their real-time aspects. After the driver handles the packet to Wi-Fi chip (doesn't matter how good/up-to-date/open the driver is), then the only thing driver can do is waiting for the packet transmission result from the chip.
  *   I haven't mentioned that in the chip there could also be multiple re-transmission processes, if the first attempts fail. The final packet delivery report only happens after all re-transmission attempts end.

In summary, in the case of Wi-Fi there are more and more complicated low-level behaviors out of the driver control, and this is not the case (most probably, or less the case) for Ethernet NIC (I guess).

Best regards,
--
Xianjun Jiao

________________________________
From: Make-wifi-fast <make-wifi-fast-bounces@lists.bufferbloat.net> on behalf of Toke Høiland-Jørgensen via Make-wifi-fast <make-wifi-fast@lists.bufferbloat.net>
Sent: Wednesday, January 10, 2024 12:23 PM
To: Bob McMahon <bob.mcmahon@broadcom.com>
Cc: Make-Wifi-fast <make-wifi-fast@lists.bufferbloat.net>
Subject: Re: [Make-wifi-fast] a cheer up tweet for y'all

Bob McMahon <bob.mcmahon@broadcom.com> writes:

> This approach is not going to work. Sun workstations as the forwarding
> planes for WiFi doesn't work nor scale and is cost & power inefficient. The
> WiFi forwarding plane needs to be all hardware and not based off of BSD. It
> has to be like a port asic in an ethernet switch. No SoC.
>
> Ethernet NICs are targeting servers where the workstation/NIC model does
> work. WiFi is never going to be the basis for cloud servers.

Well, the original context of the question was "Linux WiFi drivers are
terrible, what can we do about that", and, well, providing proper
upstream drivers at HW launch is the way to solve that.

And even so, every Linux-based CPE in existence is a contradiction of
you assertion that software-based WiFi forwarding is "not going to
work". On the contrary, the SOCs with proper open source drivers and
support are the ones that work the best, because that means we can run
OpenWrt on them instead of the vendor crapware that they ship with.

-Toke
_______________________________________________
Make-wifi-fast mailing list
Make-wifi-fast@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/make-wifi-fast

[-- Attachment #2: Type: text/html, Size: 9003 bytes --]

Re: [Make-wifi-fast] a cheer up tweet for y'all

[Dave Taht]

You have wifi6 now!? Congrats! I have often thought about getting
involved in your project, but lacking time and funding was always a
stopper. Did ardc ever lean in on your behalf?
https://github.com/open-sdr/openwifi is it, yes? What FPGA are you
recommending for wifi6?

Did you ever manage to get the fq-codel apis working? Just from
reading your code I could not figure out where to put it...

A bit more below.

On Wed, Jan 10, 2024 at 8:55 AM XianJun Jiao via Make-wifi-fast
<make-wifi-fast@lists.bufferbloat.net> wrote:
>
> Regarding the forwarding, I feel that you two are argue different thing?
>
> If I remember correctly, the mac80211 framework also re-use the ethernet data-path above some level, so the Linux kernel stack can handle both Wi-Fi data and Ethernet data. I think we all agree on this level of forwarding.
>
> I think Bob raise an interesting observation about the NIC on a server: Ethernet VS Wi-Fi.
>
> To my understanding about why Ethernet NIC works fine (or pretty good) on a server is that now a days the Ethernet NIC actually faces a non-shared media (medias are isolated by a switch). So the queue/packet in the Ethernet NIC till on the media could achieve so called 'line rate forwarding' easily?
>
> On the contrary, the Wi-Fi always faces a shared media (ISM band, CSMA/CA/etc.), and this bring a big difference compared to the Ethernet NIC. I can elaborate further on this: (because I am doing Wi-Fi chip/FPGA implementation since 2017 – the openwifi project, and now we have implemented our Wi-Fi6.)
>
> More and more complications are packed into the chip level instead of the driver level, Especially since Wi-Fi6. – it doesn't matter there is "binary firmware blobs" or not. Even there isn't "binary firmware blobs", the complications will still be there. This is somehow the requirement from the real-time behaviour/aspect defined in the standard.
> The Wi-Fi LMAC has to operate precisely in terms of the time (normally order of microsecond, or sub-microsecond). – this has to be in chip.
> The Wi-Fi LMAC in the chip will decide: when (depends on CSMA/CA, queue status/priority, packet priority/etc ) the packet should be transmitted on which sub-band (called RU in Wi-Fi6/7, OFDMA) through which spatial stream (MIMO, MU-MIMO). In summary: time, frequency and spatial for a packet.
> Again reminding: the time, frequency, spatial are controlled by chip not driver due to their real-time aspects. After the driver handles the packet to Wi-Fi chip (doesn't matter how good/up-to-date/open the driver is), then the only thing driver can do is waiting for the packet transmission result from the chip.
> I haven't mentioned that in the chip there could also be multiple re-transmission processes, if the first attempts fail. The final packet delivery report only happens after all re-transmission attempts end.

Ath9k anecdote - it has 5 queues, of 4 txops each. That is a lot of
data outstanding at even the highest rates, and worse, I have seen 30
or more retries in the field programmed in, leading to seconds of HoL
blocking.

I am really unsure as to whether anyone really got the "minstrel"
lesson related to rate control that we leveraged in that chip, either,
in successive standard implementations.
https://blog.cerowrt.org/post/minstrel/

> In summary, in the case of Wi-Fi there are more and more complicated low-level behaviors out of the driver control, and this is not the case (most probably, or less the case) for Ethernet NIC (I guess).

I agree that the hard realtime aspects of any network interface have
to be done on board, but it should punt to a more central cpu and OS
for anything larger than a txop. In this presentation (
http://www.taht.net/~d/broadcom_aug9_2018.pdf ) I said a ms, these
days, if it takes over 120us, punting the functionality to software is
beginning to make more sense with the ready availability of
multi-cores.

We had to settle for double-buffering our fq-codel-for-wifi code
because of a few missing features, and the ath10k "AQL"debacle ended
up triple buffering because we couldn´t get past some stupid design
decisions in the firmware, and dang it, the same mistake was carried
forward into the mt76 and now mt79 codebases.

Still this was orders of magnitude less queuing latency than what we
had had before, but 3x worse than what we could have achieved with
software/hw co-design.

Getting essentially to a zero copy interface where the software folk
moving packets intelligently have the right interfaces to the hardware
is really, really hard, and takes multiple iterations of the design,
and involvement considerably earlier than what generally happens today
elsewhere, where finished firmware is thrown over the wall, and that
team moves on, while the os folk patch around foolish or broken
features for another decade.

Or you can try to bundle all the needed functionality into a combined
ethernet/wifi path for some definition of need that is usually
inadaquate for some markets.

> Best regards,
> --
> Xianjun Jiao
>
> ________________________________
> From: Make-wifi-fast <make-wifi-fast-bounces@lists.bufferbloat.net> on behalf of Toke Høiland-Jørgensen via Make-wifi-fast <make-wifi-fast@lists.bufferbloat.net>
> Sent: Wednesday, January 10, 2024 12:23 PM
> To: Bob McMahon <bob.mcmahon@broadcom.com>
> Cc: Make-Wifi-fast <make-wifi-fast@lists.bufferbloat.net>
> Subject: Re: [Make-wifi-fast] a cheer up tweet for y'all
>
> Bob McMahon <bob.mcmahon@broadcom.com> writes:
>
> > This approach is not going to work. Sun workstations as the forwarding
> > planes for WiFi doesn't work nor scale and is cost & power inefficient. The
> > WiFi forwarding plane needs to be all hardware and not based off of BSD. It
> > has to be like a port asic in an ethernet switch. No SoC.
> >
> > Ethernet NICs are targeting servers where the workstation/NIC model does
> > work. WiFi is never going to be the basis for cloud servers.
>
> Well, the original context of the question was "Linux WiFi drivers are
> terrible, what can we do about that", and, well, providing proper
> upstream drivers at HW launch is the way to solve that.
>
> And even so, every Linux-based CPE in existence is a contradiction of
> you assertion that software-based WiFi forwarding is "not going to
> work". On the contrary, the SOCs with proper open source drivers and
> support are the ones that work the best, because that means we can run
> OpenWrt on them instead of the vendor crapware that they ship with.
>
> -Toke
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast



-- 
40 years of net history, a couple songs:
https://www.youtube.com/watch?v=D9RGX6QFm5E
Dave Täht CSO, LibreQos

Re: [Make-wifi-fast] a cheer up tweet for y'all

[Bob McMahon]

[-- Attachment #1.1: Type: text/plain, Size: 3037 bytes --]

> Well, the original context of the question was "Linux WiFi drivers are
> terrible, what can we do about that", and, well, providing proper
> upstream drivers at HW launch is the way to solve that.

This is out of the scope of chip makers for modern chips. The os drivers
are written by system integrators - specialization has divided these tasks.
Chip makers don't affect open vs closed source drivers of systems.  Think
of a WiFi chip now as transistors with a small microcontroller and not a
linux NIC.

One can argue that chip makers don't provide documents to open-source
developers, which is mostly accurate. But documents aren't the blocker.

I think an open source community would have to innovate to a level to drive
the use of chips coming off a foundry line for a chip maker to consider
assigning resources to support open-source teams. Old chips with 10+ year
old NRE doesn't justify any investment by anyone.

I think the server market & structure & level of cloud innovation make
things different for ethernet NICs.

Bob


On Wed, Jan 10, 2024 at 3:23 AM Toke Høiland-Jørgensen <toke@toke.dk> wrote:

> Bob McMahon <bob.mcmahon@broadcom.com> writes:
>
> > This approach is not going to work. Sun workstations as the forwarding
> > planes for WiFi doesn't work nor scale and is cost & power inefficient.
> The
> > WiFi forwarding plane needs to be all hardware and not based off of BSD.
> It
> > has to be like a port asic in an ethernet switch. No SoC.
> >
> > Ethernet NICs are targeting servers where the workstation/NIC model does
> > work. WiFi is never going to be the basis for cloud servers.
>
> Well, the original context of the question was "Linux WiFi drivers are
> terrible, what can we do about that", and, well, providing proper
> upstream drivers at HW launch is the way to solve that.
>
> And even so, every Linux-based CPE in existence is a contradiction of
> you assertion that software-based WiFi forwarding is "not going to
> work". On the contrary, the SOCs with proper open source drivers and
> support are the ones that work the best, because that means we can run
> OpenWrt on them instead of the vendor crapware that they ship with.
>
> -Toke
>

-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.

[-- Attachment #1.2: Type: text/html, Size: 3583 bytes --]

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4206 bytes --]

Re: [Make-wifi-fast] a cheer up tweet for y'all

[Dave Taht]

i so wish more of what I discussed 8 years ago, had made it to the
chipmakers. Minstrel-blues, in the end, didn't work out, but seemed so
promising (coupled rate and power control).
https://www.youtube.com/watch?v=Rb-UnHDw02o&t=1041s

On Wed, Jan 10, 2024 at 1:23 PM Bob McMahon <bob.mcmahon@broadcom.com> wrote:
>
> > Well, the original context of the question was "Linux WiFi drivers are
> > terrible, what can we do about that", and, well, providing proper
> > upstream drivers at HW launch is the way to solve that.
>
> This is out of the scope of chip makers for modern chips. The os drivers are written by system integrators - specialization has divided these tasks. Chip makers don't affect open vs closed source drivers of systems.  Think of a WiFi chip now as transistors with a small microcontroller and not a linux NIC.

Mediatek has had an "Upstream first" wifi chipset development policy
for many years now, the mt76 got competetive and the mt79 is looking
really good. It has a very minimal blob attached to it, and a decent
API (with a couple exceptions).

The offloaded cpu in the ath10k was actually quite powerful, and had
that code been more commonly available, it would not have taken so
many years and a mere *one* guy licensed to work on it, to get the
ath10k firmware up to snuff.

https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002

The R/T OS on that "microcontroller" was a nightmare of spaghetti
written by EEs on crack.

I quiver in fear about even less open firmware blobs than that.
>
> One can argue that chip makers don't provide documents to open-source developers, which is mostly accurate. But documents aren't the blocker.

Oh, they are key to understanding what the chip can be made to do
outside of the scope of the original designers.

> I think an open source community would have to innovate to a level to drive the use of chips coming off a foundry line for a chip maker to consider assigning resources to support open-source teams. Old chips with 10+ year old NRE doesn't justify any investment by anyone.

I would merely like a competent OS developer to be present from day
one of a new or being revised design to provide useful feedback from
the field about what ideas are BS and which are not. For example,
recently I turned down a gig that was trying to use offloads to speed
up crypto processing of DNS packets, which historically, has never
been worthwhile, as the overhead of handing off the co-processor was
far far greater for small packets than doing it on the cpu was. The
real innovation for crypto processing was in adding better cpu
instructions.

I also thought mu-mimo (one way broadcast to multiple stations) was a
total waste of time. I do have some hope for the more bi-directional
stuff in ofdma, but given the backoff structure of the wifi mac, and
the nature of tcp, mu-mimo introduced complexity for sub-zero benefit.
Merely firing all the people that marketed that and hiring on a few
more clued network developers instead, would have helped.

OS developers also have needs and desires for useful stuff in hardware
that are sometimes bogus, and sometimes genuinely useful. In wifi, I
have longed for a tx or rx is almost done interrupt, being able to
directly dma from/to the kernel layout of a skb, a completion
interrupt and a dozen other things that i outlined in the presentation
above. As for bogus, someone added NAPI support to the ath10k when it
is totally unneeded at even the maximum interrrupt rate. No idea why
that happened.

Lastly, hw or firmware that presents sane APIs to the overlying os
frequently does not happen due to lack of communication about what can
and should be done in software,
>
> I think the server market & structure & level of cloud innovation make things different for ethernet NICs.
>
> Bob
>
>
> On Wed, Jan 10, 2024 at 3:23 AM Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>
>> Bob McMahon <bob.mcmahon@broadcom.com> writes:
>>
>> > This approach is not going to work. Sun workstations as the forwarding
>> > planes for WiFi doesn't work nor scale and is cost & power inefficient. The
>> > WiFi forwarding plane needs to be all hardware and not based off of BSD. It
>> > has to be like a port asic in an ethernet switch. No SoC.
>> >
>> > Ethernet NICs are targeting servers where the workstation/NIC model does
>> > work. WiFi is never going to be the basis for cloud servers.
>>
>> Well, the original context of the question was "Linux WiFi drivers are
>> terrible, what can we do about that", and, well, providing proper
>> upstream drivers at HW launch is the way to solve that.
>>
>> And even so, every Linux-based CPE in existence is a contradiction of
>> you assertion that software-based WiFi forwarding is "not going to
>> work". On the contrary, the SOCs with proper open source drivers and
>> support are the ones that work the best, because that means we can run
>> OpenWrt on them instead of the vendor crapware that they ship with.
>>
>> -Toke
>
>
> This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it.



-- 
40 years of net history, a couple songs:
https://www.youtube.com/watch?v=D9RGX6QFm5E
Dave Täht CSO, LibreQos

Re: [Make-wifi-fast] a cheer up tweet for y'all

[Bob McMahon]

[-- Attachment #1.1: Type: text/plain, Size: 7867 bytes --]

I perceive a few major issues per this analysis:

   - This is still a sun workstation model - ditch the SoC, BSD & things
   like DMAs and see what comes out on the other side. Build a switch and/or
   transceivers
   - 802.11 standards set the road maps which is much about
   spectrum efficiency so that's the direction
   - Open-source sw engineers have zero visibility into hw engineering
   actuals so are woefully out of date (though it's understandable because the
   way things are structured doesn't expose the state of engineering in a
   public manner)
   - I think we need an e2e low-latency technical offering. Not sure why
   there has been so much misinformation around this but it hasn't seemed to
   help

Bob

On Thu, Jan 11, 2024, 1:31 AM Dave Taht <dave.taht@gmail.com> wrote:

> i so wish more of what I discussed 8 years ago, had made it to the
> chipmakers. Minstrel-blues, in the end, didn't work out, but seemed so
> promising (coupled rate and power control).
> https://www.youtube.com/watch?v=Rb-UnHDw02o&t=1041s
>
> On Wed, Jan 10, 2024 at 1:23 PM Bob McMahon <bob.mcmahon@broadcom.com>
> wrote:
> >
> > > Well, the original context of the question was "Linux WiFi drivers are
> > > terrible, what can we do about that", and, well, providing proper
> > > upstream drivers at HW launch is the way to solve that.
> >
> > This is out of the scope of chip makers for modern chips. The os drivers
> are written by system integrators - specialization has divided these tasks.
> Chip makers don't affect open vs closed source drivers of systems.  Think
> of a WiFi chip now as transistors with a small microcontroller and not a
> linux NIC.
>
> Mediatek has had an "Upstream first" wifi chipset development policy
> for many years now, the mt76 got competetive and the mt79 is looking
> really good. It has a very minimal blob attached to it, and a decent
> API (with a couple exceptions).
>
> The offloaded cpu in the ath10k was actually quite powerful, and had
> that code been more commonly available, it would not have taken so
> many years and a mere *one* guy licensed to work on it, to get the
> ath10k firmware up to snuff.
>
> https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002
>
> The R/T OS on that "microcontroller" was a nightmare of spaghetti
> written by EEs on crack.
>
> I quiver in fear about even less open firmware blobs than that.
> >
> > One can argue that chip makers don't provide documents to open-source
> developers, which is mostly accurate. But documents aren't the blocker.
>
> Oh, they are key to understanding what the chip can be made to do
> outside of the scope of the original designers.
>
> > I think an open source community would have to innovate to a level to
> drive the use of chips coming off a foundry line for a chip maker to
> consider assigning resources to support open-source teams. Old chips with
> 10+ year old NRE doesn't justify any investment by anyone.
>
> I would merely like a competent OS developer to be present from day
> one of a new or being revised design to provide useful feedback from
> the field about what ideas are BS and which are not. For example,
> recently I turned down a gig that was trying to use offloads to speed
> up crypto processing of DNS packets, which historically, has never
> been worthwhile, as the overhead of handing off the co-processor was
> far far greater for small packets than doing it on the cpu was. The
> real innovation for crypto processing was in adding better cpu
> instructions.
>
> I also thought mu-mimo (one way broadcast to multiple stations) was a
> total waste of time. I do have some hope for the more bi-directional
> stuff in ofdma, but given the backoff structure of the wifi mac, and
> the nature of tcp, mu-mimo introduced complexity for sub-zero benefit.
> Merely firing all the people that marketed that and hiring on a few
> more clued network developers instead, would have helped.
>
> OS developers also have needs and desires for useful stuff in hardware
> that are sometimes bogus, and sometimes genuinely useful. In wifi, I
> have longed for a tx or rx is almost done interrupt, being able to
> directly dma from/to the kernel layout of a skb, a completion
> interrupt and a dozen other things that i outlined in the presentation
> above. As for bogus, someone added NAPI support to the ath10k when it
> is totally unneeded at even the maximum interrrupt rate. No idea why
> that happened.
>
> Lastly, hw or firmware that presents sane APIs to the overlying os
> frequently does not happen due to lack of communication about what can
> and should be done in software,
> >
> > I think the server market & structure & level of cloud innovation make
> things different for ethernet NICs.
> >
> > Bob
> >
> >
> > On Wed, Jan 10, 2024 at 3:23 AM Toke Høiland-Jørgensen <toke@toke.dk>
> wrote:
> >>
> >> Bob McMahon <bob.mcmahon@broadcom.com> writes:
> >>
> >> > This approach is not going to work. Sun workstations as the forwarding
> >> > planes for WiFi doesn't work nor scale and is cost & power
> inefficient. The
> >> > WiFi forwarding plane needs to be all hardware and not based off of
> BSD. It
> >> > has to be like a port asic in an ethernet switch. No SoC.
> >> >
> >> > Ethernet NICs are targeting servers where the workstation/NIC model
> does
> >> > work. WiFi is never going to be the basis for cloud servers.
> >>
> >> Well, the original context of the question was "Linux WiFi drivers are
> >> terrible, what can we do about that", and, well, providing proper
> >> upstream drivers at HW launch is the way to solve that.
> >>
> >> And even so, every Linux-based CPE in existence is a contradiction of
> >> you assertion that software-based WiFi forwarding is "not going to
> >> work". On the contrary, the SOCs with proper open source drivers and
> >> support are the ones that work the best, because that means we can run
> >> OpenWrt on them instead of the vendor crapware that they ship with.
> >>
> >> -Toke
> >
> >
> > This electronic communication and the information and any files
> transmitted with it, or attached to it, are confidential and are intended
> solely for the use of the individual or entity to whom it is addressed and
> may contain information that is confidential, legally privileged, protected
> by privacy laws, or otherwise restricted from disclosure to anyone else. If
> you are not the intended recipient or the person responsible for delivering
> the e-mail to the intended recipient, you are hereby notified that any use,
> copying, distributing, dissemination, forwarding, printing, or copying of
> this e-mail is strictly prohibited. If you received this e-mail in error,
> please return the e-mail to the sender, delete it from your computer, and
> destroy any printed copy of it.
>
>
>
> --
> 40 years of net history, a couple songs:
> https://www.youtube.com/watch?v=D9RGX6QFm5E
> Dave Täht CSO, LibreQos
>

-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.

[-- Attachment #1.2: Type: text/html, Size: 9303 bytes --]

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4206 bytes --]