From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id E6CD63B29D for ; Wed, 10 Jan 2024 10:48:02 -0500 (EST) Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-6da4a923b1bso2181395b3a.2 for ; Wed, 10 Jan 2024 07:48:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1704901682; x=1705506482; darn=lists.bufferbloat.net; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=45vKYxY/ls7S+RuGg46YmD8rl5JobyNhrEw1jNfDDZQ=; b=l8Q7P3qX1zBLGH/BzZuszQbrtV8n07o2e5bnUhj9A73aZ0qUQXJCkc8JjBCj79Bf/h 56JiaZ0HKfnRVD39O5qvM222t/8iFOqC9MkJu8yNefj5bbTuG5HM/fLHLykd9pqNKeN7 WWzjrlCp23U+ZzmtNMbmonT43QzjX1Jw4B1mpHIyMn7UTipN3w1ZKZnWlscYmrvQHQVk dzND0+KBsyz4hVMIa/crEYWVcrqoJiHmsp1Rtq7SULwXOsvGxkEY2rQ1j5qTAajrmiFA oQiqCvbTr0uzPWIe7b/jGLF+P+IDXQ7/OXVhrQ/96SlmR8QxomE7qqPE71CZeSjFUsNu 9wFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704901682; x=1705506482; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=45vKYxY/ls7S+RuGg46YmD8rl5JobyNhrEw1jNfDDZQ=; b=M6lQMOS2D3QSrKnQcTy5oFQ60kmHgv9xUITuWB9yD6JowHXwFq5ETnX8WZgBDEelwC WOoKb73g/i2qX22ee+E9vOU1KaNnG8WcGiFLiq49/DRAIBQtBHLI/DVD00r/z/CCxx0s JAEV+xfcuYV7wfXlJCQQVBPBxoZOVZ6udd1ERVTkF10Ojhl+XarbqPsh9dpTNW0NMPsc uf7693DhEUoPviC/W/40mjl69cVgaiKc3Ke4T1UJHoPtSe3KCqohljuTiMRmnmN8bDtb ZykoY2nfzexAyohx25s1JueUsdgyIfHm18Utsrgdkcwd57NvfI7MEog7XggIf5GIIUMK pcWw== X-Gm-Message-State: AOJu0Yytt1h5439a+VawgQckqXWcr1hGaf8TLaEbeUh5BoULUiQkrpNx sszYwds5tlB0VfXlfQfDuI8LRQO8i9+HTs/UFyk= X-Google-Smtp-Source: AGHT+IEdGDJMuUhmDC31K4mPcMs0ssoimAZVuFslfE0rsm9Ek5rgbGdsP/gys6wECsD/nuOq/U3Jb4VLzHqbg864RwY= X-Received: by 2002:a05:6a21:99a1:b0:199:ea9e:79fa with SMTP id ve33-20020a056a2199a100b00199ea9e79famr878581pzb.55.1704901681598; Wed, 10 Jan 2024 07:48:01 -0800 (PST) MIME-Version: 1.0 References: <877cki4vxq.fsf@toke.dk> <875y01xwi5.fsf@toke.dk> In-Reply-To: From: Dave Taht Date: Wed, 10 Jan 2024 10:47:50 -0500 Message-ID: To: XianJun Jiao Cc: Bob McMahon , =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= , Make-Wifi-fast Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Make-wifi-fast] a cheer up tweet for y'all X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jan 2024 15:48:03 -0000 You have wifi6 now!? Congrats! I have often thought about getting involved in your project, but lacking time and funding was always a stopper. Did ardc ever lean in on your behalf? https://github.com/open-sdr/openwifi is it, yes? What FPGA are you recommending for wifi6? Did you ever manage to get the fq-codel apis working? Just from reading your code I could not figure out where to put it... A bit more below. On Wed, Jan 10, 2024 at 8:55=E2=80=AFAM XianJun Jiao via Make-wifi-fast wrote: > > Regarding the forwarding, I feel that you two are argue different thing? > > If I remember correctly, the mac80211 framework also re-use the ethernet = data-path above some level, so the Linux kernel stack can handle both Wi-Fi= data and Ethernet data. I think we all agree on this level of forwarding. > > I think Bob raise an interesting observation about the NIC on a server: E= thernet VS Wi-Fi. > > To my understanding about why Ethernet NIC works fine (or pretty good) on= a server is that now a days the Ethernet NIC actually faces a non-shared m= edia (medias are isolated by a switch). So the queue/packet in the Ethernet= NIC till on the media could achieve so called 'line rate forwarding' easil= y? > > On the contrary, the Wi-Fi always faces a shared media (ISM band, CSMA/CA= /etc.), and this bring a big difference compared to the Ethernet NIC. I can= elaborate further on this: (because I am doing Wi-Fi chip/FPGA implementat= ion since 2017 =E2=80=93 the openwifi project, and now we have implemented = our Wi-Fi6.) > > More and more complications are packed into the chip level instead of the= driver level, Especially since Wi-Fi6. =E2=80=93 it doesn't matter there i= s "binary firmware blobs" or not. Even there isn't "binary firmware blobs",= the complications will still be there. This is somehow the requirement fro= m the real-time behaviour/aspect defined in the standard. > The Wi-Fi LMAC has to operate precisely in terms of the time (normally or= der of microsecond, or sub-microsecond). =E2=80=93 this has to be in chip. > The Wi-Fi LMAC in the chip will decide: when (depends on CSMA/CA, queue s= tatus/priority, packet priority/etc ) the packet should be transmitted on w= hich sub-band (called RU in Wi-Fi6/7, OFDMA) through which spatial stream (= MIMO, MU-MIMO). In summary: time, frequency and spatial for a packet. > Again reminding: the time, frequency, spatial are controlled by chip not = driver due to their real-time aspects. After the driver handles the packet = to Wi-Fi chip (doesn't matter how good/up-to-date/open the driver is), then= the only thing driver can do is waiting for the packet transmission result= from the chip. > I haven't mentioned that in the chip there could also be multiple re-tran= smission processes, if the first attempts fail. The final packet delivery r= eport only happens after all re-transmission attempts end. Ath9k anecdote - it has 5 queues, of 4 txops each. That is a lot of data outstanding at even the highest rates, and worse, I have seen 30 or more retries in the field programmed in, leading to seconds of HoL blocking. I am really unsure as to whether anyone really got the "minstrel" lesson related to rate control that we leveraged in that chip, either, in successive standard implementations. https://blog.cerowrt.org/post/minstrel/ > In summary, in the case of Wi-Fi there are more and more complicated low-= level behaviors out of the driver control, and this is not the case (most p= robably, or less the case) for Ethernet NIC (I guess). I agree that the hard realtime aspects of any network interface have to be done on board, but it should punt to a more central cpu and OS for anything larger than a txop. In this presentation ( http://www.taht.net/~d/broadcom_aug9_2018.pdf ) I said a ms, these days, if it takes over 120us, punting the functionality to software is beginning to make more sense with the ready availability of multi-cores. We had to settle for double-buffering our fq-codel-for-wifi code because of a few missing features, and the ath10k "AQL"debacle ended up triple buffering because we couldn=C2=B4t get past some stupid design decisions in the firmware, and dang it, the same mistake was carried forward into the mt76 and now mt79 codebases. Still this was orders of magnitude less queuing latency than what we had had before, but 3x worse than what we could have achieved with software/hw co-design. Getting essentially to a zero copy interface where the software folk moving packets intelligently have the right interfaces to the hardware is really, really hard, and takes multiple iterations of the design, and involvement considerably earlier than what generally happens today elsewhere, where finished firmware is thrown over the wall, and that team moves on, while the os folk patch around foolish or broken features for another decade. Or you can try to bundle all the needed functionality into a combined ethernet/wifi path for some definition of need that is usually inadaquate for some markets. > Best regards, > -- > Xianjun Jiao > > ________________________________ > From: Make-wifi-fast on be= half of Toke H=C3=B8iland-J=C3=B8rgensen via Make-wifi-fast > Sent: Wednesday, January 10, 2024 12:23 PM > To: Bob McMahon > Cc: Make-Wifi-fast > Subject: Re: [Make-wifi-fast] a cheer up tweet for y'all > > Bob McMahon writes: > > > This approach is not going to work. Sun workstations as the forwarding > > planes for WiFi doesn't work nor scale and is cost & power inefficient.= The > > WiFi forwarding plane needs to be all hardware and not based off of BSD= . It > > has to be like a port asic in an ethernet switch. No SoC. > > > > Ethernet NICs are targeting servers where the workstation/NIC model doe= s > > work. WiFi is never going to be the basis for cloud servers. > > Well, the original context of the question was "Linux WiFi drivers are > terrible, what can we do about that", and, well, providing proper > upstream drivers at HW launch is the way to solve that. > > And even so, every Linux-based CPE in existence is a contradiction of > you assertion that software-based WiFi forwarding is "not going to > work". On the contrary, the SOCs with proper open source drivers and > support are the ones that work the best, because that means we can run > OpenWrt on them instead of the vendor crapware that they ship with. > > -Toke > _______________________________________________ > Make-wifi-fast mailing list > Make-wifi-fast@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/make-wifi-fast > _______________________________________________ > Make-wifi-fast mailing list > Make-wifi-fast@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/make-wifi-fast --=20 40 years of net history, a couple songs: https://www.youtube.com/watch?v=3DD9RGX6QFm5E Dave T=C3=A4ht CSO, LibreQos