From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mail.toke.dk; dkim=none; arc=none (Message is not ARC signed); dmarc=fail (Used From Domain Record) header.from=umbernetworks.com policy.dmarc=quarantine Received: from mail.umbernetworks.com (mail.umbernetworks.com [198.74.51.139]) by mail.toke.dk (Postfix) with ESMTPS id 4B11316C45AB for ; Mon, 21 Sep 2026 07:09:59 +0200 (CEST) Received: from webmail.umbernetworks.com (files.umbernetwork.com [198.74.51.139]) by mail.umbernetworks.com (Postfix) with ESMTPA id 5917A21BFCB; Mon, 21 Sep 2026 05:09:57 +0000 (UTC) MIME-Version: 1.0 Date: Sun, 20 Sep 2026 22:09:57 -0700 From: bob.mcmahon@umbernetworks.com To: bob.mcmahon@umbernetworks.com Cc: David Lang , D.Goncz-FAX-571-395-3575@replikon.net, bloat@lists.bufferbloat.net In-Reply-To: <197fd277c46621af6757407f5fd4dc7a@umbernetworks.com> References: <178979811506.1938.15903254559639098502@gauss> <1onq9q61-40o3-pnn6-pns7-58np93687005@ynat.uz> <817d1dce471ea24bcd552ffd5f38e240@umbernetworks.com> <72po68s0-qo56-2304-51s1-830r5qo1n6s2@ynat.uz> <45979n11-rs88-3nq6-1q09-16087491npo9@ynat.uz> <197fd277c46621af6757407f5fd4dc7a@umbernetworks.com> Message-ID: <4711c633fd374bd4120420d032517c1e@umbernetworks.com> X-Sender: bob.mcmahon@umbernetworks.com Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Message-ID-Hash: PFHJRZ4AYN2V4GXA4RXY22ALJ5NZP2R5 X-Message-ID-Hash: PFHJRZ4AYN2V4GXA4RXY22ALJ5NZP2R5 X-MailFrom: bob.mcmahon@umbernetworks.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list Subject: [Bloat] Re: Vol 185, Issue 5 List-Id: General list for discussing Bufferbloat Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: One architectural analogy I should have included: the concentrator is very much in the spirit of the venerable Cisco 7500, a production router architecture that remained in service for nearly two decades. The 7500 used CPU switching with shared memory, called MEMD, and DMA for I/O across the CyBus. We are applying that same shared-memory forwarding model to an 802.11 forwarding plane: central CPU scheduling and switching, with DMA to distributed radio heads over PCIe rather than packet forwarding through Ethernet switches. Bob > David, > > These are the right implementation questions. > > We hope to demo this at IETF 127 using the UAX-8 eval system. The 2027 > plan is up to 96 radio heads over fiber using a C-RAN-style intra-MAC > split. The fronthaul is PCIe from concentrator to radio head, with > shared memory and DMA under DPDK, and no Ethernet switch in the path or > in the concentrator. APs are deprecated in this architecture. > > The 802.11 resource being scheduled is TXOPs. The orthogonal dimensions > are time, space, including spatial eigenmodes, and frequency. More > carriers are one way to create more simultaneous TXOP opportunities. > Spatial reuse creates another. CASP can be used to coordinate > eligibility across all three. > > https://www.umbernetworks.com/ > > Bob > > >> while something like that could run on APs that are connected with >> fast enough wires, the latency in this would be a problem (not just >> wired network latency but processing latency, you are after all >> talking about scheduling many different stations for every transmit >> window) >> >> but you do not have control of when the mobile devices transmit >> (unless you do some sort of token passing protocol, which would need >> more transmit slots to tell the mobile devices when to transmit, >> compounding your airtime problems) >> >> It's not as simple as "only one station can transmit on a given >> channel at a time", if you have a good network layout over a large >> area, that channel is getting re-used many times where multiple >> stations can transmit on it at once without interference. >> >> you are taking a hard real-time task (if your are delayed, the airtime >> is wasted and you clobber someone else who wasn't delayed). wired >> ethernet networks and general purpose operating systems are not good >> at that sort of timing. This is why high speed network cards (wired >> and wireless) have their own internal queues so that the OS can pass >> the data to them as it's avaialble and the cards do the real-time >> tasks of scheduling the transmission. >> >> pushing that scheduling not just back to the main CPU/OS, but across >> the network to a central CPU/OS to make the decision for lots of >> places does not sound scalable[1] >> >> >> In practice, small cells and lots of different channels work very >> well. >> >> The new wifi standards push for a small number of very wideband >> channels that give a single device a huge amount of available >> bandwidth (and is fantastic if you live out in the country) >> >> but when you have congestion, borrow the idea from the cell companies, >> implement micro-cells, lots of low-power, low-footprint stations >> allowing you to re-use channels more overall, and have each station >> use narrow channels instead of wide ones. This is optimizing for >> overall throughput rather than individual throughput and works pretty >> darn well today. >> >> the current problem is that some mobile devices really don't like to >> change stations, so they will transmit louder to try to maintain >> connectivity to their current station rather than switching stations >> (Intel and Apple come to mind) >> >> this doesn't require expensive equipment, it just needs all the APs to >> be on the same SSID, DHCP managed centrally so that the device doesn't >> need to change IPs when it moves from one AP to another, and some >> smarts in allocating the channels to minimize contention between APs. >> >> I'm not saying that it can't be further improved (I am looking at DAWN >> and usteer to try and push devices to better APs faster) >> >> David Lang >> >> [1] as always, "the person saying something cannot be done should not >> interrupt the person doing it". I'd be happy to be proven wrong, but >> there do seem to be some fundamental limits you will run up against >> here. >> >> On Sun, 20 Sep 2026, bob.mcmahon@umbernetworks.com wrote: >> >>> Date: Sun, 20 Sep 2026 16:17:22 -0700 >>> From: bob.mcmahon@umbernetworks.com >>> To: David Lang >>> Cc: D.Goncz-FAX-571-395-3575@replikon.net, >>> bloat@lists.bufferbloat.net >>> Subject: Re: [Bloat] Re: Vol 185, Issue 5 >>> >>> David's deployment experience is exactly the kind of knowledge the >>> industry should be engineering into better products. Cell size, >>> transmit power, hidden stations, the number of active stations, and >>> the resulting contention all determine how well the shared medium >>> behaves under load. >>> >>> I've been working on an interface proposal aimed at that problem. It >>> has the working name Contended Access Scheduling Protocol (CASP), and >>> I'm starting the discussion around it here: >>> >>> https://www.umbernetworks.com/ietf-fiwi-problem-statement.html >>> >>> The introduction: >>> >>> Every 802.11 station contends for the medium independently, following >>> carrier sense multiple access with collision avoidance (CSMA/CA) >>> ahead of transmitting. It applies listen before talk (LBT), and after >>> a transmission, or on finding the medium busy, it must back off >>> regardless of whether anyone is on the air by then. Three properties >>> define the algorithm. Distributed: each station runs it alone on what >>> it can hear, answering to no arbiter anywhere in the system. >>> Stochastic: each draws its own random delay, so chance settles the >>> order. Exponential: each failure doubles the previous range, so >>> consecutive failures compound rather than add. As more stations hold >>> traffic, the odds that several draw the same slot climb steeply, and >>> collisions, retries and widening latency tails follow. The CSMA/CA >>> algorithm runs whenever the driver has packets queued for the 802.11 >>> MAC, and it goes quiet only while an upper layer withholds them. >>> >>> This note proposes an interface that lets a scheduler control when >>> traffic becomes eligible to enter that contention process. It asks >>> two things. The scheduler sends grants, and the endpoint follows >>> them, holding traffic back and releasing it when its grant opens. >>> Grant times are stated in the TSF domain that 802.11 already >>> synchronizes. The protocol has the working name CASP, and it defines >>> neither the scheduler nor any part of the 802.11 MAC. Section 6 sets >>> out the asks. >>> >>> Bob >>> >>> On 2026-09-20 15:49, David Lang wrote: >>>> lower latency lets stations transmit sooner, which could increase >>>> the airtime usage, but it's more pulling it forward in time than >>>> amplification. >>>> >>>> the thing with a guitar amp is that the speaker signal gets back >>>> into the input and the higher the output from the amp, the more gets >>>> into the input. that doesn't happen with packets, more acks do not >>>> result in more signal the same way >>>> >>>> David Lang >>>> >>>> On Sun, 20 Sep 2026, David Lang wrote: >>>> >>>>> Date: Sun, 20 Sep 2026 15:46:23 -0700 (MST) >>>>> From: David Lang >>>>> To: bob.mcmahon@umbernetworks.com >>>>> Cc: David Lang , >>>>> D.Goncz-FAX-571-395-3575@replikon.net, >>>>> bloat@lists.bufferbloat.net >>>>> Subject: Re: [Bloat] Re: Vol 185, Issue 5 >>>>> >>>>> wifi is a very different thing than general network latency (which >>>>> is what we were talking about before) >>>>> >>>>> A large part of it is that wifi is designed or weak signals and a >>>>> low number of users, a very different environment than most people >>>>> see today. >>>>> >>>>> It interprets loss as "this is a weak signal, let me do things that >>>>> will help in the weak signal case", slowing the transmission speed, >>>>> which helps in the face of general noise, but when the problem is >>>>> interference from other wifi transmitters, it instead expands the >>>>> window where the interference will clobber the retransmission. >>>>> >>>>> This is why wifi doesn't degrade gracefully, it goes from "works >>>>> well" to "doesn't work at all" in a very short time and it doesn't >>>>> recover until enough people give up (and retries time out). >>>>> >>>>> AFAIK newer protocols don't fix this, they just make it possible to >>>>> cram more data into a given transmission >>>>> >>>>> This is not a latency issue, just an airtime utilization problem. >>>>> Stations do listen and do not transmit if they hear another station >>>>> talking, but it's very easy to be in a situation where stations A >>>>> and B cannot hear each other, but station C can hear both (the >>>>> better the antennas on the AP, the more likely you are to run into >>>>> this problem, especially when combined with higher transmit levels >>>>> on the AP than on the devices it's talking to) >>>>> >>>>> This is why at the Scale conference, we use a LOT of APs, all >>>>> turned down to very low power and with default or worse antennas on >>>>> them to shrink the transmission range of the APs. >>>>> >>>>> David Lang >>>>> >>>>> On Sun, 20 Sep 2026, bob.mcmahon@umbernetworks.com wrote: >>>>> >>>>>> Date: Sun, 20 Sep 2026 12:30:40 -0700 >>>>>> From: bob.mcmahon@umbernetworks.com >>>>>> To: David Lang >>>>>> Cc: D.Goncz-FAX-571-395-3575@replikon.net, >>>>>> bloat@lists.bufferbloat.net >>>>>> Subject: Re: [Bloat] Re: Vol 185, Issue 5 >>>>>> >>>>>> David is right that packets don't superpose. The useful math here >>>>>> is feedback control. >>>>>> >>>>>> The guitar amp is a useful analogy for feedback instability. A >>>>>> squeal occurs when the loop gain and phase satisfy the conditions >>>>>> for sustained oscillation. The amplifier keeps supplying energy to >>>>>> the loop, and the oscillation grows until nonlinearities and >>>>>> saturation bound it. >>>>>> >>>>>> Networks have feedback loops too. The delay through that loop is >>>>>> the RTT. Tail-drop synchronization, congestion-window sawtooths, >>>>>> and other oscillatory behaviors arise when multiple controllers >>>>>> react to delayed information about a shared bottleneck. That is >>>>>> why control theory is the useful toolbox here. >>>>>> >>>>>> The key state is at the bottleneck queue. Endpoints need a timely >>>>>> congestion signal derived from that state. Loss communicates >>>>>> congestion after the queue has filled far enough to drop a packet. >>>>>> ECN lets the bottleneck communicate that information earlier and >>>>>> more frequently, giving the senders time to adjust their rates >>>>>> while keeping the queue shallow. >>>>>> >>>>>> The difficult segment is the wireless last hop. AQM at an AP >>>>>> manages its queue, while contention arbitrates access to the air. >>>>>> Managing the air itself requires scheduling: grants issued from a >>>>>> point with enough state to make the decision, including wireless >>>>>> state across the relevant radios and bottleneck queue state. >>>>>> >>>>>> An 802.11ax AP has detailed state about its own BSS and can use >>>>>> trigger frames to schedule uplink transmissions within that BSS. >>>>>> Building-wide scheduling requires a broader view: the air state >>>>>> across multiple radios together with the forwarding and bottleneck >>>>>> queue state. The scheduler therefore belongs at the point where >>>>>> those two kinds of state come together. >>>>>> >>>>>> Bob >>>>>> >>>>>> >>>>>> On 2026-09-19 02:16, David Lang wrote: >>>>>>> Explain how you think this resonance would happen? unlike sound, >>>>>>> packets don't 'echo', they deliver messages, which can (and >>>>>>> frequently do) trigger reply messages but unlike audio resonance, >>>>>>> the incoming packets don't add to the outgoing packets to make a >>>>>>> larger signal than the outgoing packets would if the timing was a >>>>>>> little difference. >>>>>>> >>>>>>> David Lang >>>>>>> >>>>>>> >>>>>>> On Sat, 19 Sep 2026, Replikon Research, D-U-N-S/FCN 77-387-4974 >>>>>>> wrote: >>>>>>> >>>>>>>> There are two realms of analysis where things can blow up >>>>>>>> >>>>>>>> One is synergy and synergetics meaning the interactions between >>>>>>>> presumably >>>>>>>> interchangeable parts relevant to the interactions between >>>>>>>> presumably >>>>>>>> interchangeable routers and node points >>>>>>>> >>>>>>>> The other is combinatorics and that's my thing I love >>>>>>>> combinatorics!!!! The >>>>>>>> only thing that matters is what you choose to count and how much >>>>>>>> each part >>>>>>>> of what you count matters to you to put it concisely. >>>>>>>> >>>>>>>> So while I have very little to contribute I am encouraged to >>>>>>>> hear that the >>>>>>>> bottleneck is in the latency but the causes of the latency are >>>>>>>> multi-dimensional there are at least we know where the problem >>>>>>>> is not where >>>>>>>> that would mean we knew the causes no that's not right let me >>>>>>>> know is what >>>>>>>> the problem is yes the problem is latency >>>>>>>> >>>>>>>> What comes to mind is it for certain latencies there may be an >>>>>>>> echo like >>>>>>>> the feedback of a guitar when the amplifiers turned up too high >>>>>>>> >>>>>>>> Does that have any intuitional relationship to these very high >>>>>>>> level >>>>>>>> Network analyzes that you all do everyday >>>>>>>> >>>>>>>> This question of residence no not residents got darn it in the >>>>>>>> microphone >>>>>>>> resonance like my voice yes >>>>>>>> >>>>>>>> I think resonance and spectral radius are two characteristics of >>>>>>>> behind >>>>>>>> level network analysis which apply here to this question of >>>>>>>> latency because >>>>>>>> when spectral radius is one resonance is nearly inevitable >>>>>>>> >>>>>>>> Is the internet we have built evolving to an internet in which >>>>>>>> some >>>>>>>> parameterizable characteristic has a special radius of 1? >>>>>>>> >>>>>>>> I sure as hell hope not >>>>>>>> _______________________________________________ >>>>>>>> Bloat mailing list -- bloat@lists.bufferbloat.net >>>>>>>> To unsubscribe send an email to >>>>>>>> bloat-leave@lists.bufferbloat.net >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> Bloat mailing list -- bloat@lists.bufferbloat.net >>>>>>> To unsubscribe send an email to bloat-leave@lists.bufferbloat.net >>>>>> >>>>> >>>> _______________________________________________ >>>> Bloat mailing list -- bloat@lists.bufferbloat.net >>>> To unsubscribe send an email to bloat-leave@lists.bufferbloat.net >>> >> _______________________________________________ >> Bloat mailing list -- bloat@lists.bufferbloat.net >> To unsubscribe send an email to bloat-leave@lists.bufferbloat.net > _______________________________________________ > Bloat mailing list -- bloat@lists.bufferbloat.net > To unsubscribe send an email to bloat-leave@lists.bufferbloat.net