From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 04D613CB38 for ; Sun, 10 Feb 2019 22:03:39 -0500 (EST) Received: by mail-ed1-x52f.google.com with SMTP id g4so7618188eds.4 for ; Sun, 10 Feb 2019 19:03:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=P3y+FxEJNgYe+PDbXKgKxzqUH6Gze6ccFeyX8FUyWtg=; b=Vh+Xs3/72tvr82+uPsbGw6zQRjYo7ChLgl+xALCopZ4M3GpdFg4D+S1dtyBhi8c0T1 +xWyg0qPBn4XRik79vktUw4aSyI15ILKxHQ71hR5oGfHczMSUNd3+4fHr7k/N1MqfLVK 7Wka0kTCB0h1DeaTQFY5m+8PPqqE29pFgJJjw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=P3y+FxEJNgYe+PDbXKgKxzqUH6Gze6ccFeyX8FUyWtg=; b=MrWtng7y9yzNNfV9r+yQ0f5e8OWWO0l4FbI6vlXsFMRiZLlRxYvhHrjRbrXzxFIZQs 1QebSpITafhQPCToDeO9NQGMCG/NvUgq5xR5PeujR302Eo+rQs+UvwET7fu8aGHAsQVN 7FdsCIzdMZWE1yQVMYvfDw8kNLxYeaZG70rTYzbiz/0WcaHnIspHSTR40/ZOt4MNsqJ6 WbHEhdqIW6KXMG6Yuay6JXNOnXKTfJe1I3PHjA6IyjdSbyC1JDc75tdC+nTodHZGsQeZ VP+VIr91n/jrx+4hyxSuhjCEmZdPOw2mfvzsbXrS2TfZEGUol6f+ogE2CMkYKFQGSuK0 Jkzw== X-Gm-Message-State: AHQUAuYup0DCS0yiQ/8R+o3cQOnyVcWKscsyhLGbZqJmkkdCOctQZmWJ u5+aZ4gINDHYiLIVidGsRlT6eNpFWr0wuOH11Q4JuQ== X-Google-Smtp-Source: AHgI3Ibe3WvTKB4X6kWkTGyz3IMOclwj1TcsMjw9ftJLq1PxJc0zW6H2tte4kg8j2VYPkH728DKpgFMw98mCRV4x82Y= X-Received: by 2002:a17:906:e2d5:: with SMTP id gr21mr20251679ejb.145.1549854217816; Sun, 10 Feb 2019 19:03:37 -0800 (PST) MIME-Version: 1.0 References: <3CB198EB-2844-42DB-9E5A-26708BFD7304@gmail.com> <1549825225.478726910@apps.rackspace.com> In-Reply-To: From: Bob McMahon Date: Mon, 11 Feb 2019 08:33:26 +0530 Message-ID: To: "David P. Reed" Cc: Jonathan Morton , Make-Wifi-fast Content-Type: multipart/alternative; boundary="0000000000001b94f70581958e2a" Subject: Re: [Make-wifi-fast] bloated ath10k X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Feb 2019 03:03:39 -0000 --0000000000001b94f70581958e2a Content-Type: text/plain; charset="UTF-8" Also, way off topic but the data center switch architects have proposed something called HULL. https://people.csail.mit.edu/alizadeh/papers/hull-nsdi12.pdf One model of the forwarding plane is that the chip firmware is indeed part of the forwarding and the spectrum/AP provides the mesh or "switching fabric." It's somewhat similar to how data center switches are designed. In that context a STA, besides minimizing it's processing delay, may be able to do the following interesting things: 1. If RX queues are "full" (or hit the phantom queue depth) set the CE bit to signal TCP 2. If TX queues are "full" and a standing queue (per bloat) is behind it, then drop. If not, then advantage the air access via adaptive EDCA, e.g. reduce the AIFS (within the limits of the standards) An AP might want to advertise a somewhat larger AIFS so a STA reducing its for a brief period will achieve the arbitration goals. As a side note, we've done a lot of timing work in iperf 2.0.13 to help fault isolate latency issues. It works best if the device realtime clocks are synchronized, e.g. to a GPS disciplined OCXO using PTP. We also added TCP connect times to monitor the 3WHS as well as trip times, timing the data xfer phase. Iperf 2.0.13 has been picked up by many distributions. https://sourceforge.net/projects/iperf2/ https://sourceforge.net/projects/iperf2/files/Iperf%202.0.13%20Enhancements.pdf/download Thoughts welcomed and appreciated Bob On Mon, Feb 11, 2019 at 7:48 AM Bob McMahon wrote: > Just a reminder with 802.11ax uplink OFDMA that STA A and B transmits can > be "concurrent." Also, there are companies working on full duplex per > self cancellation. Kumu networks is one > > http://kumunetworks.com/ > > Bob > > On Mon, Feb 11, 2019 at 12:30 AM David P. Reed > wrote: > >> Side note: between two stations talking through an AP, it's not half >> duplex. It's kind-of quarter-duplex. Each packet between STA A and STA B >> cannot be concurrent with the subsequent packet from A to B, and both >> transmissions of a packet from B to A. >> >> That actually has a significant effect on "queue depth". Please don't >> call it "interference", because no packet corruption happens. >> >> >> >> This is why merely fixing the queueing discipline in the AP alone doesn't >> necessarily ameliorate bufferbloat. The queues in the STA's need to be >> managed, too. >> >> >> >> You guys know that implicitly, I know I'm not telling you anything you >> don't know. But this queueing needs to be managed in such a way that the >> backoff in TCP is signaled. That is, packets need to be dropped or marked. >> You can't fix this in the forwarding paths alone. >> >> >> >> -----Original Message----- >> From: "Jonathan Morton" >> Sent: Sunday, February 10, 2019 7:38am >> To: "Adrian Popescu" >> Cc: make-wifi-fast@lists.bufferbloat.net >> Subject: Re: [Make-wifi-fast] bloated ath10k >> >> > On 10 Feb, 2019, at 2:24 pm, Adrian Popescu >> wrote: >> > >> > My attempts to use SQM and codel to reduce wifi bloat didn't seem to >> get me very far. 802.11ac seems more reliable and it seems to be more >> bloated. ath9k can go as low as 3-5 milliseconds. ath10k is usually in the >> 20-50 milliseconds range (or more, based on the number of stations). I >> usually test with a single client as I don't expect latency to improve with >> more clients. >> >> Some things are unavoidable when you move to a shared, half-duplex, noisy >> medium like wifi versus a switched, full-duplex, error-free medium like >> Ethernet. >> >> If you're getting 20-50ms under load, then I think things are working >> quite well with wifi. We can wish for better, but not long ago you could be >> looking at multiple seconds in bad cases. At the levels you're now seeing, >> ordinary interactive protocols like DNS and HTTP will work reliably and >> quickly, and even VoIP should be able to cope; you'll likely only really >> notice a problem with online games. >> >> Ath9k has some advantages over ath10k in this area. Its MAC is managed at >> a lower level by the driver, so we have much more control over it when >> trying to debloat. I think we still have more control over ath10k than most >> of the alternatives. >> >> If low latency is mission critical, though - just run a wire. >> >> - Jonathan Morton >> >> _______________________________________________ >> Make-wifi-fast mailing list >> Make-wifi-fast@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/make-wifi-fast >> _______________________________________________ >> Make-wifi-fast mailing list >> Make-wifi-fast@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/make-wifi-fast > > --0000000000001b94f70581958e2a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Also, w= ay off topic but the data center switch architects have proposed something = called HULL.

=C2=A0https://people.csail.mit.edu/alizadeh/papers/hull-n= sdi12.pdf

One model of the forward= ing plane is that the chip firmware is indeed part of the forwarding and th= e spectrum/AP provides the mesh or "switching fabric."=C2=A0 It&#= 39;s somewhat similar to how data center switches are designed.=C2=A0 =C2= =A0In that context a STA, besides minimizing it's processing delay, may= be able to do the following interesting things:
  1. If RX qu= eues are "full" (or hit the phantom queue depth) set the CE bit t= o signal TCP
  2. If TX queues are "full" and a standing queue= (per bloat) is behind it, then drop.=C2=A0 If not, then advantage the air = access via adaptive EDCA, e.g. reduce the AIFS (within the limits of the st= andards)=C2=A0 An AP might want to advertise a somewhat larger AIFS so a ST= A reducing its for a brief period will achieve the arbitration goals.
    3. <= /ol>
    As a side note, we've done a lot of timing work in iperf 2.0.1= 3 to help fault isolate latency issues.=C2=A0 It works best if the device r= ealtime clocks are synchronized, e.g. to a GPS disciplined OCXO using PTP.= =C2=A0 =C2=A0We also added TCP connect times to monitor the 3WHS as well as= trip times, timing the data xfer phase.=C2=A0 =C2=A0Iperf 2.0.13 has been = picked up by many distributions.


    Thoughts we= lcomed and appreciated

    Bob=C2=A0
<= /div>
O= n Mon, Feb 11, 2019 at 7:48 AM Bob McMahon <bob.mcmahon@broadcom.com> wrote:
J= ust a reminder with 802.11ax uplink OFDMA that STA A and B transmits can be= "concurrent."=C2=A0 =C2=A0 Also, there are companies working on = full duplex per self cancellation.=C2=A0 Kumu networks is one

On Mon, Feb 11, 2019= at 12:30 AM David P. Reed <dpreed@deepplum.com> wrote:

Side note: bet= ween two stations talking through an AP, it's not half duplex. It's= kind-of quarter-duplex. Each packet between STA A and STA B cannot=C2=A0 b= e concurrent with the subsequent packet from A to B, and both transmissions= of a packet from B to A.

That a= ctually has a significant effect on "queue depth".=C2=A0 Please d= on't call it "interference", because no packet corruption hap= pens.

=C2=A0=

This i= s why merely fixing the queueing discipline in the AP alone doesn't nec= essarily ameliorate bufferbloat. The queues in the STA's need to be man= aged, too.

=C2=A0=

You gu= ys know that implicitly, I know I'm not telling you anything you don= 9;t know. But this queueing needs to be managed in such a way that the back= off in TCP is signaled. That is, packets need to be dropped or marked. You = can't fix this in the forwarding paths alone.

=C2=A0=

-----O= riginal Message-----
From: "Jonathan Morton" <chromatix99@gmail.com>=
Sent: Sunday, February 10, 2019 7:38am
To: "Adrian Popescu"= ; <adrian= nnpopescu@gmail.com>
Cc: make-wifi-fast@lists.bufferbloat.net<= br>Subject: Re: [Make-wifi-fast] bloated ath10k

> O= n 10 Feb, 2019, at 2:24 pm, Adrian Popescu <adriannnpopescu@gmail.com> wrote:=
>
> My attempts to use SQM and codel to reduce wifi bloat did= n't seem to get me very far. 802.11ac seems more reliable and it seems = to be more bloated. ath9k can go as low as 3-5 milliseconds. ath10k is usua= lly in the 20-50 milliseconds range (or more, based on the number of statio= ns). I usually test with a single client as I don't expect latency to i= mprove with more clients.

Some things are unavoidable when you move = to a shared, half-duplex, noisy medium like wifi versus a switched, full-du= plex, error-free medium like Ethernet.

If you're getting 20-50ms= under load, then I think things are working quite well with wifi. We can w= ish for better, but not long ago you could be looking at multiple seconds i= n bad cases. At the levels you're now seeing, ordinary interactive prot= ocols like DNS and HTTP will work reliably and quickly, and even VoIP shoul= d be able to cope; you'll likely only really notice a problem with onli= ne games.

Ath9k has some advantages over ath10k in this area. Its MA= C is managed at a lower level by the driver, so we have much more control o= ver it when trying to debloat. I think we still have more control over ath1= 0k than most of the alternatives.

If low latency is mission critical= , though - just run a wire.

- Jonathan Morton

______________= _________________________________
Make-wifi-fast mailing list
Make-wif= i-fast@lists.bufferbloat.net
https://lists.bufferbloat.net/= listinfo/make-wifi-fast

_______________________________________________
Make-wifi-fast mailing list
M= ake-wifi-fast@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/make-wif= i-fast
--0000000000001b94f70581958e2a--