[Make-wifi-fast] Ubiquiti fixed framing experience

[Make-wifi-fast] Ubiquiti fixed framing experience

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 3283 bytes --]

We’ve been using some new PrismStation 5ACs at my ISP that offer 5 “TDD Framing” options:

"Flexible (legacy)” - traditional airMAX, probably
“Flexible (NEW)” - newfangled airMAX
“Fixed (5 ms)” - fixed 5 ms framing
“Fixed (8 ms)” - fixed 8 ms framing
“Fixed (10 ms)” - fixed 10 ms framing

Since installation last summer we’d been running with “Fixed (5 ms)” with a 33/66 upload/download ratio, because Ubiquiti touts the scalability of the fixed framing modes. There’s also a GPS sync option that allows APs to sync with fixed framing, allowing channel re-use. Sounds nice.

But even fixed 5ms frames mean minimum RTTs to the AP of 10ms, and on higher load, noise, etc, RTTs that can look roughly quantized to 15ms, 20ms and up. Whereas I’d see 1.3ms mean RTT to the old Rocket M5, I’d see mean 13ms to the new PrismStation 5AC.

ISP members were reporting speedtest.net <http://speedtest.net/> results of only up to about 5Mbit during evening load times and 20Mbit in the middle of the night. But tests to fast.com <http://fast.com/> and dslreports.com/speedtest <http://dslreports.com/speedtest>, which both use 8 or more flows by default, could show 40-50Mbit at the very same time that speedtest.net <http://speedtest.net/> was showing 20Mbit. Sounds like a TCP window scaling problem. Meanwhile, bi-directional throughput tests straight to the AP with Ubiquiti’s web UI speed test looked great (180 Mbit!!)

One morning I decided to try “Flexible (NEW)” framing. Right away, ping to the AP dropped to mean 6.5ms with far less variation, and late night speedtest.net <http://speedtest.net/> results jumped to 80Mbit, limited mostly by our backhaul uplink. During the day, users reported “my speed over doubled” and I could feel the latency improvement. SmokePing to the AP agreed:

https://www.heistp.net/downloads/ff_smokeping.png <https://www.heistp.net/downloads/ff_smokeping.png> (on which 3am morning did we switch to flex framing?)

IRTT to an Internet host showed large drops in send delay and send IPDV (less so on the receive path):

https://www.heistp.net/downloads/ff_send_delay.png <https://www.heistp.net/downloads/ff_send_delay.png>

https://www.heistp.net/downloads/ff_send_ipdv.png <https://www.heistp.net/downloads/ff_send_ipdv.png>

Then I found out we’re not the only ones:

https://community.ubnt.com/t5/airMAX-AC/New-Flexible-framing-and-huge-improvements-in-TCP-latency/td-p/2385132 <https://community.ubnt.com/t5/airMAX-AC/New-Flexible-framing-and-huge-improvements-in-TCP-latency/td-p/2385132>

Maybe fixed framing is useful in urban environments with hundreds of clients, but with two clients on our sector it was rather disastrous and led to a few months of bad Internet until we figured it out. 10ms chunks of discarded time can really mean a lot.

I wish we could try OpenWRT here with airtime fairness for comparison, but AirControl is too engrained as an administrative tool, so it’s tough to switch. And the AP is on top of a tower, on top of a castle on a hill, so it doesn’t get worked on often.

Anyway that’s my story, it’s 2019 and it seems that vendors are still not thinking about what latency means, even when it can start to affect TCP...

Pete


[-- Attachment #2: Type: text/html, Size: 4475 bytes --]

Re: [Make-wifi-fast] Ubiquiti fixed framing experience

[Toke Høiland-Jørgensen]

Pete Heist <pete@heistp.net> writes:

> Anyway that’s my story, it’s 2019 and it seems that vendors are still
> not thinking about what latency means, even when it can start to
> affect TCP...

Nice writeup. Have you considered writing a blog post? :)

-Toke

Re: [Make-wifi-fast] Ubiquiti fixed framing experience

[Pete Heist]

> On Jan 22, 2019, at 10:08 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Pete Heist <pete@heistp.net> writes:
> 
>> Anyway that’s my story, it’s 2019 and it seems that vendors are still
>> not thinking about what latency means, even when it can start to
>> affect TCP...
> 
> Nice writeup. Have you considered writing a blog post? :)

I’ll try to do this together with more practical stories about queueing the backhaul.

How airtime is managed is a serious thing. It’s like WiFi gold, but I probably don’t have to mention that on this list. :)

Pete

Re: [Make-wifi-fast] Ubiquiti fixed framing experience

[Dave Taht]

Pete Heist <pete@heistp.net> writes:

>> On Jan 22, 2019, at 10:08 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> 
>> Pete Heist <pete@heistp.net> writes:
>> 
>>> Anyway that’s my story, it’s 2019 and it seems that vendors are still
>>> not thinking about what latency means, even when it can start to
>>> affect TCP...
>> 
>> Nice writeup. Have you considered writing a blog post? :)
>
> I’ll try to do this together with more practical stories about queueing the backhaul.
>
> How airtime is managed is a serious thing. It’s like WiFi gold, but I probably don’t have to mention that on this list. :)

Also your
https://community.ubnt.com/t5/airMAX-AC/New-Flexible-framing-and-huge-improvements-in-TCP-latency/td-p/2385132

reference pointed at preseem which is a company started to better manage
wifi (with tools like that - and means to put fq_codel/htb/cake etc
inline) Go, preseem!

>
> Pete
>
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast

Re: [Make-wifi-fast] Ubiquiti fixed framing experience

[Pete Heist]

> On Jan 22, 2019, at 9:06 PM, Dave Taht <dave@taht.net> wrote:
> 
> Pete Heist <pete@heistp.net> writes:
> 
>>> On Jan 22, 2019, at 10:08 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>> 
>>> Pete Heist <pete@heistp.net> writes:
>>> 
>>>> Anyway that’s my story, it’s 2019 and it seems that vendors are still
>>>> not thinking about what latency means, even when it can start to
>>>> affect TCP...
>>> 
>>> Nice writeup. Have you considered writing a blog post? :)
>> 
>> I’ll try to do this together with more practical stories about queueing the backhaul.
>> 
>> How airtime is managed is a serious thing. It’s like WiFi gold, but I probably don’t have to mention that on this list. :)
> 
> Also your
> https://community.ubnt.com/t5/airMAX-AC/New-Flexible-framing-and-huge-improvements-in-TCP-latency/td-p/2385132
> 
> reference pointed at preseem which is a company started to better manage
> wifi (with tools like that - and means to put fq_codel/htb/cake etc
> inline) Go, preseem!

Yep, I also noticed that and saw them around here before. I wish them well. :)

Point taken about getting more visibility for some longer write-ups. I’ll solve it, eventually...

Re: [Make-wifi-fast] Ubiquiti fixed framing experience

[Dave Taht]

Toke Høiland-Jørgensen <toke@toke.dk> writes:

> Pete Heist <pete@heistp.net> writes:
>
>> Anyway that’s my story, it’s 2019 and it seems that vendors are still
>> not thinking about what latency means, even when it can start to
>> affect TCP...
>
> Nice writeup. Have you considered writing a blog post? :)

Great writeup.

It certainly bugs me that our email list archive is so hard to find via
google search, and blogging (or adding to the bufferbloat web site)
seems to be a better option than posts here.

>
> -Toke
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast