[Make-wifi-fast] NanoStation M5 anyone?

[Make-wifi-fast] NanoStation M5 anyone?

[Pete Heist]

This is an appeal to anyone who has access to a NanoStation M5 and a little time to try to reproduce this problem and add to this post:

https://community.ubnt.com/t5/airMAX-Installation/NanoStation-M5-ping-spikes-about-once-per-second-even-just-to/td-p/2358704

I’ve been trying unsuccessfully to demonstrate to Ubiquiti that they have a bug somewhere that causes isochronous latency spikes. It seems what’s happening is that for a fixed period of time (20ms when one Ethernet port is in use and 40ms when two are in use), once per second, the device queues packets until the pause is over. The function ar7240sw_phy_poll_reset in ag71xx_ar7240.c exactly describes the behavior of 20ms delays when one port is connected and 40ms delays when two ports are connected, so basically it appears that the Ethernet / internal switch device is being reset once per second for some reason.

So far nobody has seemed to care (after all, who’s counting 40ms here or there when it comes to WiFi?), but it does pollute my test results and may be causing latency spikes wherever these devices are deployed (like, worldwide). So if anyone else tests this and can add some weight to the discussion, I’d be glad. :)

Pete

Re: [Make-wifi-fast] NanoStation M5 anyone?

[Toke Høiland-Jørgensen]

Pete Heist <pete@heistp.net> writes:

> This is an appeal to anyone who has access to a NanoStation M5 and a little time to try to reproduce this problem and add to this post:
>
> https://community.ubnt.com/t5/airMAX-Installation/NanoStation-M5-ping-spikes-about-once-per-second-even-just-to/td-p/2358704
>
> I’ve been trying unsuccessfully to demonstrate to Ubiquiti that they
> have a bug somewhere that causes isochronous latency spikes. It seems
> what’s happening is that for a fixed period of time (20ms when one
> Ethernet port is in use and 40ms when two are in use), once per
> second, the device queues packets until the pause is over. The
> function ar7240sw_phy_poll_reset in ag71xx_ar7240.c exactly describes
> the behavior of 20ms delays when one port is connected and 40ms delays
> when two ports are connected, so basically it appears that the
> Ethernet / internal switch device is being reset once per second for
> some reason.
>
> So far nobody has seemed to care (after all, who’s counting 40ms here
> or there when it comes to WiFi?), but it does pollute my test results
> and may be causing latency spikes wherever these devices are deployed
> (like, worldwide). So if anyone else tests this and can add some
> weight to the discussion, I’d be glad. :)

Does this only happen with stock firmware? I think I have reflashed all
of mine...

-Toke

Re: [Make-wifi-fast] NanoStation M5 anyone?

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 1514 bytes --]


> On Jun 12, 2018, at 12:24 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Pete Heist <pete@heistp.net <mailto:pete@heistp.net>> writes:
> 
>> This is an appeal to anyone who has access to a NanoStation M5 and a little time to try to reproduce this problem and add to this post:
>> 
>> https://community.ubnt.com/t5/airMAX-Installation/NanoStation-M5-ping-spikes-about-once-per-second-even-just-to/td-p/2358704
> 
> Does this only happen with stock firmware? I think I have reflashed all
> of mine...

Unfortunately I haven’t confirmed that, because afaik one can no longer flash unsigned firmware on AirOS devices with signed firmware (since 6.0.6 or so) without opening the case and attaching a serial cable / using tftp, but these devices aren’t mine. I just found this thread so maybe I could try to play with first downgrading to 6.0.6, then to 6.0.6 beta, then to unsigned, just don’t need to brick the device:

https://community.ubnt.com/t5/airOS-Software-Configuration/AirOS-v6-0-6-Firmware-Issues-and-Solutions-Downgrade-Path-and-XM/m-p/1996679#M47254

If you do have time to just try a high-rate ping to the Ethernet adapter with a recent OpenWRT/LEDE installed, that would tell me something:

sudo ping -c 5000 -i 0.001 nsm5

If it reproduces, you’ll see isochronous spikes of either 20 or 40ms around once every 900-1000ms. Might be easier to see if you add preload with “-l 100”, but you would see them either way. If you don’t have time, no sweat… :)


[-- Attachment #2: Type: text/html, Size: 6186 bytes --]

Re: [Make-wifi-fast] NanoStation M5 anyone?

[Toke Høiland-Jørgensen]

Pete Heist <pete@heistp.net> writes:

>> On Jun 12, 2018, at 12:24 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> 
>> Pete Heist <pete@heistp.net <mailto:pete@heistp.net>> writes:
>> 
>>> This is an appeal to anyone who has access to a NanoStation M5 and a little time to try to reproduce this problem and add to this post:
>>> 
>>> https://community.ubnt.com/t5/airMAX-Installation/NanoStation-M5-ping-spikes-about-once-per-second-even-just-to/td-p/2358704
>> 
>> Does this only happen with stock firmware? I think I have reflashed all
>> of mine...
>
> Unfortunately I haven’t confirmed that, because afaik one can no
> longer flash unsigned firmware on AirOS devices with signed firmware
> (since 6.0.6 or so) without opening the case and attaching a serial
> cable / using tftp, but these devices aren’t mine. I just found this
> thread so maybe I could try to play with first downgrading to 6.0.6,
> then to 6.0.6 beta, then to unsigned, just don’t need to brick the
> device:
>
> https://community.ubnt.com/t5/airOS-Software-Configuration/AirOS-v6-0-6-Firmware-Issues-and-Solutions-Downgrade-Path-and-XM/m-p/1996679#M47254
>
> If you do have time to just try a high-rate ping to the Ethernet
> adapter with a recent OpenWRT/LEDE installed, that would tell me
> something:
>
> sudo ping -c 5000 -i 0.001 nsm5
>
> If it reproduces, you’ll see isochronous spikes of either 20 or 40ms
> around once every 900-1000ms. Might be easier to see if you add
> preload with “-l 100”, but you would see them either way. If you don’t
> have time, no sweat… :)

I don't, right now. But in a few weeks I might; I'll keep it in mind :)

Re: [Make-wifi-fast] NanoStation M5 anyone?

[Pete Heist]

> On Jun 12, 2018, at 2:09 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> 
>> If you do have time to just try a high-rate ping to the Ethernet
>> adapter with a recent OpenWRT/LEDE installed, that would tell me
>> something:
>> 
>> sudo ping -c 5000 -i 0.001 nsm5
>> 
>> If it reproduces, you’ll see isochronous spikes of either 20 or 40ms
>> around once every 900-1000ms. Might be easier to see if you add
>> preload with “-l 100”, but you would see them either way. If you don’t
>> have time, no sweat… :)
> 
> I don't, right now. But in a few weeks I might; I'll keep it in mind :)

I got LEDE 17.01.4 installed by going from: 6.1.7 -> 6.0.6 -> 6.0.6 beta -> 6.0.4 (unsigned) -> lede-...-squashfs-factory.bin

Surprise surprise, it doesn’t reproduce in LEDE, so it’s something in Ubiquiti's stuff or a fix they didn’t pull in.

Unfortunately I have to test with AirOS, but I’ll just have to repeat all my tests with LEDE too so we can all enjoy looking at the difference. :) Here’s hoping the flash back to factory firmware goes ok.

Thanks for the suggestion!

Re: [Make-wifi-fast] NanoStation M5 anyone?

[Toke Høiland-Jørgensen]

Pete Heist <pete@heistp.net> writes:

>> On Jun 12, 2018, at 2:09 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>> 
>>> If you do have time to just try a high-rate ping to the Ethernet
>>> adapter with a recent OpenWRT/LEDE installed, that would tell me
>>> something:
>>> 
>>> sudo ping -c 5000 -i 0.001 nsm5
>>> 
>>> If it reproduces, you’ll see isochronous spikes of either 20 or 40ms
>>> around once every 900-1000ms. Might be easier to see if you add
>>> preload with “-l 100”, but you would see them either way. If you don’t
>>> have time, no sweat… :)
>> 
>> I don't, right now. But in a few weeks I might; I'll keep it in mind :)
>
> I got LEDE 17.01.4 installed by going from: 6.1.7 -> 6.0.6 -> 6.0.6
> beta -> 6.0.4 (unsigned) -> lede-...-squashfs-factory.bin
>
> Surprise surprise, it doesn’t reproduce in LEDE, so it’s something in
> Ubiquiti's stuff or a fix they didn’t pull in.
>
> Unfortunately I have to test with AirOS, but I’ll just have to repeat
> all my tests with LEDE too so we can all enjoy looking at the
> difference. :) Here’s hoping the flash back to factory firmware goes
> ok.
>
> Thanks for the suggestion!

Yay! Why are you stuck on airos?

-Toke

Re: [Make-wifi-fast] NanoStation M5 anyone?

[Pete Heist]

> On Jun 12, 2018, at 5:52 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> 
>> Surprise surprise, it doesn’t reproduce in LEDE, so it’s something in
>> Ubiquiti's stuff or a fix they didn’t pull in.
>> 
>> Unfortunately I have to test with AirOS, but I’ll just have to repeat
>> all my tests with LEDE too so we can all enjoy looking at the
>> difference. :) Here’s hoping the flash back to factory firmware goes
>> ok.
> 
> Yay! Why are you stuck on airos?

My ISP uses it. I think there will be a hesitation to run unofficial firmware, giving up Ubiquiti’s support and management tools (I think they use AirControl). One of the techs has been testing LEDE though, so we’ll see if there’s any change in light of this… :)

It’s a little frustrating to hear “we’re unable to reproduce it” from Ubiquiti, which sounds a bit like legalese- yeah, "we’re unable to reproduce it" because we don’t have the time or interest to do it. To me this thing is clear as day, especially now...