[Cerowrt-devel] Ubiquiti QOS

Sun May 25 12:33:40 PDT 2014

On Sun, May 25, 2014 at 11:39 AM, Sebastian Moeller <moeller0 at gmx.de> wrote:
> Hi Dane,
>
>
> On May 25, 2014, at 08:17 , Dane Medic <dm70dm at gmail.com> wrote:
>
>> Is it true that devices with less than 64 MB can't handle QOS? -> https://lists.chambana.net/pipermail/commotion-dev/2014-May/001816.html
>
>         I think this means that the commotion developers think that 64MB are required.

A dev thinks so. It aint true.

(I have a deployed mesh network of mostly nano and picostations and
they never crash due to being out of memory. Sadly, they do crash for
other reasons)

As you get below 64MB some compromises are needed, and as you try to
push more bits compromises are needed, and as you add more interfaces
and queues compromises are needed. If you are using up all your memory
running some application or another, instead of having it free for
packets, you are generally in more trouble than you want to be.

So the starting factor is how much free ram you have in normal
operation with your applications running. Start with that as a
baseline. (it isn't strictly true either, you can
discard (not swap out) a great deal of program text and still have
your applications run well)

You typically want something like sqm on your gateway ethernet
interface. If you have seriously limited free ram , just run a single
fq_codel instance as in simplest.qos.

As for the mesh backbone, well, there remains so much buffering
underneath in the ath9k wifi drivers that fq_codel only takes the edge
off a little.

We just disabled 802.11e completely (getting rid of 3 out of 4 queues
per interface), and my results have always been better for that, and I
hope to catagorize them this summer -
and it also saves on memory usage.

> But it does not sound like they have first hand experience so this is either hearsay, or commotions mesh networking is memory intensive. On the openwrt side there seems no documentation of minim ram requirements. Doing a quick back-of-the-envelop calculation here:

> openWRT qos has 4 tiers which run fq_codel in both directions so we have 8 fq_codel instances, with each fq_codel having a limit of 10240 packets, so worst case we expect:
>
> 4 * 2 * 10240 = 81920 packets
>
> at 1500bytes this equals
>
> 4 * 2 * 10240 * 1500 / (1024 * 1024) = 117.1875 MB

Try to watch out for this sort of equivalence. Acks in the other
direction are 66 bytes. Arguably we should have specified fq_codel's
outside limit in bytes, not packets, and made it autotune
to some ratio around free memory.  And NONE of this memory is
pre-allocated... more on that in a paragraph

> this indeed is a bit heavy on a 32MB router, but honestly 64MB will not really help you. Then again current openwrt has a limit off 800 instead of 10240 so we end up at a worst case of:

It's the extra SSIDs that hurt most these days, followed by the queues
"needed" for 802.11e, but wait a paragraph...

> 4 * 2 * 800 * 1500 / (1024 * 1024) = 9.1552734375 MB
>
> which should still be possible with 32MB. (Note that typically fq_codel does not fill its queues up to limit, but it still would be bad if a router can easily be DOSed into OOM and rebooting…)

The principal reason for the limit is! to avoid a DOS in memory
limited routers. Otherwise I'd be perfectly happy if we could run it
at the defaults. And the limit should go up some
as we get closer to pushing gigE speeds (presently the router can only
forward at about 330mbit). I am really hating seeing people cut/paste
the limit into newer code without realizing that it was just there to
keep a 32MB box from crashing under a dos...

(and I note, that if you run out of memory to service packets, the
odds are very good your box won't crash anyway - but this exercises
code paths that are rarely touched.)

>
> (For current cerowrt with simple.qos the worst case is:
> (1001 * 4 + 1000 * 13 + 800 * 12) * 1500 / (1024 * 1024) = 38.0573272705 MB
>
> yet this still works quite well on a 64MB device (only 4 of these queues are connected to the WAN interface though)

3 up, 3 down, actually in simple.qos, 1 up, 1 down in simple.

And each wifi SSID consumes 4 queues (although right now, 3 are unused)

I want to make really clear: the FIXED overhead of fq_codel is
something like 100+ 64 bytes*flows (usually 1024) - so each fq_codel
instance eats 64K (not M!) of data to run.

The limit just keeps packet data (don't quote me, something like 200
bytes + packet size) under control, and usually only builds up on the
bottleneck device, not anywhere else.

So you might have a bottleneck on your wifi, someone dos-ing you
maybe, but the ethernet interface will be running at only a few
packets outstanding at anytime.

> One of the bigger issues with devices with small RAM is that often they have relatively weak CPUs and I seem to recall that cerowrt tops out around 60 to 70 Mbit/sec (total for ingress and egress) due to its shaping performance.

Yes. Doing considerably

>
>
> So unless you want to run commotion you might want to ask on the openwrt list…
>
> Best Regards
>         Sebastan
>
>
>
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel at lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

-- 
Dave Täht

NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article