On Fri, Aug 29, 2014 at 11:06 AM, Dave Taht <dave.taht@gmail.com> wrote:

On Fri, Aug 29, 2014 at 9:57 AM, Aaron Wood <woody77@gmail.com> wrote:
> Comcast has upped the download rates in my area, from 50Mbps to 100Mbps.
> This morning I tried to find the limit of the WNDR3800. And I found it.
> 50Mbps is still well within capabilities, 100Mbps isn't.
>
> And as I've seen Dave say previously, it's right around 80Mbps total
> (download + upload).
>
> http://burntchrome.blogspot.com/2014/08/new-comcast-speeds-new-cerowrt-sqm.html

Thank you very much, as always, for doing public benchmarking with a good setup!

No problem, I find this sort of investigation a lot of fun. Even if it is somewhat maddening at times.

Yes we hit kind of an unexpected wall on everything shipped with a processor
originally designed in 1989, and the prevalance of hardware offloads to bridge
the gap and lower costs between 100mbit and a gige is a real PITA.

Do you think this is a limitation of MIPS as a whole, or just the particular MIPS cores in use on these platforms?

OTOH, I have noticed that MIPS is losing ground to ARM as bandwidths go up. The router SoCs that I'm seeing from the usual suspects have been switching from MIPS to ARM over the last year or two. The WNDR is in the top-tier for SOHO SoCs, but at a product family is getting long in the tooth.

> I tried disabling downstream shaping to see what the result was, and it
> wasn't pretty.

Well, I'll argue that only seeing an increase of 20ms or so with the upstream
only, fq_codeled, (vs 120ms not) is not bad and within tolerances of most
applications, even voip. Secondly the characteristics of normal
traffic, as opposed
to the benchmark, make it pretty hard to hit that 100mbit download limit,
so a mere outbound rate limiter will suffice.

Well, yes... I have considered just turning it off entirely, as the the extra latency isn't awful. And frankly, the laptops (individually) never see that sort of bandwidth, but the AppleTV might when downloading video (I need to go see what the downloads are capped at by Apple).

The cpu caches are 32k/32k, the memory interface 16 bit. The rate limiter
(the thing eating all the cycles, not the fq_codel algorithm!) is
single threaded and has global locks,
and is at least partially interrupt bound at 100Mbits/sec.

This is interesting, and lines up with Sebastian's idea about perhaps using ethtool to lock the upstream interface to 100Mbps. Except that moves the bottleneck to the next upstream device... (modem), with it's buffer mgmt, so maybe that's not a great idea, either. Upstream is certainly where the biggest issues are.

> Or should I start looking for something like this:
>
> http://www.gateworks.com/product/item/ventana-gw5310-network-processor
>
> (although that's an expensive board, given the very low production volume,
> for the same cost I could probably build a small passively-cooled
> mini/micro-atx setup running x86 and dual NICs).

There is that option as well. I would certainly like to find a low end x86 box
that could rate limit + fq_codel at up to 300Mbits/sec. Toke's x86 boxes
have proven out to do 100Mbit/10Mbit correctly, but I don't remember their
specs, nor has he tried to push them past that, yet.

If I do get my hands on a Ventana board (I may still for work purposes), I'll certain set it up and see what it does in this scenario, too.

-Aaron