[Cerowrt-devel] Linksys wrt1900acs rrul traces

[Cerowrt-devel] Linksys wrt1900acs rrul traces

[Richard Smith]

A while ago I mentioned that I purchased a Linksys wrt1900acs and I was 
going to do some comparisons of speed/rrul for stock firmware and stuff 
running sqm.

It took a while to happen but I finally got it done.  I was stalled for 
a long time because for a while OpenWRT was not complete for this 
router.  It would boot but you could not (easily) return to the factory 
firmware as updates via the OpenWRT web interface did not work.

Command line updates still worked though periodically I would build dd 
head and see if it got fixed.  Finally one weekend I decided to dig in 
and figure out what was going wrong and discovered that someone else 
beat me to it and it all was working now.

The following flent rrul runs were run here on my local Gbit network 
(pretty quiet) using with OpenWrt Designated Driver r49051 / LuCI Master 
(git-16.081.38806-6b9a743) + Cake installed.

Cake installed via the instructions here:

http://www.bufferbloat.net/projects/codel/wiki/Cake#Installing-CAKE-out-of-tree-on-OpenWrt-rough-instructions

For these tests I had the inbound and outbound limits set to 975000 
kbps.  975000 was somewhat arbitrary.  I wanted it below 1Gbps enough 
that I could be sure it was the router as the limit but yet fast enough 
that I would be able to see the peak transfer rates.

Test setup is:

Laptop<-->wrt1900acs<--->Gbit Switch<--->Netperf server

Prior to running the tests I did trial runs without the wrt1900acs in 
the chain to verify every thing worked at Gbit speeds.

I ran the rrul test for each of the SQM settings at least 5 times.  For 
layer cake and piece of cake each had a run with results that were way 
off from the previous run.  I'm assuming this is some yet to be fixed 
issue and so I ran it 1 more time so that there were 5 runs with similar 
results.

All the raw files are in this tarball:

https://drive.google.com/file/d/0B-P0wCbNmKvAWmRUSzNMdXpUOFE/view?usp=sharing

Hope this is helpful.  I'll be happy to run any additional tests if 
someone wants more.

-- 
Richard A. Smith

Re: [Cerowrt-devel] Linksys wrt1900acs rrul traces

[Dave Taht]

The default firmware is quite horrificly bad, isn't it? Total
starvation of anything but the first flow, for example, in the native
tests.
Do you have the exact version number of the 1900ac's default firmware
you tested.

(I've seen it fall apart on 2 flows, too). I can hardly imagine the
number of users that don't wipe out the default firmware and just live
with this *crap*.

However, I'd have recommended you try 900mbit to see if cake/etc does
the right things, not 975mbit.


Dave Täht
Let's go make home routers and wifi faster! With better software!
https://www.gofundme.com/savewifi


On Thu, Apr 7, 2016 at 6:58 AM, Richard Smith <smithbone@gmail.com> wrote:
> A while ago I mentioned that I purchased a Linksys wrt1900acs and I was
> going to do some comparisons of speed/rrul for stock firmware and stuff
> running sqm.
>
> It took a while to happen but I finally got it done.  I was stalled for a
> long time because for a while OpenWRT was not complete for this router.  It
> would boot but you could not (easily) return to the factory firmware as
> updates via the OpenWRT web interface did not work.
>
> Command line updates still worked though periodically I would build dd head
> and see if it got fixed.  Finally one weekend I decided to dig in and figure
> out what was going wrong and discovered that someone else beat me to it and
> it all was working now.
>
> The following flent rrul runs were run here on my local Gbit network (pretty
> quiet) using with OpenWrt Designated Driver r49051 / LuCI Master
> (git-16.081.38806-6b9a743) + Cake installed.
>
> Cake installed via the instructions here:
>
> http://www.bufferbloat.net/projects/codel/wiki/Cake#Installing-CAKE-out-of-tree-on-OpenWrt-rough-instructions
>
> For these tests I had the inbound and outbound limits set to 975000 kbps.
> 975000 was somewhat arbitrary.  I wanted it below 1Gbps enough that I could
> be sure it was the router as the limit but yet fast enough that I would be
> able to see the peak transfer rates.
>
> Test setup is:
>
> Laptop<-->wrt1900acs<--->Gbit Switch<--->Netperf server
>
> Prior to running the tests I did trial runs without the wrt1900acs in the
> chain to verify every thing worked at Gbit speeds.
>
> I ran the rrul test for each of the SQM settings at least 5 times.  For
> layer cake and piece of cake each had a run with results that were way off
> from the previous run.  I'm assuming this is some yet to be fixed issue and
> so I ran it 1 more time so that there were 5 runs with similar results.
>
> All the raw files are in this tarball:
>
> https://drive.google.com/file/d/0B-P0wCbNmKvAWmRUSzNMdXpUOFE/view?usp=sharing
>
> Hope this is helpful.  I'll be happy to run any additional tests if someone
> wants more.
>
> --
> Richard A. Smith
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] Linksys wrt1900acs rrul traces

[Richard Smith]

On 04/07/2016 11:24 AM, Dave Taht wrote:
> The default firmware is quite horrificly bad, isn't it?

:)  Yeah.  I had high hopes that they might have done something better 
out of the gate.

> Total
> starvation of anything but the first flow, for example, in the native
> tests.

> Do you have the exact version number of the 1900ac's default firmware
> you tested.

1900ACS not AC.  Slightly different hardware. ACS is a faster CPU ( 
(1.6Ghz) and more RAM.

Firmware version is in the directory name. :)

'wrt1900acs_stockfw_1.0.0.169041' It was the latest firmware on the 
website when I ran the tests.

> However, I'd have recommended you try 900mbit to see if cake/etc does
> the right things, not 975mbit.

Ok. I'll use 900 next time.

-- 
Richard A. Smith

Re: [Cerowrt-devel] Linksys wrt1900acs rrul traces

[moeller0]

Hi Richard,

> For these tests I had the inbound and outbound limits set to 975000 kbps.  975000 was somewhat arbitrary.  I wanted it below 1Gbps enough that I could be sure it was the router as the limit but yet fast enough that I would be able to see the peak transfer rates.

All of the following might be old news to you, but please let me elaborate for others on this list (well, most folks here know way more about these things than I do). 
	I believe Gbit ethernet is trickier than one would guess, the 1 Gbit rate contains some overhead that one typically does not account for. Here is the equivalent on-the-wire size of a full MTU non-jumbo ethernet frame:
Layer “1+": 1500 (payload pMTU) + 6 (dest MAC) + 6 (src MAC) + 2 (ethertype) + 4 (FCS) + 7 (preamble) + 1 (start of frame delimiter) + 12 (interframe gap)) = 1500+6+6+2+4+7+1+12 = 1538
”Equivalent” in that the interframe gap is not really used, but filled with silence but it has the duration one would need for 12 bytes.

The kernel, if left to its own devices, will only account for 14 of 38 overhead bytes. But that means that each packet will carry an additional 24 bytes of unaccounted for size that still needs to be transferred:

975000 * (1538/1514) = 990455.746367 (which still is below the 1GBit Layer1+ ceiling that GbE has).  At 985000 * (1538/1514) = 1000614.26684 you would already have slowly caused the NIC’s buffer to fill ;)
Luckily sqm-scripts will allow you to specify any additional per-packet overhead so just set this to 24 and things should just work out I believe.


Best Regards
	Sebastian

Re: [Cerowrt-devel] Linksys wrt1900acs rrul traces

[Dave Taht]

There are multiple hardware queues on this ethernet interface but not
enough to avoid collisions. The driver has really deep queues. The
driver itself is buggy and lacks BQL. My own take on it was to use the
hardware queues for different forms of QoS, rather than spreading
flows across it.

There was some work on getting BQL to work on it; that stalled out.


On Thu, Apr 7, 2016 at 9:16 AM, moeller0 <moeller0@gmx.de> wrote:
> Hi Richard,
>
>> For these tests I had the inbound and outbound limits set to 975000 kbps.  975000 was somewhat arbitrary.  I wanted it below 1Gbps enough that I could be sure it was the router as the limit but yet fast enough that I would be able to see the peak transfer rates.
>
> All of the following might be old news to you, but please let me elaborate for others on this list (well, most folks here know way more about these things than I do).
>         I believe Gbit ethernet is trickier than one would guess, the 1 Gbit rate contains some overhead that one typically does not account for. Here is the equivalent on-the-wire size of a full MTU non-jumbo ethernet frame:
> Layer “1+": 1500 (payload pMTU) + 6 (dest MAC) + 6 (src MAC) + 2 (ethertype) + 4 (FCS) + 7 (preamble) + 1 (start of frame delimiter) + 12 (interframe gap)) = 1500+6+6+2+4+7+1+12 = 1538
> ”Equivalent” in that the interframe gap is not really used, but filled with silence but it has the duration one would need for 12 bytes.
>
> The kernel, if left to its own devices, will only account for 14 of 38 overhead bytes. But that means that each packet will carry an additional 24 bytes of unaccounted for size that still needs to be transferred:
>
> 975000 * (1538/1514) = 990455.746367 (which still is below the 1GBit Layer1+ ceiling that GbE has).  At 985000 * (1538/1514) = 1000614.26684 you would already have slowly caused the NIC’s buffer to fill ;)
> Luckily sqm-scripts will allow you to specify any additional per-packet overhead so just set this to 24 and things should just work out I believe.
>
>
> Best Regards
>         Sebastian
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] Linksys wrt1900acs rrul traces

[Richard Smith]

On 04/07/2016 12:16 PM, moeller0 wrote:
> Hi Richard,
>
>> For these tests I had the inbound and outbound limits set to 975000
>> kbps.  975000 was somewhat arbitrary.  I wanted it below 1Gbps
>> enough that I could be sure it was the router as the limit but yet
>> fast enough that I would be able to see the peak transfer rates.
>
> All of the following might be old news to you, but please let me
> elaborate for others on this list (well, most folks here know way
> more about these things than I do). I believe Gbit ethernet is
> trickier than one would guess, the 1 Gbit rate contains some overhead
> that one typically does not account for. Here is the equivalent
> on-the-wire size of a full MTU non-jumbo ethernet frame: Layer “1+":
> 1500 (payload pMTU) + 6 (dest MAC) + 6 (src MAC) + 2 (ethertype) + 4
> (FCS) + 7 (preamble) + 1 (start of frame delimiter) + 12 (interframe
> gap)) = 1500+6+6+2+4+7+1+12 = 1538 ”Equivalent” in that the
> interframe gap is not really used, but filled with silence but it has
> the duration one would need for 12 bytes.

I knew there was a bit of overhead but thanks for laying out the details 
and why it matters.

> 975000 * (1538/1514) = 990455.746367 (which still is below the 1GBit
> Layer1+ ceiling that GbE has).  At 985000 * (1538/1514) =
> 1000614.26684 you would already have slowly caused the NIC’s buffer
> to fill ;)

So by luck I just barely made it. :)

> Luckily sqm-scripts will allow you to specify any
> additional per-packet overhead so just set this to 24 and things
> should just work out I believe.

I knew that there might need to be some overhead accounting and I looked 
at Link Layer Adaption tab in the options when I set up the SQM but the 
info in that menu isn't quite descriptive enough for for my setup.

It has a box for "Ethernet with overhead, select for eg VSDL2".  I'm 
using Ethernet but I don't know about VSDL2.  If I select it then I get 
a 2nd box that asks for the per packet overhead.  Even if I had tried to 
fill that out then I would have gotten it wrong. :)

The other option is "ATM" which I know I'm not using.

Not knowing what I really should select I just left it at None.

Perhaps there are too many combinations to add items for everything but 
a few more options and more descriptions of scenarios might be helpful.

So now that I have that set at 24 is it worth re-running some of the tests?

-- 
Richard A. Smith

Re: [Cerowrt-devel] Linksys wrt1900acs rrul traces

[Sebastian Moeller]

Hi Richard, 


On April 8, 2016 1:51:11 PM GMT+02:00, Richard Smith <smithbone@gmail.com> wrote:
>On 04/07/2016 12:16 PM, moeller0 wrote:
>> Hi Richard,
>>
>>> For these tests I had the inbound and outbound limits set to 975000
>>> kbps.  975000 was somewhat arbitrary.  I wanted it below 1Gbps
>>> enough that I could be sure it was the router as the limit but yet
>>> fast enough that I would be able to see the peak transfer rates.
>>
>> All of the following might be old news to you, but please let me
>> elaborate for others on this list (well, most folks here know way
>> more about these things than I do). I believe Gbit ethernet is
>> trickier than one would guess, the 1 Gbit rate contains some overhead
>> that one typically does not account for. Here is the equivalent
>> on-the-wire size of a full MTU non-jumbo ethernet frame: Layer “1+":
>> 1500 (payload pMTU) + 6 (dest MAC) + 6 (src MAC) + 2 (ethertype) + 4
>> (FCS) + 7 (preamble) + 1 (start of frame delimiter) + 12 (interframe
>> gap)) = 1500+6+6+2+4+7+1+12 = 1538 ”Equivalent” in that the
>> interframe gap is not really used, but filled with silence but it has
>> the duration one would need for 12 bytes.
>
>I knew there was a bit of overhead but thanks for laying out the
>details 
>and why it matters.
>
>> 975000 * (1538/1514) = 990455.746367 (which still is below the 1GBit
>> Layer1+ ceiling that GbE has).  At 985000 * (1538/1514) =
>> 1000614.26684 you would already have slowly caused the NIC’s buffer
>> to fill ;)
>
>So by luck I just barely made it. :)the

        Yes, for full MTU-sized packets you made it, but try the same with say 150 Byte packets:
975000 * (183/164) = 1117682.9... That will nicely Show bloated Buffets in  the Ethernet driver, but should still work well for drivers using BQL. IMHO opinion proper shaping requires to take overhead into account.


>
>> Luckily sqm-scripts will allow you to specify any
>> additional per-packet overhead so just set this to 24 and things
>> should just work out I believe.
>
>I knew that there might need to be some overhead accounting and I
>looked 
>at Link Layer Adaption tab in the options when I set up the SQM but the
>
>info in that menu isn't quite descriptive enough for for my setup.
>
>It has a box for "Ethernet with overhead, select for eg VSDL2".  I'm 
>using Ethernet but I don't know about VSDL2.  If I select it then I get
>
>a 2nd box that asks for the per packet overhead.  Even if I had tried
>to 
>fill that out then I would have gotten it wrong. :)

Fair enough, we actually multiplex two related things with that drop down menu, whether to account for per packet overhead or not, and whether to also account for ATMs AAL5 cell wonkiness. In retrospect we should have separated these...

In your case select Ethernet with overhead, and manually put 24 into these packet overhead field, as the kernel already accounted for 14 of the total of 38.


>
>The other option is "ATM" which I know I'm not using.
>
>Not knowing what I really should select I just left it at None.
>
>Perhaps there are too many combinations to add items mol for everything but
>
>a few more options and more descriptions of scenarios might be helpful.
>
>So now that I have that set at 24 is it worth re-running some of the
>tests?


I assume in your case this will not change your results a lot, as you most likely used full MTU packets in the test flows, I believe your low latency under load increases show that you have not much buffer bloat left... I would still recommend to use the correct per packet overhead to be on the right side...

Best Regards
        Sebastian

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [Cerowrt-devel] Linksys wrt1900acs rrul traces

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 1016 bytes --]

Sebastian,

Recently you wrote:

In your case select Ethernet with overhead, and manually put 24 into these
> packet overhead field, as the kernel already accounted for 14 of the total
> of 38.
>

and further down:


> I assume in your case this will not change your results a lot, as you most
> likely used full MTU packets in the test flows, I believe your low latency
> under load increases show that you have not much buffer bloat left... I
> would still recommend to use the correct per packet overhead to be on the
> right side...
>

As a layman hoping to use some of this great technology on my forth coming
Omni Turris boxes how am I supposed to derive these numbers?  Is there a
simple series of questions and answers that could figure them out?  Could
those be turned into some kind of "wizard"?  Otherwise how do you expect
large numbers of users to get their systems properly configured?  Surely
you do not want to engage in an email thread with each user who even
attempts such configuration :-)

/john

[-- Attachment #2: Type: text/html, Size: 1466 bytes --]

Re: [Cerowrt-devel] Linksys wrt1900acs rrul traces ko

[Sebastian Moeller]

Hi John,

On April 8, 2016 3:55:21 PM GMT+02:00, John Yates <john@yates-sheets.org> wrote:
>Sebastian,
>
>Recently you wrote:
>
>In your case select Ethernet with overhead, and manually put 24 into
>these
>> packet overhead field, as the kernel already accounted for 14 of the
>total
>> of 38.
>>
>
>and further down: mi
>
>
>> I assume in your case this will not change your results a lot, as you
>most
>> likely used full MTU packets in the test flows, I believe your low
>latency
>> under load increases show that you have not much buffer bloat left...
>I
>> would still recommend to use the correct per packet overhead to be on
>the
>> right side...
>>
>
>As a layman hoping to use some of this great technology on my forth
>coming
>Omni Turris boxes how am I supposed to derive these num Mkbers?  Is there
>a
>simple series of questions and answers that could figure them out? 

        Well, yes and no. Yes there is an underlaying implicit decision tree that could be turned into a series of questions that allow to deduce the required per packet overhead. And no as far as I know there is no explicit version of this.

In the end it really is not as complicated as it may seem:
0) decide whether you really need an explicit traffic shaper at all.

If no you are done, else

1) research what kind of link you want to shape for, as different link technologies have different overheads. 
So typically you should ask yourself what or where the usual bottleneck link in your internet connection is located, shaping works best for links that are have fixed parameters. Good examples of technologies and links to shape for are e.g. ethernet for an active fiber link (FTTH) or ATM for old ADSL links, or PTM for vdsl2 ( FTTC):
A) Ethernet. This is the signed easiest, just figure out what is used on the wire. Typically that contains the the overhead described in my mail to Richard with the potential addition of up to two VLAN tags (4bytes each).
B) almost ethernet. Some technologies carry Ethernet frames so are similar to ethernet but don't deal in all components of Ethernet, e.g. vdsl2's PTM usesmuch of an Ethernet header but uses no preamble or ifg, but adds a few specific options.
C) ATM. This is both complicated and 'solved' at the same time, see https://github.com/moeller0/ATM_overhead_detector for references. Note this not only affects the per packet overhead but also the link rate to payload rate calculations due to quantized 48/53 encoding.
D) PTM. Similar to ethernet except it introduces a few potential overhead items that are not shared with Ethernet and it also affects payload rate calculations due to 64/65encoding.

So ideally at this point you know what true rate your bottleneck link allows and what overhead is added to each packet. Now the only step left is to figure out how much overhead the kernel already added to each packet's wire-size and reduce the configured per packet overhead by that amount.

The devil, as so often is in the details ;)
'

>Could
>those be turned into some kind of "wizard"?  

        I am pretty sure this could be turned into a series of question's or rather a sequential recipe but it will stay laymen unfriendly (with the potential exception of ATM) as it requires intermediate type technical knowledge about behaviour or links outside the user's direct control. Typically your ISP would know all this, but there does not seem to be a universal way to query an ISP to get the required information.

Otherwise how do you
>expect
>large numbers of users to get their systems properly configured? 

        Honestly, I believe the only real hope for the masses would be a universal adoptation of BQL (byte queue limit) in all CPE and CMTs/DSLAMs... That or doing a bit of research them selves.

>Surely
>you do not want to engage in an email thread with each user who even
>attempts such configuration :-)

        I agree, I believe the gist of a few of such discussions should be turned into an FAQ...

Best Regards
       Sebastian

>
>/john

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [Cerowrt-devel] Linksys wrt1900acs rrul traces

[Aaron Wood]

[-- Attachment #1.1: Type: text/plain, Size: 2687 bytes --]

These are really impressive results (flent graphs attached for the "simple"
operation).

Especially so, considering to what I see with sqm_scripts on the same
hardware with CCrc3:
http://burntchrome.blogspot.com/2015/06/sqmscripts-before-and-after-at-160mbps.html

Is this stable enough for full-time usage (as a home router)?

-Aaron

On Thu, Apr 7, 2016 at 6:58 AM, Richard Smith <smithbone@gmail.com> wrote:

> A while ago I mentioned that I purchased a Linksys wrt1900acs and I was
> going to do some comparisons of speed/rrul for stock firmware and stuff
> running sqm.
>
> It took a while to happen but I finally got it done.  I was stalled for a
> long time because for a while OpenWRT was not complete for this router.  It
> would boot but you could not (easily) return to the factory firmware as
> updates via the OpenWRT web interface did not work.
>
> Command line updates still worked though periodically I would build dd
> head and see if it got fixed.  Finally one weekend I decided to dig in and
> figure out what was going wrong and discovered that someone else beat me to
> it and it all was working now.
>
> The following flent rrul runs were run here on my local Gbit network
> (pretty quiet) using with OpenWrt Designated Driver r49051 / LuCI Master
> (git-16.081.38806-6b9a743) + Cake installed.
>
> Cake installed via the instructions here:
>
>
> http://www.bufferbloat.net/projects/codel/wiki/Cake#Installing-CAKE-out-of-tree-on-OpenWrt-rough-instructions
>
> For these tests I had the inbound and outbound limits set to 975000 kbps.
> 975000 was somewhat arbitrary.  I wanted it below 1Gbps enough that I could
> be sure it was the router as the limit but yet fast enough that I would be
> able to see the peak transfer rates.
>
> Test setup is:
>
> Laptop<-->wrt1900acs<--->Gbit Switch<--->Netperf server
>
> Prior to running the tests I did trial runs without the wrt1900acs in the
> chain to verify every thing worked at Gbit speeds.
>
> I ran the rrul test for each of the SQM settings at least 5 times.  For
> layer cake and piece of cake each had a run with results that were way off
> from the previous run.  I'm assuming this is some yet to be fixed issue and
> so I ran it 1 more time so that there were 5 runs with similar results.
>
> All the raw files are in this tarball:
>
>
> https://drive.google.com/file/d/0B-P0wCbNmKvAWmRUSzNMdXpUOFE/view?usp=sharing
>
> Hope this is helpful.  I'll be happy to run any additional tests if
> someone wants more.
>
> --
> Richard A. Smith
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>

[-- Attachment #1.2: Type: text/html, Size: 3785 bytes --]

[-- Attachment #2: dd_simple.all.png --]
[-- Type: image/png, Size: 299341 bytes --]

[-- Attachment #3: dd_simple.cdf.png --]
[-- Type: image/png, Size: 149660 bytes --]

Re: [Cerowrt-devel] Linksys wrt1900acs rrul traces

[Dave Taht]

It was not in my opinion, "ready" for use as a full time router, as of
a few weeks ago. In particular, the wifi was intolerably buggy and
needed a reboot after heavy use. The ethernet side has got to
"usable", and things like cake are working ok. I am not working very
hard on it as I'm trying to swallow the ath10k fq_codel patches and
overall ath9k drivers as I write.

If your family can put up with it, more knowledgable users such as
yourself will find more bugs and get them fixed faster.  ;)

I did pick up an edgerouter X the other day, it is kind of my hope to
get cake on that (somehow), and see how it does, and keep offloading
the wifi to another box entirely.

Dave Täht
Let's go make home routers and wifi faster! With better software!
https://www.gofundme.com/savewifi


On Thu, Apr 7, 2016 at 10:42 AM, Aaron Wood <woody77@gmail.com> wrote:
> These are really impressive results (flent graphs attached for the "simple"
> operation).
>
> Especially so, considering to what I see with sqm_scripts on the same
> hardware with CCrc3:
> http://burntchrome.blogspot.com/2015/06/sqmscripts-before-and-after-at-160mbps.html
>
> Is this stable enough for full-time usage (as a home router)?
>
> -Aaron
>
> On Thu, Apr 7, 2016 at 6:58 AM, Richard Smith <smithbone@gmail.com> wrote:
>>
>> A while ago I mentioned that I purchased a Linksys wrt1900acs and I was
>> going to do some comparisons of speed/rrul for stock firmware and stuff
>> running sqm.
>>
>> It took a while to happen but I finally got it done.  I was stalled for a
>> long time because for a while OpenWRT was not complete for this router.  It
>> would boot but you could not (easily) return to the factory firmware as
>> updates via the OpenWRT web interface did not work.
>>
>> Command line updates still worked though periodically I would build dd
>> head and see if it got fixed.  Finally one weekend I decided to dig in and
>> figure out what was going wrong and discovered that someone else beat me to
>> it and it all was working now.
>>
>> The following flent rrul runs were run here on my local Gbit network
>> (pretty quiet) using with OpenWrt Designated Driver r49051 / LuCI Master
>> (git-16.081.38806-6b9a743) + Cake installed.
>>
>> Cake installed via the instructions here:
>>
>>
>> http://www.bufferbloat.net/projects/codel/wiki/Cake#Installing-CAKE-out-of-tree-on-OpenWrt-rough-instructions
>>
>> For these tests I had the inbound and outbound limits set to 975000 kbps.
>> 975000 was somewhat arbitrary.  I wanted it below 1Gbps enough that I could
>> be sure it was the router as the limit but yet fast enough that I would be
>> able to see the peak transfer rates.
>>
>> Test setup is:
>>
>> Laptop<-->wrt1900acs<--->Gbit Switch<--->Netperf server
>>
>> Prior to running the tests I did trial runs without the wrt1900acs in the
>> chain to verify every thing worked at Gbit speeds.
>>
>> I ran the rrul test for each of the SQM settings at least 5 times.  For
>> layer cake and piece of cake each had a run with results that were way off
>> from the previous run.  I'm assuming this is some yet to be fixed issue and
>> so I ran it 1 more time so that there were 5 runs with similar results.
>>
>> All the raw files are in this tarball:
>>
>>
>> https://drive.google.com/file/d/0B-P0wCbNmKvAWmRUSzNMdXpUOFE/view?usp=sharing
>>
>> Hope this is helpful.  I'll be happy to run any additional tests if
>> someone wants more.
>>
>> --
>> Richard A. Smith
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>

Re: [Cerowrt-devel] Linksys wrt1900acs rrul traces

[moeller0]

[-- Attachment #1: Type: text/plain, Size: 3282 bytes --]

Hi Aaron,


> On Apr 7, 2016, at 19:42 , Aaron Wood <woody77@gmail.com> wrote:
> 
> These are really impressive results (flent graphs attached for the "simple" operation).
> 
> Especially so, considering to what I see with sqm_scripts on the same hardware with CCrc3:  http://burntchrome.blogspot.com/2015/06/sqmscripts-before-and-after-at-160mbps.html

	Richard has an wrt1900acs (https://wikidevi.com/wiki/Linksys_WRT1900ACS), while I believe you have an wrt1900AC (https://wikidevi.com/wiki/Linksys_WRT1900AC) so Richard’s seems a bit more recent and powerful. Or I am an idiot and can not read properly…

Best Regards
	Sebastian

> 
> Is this stable enough for full-time usage (as a home router)?
> 
> -Aaron
> 
> On Thu, Apr 7, 2016 at 6:58 AM, Richard Smith <smithbone@gmail.com> wrote:
> A while ago I mentioned that I purchased a Linksys wrt1900acs and I was going to do some comparisons of speed/rrul for stock firmware and stuff running sqm.
> 
> It took a while to happen but I finally got it done.  I was stalled for a long time because for a while OpenWRT was not complete for this router.  It would boot but you could not (easily) return to the factory firmware as updates via the OpenWRT web interface did not work.
> 
> Command line updates still worked though periodically I would build dd head and see if it got fixed.  Finally one weekend I decided to dig in and figure out what was going wrong and discovered that someone else beat me to it and it all was working now.
> 
> The following flent rrul runs were run here on my local Gbit network (pretty quiet) using with OpenWrt Designated Driver r49051 / LuCI Master (git-16.081.38806-6b9a743) + Cake installed.
> 
> Cake installed via the instructions here:
> 
> http://www.bufferbloat.net/projects/codel/wiki/Cake#Installing-CAKE-out-of-tree-on-OpenWrt-rough-instructions
> 
> For these tests I had the inbound and outbound limits set to 975000 kbps.  975000 was somewhat arbitrary.  I wanted it below 1Gbps enough that I could be sure it was the router as the limit but yet fast enough that I would be able to see the peak transfer rates.
> 
> Test setup is:
> 
> Laptop<-->wrt1900acs<--->Gbit Switch<--->Netperf server
> 
> Prior to running the tests I did trial runs without the wrt1900acs in the chain to verify every thing worked at Gbit speeds.
> 
> I ran the rrul test for each of the SQM settings at least 5 times.  For layer cake and piece of cake each had a run with results that were way off from the previous run.  I'm assuming this is some yet to be fixed issue and so I ran it 1 more time so that there were 5 runs with similar results.
> 
> All the raw files are in this tarball:
> 
> https://drive.google.com/file/d/0B-P0wCbNmKvAWmRUSzNMdXpUOFE/view?usp=sharing
> 
> Hope this is helpful.  I'll be happy to run any additional tests if someone wants more.
> 
> -- 
> Richard A. Smith
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel


[-- Attachment #2.1: Type: text/html, Size: 5140 bytes --]

[-- Attachment #2.2: dd_simple.all.png --]
[-- Type: image/png, Size: 105604 bytes --]

[-- Attachment #2.3: dd_simple.cdf.png --]
[-- Type: image/png, Size: 54360 bytes --]

Re: [Cerowrt-devel] Linksys wrt1900acs rrul traces

[Dave Taht]

[-- Attachment #1.1: Type: text/plain, Size: 3933 bytes --]

It does look like the hardware is going kaboom on a regular interval.
Shades of the old ip mis-alignment bugs we had on the 3800, where the
hardware would burp when it got a packet of a certain type.

Dave Täht
Let's go make home routers and wifi faster! With better software!
https://www.gofundme.com/savewifi

On Thu, Apr 7, 2016 at 10:50 AM, moeller0 <moeller0@gmx.de> wrote:

> Hi Aaron,
>
>
> On Apr 7, 2016, at 19:42 , Aaron Wood <woody77@gmail.com> wrote:
>
> These are really impressive results (flent graphs attached for the
> "simple" operation).
>
> Especially so, considering to what I see with sqm_scripts on the same
> hardware with CCrc3:
> http://burntchrome.blogspot.com/2015/06/sqmscripts-before-and-after-at-160mbps.html
>
>
> Richard has an wrt1900acs (https://wikidevi.com/wiki/Linksys_WRT1900ACS),
> while I believe you have an wrt1900AC (
> https://wikidevi.com/wiki/Linksys_WRT1900AC) so Richard’s seems a bit
> more recent and powerful. Or I am an idiot and can not read properly…
>
> Best Regards
> Sebastian
>
>
> Is this stable enough for full-time usage (as a home router)?
>
> -Aaron
>
> On Thu, Apr 7, 2016 at 6:58 AM, Richard Smith <smithbone@gmail.com> wrote:
> A while ago I mentioned that I purchased a Linksys wrt1900acs and I was
> going to do some comparisons of speed/rrul for stock firmware and stuff
> running sqm.
>
> It took a while to happen but I finally got it done.  I was stalled for a
> long time because for a while OpenWRT was not complete for this router.  It
> would boot but you could not (easily) return to the factory firmware as
> updates via the OpenWRT web interface did not work.
>
> Command line updates still worked though periodically I would build dd
> head and see if it got fixed.  Finally one weekend I decided to dig in and
> figure out what was going wrong and discovered that someone else beat me to
> it and it all was working now.
>
> The following flent rrul runs were run here on my local Gbit network
> (pretty quiet) using with OpenWrt Designated Driver r49051 / LuCI Master
> (git-16.081.38806-6b9a743) + Cake installed.
>
> Cake installed via the instructions here:
>
>
> http://www.bufferbloat.net/projects/codel/wiki/Cake#Installing-CAKE-out-of-tree-on-OpenWrt-rough-instructions
>
> For these tests I had the inbound and outbound limits set to 975000 kbps.
>  975000 was somewhat arbitrary.  I wanted it below 1Gbps enough that I
> could be sure it was the router as the limit but yet fast enough that I
> would be able to see the peak transfer rates.
>
> Test setup is:
>
> Laptop<-->wrt1900acs<--->Gbit Switch<--->Netperf server
>
> Prior to running the tests I did trial runs without the wrt1900acs in the
> chain to verify every thing worked at Gbit speeds.
>
> I ran the rrul test for each of the SQM settings at least 5 times.  For
> layer cake and piece of cake each had a run with results that were way off
> from the previous run.  I'm assuming this is some yet to be fixed issue and
> so I ran it 1 more time so that there were 5 runs with similar results.
>
> All the raw files are in this tarball:
>
>
> https://drive.google.com/file/d/0B-P0wCbNmKvAWmRUSzNMdXpUOFE/view?usp=sharing
>
> Hope this is helpful.  I'll be happy to run any additional tests if
> someone wants more.
>
> --
> Richard A. Smith
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>

[-- Attachment #1.2: Type: text/html, Size: 6043 bytes --]

[-- Attachment #2: dd_simple.cdf.png --]
[-- Type: image/png, Size: 54360 bytes --]

[-- Attachment #3: dd_simple.all.png --]
[-- Type: image/png, Size: 105604 bytes --]

Re: [Cerowrt-devel] Linksys wrt1900acs rrul traces

[Aaron Wood]

[-- Attachment #1.1: Type: text/plain, Size: 3525 bytes --]

Or I can't read properly... I have the first gen wrt1900ac. Dual-core
1.2GHz arm.

-Aaron

On Thu, Apr 7, 2016 at 10:50 moeller0 <moeller0@gmx.de> wrote:

> Hi Aaron,
>
>
> On Apr 7, 2016, at 19:42 , Aaron Wood <woody77@gmail.com> wrote:
>
> These are really impressive results (flent graphs attached for the
> "simple" operation).
>
> Especially so, considering to what I see with sqm_scripts on the same
> hardware with CCrc3:
> http://burntchrome.blogspot.com/2015/06/sqmscripts-before-and-after-at-160mbps.html
>
>
> Richard has an wrt1900acs (https://wikidevi.com/wiki/Linksys_WRT1900ACS),
> while I believe you have an wrt1900AC (
> https://wikidevi.com/wiki/Linksys_WRT1900AC) so Richard’s seems a bit
> more recent and powerful. Or I am an idiot and can not read properly…
>
> Best Regards
> Sebastian
>
>
> Is this stable enough for full-time usage (as a home router)?
>
> -Aaron
>
> On Thu, Apr 7, 2016 at 6:58 AM, Richard Smith <smithbone@gmail.com> wrote:
> A while ago I mentioned that I purchased a Linksys wrt1900acs and I was
> going to do some comparisons of speed/rrul for stock firmware and stuff
> running sqm.
>
> It took a while to happen but I finally got it done.  I was stalled for a
> long time because for a while OpenWRT was not complete for this router.  It
> would boot but you could not (easily) return to the factory firmware as
> updates via the OpenWRT web interface did not work.
>
> Command line updates still worked though periodically I would build dd
> head and see if it got fixed.  Finally one weekend I decided to dig in and
> figure out what was going wrong and discovered that someone else beat me to
> it and it all was working now.
>
> The following flent rrul runs were run here on my local Gbit network
> (pretty quiet) using with OpenWrt Designated Driver r49051 / LuCI Master
> (git-16.081.38806-6b9a743) + Cake installed.
>
> Cake installed via the instructions here:
>
>
> http://www.bufferbloat.net/projects/codel/wiki/Cake#Installing-CAKE-out-of-tree-on-OpenWrt-rough-instructions
>
> For these tests I had the inbound and outbound limits set to 975000 kbps.
>  975000 was somewhat arbitrary.  I wanted it below 1Gbps enough that I
> could be sure it was the router as the limit but yet fast enough that I
> would be able to see the peak transfer rates.
>
> Test setup is:
>
> Laptop<-->wrt1900acs<--->Gbit Switch<--->Netperf server
>
> Prior to running the tests I did trial runs without the wrt1900acs in the
> chain to verify every thing worked at Gbit speeds.
>
> I ran the rrul test for each of the SQM settings at least 5 times.  For
> layer cake and piece of cake each had a run with results that were way off
> from the previous run.  I'm assuming this is some yet to be fixed issue and
> so I ran it 1 more time so that there were 5 runs with similar results.
>
> All the raw files are in this tarball:
>
>
> https://drive.google.com/file/d/0B-P0wCbNmKvAWmRUSzNMdXpUOFE/view?usp=sharing
>
> Hope this is helpful.  I'll be happy to run any additional tests if
> someone wants more.
>
> --
> Richard A. Smith
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
> _______________________________________________
>
>
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>

[-- Attachment #1.2: Type: text/html, Size: 5603 bytes --]

[-- Attachment #2: dd_simple.all.png --]
[-- Type: image/png, Size: 105604 bytes --]

[-- Attachment #3: dd_simple.cdf.png --]
[-- Type: image/png, Size: 54360 bytes --]

Re: [Cerowrt-devel] Linksys wrt1900acs rrul traces

[Richard Smith]

On 04/07/2016 04:36 PM, Aaron Wood wrote:

> Or I can't read properly... I have the first gen wrt1900ac. Dual-core
> 1.2GHz arm.

The 1900acs is 1.6Ghz.  However remember my results are all local with 
no cable providers in the way and thus no buffer bloat so even if the 
hardware was the same. I'm not sure they compare apples-to-apples with 
your tests.

It does however suggest that this box can run the higher complexity 
algorithms and still maintain a lot of bandwidth.

 > Is this stable enough for full-time usage (as a home router)?

Dunno.  Currently on my network its a secondary device.  It's behind my 
primary NAT off on its own network.

It's stayed up 16d 11h so far with me running the wired flent tests and 
with a few phones and tablet devices connected to it but I would not 
call it very well used.  Not sure when I might try and swap it in over 
the WNDR3800.  Messing with the stable network makes the SO unhappy. :)

-- 
Richard A. Smith