From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id B65FD21F113 for ; Mon, 27 May 2013 16:41:57 -0700 (PDT) Received: by mail-ie0-f180.google.com with SMTP id b11so5258177iee.25 for ; Mon, 27 May 2013 16:41:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=IKM2Cds21HfeC8EhP7MYreL50OqbS3lmm0p4IsmCyIY=; b=XFdpIO0zSAdQ051R7VTRiYba9E9/Y/+Ps1/hWkOAK82BFetnx4AQqXUIh0isDUleYY FxhfC2TijpS3ORREQnA8ncxFmX8ugfGC/rIY40+3vssATvWbzydSnAfawGPQeWyutJlI 3oRBta5ej/RE4KbHqh27mpuvonh6I+KwM/GXe2Vgxc5k53wjvF7j34ik7N0sNNDaccUM yDq0yXRCBurTeRajimwg4UxYCv31AMujXG/re7sZsnUTkKYa9gyI9ZbiCmBaEuEmAG37 c1DjfejPswXDRKqvvO596to+ktlsJPCg1XGRc7OD1qSLaZ7GHqgRKoC9/A8IUVruJC4g 1weQ== MIME-Version: 1.0 X-Received: by 10.50.141.161 with SMTP id rp1mr5410143igb.11.1369698116646; Mon, 27 May 2013 16:41:56 -0700 (PDT) Received: by 10.64.35.44 with HTTP; Mon, 27 May 2013 16:41:56 -0700 (PDT) In-Reply-To: References: Date: Mon, 27 May 2013 16:41:56 -0700 Message-ID: From: Dave Taht To: Lance Hepler Content-Type: multipart/alternative; boundary=089e013cbdcab1413004ddbbb137 Cc: cerowrt-devel@lists.bufferbloat.net Subject: Re: [Cerowrt-devel] tp-link 4300 evaluation X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 May 2013 23:41:58 -0000 --089e013cbdcab1413004ddbbb137 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Mon, May 27, 2013 at 2:08 PM, Lance Hepler wrote: > On Mon, May 27, 2013 at 6:13 AM, Dave Taht wrote: > >> On Sun, May 26, 2013 at 6:23 PM, Lance Hepler wrote= : >> >>> I'd be interested in your netperf testing setup. >>> >> >> The test we've been developing is called the rrul test. The prototype >> (which is working quite well is at: >> >> https://github.com/tohojo/netperf-wrapper >> >> I use the results in my talks a lot. >> > > Ahh! Neato! I'll have to play with this! > Prior to that, everybody was testing up, down, and ping separately, and not testing TCP. I'm hoping somebody like speedtest gets around to testing all that at the same time one day soon. In the meantime it's darn useful in a wide variety of scenarios. I add the "best of" tests to: http://www.bufferbloat.net/projects/codel/wiki/RRUL_Rogues_Gallery I'm a little behind, I have some good ones on wifi that I've used in various presos, as well as of rtt fairness and of ipv6 vs ipv4. The tests continue to evolve, more recently adding equivalents to the cisco pie tests, etc. Still needed are a good VOIP and a good DASH test, and I'd like very much to start collecting statistics on how much classification is mangled on the wild and wooly internet. Toke is buried with his thesis... We have several public netperf servers available (hope to add more), but it's easy to just put up one of your own somewhere. Just compile netperf from svn with the --enable-demo option on your clients and servers. cero comes with netserver enabled by default on your internal interfaces. It does NOT scale to the highest bandwidths available but is useful for testing low rate shapers. > > With the AR71xx chips going out of style, the AR934x series is probably >>> our best bet for readily consumer-available hardware with open-source >>> friendly SoCs. >>> >> >> It seems like it. But with MIPS Technologies effectively defunct the onl= y >> real hope I have for architectural progress on that architecture would c= ome >> from a licensee with chops. >> > > You raise a good point. > Sigh. I liked MIPS. But arm has leapfrogged 3 generations to MIPS's one... Arm chips almost universally have their wifi interfaces hooked up via a very slow bus (usb or spi), and are rife with binary blobs in most of their incarnations (android, marvel, are big offenders) Few have a decent ethernet chip, or more than 1. PPC might re-enter this market. So you could look at all this as a problem or an opportunity to arduinize the edge of the net. At the moment it seems best long term to leverage all the activity on arm, particularly on the server attempts and the FPGA combos to make progress forward. Given that the first generation 802.11ac devices don't support multi-user MIMO apparently, there is room to innovate here in the next, next generation of edge hardware vs the established majors. > >> So there is certainly room to drop most of the extra stuff in the >> zedboard, design a new board around the zynq for a new-age cero router >> around it (8 ethernet phys, 2-3 of which being SFP+ and gpon capable), a= dd >> pcie for 2 radios, pins to do software radio as development continues, >> arduino headers, and dunno, battery support? >> > > Sounds like the beginning of a kickstarter project. =3D) > I was a big fan of the netfpga.org project but their current 10GigE direction is very expensive and not as useful as trying to do a standalone board < 500 dollars cost. Sadly I think that the kickstarter funding model requires a project considerably further along than "we need to design a cool board to fix the internet with the software we've already developed". Kickstarter is more of a "we are on our last gasp and we just need a little help to ship something useful" model. So angels or VCs or deep pockets would be required to get a zedboard.org-for-a-better-router project off the ground. And my verilog is very rusty. The zynq idea is a bit longer term than the immediate emergency of trying to find a 3800 replacement, and funding in general is rather scarce. Would have loved to have started the zynq project 6 months ago tho. We'd have something REALLY COOL now. > >> When I get back to california next week, I hope to find someone to talk >> to at Xilinx (suggestions, anyone?) about getting started on getting a >> board like that designed. >> > > I'm sure someone in UCSD's vast CS department does. I'll ask around. > That would be helpful. They are just down the street from me but their website makes it really hard to find someone to shoot the breeze with. > This is all pretty new stuff, perhaps some more performance can be gleane= d >>> by tuning the compiler optimizations (-march=3D74Kc?), >>> >> >> Usually compiler options are not worth all that much. I didn't much care >> for the deep pipeline in this design either... >> > > Yea, true. And the design may be deep, but it's dual-issue with > ooo-dispatch. It's definitely a lot beefier, though perhaps some basic > benchmarking (CoreMark?, Dhrystone?) might be in order to set reasonable > expectations for relative performance. > I don't believe in benchmarks like that for embedded. irq response time, dma are more important. I'll rip out the unaligned access hacks in the next build tho... those are usually hell on pipelined arches. > > We spent a lot of time oprofiling things in the early days of the ar7xx, = I >> will take a harder look as I get time this week. I'm not writing it off, >> just was very disappointed in what I got initially. The other deal kille= r >> for this platform is that 16MB of flash is a requirement for cerowrt's >> boatload of test tools. >> > > The thought of oprofiling these modules hurts. > Oh, no, it's fun. You are always surprised by the results, and sometimes easy fixes result in huge gains. It's a good way to stay in touch with the real reality instead of optimising for stuff that isn't important. > As for the flash, that's why I was looking at the NETGEAR models. Plenty > of storage if we can get past the NAND issues. > Looks like progress might be made this week! I'm rooting for everyone. Kind of deep into android at the moment myself... > > Helpfully, the WNDR4300 has 128MB of NAND flash, as does the WNDR3700v4. >>> So compiling a full CeroWRT distribution shouldn't be a problem. The fi= xeth >>> script will need to be changed, but not much else. >>> >> >> ah, you've looked deeply at the boot phase. Applause! Bringing up this >> board seems easy once the flash issues are resolved... but that requires= a >> level of time (with a scope probably) that I am personally unwilling to >> invest right now. I think there are people on the problem, tho... >> > > Yes I have, thank you! > I think you are the first person to ever have read /etc/init.d/boot. I note that I think that's the problem in getting the new procd daemon to work right.... I still remain unsure that the oddball device naming scheme in cero is a good idea. Certainly it makes better firewall rules possible... and fw3 supports the + syntax - but it's not something people have adopted to any extent, sooo... > I've dedicated this weekend to trying to grok how openwrt is architected, > how cerowrt differs, and where all its magic bits are hidden. Thankfully > everything is well-architected, so discovery has been relatively > straightforward. The hardest part was trying to figure out all those laye= rs > of nested and dynamic board configuration in target/linux/*/image/Makefil= e. > I note that cerofiles is a bit out of date. I don't think it's terribly out of date, I just haven't pushed it with the last couple builds being buggy.... > > Lance > --=20 Dave T=E4ht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html --089e013cbdcab1413004ddbbb137 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

On Mon, May 27, 2013 at 2:08 PM, Lance H= epler <nlhepler@gmail.com> wrote:
On Mon, May 27, 2013 at 6:13 AM, Dave Ta= ht <dave.taht@gmail.com> wrote:
On Sun, May 26, 2013 at 6:23 PM, Lance Hepler <nlhepler@gmail.com> wrote:
I'd be intere= sted in your netperf testing setup.
The test we've been developing is called the rrul test. The prototype (= which is working quite well is at:

https://github.com/tohojo/netperf-wrapper

I use the results in my talks a lot.
=

Ahh! Neato! I'll have to play with this!

Prior to that, everybody was testing up,= down, and ping separately, and not testing TCP. I'm hoping somebody li= ke speedtest gets around to testing all that at the same time one day soon.=

In the meantime it's darn useful in a wide variety of scenarios.
I add the "best of" tests to:

http://www.bufferbl= oat.net/projects/codel/wiki/RRUL_Rogues_Gallery

I'm a little behind, I have some good ones on wifi that I've us= ed in various presos, as well as of rtt fairness and of ipv6 vs ipv4. The t= ests continue to evolve, more recently adding equivalents to the cisco pie = tests, etc.

Still needed are a good VOIP and a good DASH test, and I'd like ver= y much to start collecting statistics on how much classification is mangled= on the wild and wooly internet. Toke is buried with his thesis...

We have several public netperf servers available (hope to add more), but it= 's easy to just put up one of your own somewhere. Just compile netperf = from svn with the --enable-demo option on your clients and servers.

cero comes with netserver enabled by default on your internal interface= s. It does NOT scale to the highest bandwidths available but is useful for = testing low rate shapers.

=A0

With the AR71xx chips going out of style, the AR934x series is pr= obably our best bet for readily consumer-available hardware with open-sourc= e friendly SoCs.

It seems like it. But with MIPS Technolog= ies effectively defunct the only real hope I have for architectural progres= s on that architecture would come from a licensee with chops.

You raise a good point.=A0

Sigh. I liked MIPS. But arm has l= eapfrogged 3 generations to MIPS's one...

Arm chips almost unive= rsally have their wifi interfaces hooked up via a very slow bus (usb or spi= ), and are rife with binary blobs in most of their incarnations (android,= =A0 marvel, are big offenders) Few have a decent ethernet chip, or more tha= n 1.

PPC might re-enter this market.

So you could look at all this as= a problem or an opportunity to arduinize the edge of the net.

At th= e moment it seems best long term to leverage all the activity on arm, parti= cularly on the server attempts and the FPGA combos to make progress forward= .

Given that the first generation 802.11ac devices don't support mult= i-user MIMO apparently, there is room to innovate here in the next, next ge= neration of edge hardware vs the established majors.

=A0
So there is certainly room to drop most of the extra stuff in the zedboard,= design a new board around the zynq for a new-age cero router around it (8 = ethernet phys, 2-3 of which being SFP+ and gpon capable), add pcie for 2 ra= dios, pins to do software radio as development continues, arduino headers, = and dunno, battery support?

Sounds like the beginnin= g of a kickstarter project. =3D)
<= br>I was a big fan of the netfpga.org pr= oject but their current 10GigE direction is very expensive and not as usefu= l as trying to do a standalone board < 500 dollars cost.

Sadly I think that the kickstarter funding model requires a project con= siderably further along than "we need to design a cool board to fix th= e internet with the software we've already developed". Kickstarter= is more of a "we are on our last gasp and we just need a little help = to ship something useful" model. So angels or VCs or deep pockets woul= d be required to get a zedboard.org-for-a-better-router project off the gro= und. And my verilog is very rusty.

The zynq idea is a bit longer term than the immediate emergency of tryi= ng to find a 3800 replacement, and funding in general is rather scarce. Wou= ld have loved to have started the zynq project 6 months ago tho. We'd h= ave something REALLY COOL now.

=A0=A0
When I get b= ack to california next week, I hope to find someone to talk to at Xilinx (s= uggestions, anyone?) about getting started on getting a board like that des= igned.

I'm sure someone in = UCSD's vast CS department does. I'll ask around.
<= /div>

That would be helpful. They are just down the st= reet from me but their website makes it really hard to find someone to shoo= t the breeze with.


This is all pretty new stuff, perhaps some more performance can b= e gleaned by tuning the compiler optimizations (-march=3D74Kc?),

Usually compiler options are not worth al= l that much. I didn't much care for the deep pipeline in this design ei= ther...

Yea, true. And th= e design may be deep, but it's dual-issue with ooo-dispatch. It's d= efinitely a lot beefier, though perhaps some basic benchmarking (CoreMark?,= Dhrystone?) might be in order to set reasonable expectations for relative = performance.

I don't believe in benchmarks l= ike that for embedded. irq response time, dma are more important. I'll = rip out the unaligned access hacks in the next build tho... those are usual= ly hell on pipelined arches.
=A0

We spent a lot of time oprofiling things in the early days of the ar7xx,= I will take a harder look as I get time this week. I'm not writing it = off, just was very disappointed in what I got initially. The other deal kil= ler for this platform is that 16MB of flash is a requirement for cerowrt= 9;s boatload of test tools.

The thought of oprofilin= g these modules hurts.

Oh, no,= it's fun. You are always surprised by the results, and sometimes easy = fixes result in huge gains. It's a good way to stay in touch with the r= eal reality instead of optimising for stuff that isn't important.
=A0
As for the flash, that's why = I was looking at the NETGEAR models. Plenty of storage if we can get past t= he NAND issues.

Looks like progress might be made t= his week! I'm rooting for everyone. Kind of deep into android at the mo= ment myself...

Helpfully, the WNDR4300 has 128MB of NAND flash, as d= oes the WNDR3700v4. So compiling a full CeroWRT distribution shouldn't = be a problem. The fixeth script will need to be changed, but not much else.=

ah, you've looked deeply at the boot = phase. Applause! Bringing up this board seems easy once the flash issues ar= e resolved... but that requires a level of time (with a scope probably) tha= t I am personally unwilling to invest right now. I think there are people o= n the problem, tho...

Yes I have, thank you! <= /div>

I think you are the first pers= on to ever have read /etc/init.d/boot. I note that I think that's the p= roblem in getting the new procd daemon to work right....

I still remain unsure that the oddball device naming scheme = in cero is a good idea. Certainly it makes better firewall rules possible..= . and fw3 supports the + syntax - but it's not something people have ad= opted to any extent, sooo...

=A0
I've dedicated this we= ekend to trying to grok how openwrt is architected, how cerowrt differs, an= d where all its magic bits are hidden. Thankfully everything is well-archit= ected, so discovery has been relatively straightforward. The hardest part w= as trying to figure out all those layers of nested and dynamic board config= uration in target/linux/*/image/Makefile.

I note that cerofiles is a bit out = of date. I don't think it's terribly out of date, I just haven'= t pushed it with the last couple builds being buggy....
=A0

Lance



--
Dave T=E4ht

Fixi= ng bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.ht= ml=20 --089e013cbdcab1413004ddbbb137--