[Cerowrt-devel] Dropping features from cerowrt

Sat Dec 24 11:35:34 PST 2011

On Sat, Dec 24, 2011 at 6:05 PM, Richard Brown
<richard.e.brown at dartware.com> wrote:
> Dave,
>
> I've been following the Cerowrt project for a few weeks because of its potential for a good replacement for my trusty, but slow, WRT54GL and DDWRT.
>
> I applaud your re-thinking the project, and the fact that you're making executive decisions about what to focus on.
> It's exactly the right thing to do when your task list stretches out so far. A few thoughts:

It was more like an executive decision... to start making executive decisions!

> - You should put a link to your "Rebasing, rethinking..." note (https://lists.bufferbloat.net/pipermail/cerowrt-devel/2011-December/000001.html) on the Cerowrt overview page (http://www.bufferbloat.net/projects/cerowrt) so that people who come to the site have a clear idea of its state.

Good Idea. Thanks. Done.

> - This will also bring more people to the cerowrt-devel list, which is a good "information radiator" about the state of the project.

Too much work was taking place on irc, and not being documented. While
irc is a good place for MANY things, I felt that too much stuff was in
my or jim's head and not out where others could see it. Not only that,
merely trying to reduce the backlog of issues to a string of smaller
emails was very head clearing for me, and I deeply desire feedback
such as yours!

> - Similarly, you could put a link on the News page, above the "Slipping dates for rc7" note.

I'll try to pull together a summary of where things are and where they
will go after the holiday.

>
> - If I made one request of cerowrt, it would be to add the fprobe package.

Adding an *optional* package - particularly a useful looking one such
as fprobe, is not a problem. In fact, I just built it as part of the
bql related smoketests I've been doing...

http://huchra.bufferbloat.net/~cero1/bql-smoketests/ -

>I like to have netflow data available (my company makes the InterMapper network monitoring software, http://intermapper.com) which has a flow analyzer built in.

Your software looks very cool and useful! And perhaps you have an
architecture in place that could do something I've been trying to do
for a while.

I'd like a tool that can analyze the burstyness of flows. We have
(with things like TSO/GSO, no AQM, wireless aggregation, and the
architecture of many TCP stacks) introduced, effectively,

So what I'm slowly prototyping is something that will let us look at
how well a set of streams are competing - showing graphically the
level of fair queueing at a really fine level, down to some graph that
is easily understood. Things like Jain's fairness index don't do this,
or rather, if you shipped two streams in two giant identical bulk
shipments, it would show as 'fair'.

I've noted my horror elsewhere at seeing the level of 'bulk shipments'
and superpacket streams we now see across the internet. We've
forgotten that packets are supposed to be small units of information.
And having a tool that demonstrates that (either standalone or in
wireshark) will help educate - and help evaluate - newer AQM
solutions.

It's still kind of hard for me to describe, but basically I'd like to
take a packet capture, graph it on a 64k vertical window and axis for
sent packets, and 64k for received below it, use, say, 10 different
colors for the top 10 streams on that capture, and then show each
packet as part of the graph, over the y axis of time.

It's hard to do - basically as TSO can send up to 64k at a time, and
the smallest packet is 64 bytes, which is a 1000x1 ratio. so, I can
get matters down to scale, using 1 pixel per 75 bytes,

So we end up with a 2000 pixel high graph that should show the level
of 'interspersedness' of a given set of streams over a time window.

And I'm really sure that for many packet captures it will look NOTHING
like a poisson distribution, or white noise. It should wake some
people up as to what we've done to packet theory in general with the
offloads, TCP stacks, and lack of AQMs.

> - I realize that adding new packages to support is in direct contravention to your stated desire to cut things back, so I would understand and honor your decision in this regard.

Oh, no, I love the idea of fprobe as an *optional* package, and I'd
like to take a look at your stuff too.

Which is better: fprobe or fprobe-ulog? What would be a good way to use it.

Most of my own concerns with the feature set boiled down to what was
needed in the 'core' - in order for cerowrt to actually boot and work
- as well as the large amounts of testing required to deal with ipv6,
mesh networking, etc -
as well as features that needed to be supported by the gui - doing all
of that is going to require whole bodies, at this point.

The main concern was that we'd wanted cerowrt to be complementary to
our efforts on the x86 front. We needed to have TWO well understood
and correctly functioning devices, an AP, and a PC, in order to get
anywhere. All year, it took, to get to where an AQM actually did
anything useful on an x86 box, as the intrinsic device driver
buffering problems got in the way, and Tianji li's algorithm (or the
implementation) didn't work out.

Several breakthroughs have been made in the kernel mainline (BQL, TiQ)
re bufferbloat that I'd like to track closely and help out on next
quarter.

So I'm still evaluating what will end up in the core of cerowrt AND
trying to allocate enough time to push along the kernel mainline in
the right direction so that it ends up in embedded (not just cerowrt!)
quickly.

I'm also hoping we can recruit more developers with more dedicated
time and/or maybe attract a bit of funding to help us get over some
humps (notably the testing is truly painful to do well).

We've got donated facilities from isc, and I just finished up a stint
at the lincs.fr lab in France. As an all-volunteer project we've done
pretty well, and I don't want to screw that up, either.  I've thought
that perhaps we should try to create a board, and governance, and
stuff like that, in the coming year, because it's going to take years
to lick bufferbloat (and I wouldn't mind trying to tackle ipv6 and
mesh networking, too)

>
> Best regards from sunny and cold Hanover, NH USA.

Merry Christmas from cold and rainy Paris, France.

> Rich Brown
>
> PS I was able to install rc6 trivially, and got it to work nicely on a WNCR3700v2. Your install instructions for OSX were impeccable. I now need to decide whether to heed your advice (not to inflict these builds on spouses and small children) or switch over to the stock Openwrt build.

Heh. I could have said this better. (At the time I was not a happy
guy). My hope would be for those interested in this project to
evaluate a stable cerowrt release for day-to-day use, make sure it's
fully compatible with their wives and small children FIRST, then
deploy for day to day. :)

Then get a second one for dad to play with! I have high hopes we'll
actually make a real dent in bufferbloat in the next two quarters, but
I can guarantee that plenty of stuff is going to break along the way.

Right now, for example, the bql-9 smoketest mentioned above works
pretty good (it's missing some features like bind and ahcp at the
moment) - but my attempts at adding several new AQMs appear to be
triggering kernel bugs. RED was proven broken for three years of the
kernel prior to 3.1.5, btw...

I'm really, really, really happy with the ar71xx based hardware at
this point, running openwrt or cerowrt. It's the best thing going in
embedded routers IMHO - and I love that there are multiple
manufacturers using this chipset so at some point or another I'll
broaden the base from the netgear 3700v2 and 3800 to buffalo, etc. I
figure it has a 2 year manufacturing run left to it, so we can focus
on higher levels of the stack for a while.

Multiple people run a final rc of cerowrt day to day with outrageous
uptimes. I've yet to get a report from anyone that their box crashed,
ever.

And that said, openwrt is mildly less complex, supports more packages
out of the box, and uses bridging, which involves less surprises
regarding some apps (like samba).

We HAVE to route in cerowrt, it's the only good way to be able to
analyze the behavior of the ethernet and two wireless radios - and
multicast works a lot better.

So, your call, but I'd like it if 'dad's router' ran cero.
>
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
FR Tel: 0638645374
http://www.bufferbloat.net