[Cerowrt-users] engaging developers and users [#314]

[Cerowrt-users] engaging developers and users [#314]

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 5942 bytes --]

our anti-phishing system kicked back on the numeric urls in this, fixed now.

The reason why cerowrt lives on the 172 dot 30 dot 42 dot X address is that
it had been my hope that others working on this project would plug *two*
routers into their home network - one for the day-to-day effort of keeping
their internet access up and running (on 192 dot 168 dot zero dot one), and
a cerowrt box for analyzing both routers behavior.

*I* don't run it as my day-to-day device at the moment. From where I sit,
it's a test tool - an increasingly good one - for coming up with solutions
to bufferbloat, and fixing the whole home router disaster with things like
ipv6, proxying, dns, etc... it has oprofile, and debugging tools by
default, etc, etc.

I had planned to get to where we had stable releases that could be used
day-to-day, but it's been a while since we had one, and I feel that we're
going to make some progress on the core bufferbloat problem next quarter,
and not have a stable release.

I'm GLAD to have users and testers - some generations of cerowrt are
running for people like jg, esr, & each, and have enormous stability and
uptimes - I don't know who else is running a generation of cerowrt
day-to-day frankly, there's been a lot of downloads - but there will always
be something broken in a smoketest or rc, that may not be able to be fixed
very quickly. Or something crazy we're doing - like routing vs bridging -
that exposes a problem that we needed to know about....

Recently, that happened with samba. And while I hope that's fixed now (in a
couple ways - wins appears to be working, and I also have a largely
untested samba 3.6.1 package, it needs to get tested at some point in next
year's development cycle)

http://www.bufferbloat.net/issues/314
http://www.bufferbloat.net/issues/303

I'd really like to use samba again personally, I used to use it a lot.
These days I tend to use sshfs, and that's zillions of times slower than
samba.

Having a user support community and people testing release candidates and
smoketests is very important to me, too! I LOVED finding out how to make
samba work right...

So, high on my list is coming up with a proper way of stressing what's on
the front page of the documentation, and setting (low!) expectations, and
keeping people engaged...

From: http://cero2.bufferbloat.net/cerowrt/

"CeroWrt <http://cero2.bufferbloat.net/cerowrt/about.html> is an
OpenWrt<http://www.openwrt.org/>router platform for use by
individuals, researchers, and students
interested in advancing the state of the art on the Internet. Specifically,
it is aimed at investigating the problems of latency under load,
bufferbloat, wireless-n<http://cero2.bufferbloat.net/cerowrt/tcp.html#wireless>,
QoS <http://cero2.bufferbloat.net/cerowrt/tcp.html#qos>, and the effects of
various TCP algorithms
<http://cero2.bufferbloat.net/cerowrt/tcp.html#tcp>on shared
networks."

If there is some place in the doc where we are not putting up large warning
signs - 'BUGS AHEAD. DANGEROUS CODE. DON'T EXPERIMENT WITH THIS ON WIVES OR
CHILDREN' - I'd to find it and fix it.

I'm perfectly happy with the hardware and core software itself at this
point. I wasn't, this time last year.

I'd like everybody in the open source and network research communities to
get TWO routers based on this chipset for christmas! Use one day to day,
running openwrt, and the other experimenting with a future outlined by the
ideas in cerowrt.

1) I'd like to come up with a good way for people to plug this in as a
'secondary' router.

Right now that requires turning off nat, and telling the upstream router to
give the cerowrt router a static ip and route to the 172 dot 30 dot 42 dot
0 slash 24 address. Perhaps we can take some screenshots of how to do that
on more common CPE?

Network renumbering involves running a couple line sed script.

http://www.bufferbloat.net/projects/cerowrt/wiki/Default_network_numbering

I hope to make renumbering a router easier with a gui, but you know, it's a
3 line sed script and a couple hundred lines of gui to write to make that
easier.
I'm also thinking of merely writing an RFC standardizing that 192 dot 168
dot zero dot 1 should be the number ALL routers come up on, and the number
all home networks should use. For april 1st.

Bridging is also possible... but not very.

2) Another thought is to do builds of the ceropackages repository for
straight openwrt, and point people at that for things like the bleeding
edge samba stuff.

I like ceropackages, it's a good way to spin up and debug a new package,
with a low barrier to entry for new people to openwrt - after which it has
always been my intent to push the stable stuff upstream. Multiple grad
students have used the ceropackages concept to get up to speed somewhat and
steve walker's been great about polishing those up. (and also submitting
packages of his own)

3) Is to more aggressively push up the stuff that works into std openwrt.
This is currently blocked by something stupid

http://www.bufferbloat.net/issues/319

or convince someone to push the stable stuff up to openwrt on a regular
basis.

5) Increase the number of people doing active development and able to fix
bugs and documentation.

Any other ideas as to accomplish these mutually incompatable goals - gain
developers, increase the userbase, gain testers,get good day to day and
long term resolve, solve bufferbloat, establish world peace, and be able to
do bleeding edge R&D... are welcomed.

I do not ever want to disappoint people with our efforts, and will work
diligently at fixing every problem exposed by the new stuff we're doing.
One of my first thoughts was pretty simple in this area though - try to do
less new stuff!

-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
FR Tel: 0638645374
http://www.bufferbloat.net

[-- Attachment #2: Type: text/html, Size: 7008 bytes --]

Re: [Cerowrt-users] engaging developers and users [#314]

[Maxim Kharlamov]

[-- Attachment #1: Type: text/plain, Size: 7475 bytes --]

Hello Dave,

I'm running cerowrt on day-to-day basis on my wndr3800 for quite a while
and it works like a charm. It is rc7-smoketest10 build. Managed to
configure everything I needed apart from dyndns client which is not a big
issue anyway.
So far wndr3800+cerowrt is the best performer out of many routers I tried.
I tested wndr3800 stock, linksys e3000 and e4200 with stock and tomatousb,
pfsense 2.0 appliance, cradlepoint mbr1400 and some others. Speedtest on
wndr3800+cerowrt is constantly at or almost at the top of the list and
during the speed test ping times are not deteriorating as badly as for the
other routers. Feature-wise cerowrt is also the best (for my particular
setup), so currently it is my main router at home even though initially I
bought it for playing and testing.

I've just received another wndr3800 and going to install openwrt on it to
compare them side-by-side and to have a playground for routers.

Overall, excellent job, Dave! I'm keeping my eye on cerowrt.


Thanks,
Max


On 15 December 2011 18:14, Dave Taht <dave.taht@gmail.com> wrote:

> our anti-phishing system kicked back on the numeric urls in this, fixed
> now.
>
> The reason why cerowrt lives on the 172 dot 30 dot 42 dot X address is
> that it had been my hope that others working on this project would plug
> *two* routers into their home network - one for the day-to-day effort of
> keeping their internet access up and running (on 192 dot 168 dot zero dot
> one), and a cerowrt box for analyzing both routers behavior.
>
> *I* don't run it as my day-to-day device at the moment. From where I sit,
> it's a test tool - an increasingly good one - for coming up with solutions
> to bufferbloat, and fixing the whole home router disaster with things like
> ipv6, proxying, dns, etc... it has oprofile, and debugging tools by
> default, etc, etc.
>
> I had planned to get to where we had stable releases that could be used
> day-to-day, but it's been a while since we had one, and I feel that we're
> going to make some progress on the core bufferbloat problem next quarter,
> and not have a stable release.
>
> I'm GLAD to have users and testers - some generations of cerowrt are
> running for people like jg, esr, & each, and have enormous stability and
> uptimes - I don't know who else is running a generation of cerowrt
> day-to-day frankly, there's been a lot of downloads - but there will always
> be something broken in a smoketest or rc, that may not be able to be fixed
> very quickly. Or something crazy we're doing - like routing vs bridging -
> that exposes a problem that we needed to know about....
>
> Recently, that happened with samba. And while I hope that's fixed now (in
> a couple ways - wins appears to be working, and I also have a largely
> untested samba 3.6.1 package, it needs to get tested at some point in next
> year's development cycle)
>
> http://www.bufferbloat.net/issues/314
> http://www.bufferbloat.net/issues/303
>
> I'd really like to use samba again personally, I used to use it a lot.
> These days I tend to use sshfs, and that's zillions of times slower than
> samba.
>
> Having a user support community and people testing release candidates and
> smoketests is very important to me, too! I LOVED finding out how to make
> samba work right...
>
> So, high on my list is coming up with a proper way of stressing what's on
> the front page of the documentation, and setting (low!) expectations, and
> keeping people engaged...
>
> From: http://cero2.bufferbloat.net/cerowrt/
>
> "CeroWrt <http://cero2.bufferbloat.net/cerowrt/about.html> is an OpenWrt<http://www.openwrt.org/>router platform for use by individuals, researchers, and students
> interested in advancing the state of the art on the Internet. Specifically,
> it is aimed at investigating the problems of latency under load,
> bufferbloat, wireless-n<http://cero2.bufferbloat.net/cerowrt/tcp.html#wireless>,
> QoS <http://cero2.bufferbloat.net/cerowrt/tcp.html#qos>, and the effects
> of various TCP algorithms<http://cero2.bufferbloat.net/cerowrt/tcp.html#tcp>on shared networks."
>
> If there is some place in the doc where we are not putting up large
> warning signs - 'BUGS AHEAD. DANGEROUS CODE. DON'T EXPERIMENT WITH THIS ON
> WIVES OR CHILDREN' - I'd to find it and fix it.
>
> I'm perfectly happy with the hardware and core software itself at this
> point. I wasn't, this time last year.
>
> I'd like everybody in the open source and network research communities to
> get TWO routers based on this chipset for christmas! Use one day to day,
> running openwrt, and the other experimenting with a future outlined by the
> ideas in cerowrt.
>
> 1) I'd like to come up with a good way for people to plug this in as a
> 'secondary' router.
>
> Right now that requires turning off nat, and telling the upstream router
> to give the cerowrt router a static ip and route to the 172 dot 30 dot 42
> dot 0 slash 24 address. Perhaps we can take some screenshots of how to do
> that on more common CPE?
>
> Network renumbering involves running a couple line sed script.
>
> http://www.bufferbloat.net/projects/cerowrt/wiki/Default_network_numbering
>
> I hope to make renumbering a router easier with a gui, but you know, it's
> a 3 line sed script and a couple hundred lines of gui to write to make that
> easier.
> I'm also thinking of merely writing an RFC standardizing that 192 dot 168
> dot zero dot 1 should be the number ALL routers come up on, and the number
> all home networks should use. For april 1st.
>
> Bridging is also possible... but not very.
>
> 2) Another thought is to do builds of the ceropackages repository for
> straight openwrt, and point people at that for things like the bleeding
> edge samba stuff.
>
> I like ceropackages, it's a good way to spin up and debug a new package,
> with a low barrier to entry for new people to openwrt - after which it has
> always been my intent to push the stable stuff upstream. Multiple grad
> students have used the ceropackages concept to get up to speed somewhat and
> steve walker's been great about polishing those up. (and also submitting
> packages of his own)
>
> 3) Is to more aggressively push up the stuff that works into std openwrt.
> This is currently blocked by something stupid
>
> http://www.bufferbloat.net/issues/319
>
> or convince someone to push the stable stuff up to openwrt on a regular
> basis.
>
> 5) Increase the number of people doing active development and able to fix
> bugs and documentation.
>
> Any other ideas as to accomplish these mutually incompatable goals - gain
> developers, increase the userbase, gain testers,get good day to day and
> long term resolve, solve bufferbloat, establish world peace, and be able to
> do bleeding edge R&D... are welcomed.
>
> I do not ever want to disappoint people with our efforts, and will work
> diligently at fixing every problem exposed by the new stuff we're doing.
> One of my first thoughts was pretty simple in this area though - try to do
> less new stuff!
>
> --
> Dave Täht
> SKYPE: davetaht
> US Tel: 1-239-829-5608
> FR Tel: 0638645374
> http://www.bufferbloat.net
>
> _______________________________________________
> Cerowrt-users mailing list
> Cerowrt-users@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-users
>
>


-- 
Thanks,
Max

[-- Attachment #2: Type: text/html, Size: 8830 bytes --]

Re: [Cerowrt-users] engaging developers and users [#314]

[Dave Taht]

On Thu, Dec 15, 2011 at 10:07 AM, Maxim Kharlamov <mcs@podsolnuh.biz> wrote:
> Hello Dave,
>
> I'm running cerowrt on day-to-day basis on my wndr3800 for quite a while and
> it works like a charm. It is rc7-smoketest10 build. Managed to configure
> everything I needed apart from dyndns client which is not a big issue
> anyway.

I should probably point people at that build as a 'stable' release, with two
known bugs (notably having to manually create the radios). We were
very close with that smoketest!

(but shortly afterwards the build, the kernel, the driver, bind, ntp, and ahcp
 all broke simultaneously, as did I. There was a ton of churn upstream,
 that is only now settling down)

So in the future, I need to come up with a good point to do a code freeze
in the release cycle. This is of course, in conflict with the idea of closely
tracking the kernel and openwrt development process,

> So far wndr3800+cerowrt is the best performer out of many routers I tried. I
> tested wndr3800 stock, linksys e3000 and e4200 with stock and tomatousb,
> pfsense 2.0 appliance, cradlepoint mbr1400 and some others. Speedtest on
> wndr3800+cerowrt is constantly at or almost at the top of the list and
> during the speed test ping times are not deteriorating as badly as for the
> other routers. Feature-wise cerowrt is also the best (for my particular
> setup), so currently it is my main router at home even though initially I
> bought it for playing and testing.

Excellent! Thank you.

Preliminary testing indicated that andrews/felix's fixes for bugs 216 and 195
got us to where openwrt was outperforming the factory firmware in most
respects...

... but I immediately turned around and started sacrificing that raw transfer
performance for lower latency. (reduced tx rings, txqueues)

I'd like to think that that, also, improved performance by a few metrics, but
I don't plan on resuming serious testing on various benchmarks until
after this coming development cycle is well underway.

In that effort thus far I've enabled a ton of debugging code to be able
to look more closely at the effects on cpu of various AQM technologies,
and what I have working at the moment just barely boots.

> I've just received another wndr3800 and going to install openwrt on it to
> compare them side-by-side and to have a playground for routers.

The core difference here at this point between cerowrt and openwrt is the vastly
reduced tx rings and txqueues - you can add these into openwrt easily by editing
/etc/init.d/boot to run ethtool at the right time,  and adding
/etc/hotplug.d/iface/00-debloat to openwrt.

I note I plan serious improvements to the basic debloat script there
adding some intelligence to it - in the upcoming development cycle.

>
> Overall, excellent job, Dave! I'm keeping my eye on cerowrt.

Thx again. Sometimes all I can see are the outstanding problems,
rather than the good stuff.


>
>
> Thanks,
> Max
>
>
> On 15 December 2011 18:14, Dave Taht <dave.taht@gmail.com> wrote:
>>
>> our anti-phishing system kicked back on the numeric urls in this, fixed
>> now.
>>
>> The reason why cerowrt lives on the 172 dot 30 dot 42 dot X address is
>> that it had been my hope that others working on this project would plug
>> *two* routers into their home network - one for the day-to-day effort of
>> keeping their internet access up and running (on 192 dot 168 dot zero dot
>> one), and a cerowrt box for analyzing both routers behavior.
>>
>> *I* don't run it as my day-to-day device at the moment. From where I sit,
>> it's a test tool - an increasingly good one - for coming up with solutions
>> to bufferbloat, and fixing the whole home router disaster with things like
>> ipv6, proxying, dns, etc... it has oprofile, and debugging tools by default,
>> etc, etc.
>>
>> I had planned to get to where we had stable releases that could be used
>> day-to-day, but it's been a while since we had one, and I feel that we're
>> going to make some progress on the core bufferbloat problem next quarter,
>> and not have a stable release.
>>
>> I'm GLAD to have users and testers - some generations of cerowrt are
>> running for people like jg, esr, & each, and have enormous stability and
>> uptimes - I don't know who else is running a generation of cerowrt
>> day-to-day frankly, there's been a lot of downloads - but there will always
>> be something broken in a smoketest or rc, that may not be able to be fixed
>> very quickly. Or something crazy we're doing - like routing vs bridging -
>> that exposes a problem that we needed to know about....
>>
>> Recently, that happened with samba. And while I hope that's fixed now (in
>> a couple ways - wins appears to be working, and I also have a largely
>> untested samba 3.6.1 package, it needs to get tested at some point in next
>> year's development cycle)
>>
>> http://www.bufferbloat.net/issues/314
>> http://www.bufferbloat.net/issues/303
>>
>> I'd really like to use samba again personally, I used to use it a lot.
>> These days I tend to use sshfs, and that's zillions of times slower than
>> samba.
>>
>> Having a user support community and people testing release candidates and
>> smoketests is very important to me, too! I LOVED finding out how to make
>> samba work right...
>>
>> So, high on my list is coming up with a proper way of stressing what's on
>> the front page of the documentation, and setting (low!) expectations, and
>> keeping people engaged...
>>
>> From: http://cero2.bufferbloat.net/cerowrt/
>>
>> "CeroWrt is an OpenWrt router platform for use by individuals,
>> researchers, and students interested in advancing the state of the art on
>> the Internet. Specifically, it is aimed at investigating the problems of
>> latency under load, bufferbloat, wireless-n, QoS, and the effects of various
>> TCP algorithms on shared networks."
>>
>> If there is some place in the doc where we are not putting up large
>> warning signs - 'BUGS AHEAD. DANGEROUS CODE. DON'T EXPERIMENT WITH THIS ON
>> WIVES OR CHILDREN' - I'd to find it and fix it.
>>
>> I'm perfectly happy with the hardware and core software itself at this
>> point. I wasn't, this time last year.
>>
>> I'd like everybody in the open source and network research communities to
>> get TWO routers based on this chipset for christmas! Use one day to day,
>> running openwrt, and the other experimenting with a future outlined by the
>> ideas in cerowrt.
>>
>> 1) I'd like to come up with a good way for people to plug this in as a
>> 'secondary' router.
>>
>> Right now that requires turning off nat, and telling the upstream router
>> to give the cerowrt router a static ip and route to the 172 dot 30 dot 42
>> dot 0 slash 24 address. Perhaps we can take some screenshots of how to do
>> that on more common CPE?
>>
>> Network renumbering involves running a couple line sed script.
>>
>> http://www.bufferbloat.net/projects/cerowrt/wiki/Default_network_numbering
>>
>> I hope to make renumbering a router easier with a gui, but you know, it's
>> a 3 line sed script and a couple hundred lines of gui to write to make that
>> easier.
>> I'm also thinking of merely writing an RFC standardizing that 192 dot 168
>> dot zero dot 1 should be the number ALL routers come up on, and the number
>> all home networks should use. For april 1st.
>>
>> Bridging is also possible... but not very.
>>
>> 2) Another thought is to do builds of the ceropackages repository for
>> straight openwrt, and point people at that for things like the bleeding edge
>> samba stuff.
>>
>> I like ceropackages, it's a good way to spin up and debug a new package,
>> with a low barrier to entry for new people to openwrt - after which it has
>> always been my intent to push the stable stuff upstream. Multiple grad
>> students have used the ceropackages concept to get up to speed somewhat and
>> steve walker's been great about polishing those up. (and also submitting
>> packages of his own)
>>
>> 3) Is to more aggressively push up the stuff that works into std openwrt.
>> This is currently blocked by something stupid
>>
>> http://www.bufferbloat.net/issues/319
>>
>> or convince someone to push the stable stuff up to openwrt on a regular
>> basis.
>>
>> 5) Increase the number of people doing active development and able to fix
>> bugs and documentation.
>>
>> Any other ideas as to accomplish these mutually incompatable goals - gain
>> developers, increase the userbase, gain testers,get good day to day and long
>> term resolve, solve bufferbloat, establish world peace, and be able to do
>> bleeding edge R&D... are welcomed.
>>
>> I do not ever want to disappoint people with our efforts, and will work
>> diligently at fixing every problem exposed by the new stuff we're doing. One
>> of my first thoughts was pretty simple in this area though - try to do less
>> new stuff!
>>
>> --
>> Dave Täht
>> SKYPE: davetaht
>> US Tel: 1-239-829-5608
>> FR Tel: 0638645374
>> http://www.bufferbloat.net
>>
>> _______________________________________________
>> Cerowrt-users mailing list
>> Cerowrt-users@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-users
>>
>
>
>
> --
> Thanks,
> Max
>



-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
FR Tel: 0638645374
http://www.bufferbloat.net

Re: [Cerowrt-users] engaging developers and users [#314]

[Dave Taht]

On Sat, Dec 17, 2011 at 9:30 AM, Dave Taht <dave.taht@gmail.com> wrote:
> On Thu, Dec 15, 2011 at 10:07 AM, Maxim Kharlamov <mcs@podsolnuh.biz> wrote:

> The core difference here at this point between cerowrt and openwrt is the vastly
> reduced tx rings and txqueues - you can add these into openwrt easily by editing
> /etc/init.d/boot to run ethtool at the right time,  and adding
> /etc/hotplug.d/iface/00-debloat to openwrt.

Another core difference is that on cerowrt's wireless, I did much more
extensive
diffserv classification than what's currently in the kernel mainline
and openwrt.

In particular, one thing you can really 'feel' is that stuff with the
IMM bit set
(notably ssh), gets dumped into the VI queue, and that helps ssh a LOT.

Unfortunately that series of patches was terribly hacky and I still haven't
found a good way to rework them for submittal upstream.

You easily get the same behavior for things like ssh via iptables.