[Cerowrt-devel] Ideas on how to simplify and popularize bufferbloat control for consideration.

Wed May 21 11:07:37 EDT 2014

On Wed, May 21, 2014 at 4:42 AM, Frits Riep <riep at riepnet.com> wrote:
> Thanks Dave for your responses.  Based on this, it is very good that qos-scripts is available now through openwrt, and as I experienced, it provides a huge advantage for most users.

I should point out that another issue with deploying fq_codel widely
is that it requires an accurate
measurement (currently) of the providers bandwidth.

My hope/expectation is that more ISPs that
provide CPE will ship something that is configured correctly by
default, following in free.fr's footsteps,
and trying to beat the cable industry to the punch, now that the core
code is debugged and documented, creating an out-of-box win.

> I would agree prioritizing ping is in and of itself not the key goal, but based on what I've read so far, fq-codel provides dramatically better responsiveness for any interactive application such as web-browsing, voip, or gaming, so it qos-scripts would be advantageous for users like your mom if she were in an environment where she had a slow and shared internet connection.  Is that a valid interpretation?

Sure. My mom has a fast, non-shared internet connection. Her biggest
problem is she hasn't
got off of windows despite my brother's decade of attempts to move her
to osx.... :)

Markets where this stuff seriously applies as a rate limiter + qos system
today are small to medium business, cybercafes, shared workspaces,
geek-zones, and so on. It also applies on ethernet and in cases where
you want to artificially have a rate limit like:

http://pieknywidok.blogspot.com/2014/05/10g-1g.html

We're ~5 years ahead of the curve here at cerowrt-central. Tools "just
work" for any sysadmin with chops. Commercial products are in the
pipeline.

While it takes time to build it into a product, I'd kind of expect
barracuda and ubnt to add fq_codel
to their products fairly soon, and for at least one switch vendor to follow.

It's in shorewall, ipfire, streamboost, everything downstream from openwrt,
linux mainline (and thus every linux distro) already. I know of a
couple cloud providers that are running
sch_fq and fq_codel already.

One thing I'm a little frustrated about, is that I'd expected sch_fq
to replace pfifo_fast by default
on more linux distros by now. It's a single sysctl...

> I am interested in further understanding the differences based on the brief differences you provide.  It is true that few devices provide DSCP marking, but if the latency is controlled for all traffic, latency sensitive traffic benefits tremendously even without prioritizing by l7 (layer 7 ?). Is this interpretation also valid?

Very, very true. Most of the need for prioritization goes away
entirely, due to the "sparse" vs "full" (or fast vs slow) queue
concept in fq_codel. In most circumstances things like voip just cut
through other traffic like butter. Videoconferencing is vastly
improved, also.

However, on very, very slow links (<3mbit), nothing helps enough. It's
not just the qos system that needs to be tuned, but that modern TCPs
and the web are optimized for much faster links and have features that
hurt at low speeds.  (what helps most is installing adblock plus!).
Torrent is something of a special case - I find it totally bearable at
20mbit/4mbit without classification - but unbearable at 8/1.

I'm pretty satisfied we have the core algorithms and theory in place,
now, to build edge devices that work much better at 3mbit to 200mbit,
at least, possibly 10gbit or higher.

> Yes, your mom wouldn't be a candidate for setting up ceroWRT herself, but if it were set up for her, or if it could be incorporated into a consumer router with automatically determining speed parameters,

That automatic speedtest thing turns out to be hard.

>she would benefit totally from the performance improvement.

Meh. She needs to get off of windows.

>So the technology ultimately needs to be taken mainstream, and yes that is a huge task.

Yep. If we hadn't given away everything perhaps there would be a
business model to fund that - streamboost is trying that route.

My hope was that the technology is merely so compelling that vendors
would be falling over themselves to answer the customer complaints.
But few have tied "bufferbloat" to the problems gamers and small
business are having with their internet uplinks as yet and more
education and demonstration seems necessary.

There is a huge backlog of potential demand for a better dslam, in
particular, as well as better firewalls and cablemodems. I don't have
a lot of hope for the two CMTS vendors to move to improve things
anytime soon.

> Frits
>
> -----Original Message-----
> From: Dave Taht [mailto:dave.taht at gmail.com]
> Sent: Tuesday, May 20, 2014 7:14 PM
> To: Frits Riep
> Cc: cerowrt-devel at lists.bufferbloat.net
> Subject: Re: [Cerowrt-devel] Ideas on how to simplify and popularize bufferbloat control for consideration.
>
> On Tue, May 20, 2014 at 3:11 PM, Frits Riep <riep at riepnet.com> wrote:
>> The concept of eliminating bufferbloat on many more routers is quite
>> appealing.  Reading some of the recent posts makes it clear there is a
>> desire to  get to a stable code, and also to find a new platform
>> beyond the current Netgear.  However, as good as some of the proposed
>> platforms maybe for developing and for doing all of the new
>> capabilities of CeroWRT, I also would like to propose that there also
>> be some focus on reaching a wider and less sophisticated audience to
>> help broaden the awareness and make control of bufferbloat more available and easier to attain for more users.
>
> I agree that reaching more users is important. I disagree we need to reach them with cerowrt. More below:
>
>>
>>
>> ·         It appears there is a desire to merge the code into an upcoming
>> OpenWRT barrier breaker release, which is excellent as it will make it
>> easier to fight buffer bloat on a wide range of platforms and provide
>> users with a much easier to install firmware release.  I’d like to be
>> able to download luci-qos-scripts and sqm-scripts and have basic
>> bufferbloat control on a much greater variety of devices and to many
>> more users.  From an awareness perspective this would be a huge win.
>> Is the above scenario what is being planned, is it likely to happen in the reasonable future?
>
> Yes, I'd submitted sqm for review upstream, got back a few comments. Intend to resubmit again when I get a chance.
>
>>
>> ·         From my perspective, it would be ideal to have this available to
>> many users in a more affordable platform, something like an 8mb flash
>> router like the TP-Link WDR-4300, which is otherwise a very capable
>> router with dual channels and good performance.
>>
>> ·         (I’ve managed to set up such a WDR-4300, with OpenWRT trunk,
>> figured how to telnet and install Luci, then luci-app-qos, and
>> qos-scripts and I thought the bufferbloat control was remarkable.)
>> How much better would it be if I were able to use luci-qos-scripts and sqm-scripts instead?
>
> You can easily add the .ipk files for sqm-scripts and luci-app-sqm to any release of openwrt. They are just scripts. They need some optional kernel modules and tools.
>
> I regard the qos-scripts as pretty good - the core differences from sqm are
>
> qos  vs sqm
> ---------------
> both use fq_codel. :)
> hfsc vs htb # A wash, hfsc mostly behaves like htb ping optimized vs de-optimized # optimizing for ping looks good in benchmarks but it's silly in the real world
> (l7) classification vs dscp # clear win to qos here, nearly nothing uses dscp no framing compensation vs comprehensive framing compensation # win here for sqm no alternate queue models vs many alternate queue models # with fq_codel the winner, who cares?
> fits in 4mb flash vs barely fits in 4mb flash
>
> The real killer problem for qos-scripts, for me, was that they didn't do ipv6. I'd like to see that fixed, at the very least.
>
>
>>
>> ·         For these target users, they need simplicity, good performance,
>> ease of setup and affordability.  They are not able to deal with the
>> routing between subnets on wireless, IPv6 setup, and any complexities
>> introduced by DNSSEC.  Marketing  the advantages of bufferbloat alone
>> requires lots of education and publicity (and as we have seen there
>> are many misleading posts by seemingly persuasive nay-sayers that it is all smoke and mirrors.
>
> Well, my intent is to make the successful bits of technology widely available.
> They are widely available. And being adopted everywhere. Win.
>
> As for the additional complexities, well, they will get less complex over time.
>
> In one respect, they are a stake in the ground. I have high hopes for the eventual success of hnetd and mdns proxy services, although they are alpha and nearly unusable right now, some are making substantial investments into them.
>
> In another the additional complexities of cero - like routing vs bridging - are there to further the research into fixing wifi technologies - which we haven't really even started yet. I'm increasingly convinced we need to do make-wifi-fast as a separate, focused project, building on a stable base.
>
>> ·         Would it be possible to have a simplified release of CeroWRT (in
>> terms of setup, and features), make It available for a reliable and
>> affordable platform, and publicize it and have it reach hopefully a
>> much wider audience?  This could potentially be through the OpenWRT channels.
>
> Possible yes. Affordable, no. Given that this has been a nearly full time project for me, for the last 38 months, with nearly zero revenue, I have no intent or interest in gaining anything other than knowledgable, clued users that want to advance the state of the art. My mom doesn't run cerowrt, nor do I want her to.
>
> If someone dropped ~1m/year on the project, that could change, but at present levels of funding I'd be better off working at mcdonalds. Even if funding appeared from the sky I'd rather spend it on R&D than GUI...
>
> Certainly IF there was some cost model that made sense, awesome! let's go for world domination!
>
> I continue to pursue the grant
> route, but the only thing that resonates even slightly with potential funders is not speed but security issues, which give me nightmares. Another model that works is actually making and selling a router, but that requires up front capital and entry into a very tight, profit-limited market.
>
> Biggest problem we have is supporting ONLY one router, even-semi-well, is a PITA.
>
> Adding a new one costs more. I'm now on my 4th day of trying to make the archer work. That's 6k of my life I'll never have back. And the ath10k in it sucks, and working to make that work well is not something I want to be doing due to the binary blob wifi firmware.
>
> I'm all in favor of handing off future cerowrt development to a nonprofit of interested users, and sitting back and focusing on fixing just the bits I care about, if anyone is interested in forming one...
>
>> ·         Part of the reason why Tomato had been so popular is that the
>> firmware upgrade,  install, configuration, and management was well
>> within the capabilities of the average weekend hacker, and there were
>> compelling features and reliability vs the factory firmwares at the time.
>
> Yep. dd-wrt is the same. And various downstream users like buffalo, meraki etc.
>
> I'm totally happy that they exist and have a working market.
>
>> ·         Even installing OpenWRT, especially Trunk, and finding,
>> downloading and enabling packages, while very powerful, and flexible,
>> is still quite complex to someone who does not spend a lot of time
>> reading wiki’s and release notes.
>
> Yes, CeroWrt is an improvement on OpenWrt in that regard. But it isn't enough. Doing serious UI improvements and simplification IS necessary, and that's not my bag. The EFF is making noises about doing something with the front end of openwrt and/or cero in the next year or so (see their owtech list for more details), that also goes after the security issue.
>
>> I’d be interested in feedback on these thoughts.
>
> There you go. I LOVE that we have a happy userbase, and love what we've accomplished, and have loved being here to help make it happen, and love that lots of people want to get it more out there to more people, it's gratifying as hell, and there are a lot of negatives too, like chasing bugs for months on end...
>
> ... but after we freeze, I need a vacation and to do something else for a while.
>
> I'm presently planning on spending the summer working on something that pays, and on improving ns3 with the GSOC, and testing a deployment of cerowrts on a modest scale, and working on a new/improved rate limiter integrated with fq_codel.  And only updating cero for CVEs or major new features.
>
> That's a full plate.
>
> If someone else wants to step up to maintain or continue to push cerowrt forward in some direction or another, I'm all for it.
>
> It's kind of my hope a clear winner on the chipset front will emerge and we can move to that, but even if that happens it will be months and months before it could be considered stable...
>
>
>
>>
>>
>> Frits Riep
>>
>>
>>
>>
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel at lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>
>
>
>
> --
> Dave Täht
>
> NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
>

-- 
Dave Täht

NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article