[Cake] the Cake stalemate

[Cake] the Cake stalemate

[Pete Heist]

Whether or not it helps with the upcoming talk or future plans, I’m thinking about the most recent experience with trying to upstream Cake and how to break the stalemate.

Cake stretches (arguably breaks) what qdiscs were designed for, which, correct me if I’m wrong, is what caused a lot of the pushback. I’m thinking of NAT support, ingress mode, host isolation, diffserv modes, and ack filtering, at a minimum. Each of those things can be useful in one case or another both individually and together, as evidenced by a satisfied user base. But when upstreaming I think each got a response with some variation on “no swimming in my bathtub.”

It seems that some would rather see Cake broken into pieces, and yet Dave points out that what makes Cake useful is the ability to easily address some common real world scenarios from one place:

> On Jun 19, 2018, at 3:46 AM, Dave Taht <dave.taht@gmail.com> wrote:
> 
> One of cake's "minor" features is the *perfect* defeat of the htb
> based shaper in cable modems. If you know the set-rate on the modem,
> you just set it to the same thing and get vastly superior performance to
> docsis 3.1, pie, or the sqm-scripts.

Some of its functionality is painful to implement using separate qdiscs, iptables rules, misc kernel modules, etc. So Cake is something useful that can’t be delivered to the world because of something like an impedance mismatch. There’s something wrong here, right? I’m just asking the question, what is it:

1) Should we accept qdiscs and Linux networking as they are and work within their borders?

or

2) Are there problems with the qdisc architecture or improvements from our experience that we can suggest for Linux networking that would help?

Or some combination?

Thoughts on this:

- The need to use an IFB device to control the ingress queue seems cumbersome, and that one should be able to control both the egress and ingress queues with one qdisc and one device, since egress and ingress traffic can be related (say for half-duplex devices).

- And let’s say you wanted to try to control the queue of a half-duplex physical layer with varying, asymmetrical rates and airtime considerations like WiFi. That would be a real challenge within the qdisc architecture. Realistically, you’d have to write a driver for every device that exists, right? It seems that situation could be better.

- Is there a right way to implement host isolation with NAT support?

- Just as a brainstorm, if I understand it right, qdiscs and tc live outside of the netfilter framework and within iproute2. Wouldn’t it have more conceptual integrity if netfilter and iproute2 were integrated? I’m thinking of a modular architecture that regardless of whether you queue, route, filter, prioritize, etc, you work in the same space. Or if that sounds like mayhem, then at least I question, does queueing belong with routing?

- Related: instead of a qdisc, could Cake rather be written as a netfilter module or series of modules with a common configuration utility?

I’m writing a bit beyond my experience here and those who’ve written a qdisc could surely have better ideas on what the path forward is…

Re: [Cake] the Cake stalemate

[Toke Høiland-Jørgensen]

Pete Heist <pete@heistp.net> writes:

> Whether or not it helps with the upcoming talk or future plans, I’m
> thinking about the most recent experience with trying to upstream Cake
> and how to break the stalemate.

Not sure there is a stalemate, actually. I just ran out of time to do
further revisions; planning to take that up again. Don't see any reason
why we shouldn't be able to succeed in getting Cake upstream in its
current form(ish) in just a few more revisions (or, who knows, maybe a
dozen more; better not jinx it ;)).

You make a few good points otherwise, that I don't have time to reply to
right now :P

-Toke

Re: [Cake] the Cake stalemate

[Pete Heist]

> On Jun 19, 2018, at 11:55 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Pete Heist <pete@heistp.net> writes:
> 
>> Whether or not it helps with the upcoming talk or future plans, I’m
>> thinking about the most recent experience with trying to upstream Cake
>> and how to break the stalemate.
> 
> Not sure there is a stalemate, actually. I just ran out of time to do
> further revisions; planning to take that up again. Don't see any reason
> why we shouldn't be able to succeed in getting Cake upstream in its
> current form(ish) in just a few more revisions (or, who knows, maybe a
> dozen more; better not jinx it ;)).

Ah, ok, so I may have mis-interpreted some of the “we won’t accept it unless” statements, which I got lost in. :)

Then just take the rest of it as other thoughts for when there’s time / energy…

Re: [Cake] the Cake stalemate

[Jonathan Morton]

> On 19 Jun, 2018, at 12:55 pm, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Not sure there is a stalemate, actually. I just ran out of time to do
> further revisions; planning to take that up again. Don't see any reason
> why we shouldn't be able to succeed in getting Cake upstream in its
> current form(ish) in just a few more revisions (or, who knows, maybe a
> dozen more; better not jinx it ;)).

My impression was that we were practically there - but we ran up against a hang bug which *may* have had a root cause elsewhere than Cake itself.  I strongly suspect we're tickling a HW driver bug, but we need someone with a different flavour of 100Gbps NIC to try (and hopefully fail) to reproduce it.

There's not much hope of my getting that calibre of hardware myself.

 - Jonathan Morton

Re: [Cake] the Cake stalemate

[Toke Høiland-Jørgensen]

Jonathan Morton <chromatix99@gmail.com> writes:

>> On 19 Jun, 2018, at 12:55 pm, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> 
>> Not sure there is a stalemate, actually. I just ran out of time to do
>> further revisions; planning to take that up again. Don't see any reason
>> why we shouldn't be able to succeed in getting Cake upstream in its
>> current form(ish) in just a few more revisions (or, who knows, maybe a
>> dozen more; better not jinx it ;)).
>
> My impression was that we were practically there - but we ran up
> against a hang bug which *may* have had a root cause elsewhere than
> Cake itself.  I strongly suspect we're tickling a HW driver bug, but
> we need someone with a different flavour of 100Gbps NIC to try (and
> hopefully fail) to reproduce it.

Yeah, that and getting the ACK filter approved by the TCP devs are the
major outstanding issues. I'm planning to go poke at the hanging issue
once I have some time, then hopefully fix whatever is causing it, and
resubmit :)

We also saw a bug on 32-bit MIPS where some combinations of 64-bit
netlink attributes would cause stats display in tc to fail. However, I
believe this is more a case of Cake exposing a latent bug somewhere in
the tc or kernel netlink code (alignment issues, perhaps?), and so I'm
not sure it is necessarily a blocker for merging Cake. However, if
someone could take a look that would be very helpful. I forget if the
current head of the cobalt branch exposes the bug, but I think it does.
It's quite obvious when it happens: no stats output whatsoever...

-Toke

Re: [Cake] the Cake stalemate

[Pete Heist]

> On Jun 19, 2018, at 1:32 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
> 
> My impression was that we were practically there - but we ran up against a hang bug which *may* have had a root cause elsewhere than Cake itself.  I strongly suspect we're tickling a HW driver bug, but we need someone with a different flavour of 100Gbps NIC to try (and hopefully fail) to reproduce it.
> 
> There's not much hope of my getting that calibre of hardware myself.

Me neither. I think that’s about the bandwidth of uncompressed 8K video, 16 bpp @ 60 Hz. I don’t think I could drive it with anything even if I had the NIC, and I don’t see an obtainable alternative to Mellanox, which is around 1000 USD, and you need two or at least the dual port version for twice the price.

I’m glad if we may still make it with Cake then. My bad on the stalemate interpretation- I was conflating some of the thorny feedback with expressions of burnout. :) The feedback still stands though, but for later...

Re: [Cake] the Cake stalemate

[Pete Heist]

> On Jun 19, 2018, at 1:54 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> We also saw a bug on 32-bit MIPS where some combinations of 64-bit
> netlink attributes would cause stats display in tc to fail. However, I
> believe this is more a case of Cake exposing a latent bug somewhere in
> the tc or kernel netlink code (alignment issues, perhaps?), and so I'm
> not sure it is necessarily a blocker for merging Cake. However, if
> someone could take a look that would be very helpful. I forget if the
> current head of the cobalt branch exposes the bug, but I think it does.
> It's quite obvious when it happens: no stats output whatsoever...

I have a 32-bit MIPS in my ER-X, but it sounds like what I saw (outrageous refcnt values) was something different:

qdisc pfifo_fast 0: dev vtun1 root refcnt 23227372115329026 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 3696 bytes 44 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc cake 800a: dev ifb4eth4 root refcnt 23227372115329026 bandwidth 30Mbit diffserv3 dual-dsthost nat ingress split-gso rtt 100.0ms raw overhead 0 
 Sent 41906985 bytes 41340 pkt (dropped 347, overlimits 57700 requeues 0)

This may be related to its old 3.10 kernel though (BTW, looks like Ubiquiti will move to 4.9 soon, alpha is out…).

However, if there’s a way I should try to reproduce something on this hardware to take a look, send any info you’ve got (how to add 64-bit netlink attributes?). I even have a spare ER-X on which I could put OpenWRT in case I need to be working with a more modern kernel…

Re: [Cake] the Cake stalemate

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 4773 bytes --]



> On 19 Jun 2018, at 13:26, Pete Heist <pete@heistp.net> wrote:
> 
> 
>> On Jun 19, 2018, at 1:54 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> 
>> We also saw a bug on 32-bit MIPS where some combinations of 64-bit
>> netlink attributes would cause stats display in tc to fail. However, I
>> believe this is more a case of Cake exposing a latent bug somewhere in
>> the tc or kernel netlink code (alignment issues, perhaps?), and so I'm
>> not sure it is necessarily a blocker for merging Cake. However, if
>> someone could take a look that would be very helpful. I forget if the
>> current head of the cobalt branch exposes the bug, but I think it does.
>> It's quite obvious when it happens: no stats output whatsoever...
> 
> I have a 32-bit MIPS in my ER-X, but it sounds like what I saw (outrageous refcnt values) was something different:
<snip>

Yes it was.  At one point iproute’s tc was doing hidden type promotions in printing from 32bit to 64bit types and neglecting to tell the printf formatter of the change, thus printf was starting at the wrong point in memory in big endian environments.  This was part of the move to JSON output.

Toke took my bug report & patch and made it acceptable to upstream where it now lives as: https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/commit/?id=4db2ff0db46f6368d89cfb3498a700e1256d2a04 and is included in iproute2 v.4.17

> However, if there’s a way I should try to reproduce something on this hardware to take a look, send any info you’ve got (how to add 64-bit netlink attributes?). I even have a spare ER-X on which I could put OpenWRT in case I need to be working with a more modern kernel…

The lack of stats on recent (ie post https://github.com/dtaht/sch_cake/commit/af1d7cde7046af55ec867b29854d754816b64bc8 May 15th) with MIPS BE & LE 32 bit arch is a mystery.  My hack workaround to that for my own personal openwrt builds is https://github.com/ldir-EDB0/openwrt/tree/tokesiproutedebug - which also includes a debug commit from Toke.

I considered bumping openwrt’s master branch to point at latest commit of ‘cobalt’ like my build does, so we could judge from the resultant screaming if it was just MIPS affected or other 32 bit arch’s.  I was dissuaded from doing so.

I got a little further into collecting info on this courtesy ‘kmod-netnl’ which allows packet capture of netlink packets as if on a network interface - captures sent to Toke IIRC but they require hand disassembly to determine where the packet formatting is going wrong.  And there $real_life intervened and I’ve not looked at since/had some more pressing bugs to ponder.

Openwrt nearly bumped to iproute v4.17 but I haven’t yet got around to seeing if that makes any difference.  It looks like netlink_parse_nested cannot cope with 64bit netlink attributes…. but this requires a person who can code rather than me to go any further.

RE: the stalemate.  I swing between an absolute hatred of anything linux/open source/mail lists and finding some people *incredibly* helpful and thinking ‘it’s not so bad, actually this is fun’.  I offer a very recent example of this where I worked with David Woodhouse on a kernel PPPoATM bug (caused by a ticking timebomb that one E Dumazet left behind ;-) that stretched me to my absolute limits but was executed in a spirit of helpfulness, curiosity & fun.  So it seems to be about finding the right person in kernel land who can both see the errors in our code but also see the value and effort in what we have achieved.  Maybe I’m being unfair and not interpreting the kernel mailing list environment correctly but to me it comes across as abrasive at best (and I swore I'd put my head in a tiger’s mouth and tickle its testicles with a spanner before I even think of trying to submit another patch upstream)

On the other hand I can also see that had we approached/involved the kernel people earlier on then some of the blind alleys we’ve travelled (I’m thinking passing of netlink stats here) could have been avoided.  Instead we’ve invested years of work and just presented a fait accompli.  Whether that would have yielded some of the layer breaking stuff we’ve ended up with I very much doubt and cobalt would have been much, much poorer as a result.

The beauty of cake/cobalt is that it does a number of sensible things all in one command line (and has to work around some of linux’s layering decisions.. IFB)

Anyway, there’s my opinion.

KDB

> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake


Cheers,

Kevin D-B

012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Cake] the Cake stalemate

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 4821 bytes --]


> On Jun 19, 2018, at 3:41 PM, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
> 
>> On 19 Jun 2018, at 13:26, Pete Heist <pete@heistp.net> wrote:
>> 
>> I have a 32-bit MIPS in my ER-X, but it sounds like what I saw (outrageous refcnt values) was something different:
> <snip>
> 
> Yes it was.  At one point iproute’s tc was doing hidden type promotions in printing from 32bit to 64bit types and neglecting to tell the printf formatter of the change, thus printf was starting at the wrong point in memory in big endian environments.  This was part of the move to JSON output.

Ok, that one sounds likely to have been squashed...

> Toke took my bug report & patch and made it acceptable to upstream where it now lives as: https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/commit/?id=4db2ff0db46f6368d89cfb3498a700e1256d2a04 <https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/commit/?id=4db2ff0db46f6368d89cfb3498a700e1256d2a04> and is included in iproute2 v.4.17
> 
>> However, if there’s a way I should try to reproduce something on this hardware to take a look, send any info you’ve got (how to add 64-bit netlink attributes?). I even have a spare ER-X on which I could put OpenWRT in case I need to be working with a more modern kernel…
> 
> The lack of stats on recent (ie post https://github.com/dtaht/sch_cake/commit/af1d7cde7046af55ec867b29854d754816b64bc8 <https://github.com/dtaht/sch_cake/commit/af1d7cde7046af55ec867b29854d754816b64bc8> May 15th) with MIPS BE & LE 32 bit arch is a mystery.  My hack workaround to that for my own personal openwrt builds is https://github.com/ldir-EDB0/openwrt/tree/tokesiproutedebug <https://github.com/ldir-EDB0/openwrt/tree/tokesiproutedebug> - which also includes a debug commit from Toke.

Ah, and I see my OpenWRT build’s kmod-sched-cake is from 2017-01-28, before the dawn of time. Really dumb question- how do you do your MIPS builds, cross-compile? My MIPS hardware is enough to run things but I don’t know if I’ll be installing dev tools and compiling on 64M RAM and 16M flash.

> I got a little further into collecting info on this courtesy ‘kmod-netnl’ which allows packet capture of netlink packets as if on a network interface - captures sent to Toke IIRC but they require hand disassembly to determine where the packet formatting is going wrong.  And there $real_life intervened and I’ve not looked at since/had some more pressing bugs to ponder.
> 
> Openwrt nearly bumped to iproute v4.17 but I haven’t yet got around to seeing if that makes any difference.  It looks like netlink_parse_nested cannot cope with 64bit netlink attributes…. but this requires a person who can code rather than me to go any further.

Ok, this just sounds ugly and I’m not likely to progress on it soon to be honest.

> RE: the stalemate.  I swing between an absolute hatred of anything linux/open source/mail lists and finding some people *incredibly* helpful and thinking ‘it’s not so bad, actually this is fun’.  I offer a very recent example of this where I worked with David Woodhouse on a kernel PPPoATM bug (caused by a ticking timebomb that one E Dumazet left behind ;-) that stretched me to my absolute limits but was executed in a spirit of helpfulness, curiosity & fun.  So it seems to be about finding the right person in kernel land who can both see the errors in our code but also see the value and effort in what we have achieved.  Maybe I’m being unfair and not interpreting the kernel mailing list environment correctly but to me it comes across as abrasive at best (and I swore I'd put my head in a tiger’s mouth and tickle its testicles with a spanner before I even think of trying to submit another patch upstream)
> On the other hand I can also see that had we approached/involved the kernel people earlier on then some of the blind alleys we’ve travelled (I’m thinking passing of netlink stats here) could have been avoided.  Instead we’ve invested years of work and just presented a fait accompli. Whether that would have yielded some of the layer breaking stuff we’ve ended up with I very much doubt and cobalt would have been much, much poorer as a result.

Thanks, I rather agree, I just think that ideally cooperation among all levels of the kernel would yield a total result with more conceptual integrity, and fewer fortified turfs. I wouldn’t stop questioning the stuff that’s been around for years, so whatever we learned in trying to get Cake out there, maybe after it _is_ out there, we should send up some constructive feedback on what could have made it better. Ideally if something useful doesn’t fit upstream, there should either be genuine solutions on how to make it fit, or a re-thinking of the upstream code so it does.


[-- Attachment #2: Type: text/html, Size: 15444 bytes --]

Re: [Cake] the Cake stalemate

[Dave Taht]

a couple notes.

"forking" code and living out of tree for a while has been a way to
spin faster on solving thorny problems, and welcome newer,
less experienced devs to a difficult codebase while encouraging folk
in the problem domain to enter. I am an increasingly
lousy programmer, (for the record my best languages were lisp and perl
15+ years back), and these days far more interested in politics and
theory, than in the grunge of programming (I do confess interest in go
which is cheering me up - I used to learn a new language every couple
of years, but python's "GIL" offended my multi-coreness and rust was
too much like BDSM).

Sometimes I'd like to investigate how "dunbars limit" shapes the linux
communities.

Had we tried to merge cake  earlier making it as feature complete is
it is now would have been harder.

Cake specifically is not modular to gain benefits - it uses one less
packet than htb + fq_codel, which was a compelling enough reason to
pursue it back on day one. I'm oft-boggled by what it has become.

It's also less modular in my (thus far forlorn) hope it could more
easily migrate to codebases outside of linux, not just bsd, but
ns2/3, fuschia, ddpk, and proprietary stuff like whatever is inside a
rdk cable modem, cmts or dslam nowadays. I'm not sure if I've ever
clearly expressed "less modularity" or why as a design goal here...

We did, always, seek some visibility into the mainline linux folk -
stephen hemminger has always been around, eric dumazet aware,
etc. General rule - Always, always have some contact with the core
folk, don't bug 'em until you've got something good - establish
personal contacts, go to a conference once in a while. Drinking beer
with the people you argue with on mailing lists helps. I wish kevin
had dropped in at london ietf. I find it difficult to physically
travel these days, I'm dreading the trip, but I'm seeking contacts in
the ftc,  planning on dropping in at comcast and having ESR and cathy
kick my ass at power grid.

a persistent sadness is we've not had the funds to get to conferences
or even make t-shirts. or do pr. or pay ourselves.

I go through long periods unsubscribed to the netdev mailing list. I
can't stand the patch volume; just watching it go by burns me out. I
feel the pain of those compelled to be on it, and have a great deal of
sympathy for dave miller's irascibility. In person eric, david,
stephen are *great*. Jesper, dave woodhouse, Jon corbet are *amazing*.
so many people are great in person but testy on mailing lists. some
are great on mailing lists but lousy in person....

The main reason why I'm on nedev is to spot interesting new features
and watch bugs get found and fixed, and I'm hoping to get off of it
again a few months after cake makes it in. With a sigh of relief.
Maybe this userspace wifi emulator will get funded. Not feeling burned
out over having to face c or the kernel again feels good.

the cake codebase has benefited hugely by finally getting more review
from the top devs in the field now, and I applaud y'alls persistence.
We'll find this last bug, and then...

everybody can have cake.



On Tue, Jun 19, 2018 at 7:48 AM, Pete Heist <pete@heistp.net> wrote:
>
> On Jun 19, 2018, at 3:41 PM, Kevin Darbyshire-Bryant
> <kevin@darbyshire-bryant.me.uk> wrote:
>
> On 19 Jun 2018, at 13:26, Pete Heist <pete@heistp.net> wrote:
>
> I have a 32-bit MIPS in my ER-X, but it sounds like what I saw (outrageous
> refcnt values) was something different:
>
> <snip>
>
> Yes it was.  At one point iproute’s tc was doing hidden type promotions in
> printing from 32bit to 64bit types and neglecting to tell the printf
> formatter of the change, thus printf was starting at the wrong point in
> memory in big endian environments.  This was part of the move to JSON
> output.
>
>
> Ok, that one sounds likely to have been squashed...
>
> Toke took my bug report & patch and made it acceptable to upstream where it
> now lives as:
> https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/commit/?id=4db2ff0db46f6368d89cfb3498a700e1256d2a04
> and is included in iproute2 v.4.17
>
> However, if there’s a way I should try to reproduce something on this
> hardware to take a look, send any info you’ve got (how to add 64-bit netlink
> attributes?). I even have a spare ER-X on which I could put OpenWRT in case
> I need to be working with a more modern kernel…
>
>
> The lack of stats on recent (ie post
> https://github.com/dtaht/sch_cake/commit/af1d7cde7046af55ec867b29854d754816b64bc8
> May 15th) with MIPS BE & LE 32 bit arch is a mystery.  My hack workaround to
> that for my own personal openwrt builds is
> https://github.com/ldir-EDB0/openwrt/tree/tokesiproutedebug - which also
> includes a debug commit from Toke.
>
>
> Ah, and I see my OpenWRT build’s kmod-sched-cake is from 2017-01-28, before
> the dawn of time. Really dumb question- how do you do your MIPS builds,
> cross-compile? My MIPS hardware is enough to run things but I don’t know if
> I’ll be installing dev tools and compiling on 64M RAM and 16M flash.
>
> I got a little further into collecting info on this courtesy ‘kmod-netnl’
> which allows packet capture of netlink packets as if on a network interface
> - captures sent to Toke IIRC but they require hand disassembly to determine
> where the packet formatting is going wrong.  And there $real_life intervened
> and I’ve not looked at since/had some more pressing bugs to ponder.
>
> Openwrt nearly bumped to iproute v4.17 but I haven’t yet got around to
> seeing if that makes any difference.  It looks like netlink_parse_nested
> cannot cope with 64bit netlink attributes…. but this requires a person who
> can code rather than me to go any further.
>
>
> Ok, this just sounds ugly and I’m not likely to progress on it soon to be
> honest.
>
> RE: the stalemate.  I swing between an absolute hatred of anything
> linux/open source/mail lists and finding some people *incredibly* helpful
> and thinking ‘it’s not so bad, actually this is fun’.  I offer a very recent
> example of this where I worked with David Woodhouse on a kernel PPPoATM bug
> (caused by a ticking timebomb that one E Dumazet left behind ;-) that
> stretched me to my absolute limits but was executed in a spirit of
> helpfulness, curiosity & fun.  So it seems to be about finding the right
> person in kernel land who can both see the errors in our code but also see
> the value and effort in what we have achieved.  Maybe I’m being unfair and
> not interpreting the kernel mailing list environment correctly but to me it
> comes across as abrasive at best (and I swore I'd put my head in a tiger’s
> mouth and tickle its testicles with a spanner before I even think of trying
> to submit another patch upstream)
>
> On the other hand I can also see that had we approached/involved the kernel
> people earlier on then some of the blind alleys we’ve travelled (I’m
> thinking passing of netlink stats here) could have been avoided.  Instead
> we’ve invested years of work and just presented a fait accompli. Whether
> that would have yielded some of the layer breaking stuff we’ve ended up with
> I very much doubt and cobalt would have been much, much poorer as a result.
>
>
> Thanks, I rather agree, I just think that ideally cooperation among all
> levels of the kernel would yield a total result with more conceptual
> integrity, and fewer fortified turfs. I wouldn’t stop questioning the stuff
> that’s been around for years, so whatever we learned in trying to get Cake
> out there, maybe after it _is_ out there, we should send up some
> constructive feedback on what could have made it better. Ideally if
> something useful doesn’t fit upstream, there should either be genuine
> solutions on how to make it fit, or a re-thinking of the upstream code so it
> does.
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] the Cake stalemate

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 3137 bytes --]


> On Jun 19, 2018, at 6:24 PM, Dave Taht <dave.taht@gmail.com> wrote:
> 
> "forking" code and living out of tree for a while has been a way to
> spin faster on solving thorny problems, and welcome newer,
> less experienced devs to a difficult codebase while encouraging folk
> in the problem domain to enter.

I hadn’t though of that. It’s important.

> I am an increasingly
> lousy programmer, (for the record my best languages were lisp and perl
> 15+ years back), and these days far more interested in politics and
> theory, than in the grunge of programming (I do confess interest in go
> which is cheering me up

Maybe give Go some more time before hanging up the editor? It looks like it will be around a while, and “asymptotically approaching boring” as they like to say about it. (an oldie but a goodie: https://www.youtube.com/watch?v=sln-gJaURzk <https://www.youtube.com/watch?v=sln-gJaURzk>).

> Cake specifically is not modular to gain benefits.

sort of like the monolithic kernel that way...

> I'm oft-boggled by what it has become.

I think it just shows there’s a need for all of it, but I still never felt like we’ve fully nailed how to explain it. It’s not easy! For the untechnical, I don’t think we can do much better than “it makes your Internet faster”, which is what most people want, to get on with their lives and any “settings” are already too many. For more technical people, it’s a series of features that deserve individual explanation, like you have in your slides, only it’s a lot to fit into 20 minutes. I’ll try to look at it again.

> I go through long periods unsubscribed to the netdev mailing list. I
> can't stand the patch volume; just watching it go by burns me out. I
> feel the pain of those compelled to be on it, and have a great deal of
> sympathy for dave miller's irascibility. In person eric, david,
> stephen are *great*. Jesper, dave woodhouse, Jon corbet are *amazing*.
> so many people are great in person but testy on mailing lists. some
> are great on mailing lists but lousy in person….
> The main reason why I'm on nedev is to spot interesting new features
> and watch bugs get found and fixed, and I'm hoping to get off of it
> again a few months after cake makes it in. With a sigh of relief.

Thanks for those insights. Kernel-space isn’t for everyone I guess...

> Maybe this userspace wifi emulator will get funded. Not feeling burned
> out over having to face c or the kernel again feels good.

Please let’s try. I think we can do a good job with it, and WiFi testing is a real pain point. My closet looks ridiculous (pictures to come), and I bet I have the most reasonable setup of any of us. Would it be an open source project?

> the cake codebase has benefited hugely by finally getting more review
> from the top devs in the field now, and I applaud y'alls persistence.
> We'll find this last bug, and then…

Wish I could help more with development but there may be too much bootstrapping for me with what little left is needed. I’ve never been able to really fault in the source.

[-- Attachment #2: Type: text/html, Size: 4638 bytes --]

Re: [Cake] the Cake stalemate

[Jonathan Morton]

> On 19 Jun, 2018, at 11:14 pm, Pete Heist <pete@heistp.net> wrote:
> 
> I don’t think we can do much better than “it makes your Internet faster”, which is what most people want

s/faster/reliable/

It would be really nice to be able to say "go out and buy this box, plug it in and run through the setup wizard" instead of "go through the frankly scary process of reflashing your router with OpenWRT".  In that case we might really get some traction.

 - Jonathan Morton

Re: [Cake] the Cake stalemate

[Dave Taht]

On Tue, Jun 19, 2018 at 1:35 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
>> On 19 Jun, 2018, at 11:14 pm, Pete Heist <pete@heistp.net> wrote:
>>
>> I don’t think we can do much better than “it makes your Internet faster”, which is what most people want
>
> s/faster/reliable/
>
> It would be really nice to be able to say "go out and buy this box, plug it in and run through the setup wizard" instead of "go through the frankly scary process of reflashing your router with OpenWRT".  In that case we might really get some traction.

Faster and more reliable is a good catchphrase.

Here's eero's implementation of sqm:
https://www.macobserver.com/tips/quick-tip/enable-sqm-eero-wifi-mesh-network/

>  - Jonathan Morton
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] the Cake stalemate

[Pete Heist]

> On Jun 19, 2018, at 10:35 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
> 
>> On 19 Jun, 2018, at 11:14 pm, Pete Heist <pete@heistp.net> wrote:
>> 
>> I don’t think we can do much better than “it makes your Internet faster”, which is what most people want
> 
> s/faster/reliable/
> 
> It would be really nice to be able to say "go out and buy this box, plug it in and run through the setup wizard" instead of "go through the frankly scary process of reflashing your router with OpenWRT".  In that case we might really get some traction.

There could be a transparent bridge available with Cake installed that just has a web interface for a few basic settings, but it would only help people with WiFi APs or switches / networks separate from their router. Probably not most people.

My secret hope is we’re heading into an era when people will want to own their data once again and move it out of the cloud and back into their residences. Small, Linux-based home devices will be so cheap, powerful and reliable that they’ll route, store, serve, control, be an AP, plus allow the installation of unprivileged containers or apps with various distributed functionalities. Just like some join cooperative ISPs, people will join a cooperative to pay for the engineering, development and support of these devices, because they’ll realize that it’s the right thing to do instead of selling themselves for “free” services that perpetuate monopolies. Seems like a long way about it, but surely then we can fund and install Cake on these devices. :) If it wouldn’t be everyone, maybe it would be for some? for enough? pipe dream?

Re: [Cake] the Cake stalemate

[Sebastian Moeller]

Well,



On June 19, 2018 10:35:03 PM GMT+02:00, Jonathan Morton <chromatix99@gmail.com> wrote:
>> On 19 Jun, 2018, at 11:14 pm, Pete Heist <pete@heistp.net> wrote:
>> 
>> I don’t think we can do much better than “it makes your Internet
>faster”, which is what most people want
>
>s/faster/reliable/
>
>It would be really nice to be able to say "go out and buy this box,
>plug it in and run through the setup wizard" instead of "go through the
>frankly scary process of reflashing your router with OpenWRT".  In that
>case we might really get some traction.
>
> - Jonathan Morton
>

I believe both evenroutes IQrouter as well as CZ.nic's turris omnia, fullfil your requirements. The IQrouter is even available on Amazon... Both allow to use cake and sqm-scripts, the IQrouter out of the box I believe while with the omnia a few missing packages need to be installed first.

Best Regards
        Sebastian

>_______________________________________________
>Cake mailing list
>Cake@lists.bufferbloat.net
>https://lists.bufferbloat.net/listinfo/cake

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [Cake] the Cake stalemate

[Sebastian Moeller]

On June 19, 2018 11:48:08 PM GMT+02:00, Dave Taht <dave.taht@gmail.com> wrote:
>On Tue, Jun 19, 2018 at 1:35 PM, Jonathan Morton
><chromatix99@gmail.com> wrote:
>>> On 19 Jun, 2018, at 11:14 pm, Pete Heist <pete@heistp.net> wrote:
>>>
>>> I don’t think we can do much better than “it makes your Internet
>faster”, which is what most people want
>>
>> s/faster/reliable/
>>
>> It would be really nice to be able to say "go out and buy this box,
>plug it in and run through the setup wizard" instead of "go through the
>frankly scary process of reflashing your router with OpenWRT".  In that
>case we might really get some traction.
>
>Faster and more reliable is a good catchphrase.
>
>Here's eero's implementation of sqm:
>https://www.macobserver.com/tips/quick-tip/enable-sqm-eero-wifi-mesh-network/

Well do we know what eero's really does under the hood? So are they using sqm or are they simply using sqm as a marketing term for their own special sauce? 

Best Regards
        Sebastian


>
>>  - Jonathan Morton
>>
>
>
>
>-- 
>
>Dave Täht
>CEO, TekLibre, LLC
>http://www.teklibre.com
>Tel: 1-669-226-2619
>_______________________________________________
>Cake mailing list
>Cake@lists.bufferbloat.net
>https://lists.bufferbloat.net/listinfo/cake

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [Cake] the Cake stalemate

[Jonathan Morton]

> On 20 Jun, 2018, at 2:38 am, Sebastian Moeller <moeller0@gmx.de> wrote:
> 
> The IQrouter is even available on Amazon...

Is it?  I just searched for it and came up empty.

 - Jonathan Morton

Re: [Cake] the Cake stalemate

[Sebastian Moeller]

Hi Jonathan,

On June 20, 2018 5:36:15 AM GMT+02:00, Jonathan Morton <chromatix99@gmail.com> wrote:
>> On 20 Jun, 2018, at 2:38 am, Sebastian Moeller <moeller0@gmx.de>
>wrote:
>> 
>> The IQrouter is even available on Amazon...
>
>Is it?  I just searched for it and came up empty.

It currently shows up on amazon.com (https://www.amazon.com/gp/product/B06WP5GTS8/ref=as_li_tl?ie=UTF8&tag=evenroute-20&camp=1789&creative=9325&linkCode=as2&creativeASIN=B06WP5GTS8&linkId=e287314e38c113de2694b7469bf0950e) but will ship to Europe at least it will ship to Germany. But you are right I do not find it on amazon.de.

Best Regards
        Sebastian


>
> - Jonathan Morton

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.