[Cake] isp economics

[Dave Taht]

On Sat, Jul 28, 2018 at 9:41 AM Pete Heist <pete at heistp.net> wrote:
>
> On Jul 28, 2018, at 10:06 AM, Jonathan Morton <chromatix99 at gmail.com> wrote:
>
> This sounds like a relatively complex network topology, in which there are a lot of different potential bottlenecks, depending on the dynamic state of the network.
>
>
> It is, which is the argument from those who want a more centralized approach. Backhaul link types include 5 GHz, 10 GHz, 60 GHz, licensed spectrum (apparently), and some fiber. Side note: I’ve heard that aligning the 60 GHz p2p links is a chore.
>
> Okay, so straight away we're in a significantly different regime from CoverFire's situation, where the subscribers each have a defined link bandwidth (which may or may not be related to the capabilities of the underlying physical link).  You simply need to share some backhaul link fairly between subscribers using it at any given moment.
>
>
> Exactly. Many members, including myself, are limited by our CPE links during off hours, and by the backhaul during high traffic hours.

3 items

1) Co-locating some essential services (like netflix) might be of help.

https://openconnect.netflix.com/en/

2) If the members voted for more backhaul, the costs are understandable...

3) Philosophically I vastly prefer the concept of "everyone sharing
the network", rather than rate plans, to create some market tension as
to the available bandwidth at any given time of day. An extreme
viewpoint was that you'd bill for bandwidth more during times of the
day when it was more valuable, and less when it was unused.

Instead we get lying rate "up to X" plans that force "building
overcapacity because thats what I bought", and arbitrary ratios of
10x1 or worse overselling actual capacity to subscribers (which got
blown up by the rise of streaming)

However, if everyone always shared equally in the fate of the network,
everyone has an incentive to make it as a good as possible, and in
off-hours anybody could get more than "X", if available.

Back in the good ole days you'd find me in the lab at 2am because "the
network was better then". Incentivtising off peak use is what torrent
did. Backups were also frequently done then. etc.

> Let's call this one a vote for "diffserv not required", since DRR++ copes well by prioritising sparse traffic.
>
>
> Yep, I don’t know if you’ll hear “diffserv required” for an ISP, but we’ll see.
>
> Could you convert that DB to eBPF rules?  This would let you use the same configuration interface as CoverFire's situation.
>
>
> That should be no problem at all.
>
> In general, I think the make-wifi-fast team has a better handle on the subtleties of half-duplex links.  If there's any way to get their work into the airOS devices, so much the better.

ubnt did add at least airtime fairness to airos a year or two back. It
was not on by default. Their "TDMA" qos system was this insane mess of
sfq rules when I tore it apart.... 8 years back.

> I wish that were the case, but am not holding my breath. As things stand now, I may still use HTB then, which is ok.

>The key is not just the dynamic rates, but also, it appears, having egress and ingress through a common IFB that’s split into sub-queues for egress and ingress. I’ll punt on WiFi for now and come back to it later.
>
> But I think there's also a use for "ISP-type" Cake in your network, especially in the wired backhauls where link bandwidth is relatively predictable, and I suspect most of these will be in the core parts of your network where the traffic is most complex.  As long as you can get around to running a suitable kernel version, there should also be no inherent problem with replicating the same dynamic adjustment of global shaper rate as "normal" Cake has, which will allow it to be used on some of your wireless links as well.
>
>
> Yes, the biggest benefit for us would probably be “member fairness” instead of “IP fairness”. One of the long-time admins was excited that this might be possible. Also, I’ll be interested to see if improving queue management gets us better utilization of the Internet link. If so, we may want fairness at the gateway, which may be a job more for an ISP Cake given the number of flows.
>
> Lastly, if ISP Cake could support SMP, that would be all the better for the dual and quad core APU backhaul routers. :)
>
> _______________________________________________
> Cake mailing list
> Cake at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619