From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-x22a.google.com (mail-qk0-x22a.google.com [IPv6:2607:f8b0:400d:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 5F16B3BA8E for ; Sat, 28 Jul 2018 13:32:17 -0400 (EDT) Received: by mail-qk0-x22a.google.com with SMTP id z74-v6so5357273qkb.10 for ; Sat, 28 Jul 2018 10:32:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=WfjR2eOWoaOz6kJecQQ3YuFPxh5zVlvjK8cGQmAMCPQ=; b=BNqd0l6yX1HmY3JvdYhvUNkHTwa5TBenKFJnKRD9ZfKGzHcuXaFe1EUwNZf748xlci 6VGaCipPJDRWuh2fWkz6FAaYNVCt+6pjVL4I6HKCtLaK1CnKCXTmpAkZiOzQlRuTg1wk qsjrx+73qFhSyfCTe+0gyo+6fxUxvwH6fA2KTifPiZ6t0//8HpGko89EpnlzAqvXCUy7 EiJkfkz9U1xLA1eatyvCGB4qOR4T2mrLDYdp8bSBgejoZZ38Y5FVkiXgy2TU4XBnxwIO IW0VCPuAgtJLfcthfn2G3jMqHPBOzP3N98IuIv76FgC6jiBeRwmxvMA9wym4LbA1nkYt D0TA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=WfjR2eOWoaOz6kJecQQ3YuFPxh5zVlvjK8cGQmAMCPQ=; b=sT5b+pTTCqRSNz84f1tpTiCqaNYtASxA1BJHFfJ+c0LfgMK97rJ6roPqZiSCZc1ymF XDi7o2IP4vVjNyadILcDzQbFCxTSf7eCC+6fmaPCisSXbyS1fPawRfHMZ9xG/cmhlHSn moAPzsYcygVc5OUncmUr8ESKQJXwmNnkj5LYZCsTqNum05q0EH6XXgIq7+FN4GvfST4U +f+aX4OiAMEQJ0PKJZsGMKypVT/H6dqUglKP1BrKVjTpxmsZ7RmrqQOH7H9VwoqOjukl +vY5+nDUzWAaMEpanS63j0C7aGtr/2XmpuUT6otLS9loG8EATQ3jXm7yPlOxwZRUpNrT qqUQ== X-Gm-Message-State: AOUpUlFD/Di+819JN0YHl5dOC3qbJTtFT8SoDgHBeOOKfXVn48Vf7SlI 8efxo1l7g+cfeitlh23DMV65nkVJLo9zYqDXE/s= X-Google-Smtp-Source: AAOMgped2I99GnNHGygCdYANgC4ThiwW7WR+moPi4L9T6hTkzbMVU7J8xzP8bkOX4/nC3z2ivnUgwYjm6SmYqda/DdI= X-Received: by 2002:a37:2121:: with SMTP id h33-v6mr10154018qkh.319.1532799136851; Sat, 28 Jul 2018 10:32:16 -0700 (PDT) MIME-Version: 1.0 References: <1357421162.31089.1531812291583@webmail.strato.de> <1c323544b3076c0ab31b887d6113f25f572e41ae.camel@coverfire.com> <87woth28rw.fsf@toke.dk> <87tvol1z6h.fsf@toke.dk> <8980FA6C-508B-43E1-8C23-6EBC4A10499A@heistp.net> <842E73E8-368B-4F53-9A6C-31C10420536E@heistp.net> In-Reply-To: <842E73E8-368B-4F53-9A6C-31C10420536E@heistp.net> From: Dave Taht Date: Sat, 28 Jul 2018 10:32:05 -0700 Message-ID: To: Pete Heist Cc: Jonathan Morton , Cake List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: [Cake] isp economics X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jul 2018 17:32:17 -0000 On Sat, Jul 28, 2018 at 9:41 AM Pete Heist wrote: > > On Jul 28, 2018, at 10:06 AM, Jonathan Morton wro= te: > > This sounds like a relatively complex network topology, in which there ar= e a lot of different potential bottlenecks, depending on the dynamic state = of the network. > > > It is, which is the argument from those who want a more centralized appro= ach. Backhaul link types include 5 GHz, 10 GHz, 60 GHz, licensed spectrum (= apparently), and some fiber. Side note: I=E2=80=99ve heard that aligning th= e 60 GHz p2p links is a chore. > > Okay, so straight away we're in a significantly different regime from Cov= erFire's situation, where the subscribers each have a defined link bandwidt= h (which may or may not be related to the capabilities of the underlying ph= ysical link). You simply need to share some backhaul link fairly between s= ubscribers using it at any given moment. > > > Exactly. Many members, including myself, are limited by our CPE links dur= ing off hours, and by the backhaul during high traffic hours. 3 items 1) Co-locating some essential services (like netflix) might be of help. https://openconnect.netflix.com/en/ 2) If the members voted for more backhaul, the costs are understandable... 3) Philosophically I vastly prefer the concept of "everyone sharing the network", rather than rate plans, to create some market tension as to the available bandwidth at any given time of day. An extreme viewpoint was that you'd bill for bandwidth more during times of the day when it was more valuable, and less when it was unused. Instead we get lying rate "up to X" plans that force "building overcapacity because thats what I bought", and arbitrary ratios of 10x1 or worse overselling actual capacity to subscribers (which got blown up by the rise of streaming) However, if everyone always shared equally in the fate of the network, everyone has an incentive to make it as a good as possible, and in off-hours anybody could get more than "X", if available. Back in the good ole days you'd find me in the lab at 2am because "the network was better then". Incentivtising off peak use is what torrent did. Backups were also frequently done then. etc. > Let's call this one a vote for "diffserv not required", since DRR++ copes= well by prioritising sparse traffic. > > > Yep, I don=E2=80=99t know if you=E2=80=99ll hear =E2=80=9Cdiffserv requir= ed=E2=80=9D for an ISP, but we=E2=80=99ll see. > > Could you convert that DB to eBPF rules? This would let you use the same= configuration interface as CoverFire's situation. > > > That should be no problem at all. > > In general, I think the make-wifi-fast team has a better handle on the su= btleties of half-duplex links. If there's any way to get their work into t= he airOS devices, so much the better. ubnt did add at least airtime fairness to airos a year or two back. It was not on by default. Their "TDMA" qos system was this insane mess of sfq rules when I tore it apart.... 8 years back. > I wish that were the case, but am not holding my breath. As things stand = now, I may still use HTB then, which is ok. >The key is not just the dynamic rates, but also, it appears, having egress= and ingress through a common IFB that=E2=80=99s split into sub-queues for = egress and ingress. I=E2=80=99ll punt on WiFi for now and come back to it l= ater. > > But I think there's also a use for "ISP-type" Cake in your network, espec= ially in the wired backhauls where link bandwidth is relatively predictable= , and I suspect most of these will be in the core parts of your network whe= re the traffic is most complex. As long as you can get around to running a= suitable kernel version, there should also be no inherent problem with rep= licating the same dynamic adjustment of global shaper rate as "normal" Cake= has, which will allow it to be used on some of your wireless links as well= . > > > Yes, the biggest benefit for us would probably be =E2=80=9Cmember fairnes= s=E2=80=9D instead of =E2=80=9CIP fairness=E2=80=9D. One of the long-time a= dmins was excited that this might be possible. Also, I=E2=80=99ll be intere= sted to see if improving queue management gets us better utilization of the= Internet link. If so, we may want fairness at the gateway, which may be a = job more for an ISP Cake given the number of flows. > > Lastly, if ISP Cake could support SMP, that would be all the better for t= he dual and quad core APU backhaul routers. :) > > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake --=20 Dave T=C3=A4ht CEO, TekLibre, LLC http://www.teklibre.com Tel: 1-669-226-2619