From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 290A33BA8E for ; Sat, 28 Jul 2018 12:41:40 -0400 (EDT) Received: by mail-wr1-x436.google.com with SMTP id j5-v6so8160381wrr.8 for ; Sat, 28 Jul 2018 09:41:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=E+jfB46Rx2aAMla6Sj/Txqd9mZ5sAdbC2s+lD+nL71Q=; b=GkBNymWEWQK3nekM1k0rrO4UPTGpqebLIEWNPKIpfImwGqBuK0+G26qOtjupIamTBJ /GMbJoVi47d+2jhQGbQnxsZ9Qv7lrqedK4u+n8TfwK3gPFHZKrQsffFjrLAC4pwsyHqr VeDPDltL79hPzdzzQDGsiYShXn9rFWHw5uuKdHeFitoyLToNFsZa4ntY1xzcrqcxCW+J ZClm7REUcy8expLaj4pE4Luzl0p+IArKC0KlvgKEQWCM3gfjMnAk8r0wLZHrch+qrfpZ 4q6wP4QrNye5lNsW9QNy/ynCbOdDbn8BOl9PVmE+0wdQXDXV4sK6dFQoDJ6AtOW2mX9H UbnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=E+jfB46Rx2aAMla6Sj/Txqd9mZ5sAdbC2s+lD+nL71Q=; b=WNOfVtfh4BriNmDZU0hrafyOyOhGQSUO6jkrQ8m6wDvkOiRwUbELnoY54DUXcSfQm2 DR0rga6FRom0Pp3KLnXn8LggL0ap/IPG91W1lQzSq51h7eHj6/JhnVM3kfJjDJ3t2NRr Kc+39oh2PgGU0ov5rRGQNL5oG2Af6RNHnqxL9F88k9toRDdJLOy7noNS3sttPucMnG3T FoHDe99nfF+RiDyRhpqU5OjefTA/oODbqaet0sCAftlIwSb1BupVZn9JudfKr+IthfoV Hj49NWcR5q32abhrA7zzwjEFNTypWhqoFqpBKKvG/v17yLKHb8ZT6cJyVZ/sa23jM2uj ob0w== X-Gm-Message-State: AOUpUlEc+v76faFBAdU0r9hbtoKlTPaSAQin85p15y+g771fnqmwWuK4 5hd8uxg7lqRkO77e0kv9lPG+qA== X-Google-Smtp-Source: AAOMgpdxixI7BPoKzivU7ddyjrPPfDYX51kVBTTsUhmjRwgUZX0jD2U0BtbIX+g1ZCxNpTph8cAY7A== X-Received: by 2002:adf:8877:: with SMTP id e52-v6mr10466492wre.30.1532796099248; Sat, 28 Jul 2018 09:41:39 -0700 (PDT) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id s2-v6sm12781355wrn.83.2018.07.28.09.41.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 28 Jul 2018 09:41:38 -0700 (PDT) From: Pete Heist Message-Id: <842E73E8-368B-4F53-9A6C-31C10420536E@heistp.net> Content-Type: multipart/alternative; boundary="Apple-Mail=_60E0DC6D-8029-40DF-836A-940107C6020E" Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Date: Sat, 28 Jul 2018 18:41:37 +0200 In-Reply-To: Cc: Cake List To: Jonathan Morton References: <1357421162.31089.1531812291583@webmail.strato.de> <1c323544b3076c0ab31b887d6113f25f572e41ae.camel@coverfire.com> <87woth28rw.fsf@toke.dk> <87tvol1z6h.fsf@toke.dk> <8980FA6C-508B-43E1-8C23-6EBC4A10499A@heistp.net> X-Mailer: Apple Mail (2.3445.9.1) Subject: Re: [Cake] =?utf-8?q?Using_cake_to_shape_1000=E2=80=99s_of_users=2E?= X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jul 2018 16:41:40 -0000 --Apple-Mail=_60E0DC6D-8029-40DF-836A-940107C6020E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jul 28, 2018, at 10:06 AM, Jonathan Morton = wrote: >=20 > This sounds like a relatively complex network topology, in which there = are a lot of different potential bottlenecks, depending on the dynamic = state of the network. It is, which is the argument from those who want a more centralized = approach. Backhaul link types include 5 GHz, 10 GHz, 60 GHz, licensed = spectrum (apparently), and some fiber. Side note: I=E2=80=99ve heard = that aligning the 60 GHz p2p links is a chore. > Okay, so straight away we're in a significantly different regime from = CoverFire's situation, where the subscribers each have a defined link = bandwidth (which may or may not be related to the capabilities of the = underlying physical link). You simply need to share some backhaul link = fairly between subscribers using it at any given moment. Exactly. Many members, including myself, are limited by our CPE links = during off hours, and by the backhaul during high traffic hours. > Let's call this one a vote for "diffserv not required", since DRR++ = copes well by prioritising sparse traffic. Yep, I don=E2=80=99t know if you=E2=80=99ll hear =E2=80=9Cdiffserv = required=E2=80=9D for an ISP, but we=E2=80=99ll see. > Could you convert that DB to eBPF rules? This would let you use the = same configuration interface as CoverFire's situation. That should be no problem at all. > In general, I think the make-wifi-fast team has a better handle on the = subtleties of half-duplex links. If there's any way to get their work = into the airOS devices, so much the better. I wish that were the case, but am not holding my breath. As things stand = now, I may still use HTB then, which is ok. The key is not just the = dynamic rates, but also, it appears, having egress and ingress through a = common IFB that=E2=80=99s split into sub-queues for egress and ingress. = I=E2=80=99ll punt on WiFi for now and come back to it later. > But I think there's also a use for "ISP-type" Cake in your network, = especially in the wired backhauls where link bandwidth is relatively = predictable, and I suspect most of these will be in the core parts of = your network where the traffic is most complex. As long as you can get = around to running a suitable kernel version, there should also be no = inherent problem with replicating the same dynamic adjustment of global = shaper rate as "normal" Cake has, which will allow it to be used on some = of your wireless links as well. Yes, the biggest benefit for us would probably be =E2=80=9Cmember = fairness=E2=80=9D instead of =E2=80=9CIP fairness=E2=80=9D. One of the = long-time admins was excited that this might be possible. Also, I=E2=80=99= ll be interested to see if improving queue management gets us better = utilization of the Internet link. If so, we may want fairness at the = gateway, which may be a job more for an ISP Cake given the number of = flows. Lastly, if ISP Cake could support SMP, that would be all the better for = the dual and quad core APU backhaul routers. :) --Apple-Mail=_60E0DC6D-8029-40DF-836A-940107C6020E Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On = Jul 28, 2018, at 10:06 AM, Jonathan Morton <chromatix99@gmail.com> wrote:

This sounds like a relatively complex network topology, in = which there are a lot of different potential bottlenecks, depending on = the dynamic state of the = network.

It = is, which is the argument from those who want a more centralized = approach. Backhaul link types include 5 GHz, 10 GHz, 60 GHz, licensed = spectrum (apparently), and some fiber. Side note: I=E2=80=99ve heard = that aligning the 60 GHz p2p links is a chore.

Okay, so straight away we're in a significantly different = regime from CoverFire's situation, where the subscribers each have a = defined link bandwidth (which may or may not be related to the = capabilities of the underlying physical link).  You simply need to = share some backhaul link fairly between subscribers using it at any = given moment.

Exactly. Many members, including myself, are = limited by our CPE links during off hours, and by the backhaul during = high traffic hours.

Let's call this one a vote for = "diffserv not required", since DRR++ copes well by prioritising sparse = traffic.

Yep, I don=E2=80=99t know if you=E2=80=99ll hear = =E2=80=9Cdiffserv required=E2=80=9D for an ISP, but we=E2=80=99ll = see.

Could you convert that DB to eBPF rules?  This would let = you use the same configuration interface as CoverFire's = situation.

That should be no problem at all.

In general, I think the make-wifi-fast team has a better = handle on the subtleties of half-duplex links.  If there's any way = to get their work into the airOS devices, so much the better.

I = wish that were the case, but am not holding my breath. As things stand = now, I may still use HTB then, which is ok. The key is not just the = dynamic rates, but also, it appears, having egress and ingress through a = common IFB that=E2=80=99s split into sub-queues for egress and ingress. = I=E2=80=99ll punt on WiFi for now and come back to it = later.

But I think there's also a use = for "ISP-type" Cake in your network, especially in the wired backhauls = where link bandwidth is relatively predictable, and I suspect most of = these will be in the core parts of your network where the traffic is = most complex.  As long as you can get around to running a suitable = kernel version, there should also be no inherent problem with = replicating the same dynamic adjustment of global shaper rate as = "normal" Cake has, which will allow it to be used on some of your = wireless links as well.

Yes, the biggest benefit for us would = probably be =E2=80=9Cmember fairness=E2=80=9D instead of =E2=80=9CIP = fairness=E2=80=9D. One of the long-time admins was excited that this = might be possible. Also, I=E2=80=99ll be interested to see if improving = queue management gets us better utilization of the Internet link. If so, = we may want fairness at the gateway, which may be a job more for an ISP = Cake given the number of flows.

Lastly, if ISP Cake could support SMP, = that would be all the better for the dual and quad core APU backhaul = routers. :)

= --Apple-Mail=_60E0DC6D-8029-40DF-836A-940107C6020E--