From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 679E03B29E for ; Sat, 28 Jul 2018 03:18:51 -0400 (EDT) Received: by mail-wm0-x229.google.com with SMTP id y9-v6so7876905wma.5 for ; Sat, 28 Jul 2018 00:18:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=c5kwgjSgRmdZnoszPwbg9ux8rDWe6mej5OO0M5NDF7w=; b=FvVnFYLddQM8k3ly2IYcL1aS8H5OeXYp3n9JhiBhgt4MRaJzbQh7824cuJ4oJK2ZNX cOwcFYLhWk4QD7Awnh+j2+C8BlISYm2iFLRtDZWX+sJ//rX2kyyuG6tg1RdHy5xoYD58 YjK8BrFdogpCxLsKa/hTSv+2bcmCNG88yQKQ010wxyE6F1ERCDMgt9HZPa6RKCOs8UGE Ac5AneW2bvp1+dDZoD/h1yDnCfJfUpe3TeKVcXnEsWFscQyT9jBRgisJ7ZnWh5WKMKd+ GPPJ9YFRB8KBYUNUKwaIATh5EUknNRkLUFG643tuFJ0VrmaK8XiPmf8hkkccpo+961DD HC/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=c5kwgjSgRmdZnoszPwbg9ux8rDWe6mej5OO0M5NDF7w=; b=U4634ZLAG6JlVG2VzJx4mqHScoHO6OoT6A9oeqOaYHmwll5Ex1LTnB7rz9BQwvf//K 81cPdWp9W6oPlNVO9bbgWXToKFX1sjb2gRPIWU/Meg1LKzf46NZezQLcPtAbf8wiNZ0k h9R0gplSUTL9ZtUuWMzHT06O1MBwAcM+X8QrkWM4XMqwL/FUSEv8r+yJ+k7dXFE8mch9 ZLBsrGCIhFVx+igokNy1SjJQbRoMdYrTzZNjg3gCbKF6O5tXYYADYYeTS9b9h3t7jvtQ 8oYh5rzIvPVNFTdEVnR5wpAFvlAcm9zju9LQFAZTUkhAhGwkABBPRLc5sDmN2BBKaffg 8pAw== X-Gm-Message-State: AOUpUlEWILb9n/HphJOpOsrpUb9UfwaLVouGSP6a9ZazDjrKOpFdErND Sg/4YSuKK/NE9XarcjrCpleRbQ== X-Google-Smtp-Source: AAOMgpfKoLBRPMnEwmmUzRn93nzJRnecAuQoQ15J3EnrE086aHe1NKukd2JegxM6ih9r37a2PkKmOg== X-Received: by 2002:a1c:2d54:: with SMTP id t81-v6mr7595983wmt.31.1532762330382; Sat, 28 Jul 2018 00:18:50 -0700 (PDT) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id f18-v6sm4393303wrt.64.2018.07.28.00.18.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 28 Jul 2018 00:18:49 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Pete Heist In-Reply-To: Date: Sat, 28 Jul 2018 09:18:48 +0200 Cc: Cake List Content-Transfer-Encoding: quoted-printable Message-Id: <8980FA6C-508B-43E1-8C23-6EBC4A10499A@heistp.net> References: <1357421162.31089.1531812291583@webmail.strato.de> <1c323544b3076c0ab31b887d6113f25f572e41ae.camel@coverfire.com> <87woth28rw.fsf@toke.dk> <87tvol1z6h.fsf@toke.dk> To: Jonathan Morton X-Mailer: Apple Mail (2.3445.9.1) Subject: Re: [Cake] =?utf-8?q?Using_cake_to_shape_1000=E2=80=99s_of_users=2E?= X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jul 2018 07:18:51 -0000 > On Jul 26, 2018, at 11:38 PM, Jonathan Morton = wrote: >=20 > It would also be valuable to have a firmer handle on the actual = requirements in the field. For example, if it is feasible to focus only = on current Linux kernels, then a lot of backwards compatibility cruft = can be excised when importing existing code from Cake. One more data point, for FreeNet Liberec (WISP co-op)... There are some older backhaul routers still with 2.6.26.8(!) although = those are being phased out so don=E2=80=99t count them. More current = ones use 3.16.7 and there=E2=80=99s some discussion but I=E2=80=99m not = sure what/when the upgrade plan is. I think the Internet router uses a = more modern Debian 9 which is likely to have a 4.9 series, but I don=E2=80= =99t have access to it. FreeNet might be unusual in that administration is distributed. = Volunteer members administer backhaul routers which carry some number of = customers, average a few dozen or so. These do = routing/firewalling/monitoring and QoS, when they=E2=80=99re a = bottleneck. They use either an ancient esfq (has per-IP fairness though) = on older kernels and sfq when it=E2=80=99s not available, which is now = more common. There are arguments (heated ones, I=E2=80=99ve heard) about = centralizing administrative functions, including QoS, at or near the = Internet gateway, but I think this would have to depend on = over-provisioning the backhaul. I=E2=80=99ve volunteered to modernize = the QoS when I can. If ISP flavored Cake doesn=E2=80=99t happen, it will = end up being HTB+sfq|fq_codel|cake depending on what the kernel = supports. > If all users will have the same link-layer technology (with the same = overhead parameters), then these can be set globally - or if not, they = can be set per-tier. Afaik almost all members use WiFi, possibly some Ethernet. There are no = tiers and no per-member rate limits. > Is the Diffserv support from Cake likely to be useful, and if so how = flexible should the configuration be? =20 Currently the root qdisc on backhaul routers is a prio with "bands 3 = priomap 2 2 2 2 2 2 0 0 2 2 2 2 2 2 2 2=E2=80=9D. I don=E2=80=99t think = we=E2=80=99d want DSCP for anything, with the possible exception of = voip, and even then there might just be a special IP/port rule for the = asterisk server. > And are there only a few discrete settings for bandwidth per user, or = do we have to be more flexible to handle a BRAS environment? =20 Mainly in this case only fairness between members is needed. There=E2=80=99= s a db mapping members to their MAC addresses (usually one to one but = not always). > Is it also necessary to account per-user traffic accurately, or will = an external tool be used for that? It would be better than what is done now (counting per-IP, which when a = customer has multiple MACs makes it harder to interpret). > Many other low-level considerations can be adjusted on the fly during = the conversion, so they are not in themselves a problem. In this = category are questions like "how many simultaneous users and flows can = be supported=E2=80=9D. In this case something that works to a few thousand users is likely to = be fine (currently, 800 members). Lastly, do you think a better shaper for point-to-point WiFi is within = the scope of this project? This might be more needed for WISPs, but = here=E2=80=99s why: - If we still have bottlenecks in the backhaul, we=E2=80=99ll need to = keep doing QoS there, and for FreeNet that means soft rate limiting, = because the WiFi devices have to keep running airOS for its management = tools. - If you only rate limit on egress, you lose at least half your = available throughput. - You can run egress and ingress through a common IFB, but in my = testing, TCP RTT (rrul_be with =E2=80=94socket-stats) gets higher. - I find it works better to use HTB at the root and have HTB+cake as = leaf queues, then TCP RTT is roughly cut in half. - But it would be even better if the shaper understood an approximation = of airtime, because aggregate throughput changes based on the balance of = up/down traffic in the case where up/down rates are stable but = asymmetric, which FreeNet sometimes has. - And, I=E2=80=99d rather not have to use HTB, and be able to use the = better deficit mode shaper in Cake. I know that WiFi has many complexities where soft rate limiting can=E2=80=99= t ever be perfect without knowledge from the driver, but I still think = it could be useful to have something that does a better approximation = than what we have today. The question is, is that something that could = be part of this project, or not=E2=80=A6 :)