From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 3A09C3B2A4 for ; Mon, 30 Jul 2018 06:09:51 -0400 (EDT) Received: from [172.16.11.53] ([134.76.241.253]) by mail.gmx.com (mrgmx101 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MbsV8-1fQlYD32gg-00JH2W; Mon, 30 Jul 2018 12:09:39 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Sebastian Moeller In-Reply-To: <305C8083-A170-42E8-BC96-120D3B0065EA@heistp.net> Date: Mon, 30 Jul 2018 12:09:37 +0200 Cc: =?utf-8?Q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Cake List Content-Transfer-Encoding: quoted-printable Message-Id: <8893BD36-72A4-4FC8-A013-BBB97F0E879B@gmx.de> References: <1357421162.31089.1531812291583@webmail.strato.de> <1c323544b3076c0ab31b887d6113f25f572e41ae.camel@coverfire.com> <87woth28rw.fsf@toke.dk> <87tvol1z6h.fsf@toke.dk> <87wotfzql8.fsf@toke.dk> <87r2jlzwew.fsf@toke.dk> <305C8083-A170-42E8-BC96-120D3B0065EA@heistp.net> To: Pete Heist X-Mailer: Apple Mail (2.3445.9.1) X-Provags-ID: V03:K1:EfBaG3j9iGqAK7zrtZ9eBiIVM8aNhKo/ID9NCCjnn9G0H+34wli 0sl3u/6fY8qcr8BoGa5OUOaHY8o2n7ROVItpWW86xWy2dI5IZRUUGXI5ilVwwjucbkcThaO 1jYYWp2lWmkhMs9Iwd+9vA+dXEEqRScPQSZoW/TWg5iNbzMLgdvH9KhO60HgRI5FBSWnlCb HXBoJImosuk7UX/QsL0cg== X-UI-Out-Filterresults: notjunk:1;V01:K0:jOV5Yyp+F4A=:Vg+U+F0mUaMKSLWNQr1we4 9FmxskhVlzYNkUdJT92gC6Qb17fKUJMTNwLXIfeQXgZbKxgJqcPobHv13KtZGm/LcRLJ892bH SDpmnnC3HndM+7Ud0CwF4OI4HbIGCWk4OeGm2ZXuH/TzO/OFgp/bihU2VL6FVnHRMl+7W7ZaM Ws8eX4u2hEBeUiK9f5UNCpSXaDYOk3jv3L4ixwf96Iqe5+yJH00QzSwvc2hOHCqKd5Q5EwWCC hcQEf+ij5g4bVjHU04u5Aar4dx8ZfxM8Dv8BfVMnoB7qkTn/2QGzKMw4ckV/nkURC/ia+eamd ZzqHiyM7N8tyEzOoappJ72oAjTsd7D7bfh/95DOVNnlaTBEyhTSsBfrzl6PJHt2j3TmxQQpQH tCT3StJtnO1sTrBpiuR+fdOqxkpdjx2F3u1hwjtGoI84/D62ojXRmK5MlQ/FAK68Lij4I567h Pv3r1byIc34T96OEc6+LL5adUDu8uZunFdF18Ik46UorLZNu5QGj2pWZZOa4iUpmNeCWE0AY5 WmtDa4pu8x2QtncItWCerJ2eNNkFC2anXwlFHgHvqzEU7gPYcKpLyWW7zqefApLgNR/XELZGh USR1ikaVdRw8G/ZVZMDl/YxmJeSSqZ3trs0dHTVB+fLz1YzmKDTWVW/GvFMQe318tRGP3+6tt 1Y/F06sMeNZCWC80dJO7dlt61F97ZXU0EL+N9uR8K7rtWlJX25rJFdOx3KYyVprBxYK6OJ8N+ +zBzMfmWFqw0ySlydZmozl6c1h/as5X8e4vVJvb7OR6eFg1otXlshg6IeVw5eM3cozrI6rQa7 TE5r2o7 Subject: Re: [Cake] =?utf-8?q?Using_cake_to_shape_1000=E2=80=99s_of_users=2E?= X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 10:09:51 -0000 Hi Pete > On Jul 30, 2018, at 11:14, Pete Heist wrote: >=20 >=20 >> On Jul 29, 2018, at 9:14 PM, Toke H=C3=B8iland-J=C3=B8rgensen = wrote: >>>=20 >>> Caveats that I know of: >>> - Limited to 1024 members >>> - No fairness between flows >>=20 >> You could assign more than one queue per customer and hash traffic >> between them in BPF=E2=80=A6 >=20 > True. There will always be that limit of 1024 (in my case I=E2=80=99ll = need 800). I believe all you needto do is change the following in scg_cake.c: #define CAKE_QUEUES (1024) Now I heard reports that above a certain number this breaks, but it = might be enough for 32K or even 64k, or ~32 to 64 queues per customer, = which might already help to spread out things a bit... Best Regards Sebastian >=20 >>> - Non-member traffic would have to be dealt with somehow, maybe put = in >>> its own queue or split among multiple queues, otherwise there can be >>> hash collisions with member queues >>=20 >> Yeah, an "overflow queue" is definitely needed in this kind of >> deployment :) >=20 > Yep, I=E2=80=99d like to hash the non-member flows across the = remaining queues. >=20 >> I actually wrote an eBPF classifier a few months back, that can = lookup >> subnets in a BPF map and map them into different classes: >> https://github.com/tohojo/tc-classifier >=20 > Nice! That along with Dave=E2=80=99s ack classifier will help me write = one for MAC addresses. I got as far as =E2=80=9Cmy first no-panic = bpf=E2=80=9D, but for starters I wasn=E2=80=99t sure of the right way to = set classid, so I see TC_H_MAKE. The documentation one finds on BPF = varies a lot in correctness, so I messed around a while. >=20 > I think regardless of whether ISP cake is a new qdisc or changes to = the current one, it would be good to provide a common tool like this for = mapping both MACs and IP subnets. Maybe I can just expand tc-classifier = a bit for my needs and try to think of others also? Here=E2=80=99s how = it could work: >=20 > Userspace tool: > - accepts as input from stdin or file, space or comma separated = mappings of one of (MAC address, IPv4 subnet or IPv6 subnet) to both = classid (flow) and priority (tin), so three fields total > - accepts as an optional argument tin to place unclassified traffic in = (defaults to 0) > - returns an error if no queues available for unclassified traffic > - puts mappings into up to three global BPF maps (for MAC, IPv4 and = IPv6) > - puts unclassified traffic tin, if non-zero, into a global > - should lock globals here so updates can be made without removing / = re-adding qdisc >=20 > BPF filter: > - tries to classify first using MAC address map, then IPv4 or IPv6 = maps for IP traffic > - spreads any unclassified traffic in unclassified traffic tin across = remaining classids from max(classid)+1 to 1023 >=20 > Lastly, although it's natural to use classid for flow (subscriber) and = priority for tin, we have a hard maximum of 2^16 subscribers in a given = tin (minor classid is 16 bits). It doesn=E2=80=99t matter now because we = only have 1024 flows per tin, but for ISP cake, is a limit of ~2^16 = subscribers in one tin enough? Otherwise we=E2=80=99d have to change the = way we specify this. >=20 > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake