From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x441.google.com (mail-wr1-x441.google.com [IPv6:2a00:1450:4864:20::441]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 68B823B29E for ; Mon, 30 Jul 2018 06:55:33 -0400 (EDT) Received: by mail-wr1-x441.google.com with SMTP id f12-v6so2451530wrv.12 for ; Mon, 30 Jul 2018 03:55:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=e6sPIwhMXx30x2OEvYEmxqlu8SGRgFyGVnU9IB2I794=; b=anQJln9QpDgS8+nJW/qe+BqAUCqL4OYo6S7LgouYkg71cWCR0oVZt85a+/YeTTHIpp fuwrMAX7o6kA7kmJys0rN6kMBaIt2oq1nFdkvdqrWbGJkob/Br0Z9FG6SsawGkMGjbbU R2wbH9c42h/cKNvCUXw1YQOdXtWZBwfGB1uIvw3j4gkX3z32Kvp6Z75qXV7WkgvsuZLa MV39f6UoC3M4b3ugGz5RL2zn0Yv06cjscHsdp4/mEAi+Y+2Lmur4uunHChheOUriepDS aG4/gsExvgdjSsIVJkwvFi+sA47XX+gwAM3u7pONd7WPQ/BN9IoGOn3vnRov3J3LPkwt 8vTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=e6sPIwhMXx30x2OEvYEmxqlu8SGRgFyGVnU9IB2I794=; b=n062kU8XMaWyAoQVGxBZghLDnSuFFtigh7Vzg6Paeff7tRlrwHSnCAFhvOjEzHKQPc LaUD6tcF11f1wXRXwHBbsft94RyMf3t2cilPjtoHUnggeYdOcZ6db6YNLvX5+KHaKI4j clyYkbVd3IDXlnnQnEgrU3MLlQD4TWEyW9a0Ucybio0ZAiPI4y2mPkyHZWs+H63h3Zj1 INJTakVlt+gehHdlOEeAnooT3eVYMysze4gItgW/UxZ7EatUeDwWUQ5MEraFGMxwzVd8 fupIyYsUz4dFW7rSlbE2GI0XdM7ChobuS6bhHC10XTwWkoJes7E9ou/7EpRJpuYXjd8P d8wA== X-Gm-Message-State: AOUpUlE84QKxsPRwbMyoZJ0HAkjX48VYG95q/kOCPk22d3HPFtF5jhHU zhOwg89jQiRF37jyo5g4LqjJgkDD2hA= X-Google-Smtp-Source: AAOMgpfD8pRAXcoJGqoXeySL0oV9jqxXUZJckH8OIDy+A23o/mGCQB0MkITZNNNX8R1KUbi1r1WnKQ== X-Received: by 2002:adf:e405:: with SMTP id g5-v6mr17669025wrm.96.1532948132431; Mon, 30 Jul 2018 03:55:32 -0700 (PDT) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id q11-v6sm11378392wrp.58.2018.07.30.03.55.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Jul 2018 03:55:31 -0700 (PDT) From: Pete Heist Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_6A08EC8F-DF61-4B08-AF7F-3EA73D1182D2" Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Date: Mon, 30 Jul 2018 12:55:30 +0200 In-Reply-To: <8893BD36-72A4-4FC8-A013-BBB97F0E879B@gmx.de> Cc: =?utf-8?Q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Cake List To: Sebastian Moeller References: <1357421162.31089.1531812291583@webmail.strato.de> <1c323544b3076c0ab31b887d6113f25f572e41ae.camel@coverfire.com> <87woth28rw.fsf@toke.dk> <87tvol1z6h.fsf@toke.dk> <87wotfzql8.fsf@toke.dk> <87r2jlzwew.fsf@toke.dk> <305C8083-A170-42E8-BC96-120D3B0065EA@heistp.net> <8893BD36-72A4-4FC8-A013-BBB97F0E879B@gmx.de> X-Mailer: Apple Mail (2.3445.9.1) Subject: Re: [Cake] =?utf-8?q?Using_cake_to_shape_1000=E2=80=99s_of_users=2E?= X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 10:55:33 -0000 --Apple-Mail=_6A08EC8F-DF61-4B08-AF7F-3EA73D1182D2 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jul 30, 2018, at 12:09 PM, Sebastian Moeller = wrote: >>>=20 >>> You could assign more than one queue per customer and hash traffic >>> between them in BPF=E2=80=A6 >>=20 >> True. There will always be that limit of 1024 (in my case I=E2=80=99ll = need 800). >=20 > I believe all you needto do is change the following in scg_cake.c: > #define CAKE_QUEUES (1024) >=20 > Now I heard reports that above a certain number this breaks, but it = might be enough for 32K or even 64k, or ~32 to 64 queues per customer, = which might already help to spread out things a bit... That=E2=80=99s true, I could try it. I could also add to the classifier = the ability to spread one subscriber across multiple queues, as Toke = suggested. You=E2=80=99d have to specify how many queues you=E2=80=99ve = compiled Cake with (defaults to 1024), and it would spread subscribers = across the available classids evenly. I suppose the userspace tool is = right place to do this even in ISP Cake, as updates can be made pretty = easily to a global BPF map without restarting the qdisc, and further = customizations are possible for those writing their own eBPF without = qdisc changes. We=E2=80=99ll eventually want to handle changing the number of queues in = ISP Cake somehow so people don=E2=80=99t have to change and re-compile. = Should become a tc parameter instead of a define? I guess it comes back = to the question of =E2=80=9Cnew qdisc or not=E2=80=9D=E2=80=A6 --Apple-Mail=_6A08EC8F-DF61-4B08-AF7F-3EA73D1182D2 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On Jul 30, 2018, at 12:09 PM, Sebastian Moeller <moeller0@gmx.de> = wrote:

You could assign more than one queue per = customer and hash traffic
between them in BPF=E2=80=A6

True. There will always be that = limit of 1024 (in my case I=E2=80=99ll need 800).

I believe all you needto do is change the following in = scg_cake.c:
#define = CAKE_QUEUES (1024)

Now I heard reports that above a certain number this breaks, = but it might be enough for 32K or even 64k, or ~32 to 64 queues per = customer, which might already help to spread out things a = bit...

That=E2=80=99s true, I could try it. I could also add to the = classifier the ability to spread one subscriber across multiple queues, = as Toke suggested. You=E2=80=99d have to specify how many queues = you=E2=80=99ve compiled Cake with (defaults to 1024), and it would = spread subscribers across the available classids evenly. I suppose the = userspace tool is right place to do this even in ISP Cake, as updates = can be made pretty easily to a global BPF map without restarting the = qdisc, and further customizations are possible for those writing their = own eBPF without qdisc changes.

We=E2=80=99ll eventually want to handle = changing the number of queues in ISP Cake somehow so people don=E2=80=99t = have to change and re-compile. Should become a tc parameter instead of a = define? I guess it comes back to the question of =E2=80=9Cnew qdisc or = not=E2=80=9D=E2=80=A6

= --Apple-Mail=_6A08EC8F-DF61-4B08-AF7F-3EA73D1182D2--