From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-il1-x129.google.com (mail-il1-x129.google.com [IPv6:2607:f8b0:4864:20::129]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 40AC83CB35 for ; Mon, 17 Feb 2020 13:21:55 -0500 (EST) Received: by mail-il1-x129.google.com with SMTP id p8so15026172iln.12 for ; Mon, 17 Feb 2020 10:21:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :content-transfer-encoding; bh=r1xdg7F2vSVGPg7HQu0O62W+qSN/TIQawgH/dIpkpKY=; b=RuMeYZ4XaHf5W0Dp9Y+0N/7k01Yq/nsF04VJE3A0sh7n3+Feum97ubB8wMQ1mv8CQe MV91IzKQDiYeQZ43nukFn1NgRRp76Mrzir/pjxLpHftUh5d+eVs4y63t04Xwondlft4m gxovp0IrMogjGV5Y91vYUxyozPqKZyw2zkfpT7bY2/jlDtzIQno5X0Kb1tjptxNsib8f 7wkLLlWPIx5PVqios6A09V5um4s+RP8MqQmHaDj8Bt1MFIIwHobHPQqOz1NU37X0sSSM xtCk+Vptvgo1dqPVYMRTy6lEyz9kmNMDulB8gGZ38605PR4pujIsY8sBHhWBKlzNCD83 cviQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:content-transfer-encoding; bh=r1xdg7F2vSVGPg7HQu0O62W+qSN/TIQawgH/dIpkpKY=; b=g5GdgRyI/vZmhEfGywYrBI2FZibLJcqfFQbiLS8L3YruaAEXtkvacEPZy07Gh4ROQT NjceqJBxTMnhX9ZpzXrtCjuoKsaAXzFEtUBHMTsrngC+7qNzwHhWxtKULkQw4mTNB4DG IbxPruX9ZAK+O9tT7TDtZDFhjGwT2+NPspCS9mginclnSQDBofjFQ3/B8qlt1k1K5l2z JYI78+ejNQAlt+uBBZnVdlk9NGyCtmchyrtBJRmKEuvkEjZxmwgGX+8m2p9ab1PJE1Ud FbyfFEBTmMD3eqCWPUvqN4lc48aPh+lt/2TTFktl2+zT95ieBWgQmAxCwGv1Yarc29D2 nq6w== X-Gm-Message-State: APjAAAVYlSafKDhvAIbepRSwPF7j+OFG1hTOqaPR06Lgey0hBlwVHxy/ swcJx9htkXT5eGoajoyVL3NhcuctHoZumvM6T+o= X-Google-Smtp-Source: APXvYqyYG2YbZ0gY4dYfAFNfookab7shacZYVBx/gBaqp07/ktoXHdrT8ZngjORJxbxmvoxA8+hjw/iqPBeF0sKlV8M= X-Received: by 2002:a92:3a95:: with SMTP id i21mr15562138ilf.249.1581963714661; Mon, 17 Feb 2020 10:21:54 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Dave Taht Date: Mon, 17 Feb 2020 10:21:46 -0800 Message-ID: To: Mike , Cake List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] Large number of Flows X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Feb 2020 18:21:55 -0000 On Mon, Feb 17, 2020 at 7:52 AM Mike wrote: > > So 1024 is the max queues that it supports as is, so if I had 1500 users = with their own traffic shaping setup per user it would be unsupported witho= ut recompiling the kernel? Is there a command to see how many is used and = available? I think we are starting from two very different reference points for what we can accomplish. Cake's primary use case is for the CPE, or customer edge router shaping egress and ingress to suit the customers needs, and usually below the ISPs (badly) shaped rate. Each cake instance only does one bandwidth, with that 1024 queue default. If you are looking to have a bandwidth per subscriber managed centrally, It is certainly feasible and desirable to use cake from the ISP premise. How I do it is I have one virtual interface per "subscriber", managed by a route table (e.g. ip route 192.168.1.99 via dev cust1; ip -6 route aa::bb:cc::/48 via dev cust1) to which I add the cake instance, bandwidth parameter, etc Others like preseem do things like a transparent bridge in between the switch and the edge (dsl bras, etc You can setup an htb or drr + one instance of cake per subscriber if you li= ke. > Also I saw on the fq_codel page they talk about issues with cores and net= em but Cake doesn=E2=80=99t seem to use netem to delay packets etc based on= the man page, so is the core issue still a factor? Depends on your requirements. htb + anything or cake tend to lock the processing to a single core. It doesn't in the case I describe above, but I've not tried to push it past 10Gbit. > > > On February 17, 2020 at 9:34:59 AM, Dave Taht (dave.taht@gmail.com) wrote= : > > fq_codel, Cake etc, supports an infinite number of flows. > > It has a limited number of "queues" that can get mapped to flows, but > it's usually ok if a collision happens. > > The 1024 queue tradeoff is based on the observation that usually a max > of a few hundred active flows exist, and furthermore, > excessive fair queueing tends to defeat the purpose of the aqm of > keeping overall flow lengths short. Collisions of two fat flows are > rare. > > You can recompile cake with more queues if you like (fq_codel has a > soft limit of 64k queues). We don't have much data on 10GigE+ > behaviors. It was kind > of my assumption more queues would help in the 40GigE world, but > that's usually got hardware mq (64 or more), and what I'm seeing there > is 64 default fq_codel instances, 64k > queues essentially, and I think that's WAY too much.... > > > On Mon, Feb 17, 2020 at 6:07 AM Mike wrote: > > > > Will cake support a large number of flows like over a thousand per linu= x box without any modifications. I did see that there was a qdisc issue for= fq_codel on a large scale. We would be using linux kernel 4.19 which has c= ake already in it. Any help or issues that might be encountered in scaling = would be appreciated. > > > > > > > > Thanks > > Mike Thompson > > > > _______________________________________________ > > Cake mailing list > > Cake@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/cake > > > > -- > Make Music, Not War > > Dave T=C3=A4ht > CTO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-831-435-0729 --=20 Make Music, Not War Dave T=C3=A4ht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-435-0729