From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-il1-x134.google.com (mail-il1-x134.google.com [IPv6:2607:f8b0:4864:20::134]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 315703B2A4 for ; Wed, 3 Mar 2021 21:51:40 -0500 (EST) Received: by mail-il1-x134.google.com with SMTP id c10so23464705ilo.8 for ; Wed, 03 Mar 2021 18:51:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=/wtKKRCyBnqETllX3GtlOVYteU34MFlydiYKBLc7A7w=; b=mz9pciv6ihmAYn6rbTojjoOe7t92Q1np+EwMDRlYrV8KrohqYPgRsK1TSwvAWzOkr0 EMrJTuDW/uwTrgMTMz/7NTKjV2ljr9OxSn4Spq7uJrfJZmzmn4HuWwUssz4r8E6uEiNX PYdUCxH6L8USHVNnOIQwp4EbD/+cyuWllpO4DA1ugQNhflD3lTpEQav748AmI6RjTd9b TFqcQxZ3B1wnZi2nVxBq/u8aSUwqpe6w+nOY7T2/NlDsRfb9XHmTtS6ruigsUI68ntNp 9QEO73JYXv3DLX5YiWCSSSWqWCG9GxdQ9EiMCLeuWW7M170cUb+A51oZ5WRgp8rM2T18 ttjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=/wtKKRCyBnqETllX3GtlOVYteU34MFlydiYKBLc7A7w=; b=Vw0yq3H7k7hc15pqqZymvyK8Lw8hft539oQcZqk0l8bFGYX1/Cv7vI1v0lV2zLI1N+ J4ICwZshscV+ZU8DLB5LLNqu03NkkzFBzfT2Vqu5eppEnGy6owN6gqzBtw08f7XCNbrJ iorxxLICx/5oEQA8EdBvNMfg3s8fle7Cz+O5hqfs0rxKUIVWCkaaUz27hJ+dwZ+u7/+a U3gdn9VsMY7mXC22DJt6gZAwcVrmAYjdqXpbtP4KwSstfaqomAVSzkG0TU4GR9W8ww+2 1bz6KcqbDSJVirPg/vNxnlec53tLHg0ZgwpyeaaH5T/0cQdlxKnHFVPnvdLjWSO7nJ9w l11w== X-Gm-Message-State: AOAM532OKKLTBZaRHSaaVvqvgs7DCtDwWSbGM0y9ye690iOBGpdzFB56 Fygl6rLpDCKugbFhB++Hz7q0C0Ffp2nlJ07V0aE= X-Google-Smtp-Source: ABdhPJwjEN9krEc4pq4zmd/Ip1X5JaUgm5SX5se0O5J526dtuDA/xc1O/z/MJITwNOxXrGHwzgpl36snRdwvJoho1Vg= X-Received: by 2002:a05:6e02:1d85:: with SMTP id h5mr2280800ila.246.1614826299657; Wed, 03 Mar 2021 18:51:39 -0800 (PST) MIME-Version: 1.0 References: <9C791D21-7FC9-425E-9167-EC7660BF38F9@gmail.com> In-Reply-To: <9C791D21-7FC9-425E-9167-EC7660BF38F9@gmail.com> From: Dave Taht Date: Wed, 3 Mar 2021 18:51:28 -0800 Message-ID: To: Jonathan Morton Cc: Thomas Croghan , Cake List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] ISP Implementation X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Mar 2021 02:51:40 -0000 recently there was a thread on another bufferbloat list about a very interesting ISP approach using massively hashed tc filters + fq_codel or cake that has code in github. I cannot for the life of me remember the name of the thread or the github right. On Wed, Mar 3, 2021 at 6:47 PM Jonathan Morton wrot= e: > > > On 4 Mar, 2021, at 3:54 am, Thomas Croghan wr= ote: > > > > So, a beta of Mikrotik's RouterOS was released some time ago which fina= lly has Cake built into it. > > > > In testing everything seems to be working, I just am coming up with som= e questions that I haven't been able to answer. > > > > Should there be any special considerations when Cake is being used in a= setting where it's by far the most significant limiting factor to a connec= tion? For example: --10 Gbps Fiber -- --10 Gbps Fib= er -- [ISP Switch] -- 1 Gbps Fiber -- <500 Mbps Customer> > > In this situation very frequently the "" could be running C= ake and do the bandwidth limiting of the customer down to 1/2 (or even less= ) of the physical connectivity. A lot of the conversations here revolve aro= und Cake being set up just below the Bandwidth limits of the ISP, but that'= s not really going to be the case in a lot of the ISP world. > > There shouldn't be any problems with that. Indeed, Cake is *best* used a= s the bottleneck inducer with effectively unlimited inbound bandwidth, as i= s typically the case when debloating a customer's upstream link at the CPE.= In my own setup, I currently have GigE LAN feeding into a 2Mbps Cake inst= ance in that direction, to deal with a decidedly variable LTE last-mile; th= is is good enough to permit reliable videoconferencing. > > All you should ned to do here is to filter each subscriber's traffic into= a separate Cake instance, configured to the appropriate rate, and ensure t= hat the underlying hardware has enough throughput to keep up. > > > Another question would be based on the above: > > > > How well does Cake do with stacking instances? In some cases our above = example could look more like this: -- [Some sort of limitation t= o 100 Mbps] -- -- 1 Gbps connection- <25 Mbps Customer X 10> > > > > In this situation, would it be helpful to Cake to have a "Parent Queue"= that limits the total throughput of all customer traffic to 99-100 Mbps th= en "Child Queues" that respectively limit customers to their 25 Mbps? Or wo= uld it be better to just setup each customer Queue at their limit and let C= ake handle the times when the oversubscription has reared it's ugly head? > > Cake is not specifically designed to handle this case. It is designed ar= ound the assumption that there is one bottleneck link to manage, though the= re may be several hosts who have equal rights to use as much of it as is av= ailable. Ideally you would put one Cake or fq_codel instance immediately u= pstream of every link that may become saturated; in practice you might not = have access to do so. > > With that said, for the above topology you could use an ingress Cake inst= ance to manage the backhaul bottleneck (using the "dual-dsthost" mode to mo= re-or-less fairly share this bandwidth between subscribers), then a per-sub= scriber array of Cake instances on egress to handle that side, as above. I= n the reverse direction you could invert this, with a per-subscriber tree o= n ingress and a backhaul-generic instance (using "dual-srchost" mode) on eg= ress. The actual location where queuing and ECN marking occurs would shift= dynamically depending on where the limit exists, and that can be monitored= via the qdisc stats. > > This sort of question has come up before, which sort-of suggests that the= re's room for a qdisc specifically designed for this family of use cases. = Indeed I think HTB is designed with stuff like this in mind, though it uses= markedly inferior shaping algorithms. At this precise moment I'm occupied= with the upcoming IETF (and my current project, Some Congestion Experience= d), but there is a possibility I could adapt some of Cake's technology to a= HTB-like structure, later on. > > - Jonathan Morton > > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake --=20 "For a successful technology, reality must take precedence over public relations, for Mother Nature cannot be fooled" - Richard Feynman dave@taht.net CTO, TekLibre, LLC Tel: 1-831-435-0729