From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id C463A3B2A4 for ; Wed, 27 Jan 2021 13:25:46 -0500 (EST) Received: by mail-io1-xd2b.google.com with SMTP id u8so2826671ior.13 for ; Wed, 27 Jan 2021 10:25:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=content-transfer-encoding:from:mime-version:subject:date:message-id :references:cc:in-reply-to:to; bh=jyTxGl5cJINU6EiVbfkV7dddqbS7KFptVKNamtu0llU=; b=iaK68dLjAzrR/a2cOsb1OtNVRtxCAvekSfH/FJlxal0DSFwouBPjJEZ0DCimuebQWq xJk3sev4JJjCykVPRXs/KLzbNR55V3eK0Nb2DJ2aKlA3U+RFqFISeuc4TsmuBHtA1aaL LgGNkKGwlbTmstq12PWMseME+IZ3yEkI3yFZOgSKGDoUuOLxzIKM+AQnVwgakTmYJK1H P05vYhmGHzab6KftnD+YBcf3A0+IhAJLDseCTe6dcpnZmRU0BtGLLD6Pfi+hF2/Zk293 Acdd/zXSb9pANNooyaW6CohW0Fhgd+M3J5OWIHn9hSuOuSCxgnlmgAh7e1Q2HXAOpHcA 3LmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:date:message-id:references:cc:in-reply-to:to; bh=jyTxGl5cJINU6EiVbfkV7dddqbS7KFptVKNamtu0llU=; b=FMZsqdRgEy4xYwY0chx5AYZieJNFKczo2aXnLWW9vujHSbnQS6ShCD6qr0JNsiRoTD h/vh9yYtonz5gkVdTx86JmN8RVkjgDsnqdAFZRIcj6Wy/mHzlZ6+FYtkyCvweyGeEHS4 /zlBchvHl1lkyoz3zB5kaihVCnvIPEPi1WlLIX3GnF/WjvIAXxnH76itOLqsaPTp/aZD QSwtF7Z70zROjqXSH4gDgU+4WUAZNV8S33ZIm1i5QSC24zfPKVJkIg/svanS7ghow9HJ V05uESZWVbjffiUVAA7tM7jBW2WTTteQs0evgz+mPl5yC/WeZIymJH48mKpiURLe0upd KVpw== X-Gm-Message-State: AOAM53206GmPu3Sz6GslATl7IB74/6UePgP/128gYUArYJgvYtVCGghw bj7gcZHHYYAAzmRAQZmmz44HM9VIvNI= X-Google-Smtp-Source: ABdhPJwyTSynhsVxNRIvrRUAFR6dFs8MaPuIuiFmogC5rcxpuVkpH05vdqs9qIkOToUqeSposOKFKQ== X-Received: by 2002:a5e:990e:: with SMTP id t14mr8674882ioj.7.1611771945784; Wed, 27 Jan 2021 10:25:45 -0800 (PST) Received: from [192.168.2.45] ([75.9.254.211]) by smtp.gmail.com with ESMTPSA id m15sm1414813ilh.6.2021.01.27.10.25.44 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 27 Jan 2021 10:25:45 -0800 (PST) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: Jeremy Marks Mime-Version: 1.0 (1.0) Date: Wed, 27 Jan 2021 12:25:43 -0600 Message-Id: References: Cc: Cake List In-Reply-To: To: Dave Taht X-Mailer: iPhone Mail (18C66) Subject: Re: [Cake] Trouble getting decent performance with CAKE X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jan 2021 18:25:46 -0000 I think the redirect command (in that alternate code block that I didn't use= ) is from older code, so it's no longer needed.=20 I will reimplement CAKE, adding the mirred filter.=20 Do you think I can safely remove the imq0 interface? The stats (thank you!) s= uggest to me that it's not being used. Removing may not be necessary anyway.= =20 After the bandwidth is not being consumed by others behind this router, I wi= ll run dsl reports and flent with and without CAKE, and report back. I really appreciate the help.=20 On Jan 27, 2021, at 11:51 AM, Dave Taht wrote: =EF=BB=BFYou need the tc mirred to get stuff into ifb0. I don't know what th= e redirect command you are using does. tc -s qdisc will show stats for the cake instance if that was all you need on this platform. On Wed, Jan 27, 2021 at 9:49 AM wrote: >=20 > Thanks so much for getting back to me. > First, I am not so expert in these things, so please excuse my questions a= bout providing the info you have requested and implementing your suggestions= . >=20 > 1. I don=E2=80=99t know what imq0 is used for, either. > I don't have a script to run CAKE at system start because I haven't go= t it working well enough. So, at reboot, the out-of-the box qdiscs are: >=20 > qdisc ingress ffff: dev eth0 parent ffff:fff1 ---------------- > qdisc pfifo_fast 0: dev imq0 root refcnt 2 bands 3 priomap 1 2 2 2= 1 2 0 0 1 1 1 1 1 1 1 1 > qdisc pfifo_fast 0: dev ifb0 root refcnt 2 bands 3 priomap 1 2 2 2= 1 2 0 0 1 1 1 1 1 1 1 1 >=20 > You can see that the imq0 device is present at startup. A quick search= suggests that Ubiquiti uses this for native QOS during egress. If it's not c= lear what it's doing and if it's interfering, I will happily remove it, if I= knew how. The worst that happens is that the system crashes and I have to r= eboot. >=20 > 2. At startup (so no CAKE), sudo tc qdisc -s show: >=20 > qdisc ingress ffff: dev eth0 parent ffff:fff1 ---------------- > Sent 2055477419 bytes 2270158 pkt (dropped 0, overlimits 0 requeue= s 0) > backlog 0b 0p requeues 0 >=20 > qdisc pfifo_fast 0: dev imq0 root refcnt 2 bands 3 priomap 1 2 2 2= 1 2 0 0 1 1 1 1 1 1 1 1 > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 >=20 > qdisc pfifo_fast 0: dev ifb0 root refcnt 2 bands 3 priomap 1 2 2 2= 1 2 0 0 1 1 1 1 1 1 1 1 > Sent 2087261885 bytes 2270207 pkt (dropped 0, overlimits 0 requeue= s 1) > backlog 0b 0p requeues 1 >=20 > 3. I set up CAKE with the following commands: > sudo tc qdisc add dev eth0 root handle 1:0 cake bandwidth...[switch= es] >=20 > sudo set interfaces input ifb0 > sudo set interfaces ethernet eth0 redirect ifb0 > sudo tc qdisc add dev ifb0 root handle 2:0 cake bandwidth...[switch= es] >=20 > Do these commands make CAKE the default, or can I choose a qdisc to us= e when running flent? If so, how do I do this? >=20 > 4. Regarding your suggestion: "the download setting up tc mirred I assume y= ou are doing with sqm? Do that." >=20 > This is outside my knowledge, and I have not done anything like that. > I have read an alternative (older?) way to set up CAKE for ingress: > sudo ip link add name ifb4eth0 type ifb > sudo tc qdisc del dev eth0 ingress > sudo tc qdisc add dev eth0 handle ffff: ingress > sudo tc qdisc del dev ifb4eth0 root > sudo tc qdisc add dev ifb4eth0 root cake bandwidth... > sudo ifconfig ifb4eth1 up > sudo tc filter add dev eth1 parent ffff: protocol all prio 10 u32 m= atch u32 0 0 flowid 1:1 action mirred egress redirect dev ifb4eth1 >=20 > I suspect the first 6 lines of this code achieve the same thing as the f= irst two lines of code that I used (see point 3, above). Am I right? > Is the filter that is specified in the last line what you are suggestin= g I add? >=20 > This is about as much as I can address with what you have suggested, witho= ut a super long email. >=20 > Please feel free to correct my mistakes in understanding. > If you have suggestions based on what I have written, I would so grateful.= I am really looking forward to hearing what you think. > This learning curve is a little steep. >=20 > Many thanks. >=20 >=20 > -----Original Message----- > From: Dave Taht > Sent: Tuesday, January 26, 2021 11:19 PM > To: jmarks2@gmail.com > Cc: Cake List > Subject: Re: [Cake] Trouble getting decent performance with CAKE >=20 > As much as I like to get the fustest with the mostest with the rrul test, i= t is often easier to test up and down separately. >=20 > do : >=20 > tc qdisc delete ifb0 root > tc qdisc delete eth0 root >=20 > I am not sure what imq is being used for.... >=20 > Can you share the output of tc -s >=20 > instead? >=20 > a simple upload test would then be: >=20 > (unless you made cake the default) >=20 > flent -t pfifo_fast -H wherever --socket-stats -x --te=3Dupload_streams=3D= 4 tcp_nup >=20 > tc qdisc add dev eth0 root cake all_your_options >=20 > flent -t cake -H wherever --socket-stats -x --te=3Dupload_streams=3D4 tcp_= nup >=20 > the download setting up tc mirred I assume you are doing with sqm? Do that= . >=20 > flent -t -H wherever --socket-stats -x --te=3Ddownload_streams=3D4 tcp_ndo= wn >=20 > then send tc -s >=20 > It's entirely feasible your so-called 100mbit service is running at far fa= r less than 100mbit in the first place. >=20 > Another thing to check is if your modem is connecting to the device at 100= Mbit or a gbit. SOME - not enough - dsl modems actually connect at 100Mbit a= nd/or joy of joys excert hw flow control. > 95Mbit is gonna be slightly too high in former case for cake to control th= ings, try 92mbit, or less. >=20 > On Tue, Jan 26, 2021 at 4:22 PM wrote: >>=20 >> Dear CAKE experts, >>=20 >> I write for your help in getting CAKE to perform on my router as I unders= tand it should. >>=20 >>=20 >>=20 >> I replaced my aging Asus home router with a Ubiquiti Edgerouter 4 specifi= cally so I could take advantage of the port of CAKE to it. >>=20 >>=20 >>=20 >> I am in the US, with VDSL2 service provided by AT&T at 100/20. I now have= all unfiltered traffic delivered by the AT&T modem to the eth0 port on the r= outer. >>=20 >>=20 >>=20 >> My desktop runs Windows. So, to take advantage of flent to quantify CAKE p= erformance, I stood up a Linux Mint VM and ran flent rrul tests before CAKE i= nstallation, and after. >>=20 >>=20 >>=20 >> The first attached datafile, labeled qdisc pfifo fast, is the performance= before CAKE. The other file, called 20/95 ack filtered bridged ptm, is my l= atest attempt with configuring CAKE flags. >>=20 >> My major issue is with Ingress. Egress seems fine. >>=20 >> For ingress, I see very little difference between performances with and w= ithout CAKE. And the overall throughput is far less than 100 mbit. >>=20 >>=20 >>=20 >> The queuing disciplines on the router are: >>=20 >>=20 >>=20 >> jeremymarks@ubnt:~$ sudo tc qdisc show >>=20 >>=20 >>=20 >> qdisc cake 1: dev eth0 root refcnt 2 bandwidth 20Mbit besteffort >> dual-srchost nat nowash ack-filter split-gso rtt 100.0ms ptm overhead >> 22 no-sce >>=20 >> qdisc ingress ffff: dev eth0 parent ffff:fff1 ---------------- >>=20 >> qdisc pfifo_fast 0: dev imq0 root refcnt 2 bands 3 priomap 1 2 2 2 1 >> 2 0 0 1 1 1 1 1 1 1 1 >>=20 >> qdisc cake 2: dev ifb0 root refcnt 2 bandwidth 95Mbit besteffort >> dual-dsthost nat nowash ingress ack-filter split-gso rtt 100.0ms ptm >> overhead 22 no-sce >>=20 >>=20 >>=20 >> I must be making a major mistake in configuring CAKE. >>=20 >>=20 >>=20 >> I would be grateful for any help or advice in getting this amazing progra= m to work as it should. >>=20 >>=20 >>=20 >> Many thanks for any help. >>=20 >> Jeremy >>=20 >>=20 >>=20 >>=20 >>=20 >> _______________________________________________ >> Cake mailing list >> Cake@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cake >=20 >=20 >=20 > -- > "For a successful technology, reality must take precedence over public rel= ations, for Mother Nature cannot be fooled" - Richard Feynman >=20 > dave@taht.net CTO, TekLibre, LLC Tel: 1-831-435-0729 >=20 --=20 "For a successful technology, reality must take precedence over public relations, for Mother Nature cannot be fooled" - Richard Feynman dave@taht.net CTO, TekLibre, LLC Tel: 1-831-435-0729