From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-il1-x144.google.com (mail-il1-x144.google.com [IPv6:2607:f8b0:4864:20::144]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id A1FE13B29E for ; Wed, 25 Mar 2020 15:18:44 -0400 (EDT) Received: by mail-il1-x144.google.com with SMTP id a6so3031367ilr.4 for ; Wed, 25 Mar 2020 12:18:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=bXXR2yeF4/2l+O+TDqw6L0o1j9MjDWzZ8mqZF8wwXFg=; b=h2OzzUOkmrxzmOLJbRO//jZYBxfogiGMvP1WwqeYnYqPLumRoDvCv+3SB5RLP93c+/ X7Svg+TrpNj/KYpq+Fu850uvpW6IiIDl6tgheF+eP5sTOVMcDFPji7889CF+S27+vpg2 DobglmdhtRTPqaJzCpuGcw3+XNQ3uMbf7DYkjCIj01Q/4SSJYHI/ZGFzFAVe+nKEO7mF b/63DJ2jZUY7U4SpCWU0elTHjwrgh2iCf58B746mzGnsHcQcCDlmr0ysr+X7JvlubyVt KGJTAatAhrEXs+oSKvef/dTH04REi9TG2Zkf+RyxLtQR2wqPJyQnH5CsSxfEBKEj9idA TbvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=bXXR2yeF4/2l+O+TDqw6L0o1j9MjDWzZ8mqZF8wwXFg=; b=ttg8EaDk15Oi1JR3vXAu4OPSpkeQ/H/DTGwz6Mzjne6bZvz1WFeqsyWJrywiugtLys 6Nmw/tBxTOogw9tBVlPKxq8d+uhx042iHJyTAHqHX+CgQrvAS+NPciIngvzZfwvzr0qY 9HomgkUbEnzndclxbLYHy4MoFtQ8qbgcmyEjsaPxpV+sUcFYsWbWhc9wRdgRlA8qR6BH XEg+dwcs7B4gJWldZnhshmJC3pnlGxtNIYOLZ6QBgd7tEfol5jLvCe4AiblYTW1Knelk /0lDsO391vboJbWd4b0kBYrs+9ZHa0t0NqtfVddvWEWPOK/pbH+Se2zlfjiiUCtyBP11 LEww== X-Gm-Message-State: ANhLgQ2NvF+rIcx+92sYe66J7PKMbNNyMvkhxu7YWqxPF+fZF140yMPF ypL8KlgjRTPsFu8LNeQcnHYdjNTjngCsvzNIAR4= X-Google-Smtp-Source: ADFU+vuYN56VN5m3eNv4Ly1fpMbIYE4L933qlFCGRS9ENhztMqX0DmyW4tiV488QnxkoB+yruKTAoHlKYy9/lbAzTX0= X-Received: by 2002:a92:dd0e:: with SMTP id n14mr5345053ilm.0.1585163923087; Wed, 25 Mar 2020 12:18:43 -0700 (PDT) MIME-Version: 1.0 References: <875zesret5.fsf@toke.dk> <87r1xgpuhm.fsf@toke.dk> In-Reply-To: From: Dave Taht Date: Wed, 25 Mar 2020 12:18:31 -0700 Message-ID: To: Aaron Wood Cc: =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= , bloat Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Bloat] Still seeing bloat with a DOCSIS 3.1 modem X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2020 19:18:44 -0000 On Wed, Mar 25, 2020 at 8:58 AM Aaron Wood wrote: > > One other thought I've had with this, is that the apu2 is multi-core, and= the i210 is multi-queue. > > Cake/htb aren't, iirc, setup to run on multiple cores (as the rate limite= rs then don't talk to each other). But with the correct tuple hashing in t= he i210, I _should_ be able to split things and do two cores at 500Mbps eac= h (with lots of compute left over). > > Obviously, that puts a limit on single-connection rates, but as the numbe= r of connections climb, they should more or less even out (I remember Dave = Taht showing the oddities that happen with say 4 streams and 2 cores, where= it's common to end up with 3 streams on the same core). But assuming that= the hashing function results in even sharing of streams, it should be fair= ly balanced (after plotting some binomial distributions with higher "n" val= ues). Still not perfect, especially since streams aren't likely to all be = elephants. We live with imperfect per core tcp flow behavior already. What I wanted to happen was the "list" ingress improvement to become more generally available ( I can't find the lwn link at the moment). It has. I thought that then we could express a syntax of tc qdisc add dev eth0 ingress cake-mq bandwidth whatever, and it would rock. I figured getting rid of the cost of the existing ifb and tc mirred, and having a fast path preserving each hardware queue, then using rcu to do a sloppy allocate atomic lock for shaped bandwidth and merge every ms or so might be then low-cost enough. Certainly folding everything into a single queue has a cost! I was (before money ran out) prototyping adding a shared shaper to mq at one point (no rcu, just There have been so many other things toss around (bpf?) As for load balancing better, google "RSS++", if you must. > On Wed, Mar 25, 2020 at 4:03 AM Toke H=C3=B8iland-J=C3=B8rgensen wrote: >> >> Sebastian Moeller writes: >> >> > Hi Toke, >> > >> > >> >> On Mar 25, 2020, at 09:58, Toke H=C3=B8iland-J=C3=B8rgensen wrote: >> >> >> >> Aaron Wood writes: >> >> >> >>> I recently upgraded service from 150up, 10dn Mbps to xfinity's gigab= it >> >>> (with 35Mbps up) tier, and picked up a DOCSIS 3.1 modem to go with i= t. >> >>> >> >>> Flent test results are here: >> >>> https://burntchrome.blogspot.com/2020/03/bufferbloat-with-comcast-gi= gabit-with.html >> >>> >> >>> tl/dr; 1000ms of upstream bufferbloat >> >>> >> >>> But it's DOCSIS 3.1, so why isn't PIE working? Theory: It's in DOC= SIS 3.0 >> >>> upstream mode based on the status LEDs. Hopefully it will go away i= f I can >> >>> convince it to run in DOCSIS 3.1 mode. >> >> >> >> I think that while PIE is "mandatory to implement" in DOCSIS 3.1, the >> >> ISP still has to turn it on? So maybe yelling at them will work? (ha!= ) >> >> >> >>> At the moment, however, my WRT1900AC isn't up to the task of dealing= with >> >>> these sorts of downstream rates. >> >>> >> >>> So I'm looking at the apu2, which from this post: >> >>> https://forum.openwrt.org/t/comparative-throughput-testing-including= -nat-sqm-wireguard-and-openvpn/44724 >> >>> >> >>> Will certainly get most of the way there. >> >> >> >> My Turris Omnia is doing fine on my 1Gbps connection (although that >> >> hardly suffers from bloat, so I'm not doing any shaping; did try it >> >> though, and it has no problem with running CAKE at 1Gbps). >> > >> > Well, doing local network flent RRUL stress tests indicated that >> > my omnia (at that time with TOS4/Openwrt18) only allowed up to >> > 500/500 Mbps shaping with bi directionally saturating traffic >> > with full MTU-sized packets. So I undirectional CAKE at 1Gbps >> > can work, but under full load, I did not manage that, what did I >> > wrong? >> >> Hmm, not sure I've actually done full bidirectional shaping. And trying >> it now, it does seem to be struggling... >> >> -Toke > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat -- Make Music, Not War Dave T=C3=A4ht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-435-0729