From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-x233.google.com (mail-qt0-x233.google.com [IPv6:2607:f8b0:400d:c0d::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 765D33BA8E for ; Tue, 1 May 2018 15:22:31 -0400 (EDT) Received: by mail-qt0-x233.google.com with SMTP id f13-v6so5885750qtp.10 for ; Tue, 01 May 2018 12:22:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=zS4tL+S4JEI6agwn0U+Izx0H6VihK8MuOXTMcRwzwBU=; b=UDaLMk4SFsg7VZyCmqVMDt+qS63jOJJ9sXPkalDPHwqFeFdEV2nTn9XaAlY6CUwXUe /UU3SocKSNw5Ror79ZroKe7vcSx2j3ej5ZCorjQehma1DSzL1MAxtS/rUjN+elpE2oiB rgaEirNAJOiIsJzixeylMCNZeppQkpn/M09y+fo9XuEQZgDu5vI58nPOY72sWJjY7ouk IbqkMFzv7bloPlhpTwpt/w9vxPTF2yPZVzmATfD5S7LaFNvoD6oPEVhM/lpl/oVx3tYD Yc5bkiQc27XJI6bW4DtYb2CKReJN9HhgTfN6LlZi3T9VlasYtIzvgCj4/Fjt05LaBHFY gPnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=zS4tL+S4JEI6agwn0U+Izx0H6VihK8MuOXTMcRwzwBU=; b=TXLcU7HzVfCnR5jTZJUKf3agBJ87KGO7AA+gwzxf4BARURJOCylhAQ8AnKsp4I1D0l toONhqAsBq9q4mqJ2Sxdsg7ns+SXmBsCfWQrJemPIpQPkSipgTe5GjdrzTNfrP/9//gX Kw/5QI5cCojqf7cJIZ6K8QLv/LtpxBkbbzM2bglPu/kVdMKU+dqgGbyISFwvL0oePOxu Ywue52iWJU+jSOJ/XNHmj9KvzFLdpK0K0DSQgWPvYIf9ZjJ5ODOiI1yG4HSKc/PDcyqu 8eq45DXPNfKcJXtMqOHhcJeNFoerCnW4Zgjb7D06r3zZMWk5ntPh8df5TPyo4IZi8Ba9 Gj8A== X-Gm-Message-State: ALQs6tAX1yCNogvGMM06G+2c+SjLZ81RhwEars5T4qghYRdx4/FAIS3d r8jxfmo0m2f572lRIk5PRSLINfpwqsujaCGAUvc= X-Google-Smtp-Source: AB8JxZpJvXDt1bmiwZcJVexUA0yV8VyK4ZBPiFZzkaQS4qgX7OtdXvaa1cgNpI1WptrePhRm8u3Bue9WQ7uRbpWKldc= X-Received: by 2002:aed:3df4:: with SMTP id j49-v6mr14530755qtf.110.1525202550882; Tue, 01 May 2018 12:22:30 -0700 (PDT) MIME-Version: 1.0 Received: by 10.12.243.147 with HTTP; Tue, 1 May 2018 12:22:30 -0700 (PDT) In-Reply-To: <878t932ont.fsf@toke.dk> References: <20180429213439.7389-1-toke@toke.dk> <878t932ont.fsf@toke.dk> From: Dave Taht Date: Tue, 1 May 2018 12:22:30 -0700 Message-ID: To: =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= Cc: Eric Dumazet , Cong Wang , Linux Kernel Network Developers , Cake List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] [PATCH net-next v6] Add Common Applications Kept Enhanced (cake) qdisc X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 May 2018 19:22:31 -0000 On Tue, May 1, 2018 at 11:53 AM, Toke H=C3=B8iland-J=C3=B8rgensen wrote: > Eric Dumazet writes: > >> On 04/30/2018 02:27 PM, Dave Taht wrote: >> >>> I actually have a tc - bpf based ack filter, during the development of >>> cake's ack-thinner, that I should submit one of these days. It >>> proved to be of limited use. >>> >>> Probably the biggest mistake we made is by calling this cake feature a >>> filter. It isn't. >>> >>> Maybe we should have called it a "thinner" or something like that? In >>> order to properly "thin" or "reduce" an ack stream >>> you have to have a queue to look at and some related state. TC filters >>> do not operate on queues, qdiscs do. Thus the "ack-filter" here is >>> deeply embedded into cake's flow isolation and queue structures. >> >> A feature eating packets _is_ a filter. >> >> Given that a qdisc only sees one direction, I really do not get why >> ack-filter is so desirable in a packet scheduler. > > The ACK filter in CAKE is there to solve a very particular use case: > Residential internet connections with bandwidths so asymmetric that it > hurts TCP performance. It is not on by default; but rather meant to be > configured by users which suffer from this particular ISP brokenness > (which, sadly, does happen; there are ISPs who believe a 50/1 bandwidth > ratio is reasonable). For those users, the ACK filter can help. > > We certainly do not advise people to turn it on in the general case! As > you point, in general TCP performance is best improved in the TCP stack..= . > >> You have not provided any numbers to show how useful it is to maintain >> this code > > You mean apart from that in the linked blog post and the paper? > Here's an executive summary: > > On a 30/1 Mbps connection with a bidirectional traffic test (RRUL), > turning on the ACK filter improves downstream throughput by ~20% and > upstream throughput by ~12% in conservative mode and ~40% in aggressive > mode, at the cost of ~5ms of inter-flow latency due to the increased > congestion. On a simulated 50/1 comcast connection, I got double the throughput on a similar test, with no obvious glitches in the tcp flow, in the early s= tages of development. http://blog.cerowrt.org/post/ack_filtering/ I was very, very dubious about the value of ack thinning until that point, but it was hard to argue with the data. > In *really* pathological cases, the effect can be a lot more; for > instance, the ACK filter increases the achievable downstream throughput > on a link with 100 Kbps in the upstream direction by an order of > magnitude (from ~2.5 Mbps to ~25 Mbps). > >> (probably still broken btw, considering it is changing some skb >> attributes). > > As you may have noticed over the last few iterations, I've actually been > trying to fix any brokenness. If you could be specific as to what is > still broken, that would be helpful. > > (I'm assuming are referring to the calls to skb_set_inner* - but do you > mean all of them, or just the ones that set inner =3D=3D outer?) > >> On wifi (or any half duplex medium), you might gain something by >> carefully sending ACK not too often, but ultimately this should be >> done by TCP stack, in well controlled environment [1], instead of >> middle-boxes happily playing/breaking some Internet standards. >> >> [1] TCP stack has the estimations of RTT, RWIN, throughput, and might >> be able to avoid flooding the network with acks under some conditions. > > You are quite right that in general, TCP performance is best improved in > the TCP stack. But home users are not generally in control of that; the > ACK filter helps in those specific deployments (again, it's off by > default, and you shouldn't turn it on in the general case!). > >> Say RTT is 100ms, and we receive 1 packet every 100 usec (no GRO >> aggregation) Maybe we do not really _need_ to send 5000 ack per second >> (or even 10,000 ack per second if a hole needs a repair) > > Yes, please do fix that. :) I really would like to see cake tested at 10GigE and above, and its performance improved overall. I tend to think we need more queues than 1024 at 40GigE+, and we presently run out of cpu (even unshaped) long before we hit that point. > >> Also on wifi, the queue builds in the driver queues anyway, not in the >> qdisc. > > Actually, we've fixed that (for some drivers); there is now no qdisc on > WiFi interfaces, and we've integrated FQ-CoDel'ed queueing into the > stack where it can be effective. But no, running CAKE with an ACK filter > on a WiFi link is not going to be effective. Don't do that. I share the belief with eric that thinning acks (either at the wifi qdisc o= r in tcp) on wifi interfaces will help - given that the underlying wifi layer is reliable and does retransmits, and the number of packets that can fit into a wifi aggregate limited, you really only need one ack per wifi aggregate per flow to keep the tcp connection running. That said, I'd much rather see fq_codel work with more wifi drivers than pursue this. > >> So ACK filtering, if _really_ successful, would need to be >> modularized. Heh. I keep hoping ISPs will ship symmetric links and wifi antennas > I really hope ACK filtering won't be "_really_ successful". Again, it is > not meant to be a feature that's on everywhere, it's targeting a very > specific use case that CAKE is optimised for: The home router use case. Please note that I find cake far more general purpose than just this, the ease of just slamming: tc qdisc add dev eth0 root cake bandwidth 50mbit on a link that needs it is far easier than the equivalent htb + fq_codel + other filters, and more effective. That mode is with nat on, some diffserv awareness (more correct than pfifo_fast), no link layer compensation, no ack-filter, and "just works". Certainly the major use case to date has been on home routers. Every feature in cake was based on extensive feedback from that part of the field. >> Please split Cake into a patch series. >> Presumably putting the ack-filter on a patch of its own. > > *sigh* - can do, I guess. Though I'm not sure what that is going to > accomplish, exactly? > > -Toke --=20 Dave T=C3=A4ht CEO, TekLibre, LLC http://www.teklibre.com Tel: 1-669-226-2619