From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-x22d.google.com (mail-oi0-x22d.google.com [IPv6:2607:f8b0:4003:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id C53BD3B25E for ; Wed, 1 Jun 2016 12:09:14 -0400 (EDT) Received: by mail-oi0-x22d.google.com with SMTP id k23so34962332oih.0 for ; Wed, 01 Jun 2016 09:09:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc; bh=e5TOYrdGXgYuuyqvrlXfkEFEtfmO44BfEfSw/txn2Hs=; b=bvWGyy1K+abhbnzNzV/2qaWlQwSceRHm7FybmmRjsZFu8oj6OkLswqAnqmwxch5S50 tUcqKDpjugEX49Q5Zeby3HCj/W5npS27zmHHA+RB2CFVSw6ZkJjyqpkHEYyfDPUs9K6G CovMYnsGiqtNw06ssRtvU4WGx3s9dTjLh7rCQMWv5OHGZr6mni2BHXGtsnftNlom5QHg tHYfGxglY6wTSqYVDOcsgdPg6u6S2zSN0jcYWm+rK2N1//P4iZFHB5jZzH21HBulm+pz rjLftOebIj1Mm7gJ7nCupFuuTFHOQcfGS6OgIX8osMxg4aEvbm0y4hQf7Qg/lHVKFyUQ SEqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to:cc; bh=e5TOYrdGXgYuuyqvrlXfkEFEtfmO44BfEfSw/txn2Hs=; b=R1GOpltKVbXWBWOjfG4HLiFNSkIkAdhUb9vLCsJxGVc1ei8rXwMFbnDCh76H4IZCcb T1zy2xrDKdxhp7UTOqxncFx0lmkbnZhdw6Qa1wod53jtRXMHIM7LjULDSCdJ7iItupDm OqGjhtSKE61nIjwWXCGViDT34hFH0xFOfv9K8rNME0+Fv9xf2sZwnbjE69Sn9HJH/MLQ DJjYIi7Ibyo7IU++aMmIGv2iiqm3JBalx2hgumeLGB0J8ToaXWOVPkk19gm2tD539VlM FlEHKWXMQbX7u+Sagd9Nk3lccPXUx/ZEizma7QStQ5gmDK2D7nByAnwa/2x4PEeHbbvZ t7+A== X-Gm-Message-State: ALyK8tLVZ6qjRLYfh2XZ9DtQYQ4+GCr8Ka/6CWg+16etAsFqzvmI3XYBpO+bjIiup91NqrmHQwzTCnrp3lAzHg== MIME-Version: 1.0 X-Received: by 10.202.83.82 with SMTP id h79mr17622012oib.139.1464797354126; Wed, 01 Jun 2016 09:09:14 -0700 (PDT) Received: by 10.202.229.210 with HTTP; Wed, 1 Jun 2016 09:09:14 -0700 (PDT) Date: Wed, 1 Jun 2016 09:09:14 -0700 Message-ID: From: Dave Taht To: Jonathan Morton Cc: Benjamin Cronce , cake@lists.bufferbloat.net Content-Type: text/plain; charset=UTF-8 Subject: [Cake] cake's flaws X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jun 2016 16:09:14 -0000 I just got back from vacation - and did not intend message 1 to come across as this cranky. Certainly my take on "finishing cake" was to get more users using it and providing feedback, and getting it into lede mainline will make for more users as well as make it possible for me and others to easily test again, and once it showed at least a few benefits, and perhaps grew or lost a few more features, push towards mainline into linux. I applaud this. Keep at it. No matter how grouchy I sound below, I am rooting for y'all to get it right. But: *My* principal design goal for "cake" was to pour the existing sqm-scripts into C, where it would be faster, and to scale well across a bandwidth range of 0-40Gbit+ with older and modern hardware. Everybody else here does not have a need for performance much higher than a few mbit, it seems, and that colors your viewpoints. I'd at least like to get inbound shaping to work well at 400Mbits... Cake *was* - last june - significantly faster than htb + fq_codel, enough so to do 100Mbit inbound queue management where htb+fq_codel fell over at 60Mbit on things like the wndr3800 and archer c7v2. No longer. It benched as slower when I last benched it (in december), and with incremental sub percentage point improvements or disimprovements, including many issues in the codel implementation, "presumed" fixed. It was a huge percentage slower than pfifo_fast on 10GigE and higher. At which point I gave up, went back to htb+fq_codel, and focused all my energies on building up my ability to work on wifi, where we are now showing comfortable order-of-magnitude gains and real progress. I do see many - undertested - features "improving" this that or the other thing have landed since I last paid attention. I do fear it is expected of me and toke to take cake through a serious string of tests, and my pushback has been to ask that those working on it and testing it first work with the fleet of flent servers worldwide to test the codel implementation, at least, first, and preferably have a few boxes locally to be able to test other features, or something in the cloud, perhaps leveraging mahi-mahi or some other framework. It will be easier for me to do drive-by tests of cake again once it hits lede mainline. Open issues: cake still lacks a "sqm" mode - 3 tiers of shaping - which makes it impossible to benchmark properly vs the sqm-scripts. I still see no proof that more tiers help in any way, nor any testing or proof that it (or the hfsc-fq_codel stuff that landed more recently) that it is any better. fq_codel's natural characteristics solve for VOIP just fine, in particular. 3 tiers has been enough for every other qdisc (:cough: pfifo_fast, mqprio) since the dawn of linux time. I also preferred to statically generate the parameters for each diffserv related model, saving tons of code AND resulting in shared data for it (increasingly important with hw mq) I also thought isps and some users would want a more strict prio queue model available, similar to what free.fr is using, which makes managing tv multicast easier. I think the quantum should be even more dynamic than it is today, scaling up to 3028 as sch_fq does (say, starting at 200-500mbit), and it should go back to peeling less hard. I am aware I am the one that ripped it out (in favor of testing better what we had)... I do not see any proof that the triple isolation mode for torrents does any better than the regular mode for torrents, against real torrents - or any other forms of normal multi-user traffic Somebody prove that cake's mode for this actually makes a difference, please. I thought the invsqrt cache was pointless, and most of the other tweaks to codel needed testing and evaluation, and all of them cost cpu. Register usage was poor on arm and mips architectures. I did not see a functional use for the rate estimator. I felt nearly all of the statistics collection could be dropped. In terms of API - the rate limiter does not work above 40GBit. *All* the new hardware I have played with of late does 4 or more hardware queues (on inbound and outbound), and finding ways to handle those within a single qdisc across those cpus sounds like an increasingly good idea. They are doing that for CPU efficiency, not QoS. That said, even basic support for BQL has been lacking in those arches (I'm looking at you linksys ac1200!) And I'd hoped that sane ways of leveraging cake from an ISP's perspective would emerge, which seems to involve lots more tc or iptables magic yet to be written. Conceptually I do love the idea of a set associative cache, but as for actual measurements of it's helpfulness, I have very little to show for it's benefits thus far. please keep banging the rocks together, and *please* benchmark the thing at higher rates and RTTs.