From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qc0-x231.google.com (mail-qc0-x231.google.com [IPv6:2607:f8b0:400d:c01::231]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 7BE2521F403 for ; Thu, 16 Apr 2015 06:48:54 -0700 (PDT) Received: by qcrf4 with SMTP id f4so6329094qcr.0 for ; Thu, 16 Apr 2015 06:48:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=1pz+Jyx1d2r2P74xe/0X42h+TIQ8kM8qUeU4gp0hxR4=; b=X3TFQNlLgKHIGiWYrtylzHKx9EyYz5vPd75uArFNVUHhNu24QnCXDW6z0dR7NMJIyb jb+iQFogN7jvBbcK4GMzJZB0WDZK6DjoKoAGn/d+d4uoHOnVaieDS1hSE24JBETgPJt6 d8AjRylaZLp+sshrkXeScPKEQjon6Ls/3EZYNm5Fyla0MIJQ/DcPPEu4LstsAElAAjVM eCVBllKW5CEy6IbOxLk3MMqMmvGYKdNHTS7b7FaiQILyRoYdfmbxKPz6T/DHl8kbHUY2 3fTUakFnDrQwl5n16gUocadaqXE0940KHUyz0QOmTUTzqgvunWeQKIo8si1/JbzjAoV5 c4ig== MIME-Version: 1.0 X-Received: by 10.140.82.144 with SMTP id h16mr28877472qgd.29.1429192133130; Thu, 16 Apr 2015 06:48:53 -0700 (PDT) Received: by 10.140.30.52 with HTTP; Thu, 16 Apr 2015 06:48:53 -0700 (PDT) In-Reply-To: <5BC1CA30-289D-42B4-95CD-3AE5D7B96F09@gmail.com> References: <5BC1CA30-289D-42B4-95CD-3AE5D7B96F09@gmail.com> Date: Thu, 16 Apr 2015 16:48:53 +0300 Message-ID: From: Adrian Popescu To: Jonathan Morton Content-Type: text/plain; charset=UTF-8 Cc: cake@lists.bufferbloat.net Subject: Re: [Cake] Cake3 - source code and some questions X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Apr 2015 13:49:24 -0000 I've discovered there are other problems in the Linux networking stack which don't seem to be related to fq_codel, qdiscs, AQM and HTB. There are latency inducing issues bugs in the Ethernet network drivers of many network adapters, including e1000e, or in the kernel itself. Some kernels are better. The newest ones have severe regressions in this area. I was under the impression there's a problem in codel or fq_codel that lead to very frequent latency micro-spikes of between 1 and 3 milliseconds. It also seemed to produce bigger latency spikes under moderate load. No amount of tuning and disabling of offloads helped with this. Imagine having 2 milliseconds of latency to your ISP and having your router induce between 3 to 5 milliseconds of latency for every flow. It's not particularly helpful for low latency paths on high bandwidth links. Adding more latency in both directions to high latency paths is even worse. These problems are the reason behind starting this thread. I believed these problems to be related to fq_codel or to the codel algorithm itself. My question about porting these improvements to codel and fq_codel was strictly about the tighter recovery, better invsqrt and other codel enhancements mentioned on the wiki page. The solution to this unstable latency will necessitate migration to another platform without Linux. I'm aware cake and fq_codel won't fix this problem on Linux.