From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-x22d.google.com (mail-pf0-x22d.google.com [IPv6:2607:f8b0:400e:c00::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id DEA583BA8E for ; Wed, 25 Apr 2018 12:29:05 -0400 (EDT) Received: by mail-pf0-x22d.google.com with SMTP id l27so15664287pfk.12 for ; Wed, 25 Apr 2018 09:29:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=awLPA6XMVgNjFPzwZJlEiqtd/NLZo53atnEOBtY2wLw=; b=O5ONtSLl3K9VKealO1qmcH4prnieq9uikNfn3bXklRHs9YYgkqhFE/xyKoTLsaclQ4 ltc7q01YNcP1Jp69+UwL5pWvfworujFbQkZ3b01bitwpkYqnjp+KYUzGEjU+LA9FFhIi YbSGDwNueZrwjShsTQwXzKlGAJRO/w39C7Re34zhJJXHDqhopNLL4MN13Fu4HdXFQyOn 0HhCvCZh0piE5vjr+2B2I+tBHS4rKsdgupIaynh4t30zq/oN21UeDvwddQVrwXVxhSUc 3T+py9GEKtnn6ka+9eej4eDoZ+uMvvrUOypI16Gt9UMaL4aX0DkeQDokbVsh7zN3iwE7 QQ4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=awLPA6XMVgNjFPzwZJlEiqtd/NLZo53atnEOBtY2wLw=; b=SKXspvfGRYxVwfirvJbIxYXIoaUdsQFQ7UW7+V8qrBrviSbrNDMqhVliHY7u4F5Vmo lfHLnbmrHvvOjpJma+5rZ1qqsQoV+ASQdvwifqIHz2dYUZWH3lcPLHbF7egaP3mQN5BK oOZXSMJqYEbs6X/vB8ZhjQN20XfQQdBgnZk88qzp4Uh+NJTahwH8i0Y/JsHM8siBoxFh pG4HvFyHMZYT4TgptMJ1TB04IoJCvZeMbmu2QDQ7X44vUCOq36//o8VHLH1NSec5ZbgC Oi2e/1mmK2QR5erfa+8O3h4rHfH0szPunaduqz8JETf+1FwRCfry0NF2AjY5rMkHx0Bs Xe2w== X-Gm-Message-State: ALQs6tDixdlLqfH/mr2BvC6XDo4h4FbPTlkje+kY+9zabQ82m7H2ZiBB NaaItegFs3U3s6xOv8cIqVo= X-Google-Smtp-Source: AIpwx48++U7Wm8CqmQE2UVt5BcIjcgbaMekFuCXUYVvefmhJJemPzMH9gWzp3axd5ky/8VVAOjNcNg== X-Received: by 10.98.131.69 with SMTP id h66mr18761060pfe.0.1524673745223; Wed, 25 Apr 2018 09:29:05 -0700 (PDT) Received: from ?IPv6:2620:15c:2c1:200:55c7:81e6:c7d8:94b? ([2620:15c:2c1:200:55c7:81e6:c7d8:94b]) by smtp.gmail.com with ESMTPSA id e87sm31869429pfl.65.2018.04.25.09.29.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Apr 2018 09:29:04 -0700 (PDT) To: =?UTF-8?Q?Toke_H=c3=b8iland-J=c3=b8rgensen?= , Eric Dumazet , netdev@vger.kernel.org Cc: cake@lists.bufferbloat.net, Dave Taht References: <20180425134249.21300-1-toke@toke.dk> <878t9b5n0q.fsf@toke.dk> <6bc11ded-028f-6c8f-964e-a569b4e10813@gmail.com> <8736zj6zj2.fsf@toke.dk> From: Eric Dumazet Message-ID: <8bae2ee1-efcc-1571-2a30-5b7779de2c88@gmail.com> Date: Wed, 25 Apr 2018 09:29:03 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <8736zj6zj2.fsf@toke.dk> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [Cake] [PATCH net-next v3] Add Common Applications Kept Enhanced (cake) qdisc X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2018 16:29:06 -0000 On 04/25/2018 09:06 AM, Toke Høiland-Jørgensen wrote: > Eric Dumazet writes: > >> On 04/25/2018 08:22 AM, Toke Høiland-Jørgensen wrote: >>> Eric Dumazet writes: >> >>>> What performance number do you get on a 10Gbit NIC for example ? >>> >>> Single-flow throughput through 2 hops on a 40Gbit connection (with CAKE >>> in unlimited mode vs pfifo_fast on the router): >>> >>> MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to testbed-40g-2 () port 0 AF_INET : demo >>> Recv Send Send >>> Socket Socket Message Elapsed >>> Size Size Size Time Throughput >>> bytes bytes bytes secs. 10^6bits/sec >>> >>> 87380 16384 16384 10.00 18840.40 >>> >>> MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to testbed-40g-2 () port 0 AF_INET : demo >>> Recv Send Send >>> Socket Socket Message Elapsed >>> Size Size Size Time Throughput >>> bytes bytes bytes secs. 10^6bits/sec >>> >>> 87380 16384 16384 10.00 24804.77 >> >> CPU performance would be interesting here. (netperf -Cc) > > > $ sudo tc qdisc replace dev ens2 root cake > $ netperf -cC -H 10.70.2.2 > MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.70.2.2 () port 0 AF_INET : demo > Recv Send Send Utilization Service Demand > Socket Socket Message Elapsed Send Recv Send Recv > Size Size Size Time Throughput local remote local remote > bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB > > 87380 16384 16384 10.00 15450.35 13.35 6.68 0.849 0.283 > > $ sudo tc qdisc del dev ens2 root > $ netperf -cC -H 10.70.2.2 > MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.70.2.2 () port 0 AF_INET : demo > Recv Send Send Utilization Service Demand > Socket Socket Message Elapsed Send Recv Send Recv > Size Size Size Time Throughput local remote local remote > bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB > > 87380 16384 16384 10.00 36414.23 8.20 14.30 0.221 0.257 > > > (In this test I'm running netperf on the machine that was a router > before, which is why the base throughput is higher; the other machine > runs out of CPU on the sender side). We can see here the high cost of forcing software GSO :/ Really, this should be done only : 1) If requested by the admin ( tc .... gso ....) 2) If packet size is above a threshold. The threshold could be set by the admin, and/or based on a fraction of the bandwidth parameter. I totally understand why you prefer to segment yourself for < 100 Mbit links. But this makes no sense on 10Gbit+