From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f169.google.com (mail-wi0-f169.google.com [209.85.212.169]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 35FDA21F095 for ; Wed, 11 Jul 2012 16:38:41 -0700 (PDT) Received: by wibhm2 with SMTP id hm2so4833738wib.4 for ; Wed, 11 Jul 2012 16:38:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:from:to:cc:in-reply-to:references:content-type:date :message-id:mime-version:x-mailer:content-transfer-encoding; bh=akZjMY6PfritIyYFmNb8gvYKf4co079WrWoSr+gG8SA=; b=M1ULqwnwRg3D2nDIv1mXnZ1qYXsn4/5AsLLkOQY+ksPU7Cl0UtOBLuh0uRIQVTfYz7 3jK5lGjhZJaaqq/bD11QwUvPNFbYvhUV6nvTg1BY97nxo1S144MJQcKMbSKy88qT6q++ HXw3RhMfS/tLVObR8mf2z6IYbanl398WVsjxYBkYeidfYPs6JP7p+q17xdxKB/bBde0Q 1KJpPnA/HFjPBazFhU25u/NELKarkfMbYsZoX4fwSCjVuffOzGO7OZDp8RMuHS4eOK8J EsixGuJG/xxchpUoMONMcIx64cZkfgej6P11+/Yo/RjX5LyC/57wn9CI+isWb+GKZ2Gr 1iBA== Received: by 10.216.54.212 with SMTP id i62mr17574093wec.98.1342049918916; Wed, 11 Jul 2012 16:38:38 -0700 (PDT) Received: from [172.28.88.151] ([74.125.122.49]) by mx.google.com with ESMTPS id l5sm8706313wix.5.2012.07.11.16.38.36 (version=SSLv3 cipher=OTHER); Wed, 11 Jul 2012 16:38:37 -0700 (PDT) From: Eric Dumazet To: Rick Jones In-Reply-To: <4FFDC48D.2030606@hp.com> References: <1340945457.29822.7.camel@edumazet-glaptop> <1341396687.2583.1757.camel@edumazet-glaptop> <20120709.000834.1182150057463599677.davem@davemloft.net> <1341845722.3265.3065.camel@edumazet-glaptop> <1341933215.3265.5476.camel@edumazet-glaptop> <1342019518.3265.8116.camel@edumazet-glaptop> <4FFDC48D.2030606@hp.com> Content-Type: text/plain; charset="UTF-8" Date: Thu, 12 Jul 2012 01:38:35 +0200 Message-ID: <1342049915.3265.8184.camel@edumazet-glaptop> Mime-Version: 1.0 X-Mailer: Evolution 2.28.3 Content-Transfer-Encoding: 7bit Cc: nanditad@google.com, netdev@vger.kernel.org, mattmathis@google.com, codel@lists.bufferbloat.net, ncardwell@google.com, David Miller Subject: Re: [Codel] [RFC PATCH v2] tcp: TCP Small Queues X-BeenThere: codel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: CoDel AQM discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jul 2012 23:38:41 -0000 On Wed, 2012-07-11 at 11:23 -0700, Rick Jones wrote: > On 07/11/2012 08:11 AM, Eric Dumazet wrote: > > > > > > Tests using a single TCP flow. > > > > Tests on 10Gbit links : > > > > > > echo 16384 >/proc/sys/net/ipv4/tcp_limit_output_bytes > > OMNI Send TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.99.2 (192.168.99.2) port 0 AF_INET > > tcpi_rto 201000 tcpi_ato 0 tcpi_pmtu 1500 tcpi_rcv_ssthresh 14600 > > tcpi_rtt 1875 tcpi_rttvar 750 tcpi_snd_ssthresh 16 tpci_snd_cwnd 79 > > tcpi_reordering 53 tcpi_total_retrans 0 > > I take it you hacked your local copy of netperf to emit those? Or did I > leave some cruft behind in something I committed to the repository? > Yep, its a netperf-2.5.0 with a one line change to output these TCP_INFO bits > What was the ultimate limiter on throughput? I notice it didn't achieve > link-rate on either 10 GbE nor 1 GbE. > My lab has one fast machine (source in this 10Gb test), and one slow machine (Intel Q6600 quad core), both with ixgbe cards. On Gigabit test, the receiver is a laptop. > > Thats the plan : limiting numer of bytes in Qdisc, not number of bytes > > in socket write queue. > > So the SO_SNDBUF can still grow rather larger than necessary? It is > just that TCP will be nice to the other flows by not dumping all of it > into the qdisc at once. Latency seen by the application itself is then > unchanged since there will still be (potentially) as much stuff queued > in the SO_SNDBUF as before right? Of course SO_SNDBUF can grow if autotuning is enabled. I think there is a bit of misunderstanding about this patch and what it does. It only makes sure the amount of packets (from socket write queue) are cloned in qdisc/device queue in a limited way, not "as much as allowed"