From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id D95C121F21F for ; Wed, 2 Apr 2014 12:12:18 -0700 (PDT) Received: by mail-wi0-f179.google.com with SMTP id z2so1156268wiv.6 for ; Wed, 02 Apr 2014 12:12:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=TAl6e2wRhp3UxkrGMHjJfcOOavR1DyPbRwxpQmnRunY=; b=f6IJZU4wEQ8+dByfy/Tcm5TpVRBNuxHaBpGETk7a70jp8aqFSBXvPdhEdFSPvcZJ37 c8YU6sdZttHAkNAO3RaQqcY2sLbK6yXyS3LMyojrdkaENpIYJDeg1Jqjl8R4r6sdk+/N C+lGsCcIuhM8FSBG0Eovu55yz96/6r2z0Tn5dhlQuOVogL+Aq8pqulIxaIWZEyEO7bvz wsE+SGSG8ZR+Xr/tOwlCKXxAhw9CBPxoidSxLw6A9J/bD+2tl+AwOoXx8H7JUVhZhWL3 PY359dw+rR61Pc8FaTEf0l4/OWsmkCbICu1+QB/AQ5SwlKfjK+juR3smXt2b/LtfImmo gL9Q== MIME-Version: 1.0 X-Received: by 10.194.175.70 with SMTP id by6mr3250869wjc.3.1396465937022; Wed, 02 Apr 2014 12:12:17 -0700 (PDT) Received: by 10.216.8.1 with HTTP; Wed, 2 Apr 2014 12:12:16 -0700 (PDT) Date: Wed, 2 Apr 2014 12:12:16 -0700 Message-ID: From: Dave Taht To: bloat Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: [Bloat] debloating TCP further in linux 3.14 X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 19:12:19 -0000 In addition to the SQM work ongoing... people keep improving the deployed TCPs... a multitude of improvements land= ed in Linux 3.14, notably these two caught my eye. usec resolution for tcp in 3.15, oy, vey! I am planning on stabilizing my summer's testbed on 3.14 (and having a 3.4 box lying around for comparison)... unless any other cool patches arrive! I've seen a very similar problem (I think) on OSX, where the TCP cwnd stays stuck at 2*TSO on some kinds of paths where it shouldn't. I am under the impression that TCP small queue limits is not fully enabled = by default due to some devices like wifi and non-bqled ethernet not working well. While the subsystems exist the driver writers are lagging... commit d10473d4e3f9d1b81b50a60c8465d6f59a095c46 Author: Eric Dumazet Date: Sat Feb 22 22:25:57 2014 -0800 tcp: reduce the bloat caused by tcp_is_cwnd_limited() tcp_is_cwnd_limited() allows GSO/TSO enabled flows to increase their cwnd to allow a full size (64KB) TSO packet to be sent. Non GSO flows only allow an extra room of 3 MSS. For most flows with a BDP below 10 MSS, this results in a bloat of cwnd reaching 90, and an inflate of RTT. Thanks to TSO auto sizing, we can restrict the bloat to the number of MSS contained in a TSO packet (tp->xmit_size_goal_segs), to keep original intent without performance impact. Because we keep cwnd small, it helps to keep TSO packet size to their optimal value. Example for a 10Mbit flow, with low TCP Small queue limits (no more tha= n 2 skb in qdisc/device tx ring) Before patch : lpk51:~# ./ss -i dst lpk52:44862 | grep cwnd cubic wscale:6,6 rto:215 rtt:15.875/2.5 mss:1448 cwnd:96 ssthresh:96 send 70.1Mbps unacked:14 rcv_space:29200 After patch : lpk51:~# ./ss -i dst lpk52:52916 | grep cwnd cubic wscale:6,6 rto:206 rtt:5.206/0.036 mss:1448 cwnd:15 ssthresh:14 send 33.4Mbps unacked:4 rcv_space:29200 commit 4a5ab4e224288403b0b4b6b8c4d339323150c312 Author: Eric Dumazet Date: Thu Feb 6 15:57:10 2014 -0800 tcp: remove 1ms offset in srtt computation TCP pacing depends on an accurate srtt estimation. Current srtt estimation is using jiffie resolution, and has an artificial offset of at least 1 ms, which can produce slowdowns when FQ/pacing is used, especially in DC world, where typical rtt is below 1 ms. We are planning a switch to usec resolution for linux-3.15, but in the meantime, this patch removes the 1 ms offset. All we need is to have tp->srtt minimal value of 1 to differentiate the case of srtt being initialized or not, not 8. The problematic behavior was observed on a 40Gbit testbed, where 32 concurrent netperf were reaching 12Gbps of aggregate speed, instead of line speed. This patch also has the effect of reporting more accurate srtt and send rates to iproute2 ss command as in : $ ss -i dst cca2 Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port tcp ESTAB 0 0 10.244.129.1:56984 10.244.129.2:12865 cubic wscale:6,6 rto:200 rtt:0.25/0.25 ato:40 mss:1448 cwnd:10 sen= d 463.4Mbps rcv_rtt:1 rcv_space:29200 tcp ESTAB 0 390960 10.244.129.1:60247 10.244.129.2:50204 cubic wscale:6,6 rto:200 rtt:0.875/0.75 mss:1448 cwnd:73 ssthresh:= 51 send 966.4Mbps unacked:73 retrans:0/121 rcv_space:29200 --=20 Dave T=E4ht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.= html