From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lb0-x232.google.com (mail-lb0-x232.google.com [IPv6:2a00:1450:4010:c04::232]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 3A5C221F6F7 for ; Thu, 5 Nov 2015 11:30:52 -0800 (PST) Received: by lbblt2 with SMTP id lt2so23868282lbb.3 for ; Thu, 05 Nov 2015 11:30:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Ip8m0mIXANtzXG5koCrLt5iBo9P04YomfeBc40JGyjg=; b=osNf42IIaqlw1hb/gV9DMHuc3X3H3G+Lu5iysjdMELbssHTJiMYmGQ9IpINaXlsV7g G12tMeypf1ocHtV5f7jhKVdzFnT99Z+1+UvbTw5+JKO3Vdsi6Q8oA2zs1YSPWS0iML82 DJAiX/9TRJl8ZeT57FvlfD9E1DHo3BqQbnqdSWq4BDfp0ovIebTBrOWnqzz2TzfSyiPm bGxe/rjN+/DZxszQx4L72tCVZl+jS6wCkPeLxq42B/lWHMSosDXh0VroBaZo3GNNRtDu 1czZ/OFpSwB8LNweFnfPxA6+02lHO0ejWJXhUfPwX3dXRpNNCac0uPQ2+q1zIvBz/ZvN pVnQ== X-Received: by 10.112.156.2 with SMTP id wa2mr4747357lbb.39.1446751850036; Thu, 05 Nov 2015 11:30:50 -0800 (PST) Received: from bass.home.chromatix.fi (83-245-237-101-nat-p.elisa-mobile.fi. [83.245.237.101]) by smtp.gmail.com with ESMTPSA id kf8sm1157795lbc.46.2015.11.05.11.30.48 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 05 Nov 2015 11:30:49 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) From: Jonathan Morton In-Reply-To: <87ziysldij.fsf@toke.dk> Date: Thu, 5 Nov 2015 21:30:47 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <87pozspckj.fsf@toke.dk> <6A2609D9-7747-487B-9484-ECC69C50DE96@gmx.de> <874mh3pai9.fsf@toke.dk> <50C2A7B7-1B81-41E1-B534-CA449296FE77@gmail.com> <87ziysldij.fsf@toke.dk> To: =?utf-8?Q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Mailer: Apple Mail (2.3096.5) Cc: cake@lists.bufferbloat.net Subject: Re: [Cake] Long-RTT broken again X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 19:31:15 -0000 > On 5 Nov, 2015, at 16:36, Toke H=C3=B8iland-J=C3=B8rgensen = wrote: >=20 > Jonathan Morton writes: >=20 >> So, again - what=E2=80=99s going on? Are there any clues in packet = traces >> with sequence analysis? >=20 > So, reran and took traces. Data and packet dumps at: > https://kau.toke.dk/experiments/cake/longrtt/ >=20 > There's two runs: the first one is what's in git now, the second one > patches the byte accounting to use the packet length rather than the > truesize, which helps a bit but is still inadequate. >=20 > The packet dumps and cake stats output agree: cake is running out of > buffer size. ECN is enabled on this test, and there's not a single = mark > performed, but several thousand drops. >=20 > So something is definitely wrong with the buffer sizing. Firstly: I can=E2=80=99t read the flent output with the version of flent = currently available to pip. It complains about =E2=80=9Cversion 3=E2=80=9D= files being unsupported as yet. Running the captures through tcptrace shows that the traffic on each = flow is very bursty, and that the bursts are roughly synchronised, = leading to synchronised drop bursts. It reads like a massive = advertisement for TCP Pacing, and this time cake=E2=80=99s shaper = isn=E2=80=99t enough to emulate it. (Perhaps a bigger buffer would sort = that out.) Taking one such synchronised drop burst as representative, the sum of = outstanding data on all four flows in that direction is less than 6MB at = that moment. This is on the version which I think you=E2=80=99ve = patched, and I assume the figure also includes data =E2=80=9Cin = flight=E2=80=9D in the delay line, not just in cake=E2=80=99s buffer. = It=E2=80=99s difficult to reconcile that with the nominal 15MB limit. = The absolute peak within-flow induced delay is 150ms, which corresponds = to 1.5MB, not 15MB - that=E2=80=99s suspicious. I also get the impression that CUBIC=E2=80=99s RTT-insensitive behaviour = isn=E2=80=99t helping matters. I=E2=80=99d be interested to see how = more traditional algorithms (Reno and Westwood+ in particular) cope with = this situation. I=E2=80=99ll try to reproduce the problem locally, then see what I can = do to fix it. - Jonathan Morton