From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 2C23B21F280 for ; Tue, 3 Nov 2015 09:11:39 -0800 (PST) Received: from hms-beagle.home.lan ([217.237.68.126]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0LkP8Z-1aRKyl1Q8x-00cPbL; Tue, 03 Nov 2015 18:11:32 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Sebastian Moeller In-Reply-To: <87a8qvc8tz.fsf@toke.dk> Date: Tue, 3 Nov 2015 18:11:30 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <328DEF4F-F149-42C5-920E-53D16DCF544C@gmx.de> References: <87pozspckj.fsf@toke.dk> <6A2609D9-7747-487B-9484-ECC69C50DE96@gmx.de> <874mh3pai9.fsf@toke.dk> <50C2A7B7-1B81-41E1-B534-CA449296FE77@gmail.com> <87a8qvc8tz.fsf@toke.dk> To: =?windows-1252?Q?Toke_H=F8iland-J=F8rgensen?= X-Mailer: Apple Mail (2.1878.6) X-Provags-ID: V03:K0:3pRdLs1EMkuzIH8pboM3iD8wqcikHSdU5WYrOseq0IeXd9MqdRh bfhoBPi6y085sjohB8RoQefAXyYMdfVlPND9+4emevRkF32ipmS0tA7rBPVWFnS0j8PHOau yL4FV3ALbJ+YS43nvPoUrZYlecAkPaZJs36LygY7XrZK+lxVn58tn4w05VwYjMyrfdX9w1U HJVcoVgsxZ0YbTTtLrj+w== X-UI-Out-Filterresults: notjunk:1;V01:K0:/b8TCnp/GXw=:emS6RjM6GSyVVLhGI9w2hk 4Fur+IZZkxcu7sL1tp/lEzUDEyHVqN0tEB7fQ8l3mNyI8mFyJllRCiHLz6+nA+CDkF4WA7hxl OIcbOwkxhZu7uZ7ZMNWMf6ai8yFhRjhF9Ypat83gKbib6s6J/NW4q5/S5zlLhuIKQY8H+xQPn qLiQoJZ+b1jmElNCuRpiJtlsMJr+ga+Me5bEslqmm7+LEBBpOURrmOerPfh2oJDHQLTkpAuHb A3vMb8RhgQOYLm6yG5JHRuWnW+B47y77QHp2cLHPx/1HqUZs2KxhWYkrUxb7Mt5DEselADKtx 9USe1PG5y0kRphlNCeeG8k+36svN7ug4Xw2SBA2MLPTkFZukuDkm/m+SHQAHMhBVlD43ou6Ej L2HkfBRUHlk5Hnfb6Ex71vTJViyVrl7vhWs7F3tyDVKL1WJwrSFW6WR9FSjYNXjKhbuE21hbw fYzF38DA616HSwgnqjEc21HJM7nGzypL9A3F1TuYbb6E6STKAq4sFupWM6sVVTquzenY8kLfL xt9jb952D1jt8t9CXpEehp3wQK4mQiP8MmvybvEyM1BApEOHGiRcH8RwIsBfb7ZeoL5XRVsoc 81oBlcXPGvEEnLBhF0rdtXaglDDVjIzd3awKD84eAHRWzN0Z/4XwVVLphhWjGrzlIjSN7BP4m EvO68YzIJctLTOl9xbt5sMbfUhMf3hnpLNckO4rHtKzBI1e5RZQiaUpril67qKwzB+zkZJ4V1 QmxPugimKup3o5F5yHR68Egrl4tn9cwml1WgMgT9d3JdZnCANHVm7OnyOccMCxDbZmmSAtz9S h10jNZT Cc: cake@lists.bufferbloat.net Subject: Re: [Cake] Long-RTT broken again X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Nov 2015 17:12:02 -0000 Hi Toke, On Nov 3, 2015, at 18:05 , Toke H=F8iland-J=F8rgensen = wrote: > Jonathan Morton writes: >=20 >> Cake does the queue accounting in bytes, and calculates 15MB (by >> default) as the upper limit. It=92s *not* meant to be a packet = buffer. >=20 > Ah, good. >=20 >> note the different behaviour of the upload and download streams in = the >> results given. >=20 > This is not a result of ingress/egress shaping, though; the upstream = and > downstream shaping is done on each side of the bottleneck on separate > boxes. >=20 >> The only way this could behave like a =93packet buffer=94 instead of = a >> byte-accounted queue is if there is a fixed size allocation per >> packet, regardless of the size of said packet. There are hints that >> this might actually be the case, and that the allocation is a hugely >> wasteful (for an ack) 2KB. (This also means that it=92s not a 10240 >> packet buffer, but about 7500.) >=20 > Right, well, in that case fixing the calculation to use the actual > packet size would make sense in any case? Would it? I thought actually using the amount of =93pinned=94 = kernel memory would be more relevant, it a ACK packet pins 2KB then it = should be accounted at 2KB, IF the goal of the accounting is to avoid = un-scheduled OOM, no? And if something like Dave=92s patch kicks in that = copies larger mostly empty skbs to smaller sizes, these packets then = should be accounted with that smaller size. In anyway I believe with = default kernels there is a strong correlation between a packet count = limit and a byte count limit=85 >=20 >> But in a bidirectional TCP scenario with ECN, only about a third of >> the packets should be acks (ignoring the relatively low number of = ICMP >> and UDP probes); ECN causes an ack to be sent immediately, but then >> normal delayed-ack processing should resume. This makes 6KB allocated >> per ~3KB transmitted. The effective buffer size is thus 7.5MB, which >> is still compliant with the traditional rule of thumb (BDP / >> sqrt(flows)), given that there are four bulk flows each way. >>=20 >> This effect is therefore not enough to explain the huge deficit Toke >> measured. The calculus also changes by only a small factor if we >> ignore delayed acks, making 8KB allocated per 3KB transmitted. >=20 > Well, there are also several UDP measurement flows and a ping, which = all > send really small packets; so it's not just the acks. >=20 >> So, again - what=92s going on? Are there any clues in packet traces >> with sequence analysis? >=20 > Not really; the qdisc stats show ~6000 packets dropped over the 300 > second test; and lots and lots of overlimits. So my guess is that the > queue is in fact overflowing. >=20 > Will post a trace when I get a chance tomorrow. >=20 >> I=92ll put in a configurable memory limit anyway, but I really do = want >> to understand why this is happening. >=20 > As I said before: a configurable limit is not a fix for this; we need > the default behaviour to be sane. As much as I push for a configurable limit, I fully agree that = cake=92s auto-tuning smarts should actually be smart and do the right = thing. Best Regards Sebastian >=20 > -Toke