From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.toke.dk (mail.toke.dk [52.28.52.200]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 78A353BA8E for ; Fri, 6 Jul 2018 05:29:37 -0400 (EDT) From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=toke.dk; s=20161023; t=1530869375; bh=CjdFuhuyfxzP8Bb5m/1k6E7oxGSbUqGcFEo+WxyWdBM=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=EOJzngJaBDGDAweimMTg3G5cJUgyo0s9mjyz5WWIgx41BiBxk42corswHhzzx8xvw u7tvGH9VvuBhaUNHvgiHEe71HPFcNs8nTIVgab4f/QDp1gl75c2Fd7sMrJWQb7tZ7h Ndgd1ObXfT7CrCyuFO099OgZqjNOlmeCWZ84eNH/iWmq2KqPKU+U0Nwpf/2M1dKTS5 Z/Yueqx/bJNPIYsR+uo32mcnkZ45JITD/KmI/BN++pva63upLxAk6nmZbpPsbxesJ1 ++8r8WM8BMeHA9+USE0EVmj/ATUHwGVLE9SR54U0jg32QhkXcWdcW0K+D8bosmpkz3 yh4fLpE2r8G+A== To: Pete Heist Cc: Jonathan Morton , Cake List In-Reply-To: References: <17AF79A0-0213-44E3-95B9-62795A644A47@heistp.net> <87lgatj13k.fsf@toke.dk> <87fu11ipir.fsf@toke.dk> <871scligay.fsf@toke.dk> <2AE036E5-BD3D-4176-9476-9EC824EC1D18@darbyshire-bryant.me.uk> <87r2klh1fz.fsf@toke.dk> <87lgath01v.fsf@toke.dk> <52B2B44D-4382-404C-8F6D-03F12A72B11F@heistp.net> <31667353-48F2-4FAB-AC05-163680451719@toke.dk> <48ECB6C8-5D22-4785-A6CE-696D87EC5496@toke.dk> <73DD74AD-C2E7-4A12-AE49-C06D4486660E@gmail.com> <87fu10haw7.fsf@toke.dk> <8736wxco28.fsf@toke.dk> Date: Fri, 06 Jul 2018 11:29:33 +0200 X-Clacks-Overhead: GNU Terry Pratchett Message-ID: <87o9fkbtky.fsf@toke.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] cake at 60gbit X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2018 09:29:37 -0000 Pete Heist writes: >> On Jul 6, 2018, at 12:31 AM, Toke H=C3=B8iland-J=C3=B8rgensen wrote: >>=20 >> I suspected that cake_dequeue() was looping forever, so I added some >> debug statements to investigate this; and turns out I was right. Using >> the debug patch below, in unlimited mode I get loop aborts on loop 'i' >> for unlimited mode and loop 'l' if I enable the shaper at 70 gbit. It >> happens pretty reliably, but only when I load up the link sufficiently >> (need 4-6 TCP flows which get ~50 Gbps of total throughput). >>=20 >> The weird thing is that what appears to be happening, is that cake >> somehow gets into a state where sch->q.qlen is >0 while all tin backlogs >> are 0. I have no clue how this happens; as far as I can tell, all >> changes to tin_backlog are paired with a change to q.qlen. The only >> thing outside of cake itself that modifies q.qlen is peek(), which is >> not being used here. > > I=E2=80=99m not sure I=E2=80=99ll be much help, but here are the thoughts= I start having with loop =E2=80=98i=E2=80=99: > > - is tin_deficit overflowing at these rates? at 50gbit, 2^31-1 bytes > happen in 344 ms (involuntary chuckle) > - what=E2=80=99s the value of tin_quantum_band here? but I suspect it=E2= =80=99s ok. I thought about overflows, but I don't get any "weird" values, and everything ends up back at zero when the flows stop. And it's not actually tin_backlog that's causing the looping... > - I=E2=80=99m assuming sparse_flow_count + bulk_flow_count wouldn=E2=80= =99t be 0=E2=80=A6 Yeah, they are; that's why it keeps looping. I've been looking at both tin_backlog and the *_flow_count vars as different ways of checking whether the tins are actually empty... they are all 0 when this happens. -Toke