From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x22d.google.com (mail-wm0-x22d.google.com [IPv6:2a00:1450:400c:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id B9D033BA8E for ; Fri, 6 Jul 2018 04:55:56 -0400 (EDT) Received: by mail-wm0-x22d.google.com with SMTP id w16-v6so14040883wmc.2 for ; Fri, 06 Jul 2018 01:55:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=jhp128Lcne5nZQ11/PNocbsKsFVeiZ3WZRVPErvcX+c=; b=fv2TUM3dQGylCDj4k/U7rC4rWRVsqQiI5/5iHqy+nIRRmYWZHps7/KMqdgjx6HA+gF tIGhn1CLIeHgIKE9YLR4uwAGHj28kO2NEP5n+TBuwMHnPI7JHjKxDBKJc0ktui4gTNPa uIemu1JbZ13eikKpehYYngGU+Hm5+PLZbBslDkFgnToEpS6vi4+EEYjkgC/cp4xWNnsO Wv6L/kAZFUjKn4rHBvUMDsI7dTT3iK8/vrL2B8c4RszVZ+0DlYiZT5C0yo3RZlCGsgKY jLHZqoe5gSoG3/a6nh3YRPgR0XAgwUUMP7z7DP/VzNFzNIh3gyJ3W1aZsJzEQa9O5nGr KztA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=jhp128Lcne5nZQ11/PNocbsKsFVeiZ3WZRVPErvcX+c=; b=htCJGpN33+xdEuMDB2Wazjy9XvtxnGp+ZBXpXmL9//xRYuyLlk3cNKdBOUxAYH4pOF NGT0FthR8Fuwk1srggqZKL9o7Wdr5k6dZntxAKdWzLcncTLBYmOUTRHPBdIkTGZNMPpY owZeJmrEirirpti+c0B47K8e7ffWdsTIpfmr9wGjw3T3MI5T46JzK/w0qJg1UsnBtPd9 Py9YFPsTqRHIv+WA8oy5yzuae+TXD7rZjQrlmS693j1PHuvmn3FafISei98ZPVLqfqYl OsTHIgA6huiP6sR1KGAo2MJ/sn/WlSzyg592PHqn9b3/L1DEcXpRmToR/yIWUHXauv5z 9hCA== X-Gm-Message-State: APt69E2hygg+Wlkx+t6XbnsrN+eGh3dLZdnUe9NlixxjidffPagICFg7 v7y7HJfALrNssQ4feVj4fXUdRw== X-Google-Smtp-Source: AAOMgpcAuEaMv7Adczbd8o5B158qUE1mXRDnsnJYvXAPDAuvUlJ7eG/pAj1RXtmavzJCh80QaMDxZw== X-Received: by 2002:a1c:d1c1:: with SMTP id i184-v6mr5863736wmg.111.1530867355953; Fri, 06 Jul 2018 01:55:55 -0700 (PDT) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id x16-v6sm13082632wme.12.2018.07.06.01.55.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Jul 2018 01:55:55 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.4 \(3445.8.2\)) From: Pete Heist In-Reply-To: <8736wxco28.fsf@toke.dk> Date: Fri, 6 Jul 2018 10:55:54 +0200 Cc: Jonathan Morton , Cake List Content-Transfer-Encoding: quoted-printable Message-Id: References: <17AF79A0-0213-44E3-95B9-62795A644A47@heistp.net> <87lgatj13k.fsf@toke.dk> <87fu11ipir.fsf@toke.dk> <871scligay.fsf@toke.dk> <2AE036E5-BD3D-4176-9476-9EC824EC1D18@darbyshire-bryant.me.uk> <87r2klh1fz.fsf@toke.dk> <87lgath01v.fsf@toke.dk> <52B2B44D-4382-404C-8F6D-03F12A72B11F@heistp.net> <31667353-48F2-4FAB-AC05-163680451719@toke.dk> <48ECB6C8-5D22-4785-A6CE-696D87EC5496@toke.dk> <73DD74AD-C2E7-4A12-AE49-C06D4486660E@gmail.com> <87fu10haw7.fsf@toke.dk> <8736wxco28.fsf@toke.dk> To: =?utf-8?Q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Mailer: Apple Mail (2.3445.8.2) Subject: Re: [Cake] cake at 60gbit X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2018 08:55:57 -0000 > On Jul 6, 2018, at 12:31 AM, Toke H=C3=B8iland-J=C3=B8rgensen = wrote: >=20 > I suspected that cake_dequeue() was looping forever, so I added some > debug statements to investigate this; and turns out I was right. Using > the debug patch below, in unlimited mode I get loop aborts on loop 'i' > for unlimited mode and loop 'l' if I enable the shaper at 70 gbit. It > happens pretty reliably, but only when I load up the link sufficiently > (need 4-6 TCP flows which get ~50 Gbps of total throughput). >=20 > The weird thing is that what appears to be happening, is that cake > somehow gets into a state where sch->q.qlen is >0 while all tin = backlogs > are 0. I have no clue how this happens; as far as I can tell, all > changes to tin_backlog are paired with a change to q.qlen. The only > thing outside of cake itself that modifies q.qlen is peek(), which is > not being used here. I=E2=80=99m not sure I=E2=80=99ll be much help, but here are the = thoughts I start having with loop =E2=80=98i=E2=80=99: - is tin_deficit overflowing at these rates? at 50gbit, 2^31-1 bytes = happen in 344 ms (involuntary chuckle) - what=E2=80=99s the value of tin_quantum_band here? but I suspect = it=E2=80=99s ok. - I=E2=80=99m assuming sparse_flow_count + bulk_flow_count wouldn=E2=80=99= t be 0=E2=80=A6 Dave=E2=80=99s point about the possibility of 0 length packets could = also get sch->q.qlen and tin_backlog out of sync. I=E2=80=99ll still look at loop =E2=80=98l=E2=80=99, but it starting = with while(1) does open a door of sorts=E2=80=A6 ;)