From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id E48E83BA8E for ; Fri, 6 Jul 2018 06:00:58 -0400 (EDT) Received: by mail-wr1-x436.google.com with SMTP id c13-v6so3552240wrt.1 for ; Fri, 06 Jul 2018 03:00:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=YWybZ7aqv1QeKWS12vJVuzUQWj/zmHVRw5B2Sk7HoyU=; b=gFE8yB8RaN9SL9kgEZ7VkHej3rrnBWXsL5SXMcT++p+ume3D+pOeeopQVSu4Jz5iUM pDRE2ZUhnlbQjyTZyvSNfeg1wURHtVf9vKul8B1J0o2kjqculRMMve6xjQZPxv7RK2VW uDGuIKERnDupCDEkP7s/a/WisWAOrG1z8oVfB+iQ1NLVO1SNzcbeddO8FMEcQFadEWi2 tuxTpIQsUAMKv8hTt/vFbSD5oC6kaR9GECPgAtxpU4yfqw+szl92kqtz6W1p2xu5ulvP xN5EP4vsZze3vWgOv/5E0DT9WkdSkYOQ1w23FI1Y4bhkWoSio5phZqkv1xAYtxugDLKR V8qA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=YWybZ7aqv1QeKWS12vJVuzUQWj/zmHVRw5B2Sk7HoyU=; b=PtqZ3/1kRsO+QqxAGu1FEw6D57VSynkqRZekKM4shc3/WTadK4FhrPJNex8fE68nNJ Q+q6EGCIzDUraRjhQVNJtEemZBO7RQigVRpgIX17hCg4FdOl3uGZCxFhilTiuVHKvX1A H6WEPYtHWA9PovR7boUaEzJJiAsS0sujc+//sLcQZWsjxqf1ndjRtSXX39J5ya37SWX4 1grCQyiGJ2XkunNJPDQoj8GPfcY1rZCQ5pD7MSxULhcks243Uk2MzU6CM99uoDWRM7BG CJwsC8bOSVnsBwnwBMgPlwrKeFfujlI8WLmPQXp3b3DnXdYboEJEOElaXnS7/7rPgufx Cn2w== X-Gm-Message-State: APt69E2ARH2hE0I9OzY8uEQGukyVt24WK5KMu+giRcEnhZ8wa6dB+FJ6 ozTtxhzGVHtmcnpEdhS03PT+fkqIJJo= X-Google-Smtp-Source: AAOMgpfcKn9cQOh0KAWK6xdm6hITrdxUZoNq51GIDcyrCnJKrMCSQYjPJVamqVhCnUoXldeLXsfp6Q== X-Received: by 2002:adf:820a:: with SMTP id 10-v6mr6715950wrb.144.1530871257522; Fri, 06 Jul 2018 03:00:57 -0700 (PDT) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id i4-v6sm12565095wmf.4.2018.07.06.03.00.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Jul 2018 03:00:56 -0700 (PDT) From: Pete Heist Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_FD4523A0-0FD4-430E-952F-D8CEBEC0DD49" Mime-Version: 1.0 (Mac OS X Mail 11.4 \(3445.8.2\)) Date: Fri, 6 Jul 2018 12:00:55 +0200 In-Reply-To: <87o9fkbtky.fsf@toke.dk> Cc: Jonathan Morton , Cake List To: =?utf-8?Q?Toke_H=C3=B8iland-J=C3=B8rgensen?= References: <17AF79A0-0213-44E3-95B9-62795A644A47@heistp.net> <87lgatj13k.fsf@toke.dk> <87fu11ipir.fsf@toke.dk> <871scligay.fsf@toke.dk> <2AE036E5-BD3D-4176-9476-9EC824EC1D18@darbyshire-bryant.me.uk> <87r2klh1fz.fsf@toke.dk> <87lgath01v.fsf@toke.dk> <52B2B44D-4382-404C-8F6D-03F12A72B11F@heistp.net> <31667353-48F2-4FAB-AC05-163680451719@toke.dk> <48ECB6C8-5D22-4785-A6CE-696D87EC5496@toke.dk> <73DD74AD-C2E7-4A12-AE49-C06D4486660E@gmail.com> <87fu10haw7.fsf@toke.dk> <8736wxco28.fsf@toke.dk> <87o9fkbtky.fsf@toke.dk> X-Mailer: Apple Mail (2.3445.8.2) Subject: Re: [Cake] cake at 60gbit X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2018 10:00:59 -0000 --Apple-Mail=_FD4523A0-0FD4-430E-952F-D8CEBEC0DD49 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jul 6, 2018, at 11:29 AM, Toke H=C3=B8iland-J=C3=B8rgensen = wrote: >=20 > Pete Heist > writes: >=20 >> - is tin_deficit overflowing at these rates? at 50gbit, 2^31-1 bytes >> happen in 344 ms (involuntary chuckle) >> - what=E2=80=99s the value of tin_quantum_band here? but I suspect = it=E2=80=99s ok. >=20 > I thought about overflows, but I don't get any "weird" values, and > everything ends up back at zero when the flows stop. And it's not > actually tin_backlog that's causing the looping=E2=80=A6 Ok, I think tin_deficit is meant here, esp. in light of what follows = regarding *_flow_count. Once we do get past this infinite loop, which it sounds like is not = caused by overflow here, I guess it=E2=80=99s still worth reviewing = whether tin_backlog or other values _could_ overflow in certain = conditions. In your case rtt is probably low, but what if it weren=E2=80=99= t? Adding delay with netem might coax something out. In fact, I=E2=80=99ll= see if I can add some delay to the 30-40gbit local testing that I _can_ = do to see if I notice anything... >> - I=E2=80=99m assuming sparse_flow_count + bulk_flow_count wouldn=E2=80= =99t be 0=E2=80=A6 >=20 > Yeah, they are; that's why it keeps looping. I've been looking at both > tin_backlog and the *_flow_count vars as different ways of checking > whether the tins are actually empty... they are all 0 when this = happens. Aha, ok. It does look physically possible for these to both be 0 since = there appear to be cases where one is decremented without the other = being incremented. That _all_ *_flow_count vars are 0 seems strange = logically. I=E2=80=99ll leave this alone now though as don=E2=80=99t yet = understand what the values represent well enough=E2=80=A6 :)= --Apple-Mail=_FD4523A0-0FD4-430E-952F-D8CEBEC0DD49 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On Jul 6, 2018, at 11:29 AM, Toke H=C3=B8iland-J=C3=B8rgensen = <toke@toke.dk> = wrote:

Pete Heist <pete@heistp.net> writes:

- is = tin_deficit overflowing at these rates? at 50gbit, 2^31-1 bytes
happen in 344 ms (involuntary chuckle)
- = what=E2=80=99s the value of tin_quantum_band here? but I suspect it=E2=80=99= s ok.

I thought about overflows, but I don't get any "weird" = values, and
everything = ends up back at zero when the flows stop. And it's not
actually tin_backlog that's = causing the looping=E2=80=A6

Ok, I think tin_deficit is meant here, esp. in = light of what follows regarding *_flow_count.

Once we do get past this infinite loop, which it = sounds like is not caused by overflow here, I guess it=E2=80=99s still = worth reviewing whether tin_backlog or other values _could_ overflow in = certain conditions. In your case rtt is probably low, but what if it = weren=E2=80=99t? Adding delay with netem might coax something out. In = fact, I=E2=80=99ll see if I can add some delay to the 30-40gbit local = testing that I _can_ do to see if I notice anything...

- = I=E2=80=99m assuming sparse_flow_count + bulk_flow_count wouldn=E2=80=99t = be 0=E2=80=A6

Yeah, they are; that's why it keeps looping. I've been = looking at both
tin_backlog and the *_flow_count vars as different ways of = checking
whether the = tins are actually empty... they are all 0 when this happens.

Aha, = ok. It does look physically possible for these to both be 0 since there = appear to be cases where one is decremented without the other being = incremented. That _all_ *_flow_count vars are 0 seems strange logically. = I=E2=80=99ll leave this alone now though as don=E2=80=99t yet understand = what the values represent well enough=E2=80=A6 :)
= --Apple-Mail=_FD4523A0-0FD4-430E-952F-D8CEBEC0DD49--