From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 85BA63B29E for ; Fri, 28 Dec 2018 07:58:26 -0500 (EST) Received: by mail-wr1-x431.google.com with SMTP id c14so20903619wrr.0 for ; Fri, 28 Dec 2018 04:58:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=from:content-transfer-encoding:mime-version:subject:date:references :to:in-reply-to:message-id; bh=EWnYLii4UfB8zrtbkarJefTWil6vEAwPt6nioLGD2B0=; b=VqMiln+fx16D9IISVZVPxR/U+41q21mZStE02UH1cldYjN3bmbiWwBsUcX75tmhEO4 8gTcitt8LfyvOnnC8wjUJhY4jCpUgUf+uyYp8TksSQJnYl6GqBeTgNFdozHqktJ6+lKw XZpevvo8lR3MHFzUAPKtE5oeG0gqBNduTTAupPoFkO+v239QEz/gmESw4Iux80DoTx72 6xgGDUKaLLvEOR00ixWw+4sfod7eUeKRALeWSP+mKTnfV/JovZRRz5DksaPDdzuwQA2I y40LAbbtME/aaXuqIGVUUW8JlJnJTXV7R7HkCqBKA94FdI8I0qUeXe0Lodus6d2X1Hmm +qmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:date:references:to:in-reply-to:message-id; bh=EWnYLii4UfB8zrtbkarJefTWil6vEAwPt6nioLGD2B0=; b=NvK87MErS8N6agsEzpIWLJMqkNeR6u2Cxff77PP8qlF3KH2fsxTXP7mfClO21zeRpA +V/jaUYJmV8AsS1UTSybU7AJ3BPE1N6KL2vuO33gq0u1ZfoGhCFNugvpnJui3F02ZaKS 6GD+NJFiJtBWqdrYexRLR2AF+lZFFkZO+VR2p+LG2tkOHJaJ2j9KBdEZMiYMWbRXkAO6 WmNhw6ptZ0Uqyl/M7qBB7EVHzdxTpCY6xlGUeg7ztu8meVVAX+8qVvrqzaWe0wkRHfYK 2AYxm3602e8Kgy4WYR5mQd6fzkzZpvEQLbM4KDUwg96jLwV48DvRV1jS2u3bipMBGWGP rBBQ== X-Gm-Message-State: AJcUukfxkr2OZ5trImelDAZTRgLVcjdW0S5OAGwjr4kWh7IRmEI7oECG +FyootbwcrtymsxEHO/8H7DAkKuNkKA= X-Google-Smtp-Source: ALg8bN6BH8APIY7l5caW2OGze4R9AmnzZGJtvU4NZ/E6UQCsBb+ldNNgC5daJc7EXURtXigi+8fCXw== X-Received: by 2002:adf:8264:: with SMTP id 91mr24282792wrb.312.1546001905390; Fri, 28 Dec 2018 04:58:25 -0800 (PST) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id y13sm23609295wrw.85.2018.12.28.04.58.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 28 Dec 2018 04:58:24 -0800 (PST) From: Pete Heist Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Date: Fri, 28 Dec 2018 13:58:23 +0100 References: To: Cake List In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3445.9.1) Subject: Re: [Cake] cake infinite loop(?) with hfsc on one-armed router X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Dec 2018 12:58:26 -0000 Note that this doesn=E2=80=99t happen when prio is used in place of hfsc = and cake is used in the leafs to do the rate limiting, i.e.: tc qdisc add dev eth0 root handle 1: prio bands 2 priomap 1 1 1 1 1 1 1 = 1 1 1 1 1 1 1 1 1 tc qdisc add dev eth0 parent 1:1 handle 10: cake besteffort bandwidth = 100mbit ethernet tc qdisc add dev eth0 parent 1:2 handle 11: cake besteffort bandwidth = 100mbit ethernet ether-vlan tc filter add dev eth0 protocol all parent 1:0 prio 1 basic match = "meta(vlan mask 0xfff eq 0xce4)" flowid 1:2 tc filter add dev eth0 protocol all parent 1:0 prio 2 u32 match u32 0 0 = flowid 1:1 But it does happen when drr is used instead of prio: tc qdisc add dev eth0 root handle 1: drr tc class add dev eth0 parent 1: classid 1:1 tc class add dev eth0 parent 1: classid 1:2 tc qdisc add dev eth0 parent 1:1 handle 10: cake besteffort bandwidth = 100mbit tc qdisc add dev eth0 parent 1:2 handle 11: cake besteffort bandwidth = 100mbit ether-vlan tc filter add dev eth0 protocol all parent 1:0 prio 1 basic match = "meta(vlan mask 0xfff eq 0xce4)" flowid 1:2 tc filter add dev eth0 protocol all parent 1:0 prio 2 u32 match u32 0 0 = flowid 1:1 drr might ultimately be what I want to use for this, so I can use cake = to do the rate limiting instead of htb. prio works well but leads to = starvation when the rate limit is above what the CPU can handle. Meanwhile, using htb classes with rate limits way above the actual, then = rate limiting in the cake leafs, works as well, but this seems like a = hack: tc qdisc add dev eth0 root handle 1: htb default 10 tc class add dev eth0 parent 1: classid 1:1 htb rate 10gbit tc class add dev eth0 parent 1:1 classid 1:10 htb rate 5gbit tc class add dev eth0 parent 1:1 classid 1:11 htb rate 5gbit tc filter add dev eth0 protocol ip parent 1:0 prio 1 basic match = "meta(vlan mask 0xfff eq 0xce4)" flowid 1:11 tc qdisc add dev eth0 parent 1:10 handle 20: cake besteffort bandwidth = 100mbit ethernet tc qdisc add dev eth0 parent 1:11 handle 21: cake besteffort bandwidth = 100mbit ethernet ether-vlan > On Dec 28, 2018, at 12:30 AM, Pete Heist wrote: >=20 > I=E2=80=99m seeing what I think it an infinite loop when cake is used = in a one-armed router configuration with hfsc as the rate limiter. Three = APUs are connected to the same switch and the =E2=80=9Cmiddle=E2=80=9D = APU (apu1a) routes between the default VLAN and a tagged VLAN. >=20 > apu2a <=E2=80=94 default VLAN =E2=80=94> apu1a <=E2=80=94 VLAN = 3300 =E2=80=94> apu2b >=20 > After qos is set up, ping from apu2a to apu2b still works fine. When = iperf3 is run from apu2a to apu2b it works fine, but when it goes in = reverse (apu2b to apu2a), all traffic stops flowing from apu1a on the = default VLAN. Traffic still flows from apu1a on VLAN 3300 however, with = very high RTT (mean 500ms), leading me to believe that the cake instance = on the default VLAN is in an infinite loop. >=20 > It does not happen with hfsc+fq_codel, or with htb+cake in the same = configuration. >=20 > Here are the commands that set up qos, and it only locks up when cake = is used as the instance at handle 20, not at handle 21: >=20 > ----- > tc qdisc add dev eth0 root handle 1: hfsc default 10 > tc class add dev eth0 parent 1: classid 1:1 hfsc sc rate 200mbit ul = rate 200mbit > tc class add dev eth0 parent 1:1 classid 1:10 hfsc sc rate 100mbit ul = rate 100mbit > tc class add dev eth0 parent 1:1 classid 1:11 hfsc sc rate 100mbit ul = rate 100mbit > tc filter add dev eth0 protocol ip parent 1:0 prio 1 \ > basic match "meta(vlan mask 0xfff eq 0xce4)" flowid 1:11 > tc qdisc add dev eth0 parent 1:10 handle 20: fq_codel # using cake = here locks up !!! > tc qdisc add dev eth0 parent 1:11 handle 21: cake > =E2=80=94=E2=80=94 >=20 > I=E2=80=99m using sch_cake and tc-adv from the current HEAD, on kernel = 3.16.7 (yeah, I know). >=20 > root@apu1a:~/qos# uname -a > Linux apu1a 3.16.7-ckt9-voyage #1 SMP Thu Apr 23 11:10:44 HKT 2015 = i686 GNU/Linux >=20 > Any ideas just from just this? Otherwise, I can only think to hook up = the serial cable and start with the printk=E2=80=99s=E2=80=A6 >=20