From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 3987A3B2A4 for ; Thu, 3 Jan 2019 05:46:07 -0500 (EST) Received: by mail-wm1-x336.google.com with SMTP id n190so28915160wmd.0 for ; Thu, 03 Jan 2019 02:46:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=h0JVlImAAPzO3ynXQufbwRtORQwvmgzrFKvFk/LRTis=; b=PQEyejMISm1afB77Ou/IOMacg983YRIbFLuxZzLw03ccEwBBzqimwC2NsmKNkypmku gQJfx72I24xed4I9BoYvoNeUJA7LomJMyDQ4I+87C8n6x53l1E6co9RfNQkm8jqEWTGH /pcK/C+TQ44huz5vIvpAcTzFC9pKR6FwQqDQ+l0AWqFHcZVYwtFZf1i9XhAUmcZedC87 smjt545JHWi7HnKeRB9Ippnb2ge1E9o0bW85baHJXnmDGy868cVnKUhrWTzutiq5rCT1 k5sbPcbqNIi7jJplvszaSho8fAJMP/sjaLRpPraivrn7qbSI8VZQ5RvWYuNQeUVy6+yw 7Ffw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=h0JVlImAAPzO3ynXQufbwRtORQwvmgzrFKvFk/LRTis=; b=pYEnyBLOMMucvLoLVPU5qrkz5K8aE9d1TlSxj4gAhAq7lJzP7vjj+ebDFCAawFx8vm kRRYaLkTXyKDi30k21smj2l3xqBl1rSyQJq/s3+n2Gvbci6CfCPVJX54hLA5riFZ04KO PHj4/lWupjoJ3UwOHCEu4KUANt08H7+tJu9E8n0BgDLMiSd8scKAP80W9Pogli9EmsxA M1zsK6qX5sGU0o+q4/+F2oKxKuFPkbvCnpydxNI68A6yawKopOyVleh4wgr33xaeuBMW atwpkmJEuFf7bHW/oumrM3mnc3fBFRZUcO6ruPYl4bSOywSPfi628EZvFYkYiSh4tH8o +shg== X-Gm-Message-State: AJcUukfHBD468+Ab4W2V6qAG6D5ZFkjoqGHj3S/5QQpjFUwjb4rTx5WV DjerJiXoW39SnyxwE75fdgcvUQ== X-Google-Smtp-Source: ALg8bN7/H8Vknn/WZfPWwD2CWHNfffCtFl82szNsCqGxB5LmdYUa7cagQRDVle6PeynFMiWiP49wFQ== X-Received: by 2002:a1c:7a16:: with SMTP id v22mr5224885wmc.131.1546512366121; Thu, 03 Jan 2019 02:46:06 -0800 (PST) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id 127sm42529305wmm.45.2019.01.03.02.46.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 03 Jan 2019 02:46:05 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Pete Heist In-Reply-To: <3650A136-97A6-43F5-ADD3-B94A19775379@gmail.com> Date: Thu, 3 Jan 2019 11:46:03 +0100 Cc: Georgios Amanakis , Cake List Content-Transfer-Encoding: quoted-printable Message-Id: <99C93851-3539-4CB6-BED1-193B56658486@heistp.net> References: <8F9DE6A8-8614-46A8-9E9B-7B7E4CC7414F@heistp.net> <43a8ddec5beb962c53fe828363ecc839832de2c0.camel@gmail.com> <3650A136-97A6-43F5-ADD3-B94A19775379@gmail.com> To: Jonathan Morton X-Mailer: Apple Mail (2.3445.9.1) Subject: Re: [Cake] dual-src/dsthost unfairness, only with bi-directional traffic X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jan 2019 10:46:07 -0000 > On Jan 3, 2019, at 6:18 AM, Jonathan Morton = wrote: >=20 >> On 3 Jan, 2019, at 6:15 am, Georgios Amanakis = wrote: >>=20 >> It seems if both clients are having bidirectional traffic, dual- >> {dst,src}host has the same effect as triple-isolate (on both lan and >> wan interfaces) on their bandwidth. Exactly what I=E2=80=99m seeing- thanks for testing George... > I'm left wondering whether the sense of src and dst has got = accidentally reversed at some point, or if the dual modes are being = misinterpreted as triple-isolate. To figure that out, I'd need to look = carefully at several related parts of the code. Can anyone reproduce it = from the latest kernels' upstream code, or is it only in the module? = And precisely which version of iproute2 is everyone using? It will be a while before I can try this on 4.19+, but: = iproute2/oldstable,now 3.16.0-2 i386. I compile tc-adv from HEAD. Here are more bi-directional tests with 8 up / 1 down on IP1 and 1 up / = 8 down on IP2: dual-srchost/dual-dsthost: IP1: 83.1 / 10.9, IP2: 10.7 / 83.0 dual-dsthost/dual-srchost (sense flipped): IP1: 83.0 / 10.5, IP2: 10.7 / 82.9 triple-isolate: IP1: 83.1 / 10.5, IP2: 10.7 / 82.9 srchost/dsthost (sanity check): IP1: 47.6 / 43.8, IP2: 44.2 / 47.4 dsthost/srchost (sanity check, sense flipped): IP1: 81.3 / 9.79, IP2: 11.0 / 80.7 flows: IP1: 83.0 / 10.4, IP2: 10.5 / 82.9 I also tried testing shaping on eth0.3300 and ingress of eth0.3300 = instead of egress of both eth0 and eth0.3300, because that=E2=80=99s = more like what I tested before. There was no significant change from the = above results. I managed to compile versions all the way back to July 15, 2018 = (1e2473f702cf253f8f5ade4d622c6e4ba661a09d) and still see the same = result. I=E2=80=99ll try to go earlier. As far as the code goes, the easy stuff: - flow_mode values in cake_hash are 5 for dual-srchost, 6 for = dual-dsthost and 7 for triple-isolate - the values from cake_dsrc(flow_mode) and cake_ddst(flow_mode) are as = expected in all three cases - flow_override and host_override are both 0 - looks correct: !(flow_mode & CAKE_FLOW_FLOWS) =3D=3D 0 - this looks normal to me (shows reply packets on eth0): IP1 ping: dsthost_idx =3D 450, reduced_hash =3D 129 IP1 irtt: dsthost_idx =3D 450, reduced_hash =3D 158 IP2 ping: dsthost_idx =3D 301, reduced_hash =3D 78 IP2 irtt: dsthost_idx =3D 301, reduced_hash =3D 399 Jon, is there anything I can check by instrumenting the code somewhere = specific?