From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt1-x82e.google.com (mail-qt1-x82e.google.com [IPv6:2607:f8b0:4864:20::82e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 412673B2A4 for ; Wed, 2 Jan 2019 23:15:19 -0500 (EST) Received: by mail-qt1-x82e.google.com with SMTP id k12so35708761qtf.7 for ; Wed, 02 Jan 2019 20:15:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:subject:from:to:date:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=YyF7bYUc2QT6QtFodgpbFKUkWM48UScM0vrvhol6pAI=; b=VljJeEbkGwy0D/afyjGswfWv+Crdsty+kScZLHpOiEvwtjJFQGYDRCsLqfhW7h3AI7 TL5aV50yWW9dEUAluWgaerKFQdBKDklOKFg2HQZm1If2UBOeXnnsheiVWVG+amvmmM0S ZRy/D+qGOto+uXVqRz2KUqK54KQwHJlZFYlHKD6VLzBbBx8r31CLMgtU0bkYXx3kw1Of C0T9XQWWm1G5UyGXpFsd6k0cJeaKwctiBXoMigKopMNW0X1ep0GVGlpdFqe+zvYeGsBh h3YjVNhZ7mJdfbcJWB+8JVnw0NL9jBR3AmuGIuXQoQMW5/98C+p08b4VYcKiMq68qOYu hLJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=YyF7bYUc2QT6QtFodgpbFKUkWM48UScM0vrvhol6pAI=; b=Z3JNrf22GrUa3jqb4rJ2PyhQwj8ZFKHU4DzP0Xh/DLvGhUiKtX8rGvBhynd6zhGe4u CywJIacHbyBBqFxROqCARa60ZLgU9FmWazPQiF46MxPmNkaUgw2D5VGSZeX4mrug51iP Pl9k/FvC0Bim8KE7hW/2cWocoyyjoCTFVVzWUgPCPkuwoyWyAst4vL5DWYXG4YTY92tB 2RMY0XM08P8BROBmXnqZMXktiW7S8E3WAtmAqopfMX6HjB6R9xEMQqidGpxhoBy3dEfN Xqhc1TlzcNG/N40T1TuiYCgmisdawFxXpm9tAFaLcXSx47Uhx1ddch8+RzTzHH8GL2wS nOBQ== X-Gm-Message-State: AJcUukcw2XUo1Kw4u67+v8bcLbTSXaES0wx5dVHZXPZc4V7KsaI8L2Xs gMZ6WWHrnbf1Gjigy1UdcpRtFRKW X-Google-Smtp-Source: ALg8bN55YMyxHfGqe2/bGUm7ZyzbN6vPVDkaqV154VNElzE0Vfo2XkL5wVZ64JoZjW3VJbPcEV1R5w== X-Received: by 2002:a0c:ae76:: with SMTP id z51mr46286739qvc.103.1546488918613; Wed, 02 Jan 2019 20:15:18 -0800 (PST) Received: from x3200.lan ([2601:152:4302:ae1f::16e]) by smtp.googlemail.com with ESMTPSA id 32sm31271341qto.55.2019.01.02.20.15.17 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 02 Jan 2019 20:15:17 -0800 (PST) Message-ID: From: Georgios Amanakis To: Pete Heist , Cake List Date: Wed, 02 Jan 2019 23:15:17 -0500 In-Reply-To: <43a8ddec5beb962c53fe828363ecc839832de2c0.camel@gmail.com> References: <8F9DE6A8-8614-46A8-9E9B-7B7E4CC7414F@heistp.net> <43a8ddec5beb962c53fe828363ecc839832de2c0.camel@gmail.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.30.3 Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: Re: [Cake] dual-src/dsthost unfairness, only with bi-directional traffic X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jan 2019 04:15:19 -0000 It seems if both clients are having bidirectional traffic, dual- {dst,src}host has the same effect as triple-isolate (on both lan and wan interfaces) on their bandwidth. This shouldn't happen though, or am I wrong? On Wed, 2019-01-02 at 22:57 -0500, Georgios Amanakis wrote: > I can reproduce this one to my surprise, too. > I tested on my Comcast connection, with a WRT1900ACS, running openwrt > (r8082-95b3f8ec8d, 4.14.70), with two interfaces br-lan and > eth0(wan). > > IP1=1 up / 8 down IP2=4 up / 4 down > src/dst, bidir: IP1=0.88 / 8.44, IP2=0.66 / 7.75 (ok) > dualsrc/dualdst, bidir: IP1=0.27 / 10.56, IP2=1.41 / 6.42 (unfair) > > No VLANs, no other schedulers on eth0 and br-lan apart from cake. > > > > On Wed, 2019-01-02 at 00:04 +0100, Pete Heist wrote: > > In my one-armed router setup I’m seeing host fairness work > > perfectly > > with srchost or dsthost, but with dual-srchost or dual-dsthost, > > host > > fairness deviates from the ideal, _only_ when there's bi- > > directional > > traffic. The deviation is then dependent on the number of flows. Is > > this expected? > > > > I had thought that dual-src/dsthost worked the same as src/dsthost > > (fairness between hosts) with the exception that there is also > > fairness of flows within each host. > > > > Here are some results (all rates aggregate throughput in Mbit): > > > > IP1=8 up / 1 down IP2=1 up / 8 down (post-test tc stats > > attached): > > srchost/dsthost, upload only: IP1=48.1, IP2=47.9 (OK) > > srchost/dsthost, download only: IP1=47.8, IP2=47.8 (OK) > > srchost/dsthost, bi-directional: IP1=47.5 up / 43.9 down, > > IP2=44.7 up / 46.7 down (OK) > > > > dual-srchost/dual-dsthost, upload only: IP1=48.1, > > IP2=48.0 (OK) > > dual-srchost/dual-dsthost, download only: IP1=47.9, > > IP2=47.9 (OK) > > dual-srchost/dual-dsthost, bi-directional: IP1=83.0 up / 10.7 > > down, IP2=10.6 up / 83.0 down (*** asymmetric ***) > > > > Dual-srchost/dual-dsthost, bi-directional tests with different flow > > counts: > > > > IP1=4 up / 1 down IP2=1 up / 4 down: > > IP1=74.8 up / 18.8 down, IP2=18.8 up / 74.8 down > > > > IP1=2 up / 1 down IP2=1 up / 2 down: > > IP1=62.4 up / 31.3 down, IP2=31.3 up / 62.4 down > > > > IP1=4 up / 1 down IP2=1 up / 8 down: > > IP1=81.8 up / 11.5 down, IP2=17.4 up / 76.3 down > > > > IP1=2 up / 1 down IP2=1 up / 8 down: > > IP1=79.9 up / 13.5 down, IP2=25.7 up / 68.1 down > > > > The setup: > > > > apu2a (kernel 4.9) <— default VLAN —> apu1a (kernel > > 3.16.7) <— VLAN 3300 —> apu2b (kernel 4.9) > > > > - apu1a is the router, and has cake only on egress of both eth0 and > > eth0.3300, rate limited to 100mbit for both > > - it has no trouble shaping at 100mbit up and down simultaneously, > > so > > that should not be a problem > > - the same problem occurs at 25mbit or 50mbit) > > - since apu2a is the client [dual-]dsthost is used on eth0 and > > [dual- > > ]srchost is used on eth0.3300 > > - the fairness test setup seems correct, based on the results of > > most > > of the tests, at least. > > - note in the qdisc stats attached there is a prio qdisc on eth0 > > for > > filtering out VLAN traffic so it isn’t shaped twice > > - I also get the exact same results with an htb or hfsc hierarchy > > on > > eth0 instead of adding a qdisc to eth0.3300 > > - printk’s in sch_cake.c shows values of flow_mode, srchost_hash > > and > > dsthost_hash as expected > > - I also see it going into allocate_src and allocate_dst as > > expected, > > and later ending up in found_src and found_dst > > > > I’m stumped. I know I’ve tested fairness of dual-src/dsthost > > before, > > but that was from the egress of client and server, and it was on a > > recent kernel. Time to sleep on it... > > > > _______________________________________________ > > Cake mailing list > > Cake@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/cake