From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-x242.google.com (mail-pf0-x242.google.com [IPv6:2607:f8b0:400e:c00::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 471A03B2A4 for ; Mon, 18 Jun 2018 12:08:23 -0400 (EDT) Received: by mail-pf0-x242.google.com with SMTP id w7-v6so8404617pfn.9 for ; Mon, 18 Jun 2018 09:08:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language:content-transfer-encoding; bh=T5ifEzUsQFgWgWlTHe2aUkkiLqWvjr8KujUFLz69mnM=; b=XtLjxeNywTb0J5fGjm0hgS8lvRKWpcWH/aBQ+FbaLDTNrrW5W5mGTntIEIyG9JYIV2 L/4y6gyzbixz8DLZTqjwYYyso+fsS9BCdL6gJoVvtbXTU6SffrYCtoVm5L19c/K6c072 lC5fC653FKqfnVaymqzkD9fplfHP0I13et426Ti0jkxbCacKz20IY3jLNriAt1BOwAwJ ogFsCLBfqnvcKGFXCkE+Ka7KZARPYW8OE/7TQgvmGhu8V5n2tgPRiAhcAoEc4Jy8BYOL Zp2ENfekreDODBEHr1skdueXX4KV0GYXVRk5tucy2g5ylU+K88dgk40lNF5NU2JyXmKW JNFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=T5ifEzUsQFgWgWlTHe2aUkkiLqWvjr8KujUFLz69mnM=; b=GIuqv1nqAiw26rQ1vPd6UeY+Cy4SfZh3wS10w3hgLWPuWz1qpPgVVea4EUkqC7qoB9 tp0ija1FU1dALfdwPF3/c0ubT5wvBGzMWJIMrQAhdzp5GG6x/YN80ma6SbYl9RdZ7pID cKZGztAmfQtAbF30YFriNAVAP2A0jTrFjBvq7nBXK1Lk5DJ6w+DWKGnjXvPL3T7Chm7h hIgpXsdcag4Ez6gAspcjgj+cahQJ52XCTY38fxolbwfXsk6MDVGnFj6mCN7R29wcmek4 TMVpealzFTchOIDWCHkQB853M9J2P+SaDBSE9BVSL2U9F9buuzPvFxJ3aStB+GCKHMbR su8w== X-Gm-Message-State: APt69E3bfJtmez0yJAcc4TllzKoZL901zn+ZEg/Q8FT3DG/W1Ly+RKIu 3dnCCTI9A+B1ea8/S0RnIJFJ35UD X-Google-Smtp-Source: ADUXVKIJrhgQUEojQuhdQt3SFDpJAzZyRgVh30wMEAYh/VJ4STBR+ZxYvdzS5PzGBq0ZylOibmwXsg== X-Received: by 2002:a65:6008:: with SMTP id m8-v6mr11695099pgu.134.1529338102230; Mon, 18 Jun 2018 09:08:22 -0700 (PDT) Received: from ?IPv6:2620:15c:2c1:200:55c7:81e6:c7d8:94b? ([2620:15c:2c1:200:55c7:81e6:c7d8:94b]) by smtp.gmail.com with ESMTPSA id u2-v6sm12485139pfk.3.2018.06.18.09.08.21 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 18 Jun 2018 09:08:21 -0700 (PDT) To: make-wifi-fast@lists.bufferbloat.net References: <1527721073.171416827@apps.rackspace.com> <150ABF21-FAFC-48E2-9E55-CAA609EAE449@heistp.net> <20180617131921.09bf5353@redhat.com> <5CC11C44-6C78-410D-B699-B4B1A6F5FBDD@heistp.net> From: Eric Dumazet Message-ID: <8f5915f8-7dad-f881-b0c8-f6b03165c675@gmail.com> Date: Mon, 18 Jun 2018 09:08:20 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <5CC11C44-6C78-410D-B699-B4B1A6F5FBDD@heistp.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [Make-wifi-fast] emulating wifi better - coupling qdiscs in netem? X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Jun 2018 16:08:23 -0000 On 06/18/2018 02:24 AM, Pete Heist wrote: > >> On Jun 17, 2018, at 8:47 PM, Jonathan Morton > wrote: >> >> If you have instructions for setting up a test, I could try it. >> > Ok, thanks for that, code and scripts are attached, see README.txt. > > I now use plain netns (no lxc containers), which is easier to set up and has lower RTTs and higher throughputs, probably due to no bridge device. > > Per Eric’s tip, the nfq no-op code is run with chrt -rr 99, which reduces RTTs somewhat and increases throughputs ~2-3x. No busy polling yet. > > I also tried it on VMWare with a 2011 MBP, which looks radically better than the APU2 (for nfq, RTTs ~16% of APU2 and throughput 4x higher in the non-GSO case, 11x with GSO). Results attached, and to summarize: > > ping mean (min-max) RTTs: > > APU2, no nfq: 35 us (23-291) > APU2, nfq without GSO: 80 us (53-288) > APU2, nfq with GSO: 85 us (56-270) > 2011 MBP, no nfq: 4 us (4-529) [11% of APU2] > 2011 MBP, nfq without GSO: 13 us (11-197) [16% of APU2] > 2011 MBP, nfq with GSO: 14 us (11-1568) [16% of APU2] > > iperf3 throughputs (Gbps): > > APU2, no nfq: 5.01 Gbps > APU2, nfq without GSO: 391 Mbps > APU2, nfq with GSO: 3.35 Gbps > 2011 MBP, no nfq: 39.8 Gbps [7.9x APU2] > 2011 MBP, nfq without GSO: 1.48 Gbps [3.8x APU2] > 2011 MBP, nfq with GSO: 38.0 Gbps [11.3x APU2] > > Results from a decent physical box instead of a VM may be interesting to see. If all you want to achieve is to delay packets given a set of rules, you do not want the content of them. Copying the packets to user space is not needed, you are not doing deep packet inspection. Really the throughput should be the same, only latencies should be somehow increased, nfq_set_mode(qh, NFQNL_COPY_PACKET, 0xffff) -> nfq_set_mode(qh, NFQNL_COPY_PACKET, 128); // assuming you want to inspect headers