From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id D989E3B29E for ; Fri, 16 Aug 2019 08:45:16 -0400 (EDT) Received: by mail-io1-xd35.google.com with SMTP id 18so5878377ioe.10 for ; Fri, 16 Aug 2019 05:45:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :content-transfer-encoding; bh=NtbR+5PGrMNYJqDJ2GZ5Jrgz+E9D00wYouLCv5enaaM=; b=KFoDFiRT22UI7xW9Mm4lGiqu9wjQtDdsaHcpaaGnRQCOs64EVG5VEAxauvCPf0D3z3 hQcB54HIz78K6fT8/P43FxRP4ntJp+yNTIJFht9ERayDfmj79sSqdBjRMstEnGpjyDda KlmmNrYOJl+FZx7WWMlNhifETS+zJ9xc1U+/ebKPzJuT285mgUg+GFBvdBxlOC5cfMb9 X8anj8gZzGLMyP3QQRXWceJz1ExdYjj6aHl53AmiH6+/tDSwdOjzpo5Aa/KlR827fH29 KpRee0cMTo0F+aC7d+bKZKY6X/ZMIdWT37+McYZmFfhut6whIFA1/GgJckpiWilxCX3W umBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:content-transfer-encoding; bh=NtbR+5PGrMNYJqDJ2GZ5Jrgz+E9D00wYouLCv5enaaM=; b=HAR7tB01eNuHeKL4TXzlGmiNRsqieQm1zG2+SHVI3zJZFo1GCGcbx4n8C5FLaORuyq El/dtCIA3LJhjEhSoQFbfwxx7cYb78+1W3jWHMGz9GbQP3hr3fcsj7c0yyCpKr+0Af// OkEmnl8SP0IQgU16wy+V0IOG++fLhxVa8IpbWK2zTljcvQc4bodDSB3v1ANRd41NY/gN VCgcnqVW7lIYOoCuQ99oLFd/5SCyOX624nAnEU8Tg6Vja6glCcNjx2fjPITPX0XSLWbY yfBxNZdguhZ4coyfGEymvmOpyPxwkGC04SaeFPaV2zO+gIqffnBnAdbg2nJAN0ClBINm t09A== X-Gm-Message-State: APjAAAVPKMzjnQqqyTivBvy3oczAD6Witk12m8guQwq/ZqhhJqLfSf2Y yD8SjezXLtMCh0G3dbVz8GsmafhbNnaMMgvglQCBdg== X-Google-Smtp-Source: APXvYqxbCh2hjybbZxHNiepT1p/Xe6lZHaCw/2pPAqlsNWl7Cw5wntvbq5p7/lbwgr8upEWTFwckqb7n97ZlOmViXB4= X-Received: by 2002:a6b:7a0a:: with SMTP id h10mr3851686iom.45.1565959516065; Fri, 16 Aug 2019 05:45:16 -0700 (PDT) MIME-Version: 1.0 References: <72898d5b-9424-0bcd-3d8a-fc2e2dd0dbf1@intra2net.com> In-Reply-To: <72898d5b-9424-0bcd-3d8a-fc2e2dd0dbf1@intra2net.com> From: Dave Taht Date: Fri, 16 Aug 2019 05:45:04 -0700 Message-ID: To: Cake List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: [Cake] Fwd: r8169: Performance regression and latency instability X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Aug 2019 12:45:17 -0000 ---------- Forwarded message --------- From: Juliana Rodrigueiro Date: Fri, Aug 16, 2019 at 5:20 AM Subject: r8169: Performance regression and latency instability To: Cc: , Greetings! During migration from kernel 3.14 to 4.19, we noticed a regression on the network performance. Under the exact same circumstances, the standard deviation of the latency is more than double than before on the Realtek RTL8111/8168B (10ec:8168) using the r8169 driver. Kernel 3.14: # netperf -v 2 -P 0 -H ,4 -I 99,5 -t omni -l 1 -- -O STDDEV_LATENCY -m 64K -d Send 313.37 Kernel 4.19: # netperf -v 2 -P 0 -H ,4 -I 99,5 -t omni -l 1 -- -O STDDEV_LATENCY -m 64K -d Send 632.96 In contrast, we noticed small improvements in performance with other non-Realtek network cards (igb, tg3). Which suggested a possible driver related bug. However after bisecting the code, I ended up with the following patch, which was introduced in kernel 4.17 and modifies net/ipv4: commit 0a6b2a1dc2a2105f178255fe495eb914b09cb37a Author: Eric Dumazet Date: Mon Feb 19 11:56:47 2018 -0800 tcp: switch to GSO being always on Could you please help me to clarify, should GSO be always on on my device? Or does it just affect TCP? According to ethtool it is always off, "ethtool -K eth0 gso on" has no effect, unless I switch SG on. # ethtool -k eth0 Offload parameters for eth0: Cannot get device udp large send offload settings: Operation not supported rx-checksumming: on tx-checksumming: off scatter-gather: off tcp-segmentation-offload: off udp-fragmentation-offload: off generic-segmentation-offload: off generic-receive-offload: on large-receive-offload: off I validated that reverting "tcp: switch to GSO being always on" successfully brings back the better performance for the r8169 driver. I'm sure that reverting that commit is not the optimal solution, so I would like to kindly ask for help to shed some light in this issue. Best regards, Juliana. --=20 Dave T=C3=A4ht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740