From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 6FEDE3B2A4 for ; Fri, 13 Nov 2020 01:31:30 -0500 (EST) Received: by mail-wr1-x42e.google.com with SMTP id d12so8424706wrr.13 for ; Thu, 12 Nov 2020 22:31:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=creamfinance.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version; bh=SiGYTFrVn1uCE5uRLRKfrXiXU24v8cNHQVaYuV9MyLc=; b=cTUcVx57VizDRnP71/RBXrkZNELiKuqqRi1ZmM4PX0PU0mi7k42ivO51xomsEcG4qF 2TRr51G1huFmyIVshCDsdosrFHkwQgQT7M2qetn1U/6DRYqmHYgwTZvLltGZUtp4Yji2 eb2bPQ+bJE7Qf37kbpJNFqNasUvJaoaNMuGgY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version; bh=SiGYTFrVn1uCE5uRLRKfrXiXU24v8cNHQVaYuV9MyLc=; b=I4R/sjacgC46nDMToN4tbVT/Kl21IC3a0hFC9Sk2xRtG1+BkMQzj++N0f5z6lmkBB9 vw8+nKE6+cjksxKOuEpPVh0Xbt9H1SWKEFzHItapsfal5ktuyV0OrFh+AK+CcGMxJwov nqiSogkUtOd+GhqlvCqXS6sALTY2hkEVShdG0tfu4NUa8XivLlFGnMR1ZObPywxbnEjJ 6xvBPOLsuVd781vF4KGJBG4TsnVHn+KHEtv0Rwrl3mm8tYqMDAZw4OaSNZa3rn8IdsWV h1KFmyf3xgZ/KRpHT3XBhNg5383MN4vY4Cep5ogB7jqB7Llc2TPr2EsIL9RyAEQ4b0Mb MBsA== X-Gm-Message-State: AOAM530vrmFn0D5G7tcbx5hpg77gR7TD1Kh9pqTImPPiQVi8zxfmxsUa xFNv30nLk1kW2sfGz3hbioza X-Google-Smtp-Source: ABdhPJyQ8WoYm2AJtHMyk2R6T9JPHn9LAKyQ/qC76wbLqp2Vezq9kLnysx0qVwoomfTZ8X0iRLhCIw== X-Received: by 2002:a5d:4ec4:: with SMTP id s4mr1278298wrv.31.1605249089055; Thu, 12 Nov 2020 22:31:29 -0800 (PST) Received: from [10.37.129.2] (ip-185.208.132.9.cf-it.at. [185.208.132.9]) by smtp.gmail.com with ESMTPSA id q12sm8348934wrx.86.2020.11.12.22.31.27 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Nov 2020 22:31:28 -0800 (PST) From: "Thomas Rosenstein" To: "Jesper Dangaard Brouer" Cc: Bufferbloat , "Saeed Mahameed" , "Tariq Toukan" Date: Fri, 13 Nov 2020 07:31:26 +0100 X-Mailer: MailMate (1.13.2r5673) Message-ID: <8B5A1292-72E2-4715-838C-5823AFA6B7BD@creamfinance.com> In-Reply-To: <20201112164236.3f94702a@carbon> References: <20201105143317.78276bbc@carbon> <11812D44-BD46-4CA4-BA39-6080BD88F163@creamfinance.com> <20201106121840.7959ae4b@carbon> <87blgaso84.fsf@toke.dk> <20201106135358.09f6c281@carbon> <20201106151324.5f506574@carbon> <1E70B6D2-1212-43FA-989A-03B657EEE2F2@creamfinance.com> <20201106211940.4c30ccc9@carbon> <6963be0e-3eb5-5875-b53c-66033f50dc2d@gmail.com> <12D28386-7C00-4A31-91E4-37083C1674F9@creamfinance.com> <20201109092428.293104ea@carbon> <7723D882-4DAB-4A70-9D00-DF1976872AC2@creamfinance.com> <20201109124030.71216677@carbon> <27110D8E-77DF-4D10-A5EA-6430DBD55BC7@creamfinance.com> <20201112110508.59f402e7@carbon> <20201112143111.2bb697bf@carbon> <20201112164236.3f94702a@carbon> MIME-Version: 1.0 Content-Type: text/plain; format=flowed; markup=markdown Subject: Re: [Bloat] Router congestion, slow ping/ack times with kernel 5.4.60 X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2020 06:31:30 -0000 On 12 Nov 2020, at 16:42, Jesper Dangaard Brouer wrote: > On Thu, 12 Nov 2020 14:42:59 +0100 > "Thomas Rosenstein" wrote: > >>> Notice "Adaptive" setting is on. My long-shot theory(2) is that >>> this >>> adaptive algorithm in the driver code can guess wrong (due to not >>> taking TSO into account) and cause issues for >>> >>> Try to turn this adaptive algorithm off: >>> >>> ethtool -C eth4 adaptive-rx off adaptive-tx off >>> > [...] >>>> >>>> rx-usecs: 32 >>> >>> When you run off "adaptive-rx" you will get 31250 interrupts/sec >>> (calc 1/(32/10^6) = 31250). >>> >>>> rx-frames: 64 > [...] >>>> tx-usecs-irq: 0 >>>> tx-frames-irq: 0 >>>> >>> [...] >> >> I have now updated the settings to: >> >> ethtool -c eth4 >> Coalesce parameters for eth4: >> Adaptive RX: off TX: off >> stats-block-usecs: 0 >> sample-interval: 0 >> pkt-rate-low: 0 >> pkt-rate-high: 0 >> >> rx-usecs: 0 > > Please put a value in rx-usecs, like 20 or 10. > The value 0 is often used to signal driver to do adaptive. Ok, put it now to 10. Goes a bit quicker (transfer up to 26 MB/s), but discards and pci stalls are still there. Ping times are noticable improved: 64 bytes from x.x.x.x: icmp_seq=39 ttl=64 time=0.172 ms 64 bytes from x.x.x.x: icmp_seq=40 ttl=64 time=0.414 ms 64 bytes from x.x.x.x: icmp_seq=41 ttl=64 time=0.183 ms 64 bytes from x.x.x.x: icmp_seq=42 ttl=64 time=1.41 ms 64 bytes from x.x.x.x: icmp_seq=43 ttl=64 time=0.172 ms 64 bytes from x.x.x.x: icmp_seq=44 ttl=64 time=0.228 ms 64 bytes from x.x.x.x: icmp_seq=46 ttl=64 time=0.120 ms 64 bytes from x.x.x.x: icmp_seq=47 ttl=64 time=1.47 ms 64 bytes from x.x.x.x: icmp_seq=48 ttl=64 time=0.162 ms 64 bytes from x.x.x.x: icmp_seq=49 ttl=64 time=0.160 ms 64 bytes from x.x.x.x: icmp_seq=50 ttl=64 time=0.158 ms 64 bytes from x.x.x.x: icmp_seq=51 ttl=64 time=0.113 ms > >> rx-frames: 32 >> rx-usecs-irq: 0 >> rx-frames-irq: 0 > > > > -- > Best regards, > Jesper Dangaard Brouer > MSc.CS, Principal Kernel Engineer at Red Hat > LinkedIn: http://www.linkedin.com/in/brouer