From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-x241.google.com (mail-pg0-x241.google.com [IPv6:2607:f8b0:400e:c05::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 7CA473B2A4 for ; Thu, 30 Nov 2017 10:52:00 -0500 (EST) Received: by mail-pg0-x241.google.com with SMTP id y6so3153768pgp.4 for ; Thu, 30 Nov 2017 07:52:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:subject:from:to:cc:date:in-reply-to:references :mime-version:content-transfer-encoding; bh=nMZ4845tEPsa0kFoOs1ZFOXy8xvd50lf8Eyn92tpLAk=; b=OQbiBmCqAH3F7PmsDfpiYKHCtnqyzosEfUe1UzZxHpp4F1phLdPBsl4bIHDn7ZXh1C Z1606EN+ZWiG8LMR+xBS7qqsO0RTgAqcMX/yZSRz9C3RgI80yrp8gjNxLxRWVSoDCpf3 8OLhfJLAc2YUEEkrqmfdkv74e7zz+rkQsBeuu5AXcZIJGT0kplug0DE7I/DrVh/VYXzH fP67jyeOun4PoQlkBFsbU9kWU4WM363c9+mVTdMpyBcBLMaYhrwt7r07IQ6VH3i5KnFf FR7vOeSuw1CLh1MtpHlT3G3ENTfyX7hUrlyvjAqjEck9zE/rzwUW4UtsztG0q2SjJQoR WCng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:mime-version:content-transfer-encoding; bh=nMZ4845tEPsa0kFoOs1ZFOXy8xvd50lf8Eyn92tpLAk=; b=g8uYvGC5CoVfXEWtutNTJ2lSZUCBpUQX20+W/o6C6znHVhNW/GoZ6eXMYeEEeUZMtO t31YW2ovnivQNBEGNhdRSGKL8cQvhYjZ/Bz3N8EXvodFuvLmh4Rj4CHf8lD7CzW5eYCA ZuPgPPbVT3TR6S+j4fMiX45bDBVAxdK/+7Tj6B7IEibrHvhlLKL4L3x5w1XbxMYQS0nu uW7K4B6Eh4p4jWRlSZY0Ixiw4jwgt7iNzFAZ+ACvakDbxtQgDQaeEdqQLP/jJcCIuEck 8DFylEVNiC/boA+wLxcZUCnzN86+bP/hYH7fL92P6ehoeg5NK6egBW21B5cvDvq+JfNy +sTg== X-Gm-Message-State: AJaThX6l3o3/QgCwCemzEoxXmJ/jQ1wRE5iOTS397g/BkPfKjBMYw0Hl uxwYMDZGv9lzVn3oGoWjXXV8VQ== X-Google-Smtp-Source: AGs4zMafAijyOIGIJ/gW/pDo5kVv3EC+JKbI63DjOANyGBWbQtDAFT8j51yc2gEg4HY54QVSc9U4Lw== X-Received: by 10.101.101.215 with SMTP id y23mr2776655pgv.391.1512057119721; Thu, 30 Nov 2017 07:51:59 -0800 (PST) Received: from [10.1.104.90] ([207.198.105.19]) by smtp.googlemail.com with ESMTPSA id g9sm8562585pfk.0.2017.11.30.07.51.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 30 Nov 2017 07:51:58 -0800 (PST) Message-ID: <1512057116.19682.14.camel@gmail.com> From: Eric Dumazet To: Mikael Abrahamsson Cc: bloat Date: Thu, 30 Nov 2017 07:51:56 -0800 In-Reply-To: References: <4D0E907C-E15D-437C-B6F7-FF348346D615@gmx.de> <7B92DF4D-B6B5-4A64-9E10-119DCA2D4A6F@ifi.uio.no> <1512037480.19682.10.camel@gmail.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.22.6-1+deb9u1 Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: Re: [Bloat] benefits of ack filtering X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Nov 2017 15:52:00 -0000 On Thu, 2017-11-30 at 14:04 +0100, Mikael Abrahamsson wrote: > On Thu, 30 Nov 2017, Eric Dumazet wrote: > > > I agree that TCP itself should generate ACK smarter, on receivers > > that  > > are lacking GRO. (TCP sends at most one ACK per GRO packets, that > > is why  > > we did not feel an urgent need for better ACK generation) > > Could you elaborate a bit more on the practical implications of the > above  > text? What is the typical GRO size used when doing gigabit ethernet  > transmissions? Assuming NAPI handler receives a big packet train in one go [1], GRO packets can be full size (45 MSS -> 65160 bytes of payload assuming 1448 bytes per frame) [1] GRO engine has an opt-in high res timer helping to extend NAPI poll if desired. https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-n ext.git/commit/?id=3b47d30396bae4f0bd1ff0dbcd7c4f5077e7df4e > > So if we're receiving 70kPPS of 1500 byte packets containing 1460 > MSS  > sized packet (~100 megabyte/s), what would a typical ACK rate be in > that  > case? 1) Assuming receiver handles GRO. 2) Assuming few PSH flag set on incoming frames. 3) A default GRO engine on a 10Gbit NIC would probably not aggregate packets, since 14 usec delay between each packet is too big to let NAPI handler catch more than 1 packet per NIC RX interrupt. But setting /sys/class/net/ethX/gro_flush_timeout to 14000 would allow to build full size GRO packets (45 MSS) -> TCP receiver would then send 1555 ACK per second instead of 70,000 > > In response to some other postings here, my question regarding "is > 35kPPS  > really needed" my proposal is not "let's send 50 PPS of ACKs". My > proposal  > is if we can't come up with a smarter algorithm than something from > the  > 90ties that says "let's send one ACK per 2*MSS" when we today have  > magnitudes higher rates of forwarding. Also, on for instance DOCSIS  > networks then you're going to get several ACKs back-to-back anyway  > (because if they're not pruned by the DOCSIS network, they're anyway > sent  > in "bursts" within a single DOCSIS transmit opportunity), so > imagining  > that 35kPPS gives you higher resolution than 1kPPS of ACKs is just > an  > illusion. > > So if GRO results in (I'm just speculating here) "we're only sending > one  > ACK per X kilobytes received if the packets arrived in the same  > millisecond" and X is in the 16-64 kilobyte range, then that's fine > by me. > > Any network worth anything should be able to smooth out "bursts" of > 16-64  > kilobytes at line rate anyway, in case of egress and the line rate > there  > is lower than the sending end is transmitting packets at. >