From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 99BE53B2A4 for ; Thu, 30 Nov 2017 09:52:19 -0500 (EST) Received: by mail-wm0-x235.google.com with SMTP id 64so12854368wme.3 for ; Thu, 30 Nov 2017 06:52:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=kR5BbFh4x107cZAasz3kwFIhJoKORgDnxYX0PyWF8Jk=; b=Qka95wcWkPVSCI4gtv1aK7U1X179bHkiGr/IkCiThu+KCWDp3xKYwAzMudFjbHFeo6 5dYBGr5opmj140ElvfzSlBXqlNblfpiMwKE2BHNCWVKPINYIuTpp+rB5rgLf6TxMaWJU GXrsxdZ3AsS/ccXq9w8E3F97t6yf26o+CfTy+WbfVmB8YyzYg9gZ2qii5x/cxqV79SzE UeuW0i17tp+QZ/GIAvXtrAWdZHlZ9sy88EKTmqhKJYHTgbgEkbRuGmWsO1fR54MEuSr5 MoFW/SH0UZz/8RDll6jIiuUyt2BLgvmg5igPSSiozgRNTj9yqRjLOW7QzXuGHworEJIm 7r7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=kR5BbFh4x107cZAasz3kwFIhJoKORgDnxYX0PyWF8Jk=; b=g6StAKa2tVee/sWAoNirGq8aA51ZVvGzzzDPw9evxhpFX7rc65kyGXNevn3O6mHiaA zRAQ2ipXEhYQdIbDC4YZq8gDdTP6/Yh7csmz6Zc2MfabWe5GNOWu5RjwYQ9xW9WhC/Fa 28CLbvmlM2Eh4gxoCgYQasy71qOUmyIcK9ieKMLiFwNMIUEMkjTdysWROdVnsDPSgQLI v/Y32e1mz2CMmwanffgt55isNPsXyBWC9kabmeO7zr0dMMcUDgRU6aS0yHV0ewgrnnb0 HnRwRCLqQbG0Zs6/FWw6IJzZk8l4ze0vxkaXm/DelKh3EAVyyNvr88ulEgpOuIVr5F40 4d/Q== X-Gm-Message-State: AJaThX4OBoXIpGUzpslBaUfM9QTIwdhk4ssqsj7j9a584pRV5B1ELRqT 1w7Pn+OO+27vtXj3kE0P3FPMh9wgh1BBj9blKkRpHw== X-Google-Smtp-Source: AGs4zMabmHztNkXT2Fn/Cb7PHKGywp4Q+RR+Jwc7RUuWz7VdV/yvEhEKSjuuj1Y5oK6DArJ/KTSPZ4CREfimVSzjClc= X-Received: by 10.28.73.196 with SMTP id w187mr801622wma.17.1512053538374; Thu, 30 Nov 2017 06:52:18 -0800 (PST) MIME-Version: 1.0 Received: by 10.28.203.140 with HTTP; Thu, 30 Nov 2017 06:51:56 -0800 (PST) In-Reply-To: <1512037480.19682.10.camel@gmail.com> References: <4D0E907C-E15D-437C-B6F7-FF348346D615@gmx.de> <7B92DF4D-B6B5-4A64-9E10-119DCA2D4A6F@ifi.uio.no> <1512037480.19682.10.camel@gmail.com> From: Neal Cardwell Date: Thu, 30 Nov 2017 09:51:56 -0500 Message-ID: To: Eric Dumazet Cc: Jonathan Morton , Michael Welzl , David Ros , bloat , Yuchung Cheng , Wei Wang Content-Type: multipart/alternative; boundary="001a114b30920a2265055f346674" Subject: Re: [Bloat] benefits of ack filtering X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Nov 2017 14:52:19 -0000 --001a114b30920a2265055f346674 Content-Type: text/plain; charset="UTF-8" On Thu, Nov 30, 2017 at 5:24 AM, Eric Dumazet wrote: > I agree that TCP itself should generate ACK smarter, on receivers that > are lacking GRO. (TCP sends at most one ACK per GRO packets, that is > why we did not feel an urgent need for better ACK generation) > > It is actually difficult task, because it might need an additional > timer, and we were reluctant adding extra complexity for that. > How about just using the existing delayed ACK timer, and just making the delayed ACK logic a bit smarter? We could try using the existing logic and timers, but using something adaptive instead of the magic "2" MSS received to force an ACK. > An additional point where huge gains are possible is to add TSO > autodefer while in recovery. Lacking TSO auto defer explains why TCP > flows enter a degenerated behavior, re-sending 1-MSS packets in > response to SACK flood. > Yes, agreed. I suspect there is some simple heuristic that could be implemented to allow TSO deferral for most packets sent in recovery. For example, allowing TSO deferral once the number of packet bursts (TSO skbs) sent in recovery is greater than some threshold. Perhaps TSO deferral would be fine in Recovery if we have sent, say, 10 skbs, because at that point if the ACK stream from the original flight dries up due to massive/tail loss, we have probably sent enough data in the new flight in Recovery to ensure some kind of ACKs come back to keep the ACK clock going. neal > > > On Thu, 2017-11-30 at 09:48 +0200, Jonathan Morton wrote: > > I do see your arguments. Let it be known that I didn't initiate the > > ack-filter in Cake, though it does seem to work quite well. > > With respect to BBR, I don't think it depends strongly on the return > > rate of acks in themselves, but rather on the rate of sequence number > > advance that they indicate. For this purpose, having the receiver > > emit sparser but still regularly spaced acks would be better than > > having some middlebox delete some less-predictable subset of them. > > So I think BBR could be a good testbed for AckCC implementation, > > especially as it is inherently paced and thus doesn't suffer from > > burstiness as a conventional ack-clocked TCP might. > > The real trouble with AckCC is that it requires implementation on the > > client as well as the server. That's most likely why Google hasn't > > tried it yet; there are no receivers in the wild that would give them > > valid data on its effectiveness. Adding support in Linux would help > > here, but aside from Android devices, Linux is only a relatively > > small proportion of Google's client traffic - and Android devices are > > slow to pick up new kernel features if they can't immediately turn it > > into a consumer-friendly bullet point. > > Meanwhile we have highly asymmetric last-mile links (10:1 is typical, > > 50:1 is occasionally seen), where a large fraction of upload > > bandwidth is occupied by acks in order to fully utilise the download > > bandwidth in TCP. Any concurrent upload flows have to compete with > > that dense ack flow, which in various schemes is unfair to either the > > upload or the download throughput. > > That is a problem as soon as you have multiple users on the same > > link, eg. a family household at the weekend. Thinning out those acks > > in response to uplink congestion is a solution. Maybe not the best > > possible solution, but a deployable one that works. > > - Jonathan Morton > > _______________________________________________ > > Bloat mailing list > > Bloat@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/bloat > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > --001a114b30920a2265055f346674 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On T= hu, Nov 30, 2017 at 5:24 AM, Eric Dumazet <eric.dumazet@gmail.com= > wrote:
I agree that TCP itsel= f should generate ACK smarter, on receivers that
are lacking GRO. (TCP sends at most one ACK per GRO packets, that is
why we did not feel an urgent need for better ACK generation)

It is actually difficult task, because it might need an additional
timer, and we were reluctant adding extra complexity for that.

How about just using the existing delayed ACK timer= , and just making the delayed ACK logic a bit smarter? We could try using t= he existing logic and timers, but using something adaptive instead of the m= agic "2" MSS received to force an ACK.
=C2=A0
=
An additional point where huge gains are possible is to add TSO
autodefer while in recovery. Lacking TSO auto defer explains why TCP
flows enter a degenerated behavior, re-sending 1-MSS packets in
response to SACK flood.

Yes, agreed. I = suspect there is some simple heuristic that could be implemented to allow T= SO deferral for most packets sent in recovery. For example, allowing TSO de= ferral once the number of packet bursts (TSO skbs) sent in recovery is grea= ter than some threshold. Perhaps TSO deferral would be fine in Recovery if = we have sent, say, 10 skbs, because at that point if the ACK stream from th= e original flight dries up due to massive/tail loss, we have probably sent = enough data in the new flight in Recovery to ensure some kind of ACKs come = back to keep the ACK clock going.

neal
= =C2=A0


On Thu, 2017-11-30 at 09:48 +0200, Jonathan Morton wrote:
> I do see your arguments.=C2=A0 Let it be known that I didn't initi= ate the
> ack-filter in Cake, though it does seem to work quite well.
> With respect to BBR, I don't think it depends strongly on the retu= rn
> rate of acks in themselves, but rather on the rate of sequence number<= br> > advance that they indicate.=C2=A0 For this purpose, having the receive= r
> emit sparser but still regularly spaced acks would be better than
> having some middlebox delete some less-predictable subset of them.=C2= =A0
> So I think BBR could be a good testbed for AckCC implementation,
> especially as it is inherently paced and thus doesn't suffer from<= br> > burstiness as a conventional ack-clocked TCP might.
> The real trouble with AckCC is that it requires implementation on the<= br> > client as well as the server.=C2=A0 That's most likely why Google = hasn't
> tried it yet; there are no receivers in the wild that would give them<= br> > valid data on its effectiveness.=C2=A0 Adding support in Linux would h= elp
> here, but aside from Android devices, Linux is only a relatively
> small proportion of Google's client traffic - and Android devices = are
> slow to pick up new kernel features if they can't immediately turn= it
> into a consumer-friendly bullet point.
> Meanwhile we have highly asymmetric last-mile links (10:1 is typical,<= br> > 50:1 is occasionally seen), where a large fraction of upload
> bandwidth is occupied by acks in order to fully utilise the download > bandwidth in TCP.=C2=A0 Any concurrent upload flows have to compete wi= th
> that dense ack flow, which in various schemes is unfair to either the<= br> > upload or the download throughput.
> That is a problem as soon as you have multiple users on the same
> link, eg. a family household at the weekend.=C2=A0 Thinning out those = acks
> in response to uplink congestion is a solution.=C2=A0 Maybe not the be= st
> possible solution, but a deployable one that works.
> - Jonathan Morton
> __________________= _____________________________
> Bloat mailing list
> Bloat@lists.bufferbloat= .net
> https://lists.bufferbloat.net/listinfo/bloat
_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net<= /a>
https://lists.bufferbloat.net/listinfo/bloat

--001a114b30920a2265055f346674--