From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-out02.uio.no (mail-out02.uio.no [IPv6:2001:700:100:8210::71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 16DA63BA8E for ; Thu, 30 Nov 2017 02:02:38 -0500 (EST) Received: from mail-mx02.uio.no ([129.240.10.43]) by mail-out02.uio.no with esmtp (Exim 4.82_1-5b7a7c0-XX) (envelope-from ) id 1eKIrk-00037G-HC; Thu, 30 Nov 2017 08:02:36 +0100 Received: from 93-58-133-64.ip158.fastwebnet.it ([93.58.133.64] helo=[10.0.0.4]) by mail-mx02.uio.no with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) user michawe (Exim 4.82_1-5b7a7c0-XX) (envelope-from ) id 1eKIrf-0008Zi-HK; Thu, 30 Nov 2017 08:02:36 +0100 From: Michael Welzl Message-Id: <7B92DF4D-B6B5-4A64-9E10-119DCA2D4A6F@ifi.uio.no> Content-Type: multipart/alternative; boundary="Apple-Mail=_C284508E-43F0-4B42-A567-840823936F85" Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Date: Thu, 30 Nov 2017 08:03:40 +0100 In-Reply-To: Cc: David Ros To: bloat References: <4D0E907C-E15D-437C-B6F7-FF348346D615@gmx.de> X-Mailer: Apple Mail (2.3273) X-UiO-SPF-Received: Received-SPF: neutral (mail-mx02.uio.no: 93.58.133.64 is neither permitted nor denied by domain of ifi.uio.no) client-ip=93.58.133.64; envelope-from=michawe@ifi.uio.no; helo=[10.0.0.4]; X-UiO-Spam-info: not spam, SpamAssassin (score=-4.9, required=5.0, autolearn=disabled, AWL=0.052, HTML_MESSAGE=0.001, TVD_RCVD_IP=0.001, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO) X-UiO-Scanned: 67A700D90954E1996BE789D48AA662FC5F9F350E Subject: Re: [Bloat] benefits of ack filtering X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Nov 2017 07:02:38 -0000 --Apple-Mail=_C284508E-43F0-4B42-A567-840823936F85 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi Bloaters, I=E2=80=99d like to give offer some information and thoughts on AckCC, = at the bottom of this email. > On Nov 29, 2017, at 4:53 PM, Luca Muscariello = wrote: >=20 >=20 >=20 > On Wed, Nov 29, 2017 at 3:31 PM, Mikael Abrahamsson > wrote: > On Wed, 29 Nov 2017, Luca Muscariello wrote: >=20 > Why does it say to do this? What benefit is there to either end system = to > send 35kPPS of ACKs in order to facilitate a 100 megabyte/s of TCP = transfer? >=20 > Did you check RFC 3449 ? > https://tools.ietf.org/html/rfc3449#section-5.2.1 = >=20 > RFC3449 is all about middleboxes doing things. >=20 > I wanted to understand why TCP implementations find it necessary to = send one ACK per 2xMSS at really high PPS. Especially when NIC offloads = and middleboxes frequently strip out this information anyway so it never = reaches the IP stack (right?). >=20 >=20 > I would say because it is complex to guess at which PPS to work. You = would need an adaptation mechanism. Need also to change the client and = the server sides. The AckCC Jonathan has mentioned=20 > might be a solution to that. > Probably an ACK pacer in the end host, out of the TCP stack, doing Ack = filtering and decimation can be simpler to implement than the proper = adaptation mechanism in TCP. > Maybe inside sch_fq it would be doable. Maybe not. I=E2=80=99m adding the response from Jonathan Morton here to make this = more self-contained: *** Given an RTT estimate and knowledge of the congestion window, the AckCC = option could be used to target a handful of acks (maybe 4 to 10) per = RTT. As usual, extra acks would be sent when loss is suspected, on ECN = events, and when the push flag is set. That would be perfectly sufficient. - Jonathan Morton *** A few years ago, David Ros, whom I=E2=80=99m adding in cc, one of the = original authors of RFC 5690 did a sabbatical with me at the University = of Oslo. As part of that, we advised a master student to carry out tests = with AckCC, and analyze the RFC to understand how it would have to = change if we were to proceed to Proposed Standard. The result of his = investigation is here: = http://heim.ifi.uio.no/michawe/teaching/dipls/marius-olsen/mastersthesis-m= ariusno.pdf = and his code is here: http://folk.uio.no/mariusno/master/ = Now, after finishing the thesis, when it came to writing a paper about = it, we got stuck in the discussion of =E2=80=9Chow are we going to = explain that this is really necessary?=E2=80=9D - we didn=E2=80=99t want to submit a =E2=80=9Csolution searching for a = problem=E2=80=9D paper and didn=E2=80=99t want to get rejected for not = having shown that the problem truly exists. Searching for this a little = in the academic world (papers) gave us no result, at least back then. Interestingly, at IETF 98, not so long ago, Ingemar Johansson explained = to folks at TSVWG that the problem IS real: = https://datatracker.ietf.org/meeting/98/materials/slides-98-tsvwg-sessb-7-= transport-protocol-feedback-overhead-issues-and-solutions/ = So, let me now try to answer =E2=80=9Cwhy is TCP not doing that?=E2=80=9D.= - First, AFAIK, AckCC isn=E2=80=99t implemented anywhere (except that = we have this old patch - please feel free to download, adapt, and play = with it !!) - Second, if someone was to update TCP to support this, a bit more than = simple statements about the amount of traffic being large would be good = IMO - I mean, some convincing proof that the large number of ACKs = *really* is a problem. - Third, once this is implemented and deployed and found to be = beneficial, it would be useful to follow up in the IETF and update RFC = 5690. Since nobody seems to be doing any of these things, nothing changes. But = consider this: I see folks from Google doing a lot of TCP updates in the = IETF for which they themselves appear to have an immediate need. Given = the heterogeneity and magnitude of traffic produced by Google, if they = don=E2=80=99t see a pressing need for it, I suspect that, indeed, the = problem might not be so real after all?! Also, a word of caution. In this thread, there seems to be general = agreement that TCP sends way too many ACKs, and that reducing that = number would be fantastic. I=E2=80=99m not so convinced. Okay, even if TCP isn=E2=80=99t that = ACK-clocked anymore in Linux: 1) there isn=E2=80=99t only Linux in this = world, 2) ACKs are still quite important in Fast Recovery, 3) BBR might = not need to clock out ACKs, but it measures their incoming rate. For = another example, consider a non-BBR sender in slow start: without ABC, = missing ACKs would let it grow its cwnd too cautiously. Thanks to ABC, = this can be done more aggressively - but ABC recommends a limit on how = quickly to =E2=80=9Cjump=E2=80=9D in the rate in response to a single = ACK, for good reason - to avoid producing even heavier bursts. But with = this limit, again, the TCP sender is unnecessarily cautious in Slow = Start just because it misses ACKs. My point is: the ACKs ARE the feedback that TCP works on; when you take = them away, TCP becomes =E2=80=9Cblind=E2=80=9D, and whatever improvement = is made to TCP will have to be developed on that basis. I=E2=80=99m not saying that 1 ACK for every two packets is really = necessary=E2=80=A6 but unless there=E2=80=99s hard proof that this = really is a problem, I=E2=80=99d caution against a =E2=80=9Cdownward = spiral=E2=80=9D here: the level of asymmetry offered to users today is = probably somehow related to the commonly seen TCP ACK rate - so if TCP = starts to reduce the ACK rate, folks may decide to make links even more = asymmetric, etc. etc. =E2=80=A6 I=E2=80=99m not sure this is a good = direction. Just some thoughts, and some context. Cheers, Michael --Apple-Mail=_C284508E-43F0-4B42-A567-840823936F85 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi Bloaters,

I=E2=80=99d like to give offer some information and thoughts = on AckCC, at the bottom of this email.


On Nov 29, 2017, at 4:53 PM, = Luca Muscariello <luca.muscariello@gmail.com> wrote:



On Wed, Nov 29, 2017 at 3:31 PM, Mikael = Abrahamsson <swmike@swm.pp.se> wrote:
On = Wed, 29 Nov 2017, Luca Muscariello wrote:

Why does it say to do this? What benefit is there to either end system = to
send 35kPPS of ACKs in order to facilitate a 100 megabyte/s of TCP = transfer?

 Did you check RFC 3449 ?
https://tools.ietf.org/html/rfc3449#section-5.2.1

RFC3449 is all about middleboxes doing things.

I wanted to understand why TCP implementations find it necessary to send = one ACK per 2xMSS at really high PPS. Especially when NIC offloads and = middleboxes frequently strip out this information anyway so it never = reaches the IP stack (right?).


I would say because it is complex to = guess at which PPS to work. You would need an adaptation mechanism. Need = also to change the client and the server sides. The AckCC Jonathan has = mentioned 
might be a solution to = that.
Probably an ACK pacer in the end host, out of = the TCP stack, doing Ack filtering and decimation can be simpler to = implement than the proper adaptation mechanism in TCP.
Maybe inside sch_fq it would be doable. Maybe = not.

I=E2=80=99m adding the response from Jonathan = Morton here to make this more = self-contained:
***
Given an RTT estimate and = knowledge of the congestion window, the AckCC option could be used to = target a handful of acks (maybe 4 to 10) per RTT.  As usual, extra = acks would be sent when loss is suspected, on ECN events, and when the = push flag is set.

That would be = perfectly sufficient.

- Jonathan = Morton

***

A few years ago, David Ros, whom I=E2=80=99= m adding in cc, one of the original authors of RFC 5690 did a sabbatical = with me at the University of Oslo. As part of that, we advised a master = student to carry out tests with AckCC, and analyze the RFC to understand = how it would have to change if we were to proceed to Proposed Standard. = The result of his investigation is here:
http://heim.ifi.uio.no/michawe/teaching/dipls/marius-olsen/mast= ersthesis-mariusno.pdf
and his code is = here: http://folk.uio.no/mariusno/master/

Now, after finishing the = thesis, when it came to writing a paper about it, we got stuck in the = discussion of =E2=80=9Chow are we going to explain that this is really = necessary?=E2=80=9D
- we didn=E2=80=99t want to = submit a =E2=80=9Csolution searching for a problem=E2=80=9D paper and = didn=E2=80=99t want to get rejected for not having shown that the = problem truly exists. Searching for this a little in the academic world = (papers) gave us no result, at least back then.

Interestingly, at IETF 98, not so long = ago, Ingemar Johansson explained to folks at TSVWG that the problem IS = real:
https://datatracker.ietf.org/meeting/98/materials/slides-98-tsv= wg-sessb-7-transport-protocol-feedback-overhead-issues-and-solutions/<= /div>

So, let me now = try to answer =E2=80=9Cwhy is TCP not doing that?=E2=80=9D.
- First, AFAIK, AckCC isn=E2=80=99t implemented anywhere =  (except that we have this old patch - please feel free to = download, adapt, and play with it !!)
- Second, if = someone was to update TCP to support this, a bit more than simple = statements about the amount of traffic being large would be good IMO - I = mean, some convincing proof that the large number of ACKs *really* is a = problem.
- Third, once this is implemented and = deployed and found to be beneficial, it would be useful to follow up in = the IETF and update RFC 5690.

Since nobody seems to be doing any of = these things, nothing changes. But consider this: I see folks from = Google doing a lot of TCP updates in the IETF for which they themselves = appear to have an immediate need. Given the heterogeneity and magnitude = of traffic produced by Google, if they don=E2=80=99t see a pressing need = for it, I suspect that, indeed, the problem might not be so real after = all?!

Also, a = word of caution. In this thread, there seems to be general agreement = that TCP sends way too many ACKs, and that reducing that number would be = fantastic.
I=E2=80=99m not so convinced. Okay, even = if TCP isn=E2=80=99t that ACK-clocked anymore in Linux: 1) there isn=E2=80= =99t only Linux in this world, 2) ACKs are still quite important in Fast = Recovery, 3) BBR might not need to clock out ACKs, but it measures their = incoming rate.  For another example, consider a non-BBR sender in = slow start: without ABC, missing ACKs would let it grow its cwnd too = cautiously. Thanks to ABC, this can be done more aggressively - but ABC = recommends a limit on how quickly to =E2=80=9Cjump=E2=80=9D in the rate = in response to a single ACK, for good reason - to avoid producing even = heavier bursts. But with this limit, again, the TCP sender is = unnecessarily cautious in Slow Start just because it misses = ACKs.

My point = is: the ACKs ARE the feedback that TCP works on; when you take them = away, TCP becomes =E2=80=9Cblind=E2=80=9D, and whatever improvement is = made to TCP will have to be developed on that basis.

I=E2=80=99m not saying = that 1 ACK for every two packets is really necessary=E2=80=A6  but = unless there=E2=80=99s hard proof that this really is a problem, I=E2=80=99= d caution against a =E2=80=9Cdownward spiral=E2=80=9D here: the level of = asymmetry offered to users today is probably somehow related to the = commonly seen TCP ACK rate - so if TCP starts to reduce the ACK rate, = folks may decide to make links even more asymmetric, etc. etc. =E2=80=A6 = I=E2=80=99m not sure this is a good direction.

Just some thoughts, and some = context.

Cheers,
Michael

= --Apple-Mail=_C284508E-43F0-4B42-A567-840823936F85--