From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-out02.uio.no (mail-out02.uio.no [IPv6:2001:700:100:8210::71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 955093B29E for ; Mon, 26 Nov 2018 16:56:54 -0500 (EST) Received: from mail-mx12.uio.no ([129.240.10.84]) by mail-out02.uio.no with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91) (envelope-from ) id 1gROs9-0005WX-15 for bloat@lists.bufferbloat.net; Mon, 26 Nov 2018 22:56:53 +0100 Received: from 58.116.34.95.customer.cdi.no ([95.34.116.58] helo=[10.0.0.3]) by mail-mx12.uio.no with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) user michawe (Exim 4.91) (envelope-from ) id 1gROs7-000C7J-O0 for bloat@lists.bufferbloat.net; Mon, 26 Nov 2018 22:56:52 +0100 From: Michael Welzl Content-Type: multipart/alternative; boundary="Apple-Mail=_F22D3651-76DC-421A-8052-A6066CB70AB6" Mime-Version: 1.0 (Mac OS X Mail 12.1 \(3445.101.1\)) Date: Mon, 26 Nov 2018 22:56:51 +0100 References: <65EAC6C1-4688-46B6-A575-A6C7F2C066C5@heistp.net> To: bloat In-Reply-To: <65EAC6C1-4688-46B6-A575-A6C7F2C066C5@heistp.net> Message-Id: <38535869-BF61-4FC4-A0FB-96E91CC4F076@ifi.uio.no> X-Mailer: Apple Mail (2.3445.101.1) X-UiO-SPF-Received: Received-SPF: neutral (mail-mx12.uio.no: 95.34.116.58 is neither permitted nor denied by domain of ifi.uio.no) client-ip=95.34.116.58; envelope-from=michawe@ifi.uio.no; helo=[10.0.0.3]; X-UiO-Spam-info: not spam, SpamAssassin (score=-5.0, required=5.0, autolearn=disabled, HTML_MESSAGE=0.001, TVD_RCVD_IP=0.001, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO) X-UiO-Scanned: 40D09B0039C200F463123A544AB4E9F2096364C5 Subject: Re: [Bloat] when does the CoDel part of fq_codel help in the real world? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Nov 2018 21:56:54 -0000 --Apple-Mail=_F22D3651-76DC-421A-8052-A6066CB70AB6 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi folks, That =E2=80=9CMichael=E2=80=9D dude was me :) About the stuff below, a few comments. First, an impressive effort to = dig all of this up - I also thought that this was an interesting = conversation to have! However, I would like to point out that thesis defense conversations are = meant to be provocative, by design - when I said that CoDel doesn=E2=80=99= t usually help and long queues would be the right thing for all = applications, I certainly didn=E2=80=99t REALLY REALLY mean that. The = idea was just to be thought provoking - and indeed I found this = interesting: e.g., if you think about a short HTTP/1 connection, a large = buffer just gives it a greater chance to get all packets across, and the = perceived latency from the reduced round-trips after not dropping = anything may in fact be less than with a smaller (or CoDel=E2=80=99ed) = buffer. But corner cases aside, in fact I very much agree with the answers to my = question Pete gives below, and also with the points others have made in = answering this thread. Jonathan Morton even mentioned ECN - after = Dave=E2=80=99s recent over-reaction to ECN I made a point of not = bringing up ECN *yet* again, but=E2=80=A6 yes indeed, being able to use = ECN to tell an application to back off instead of requiring to drop a = packet is also one of the benefits. (I think people easily miss the latency benefit of not dropping a = packet, and thereby eliminating head-of-line blocking - packet drops = require an extra RTT for retransmission, which can be quite a long time. = This is about measuring latency at the right layer...) BTW, Anna Brunstrom was also very quick to also give me the HTTP/2.0 = example in the break after the defense. Also, TCP will generally not = work very well when queues get very long=E2=80=A6 the RTT estimate gets = way off. All in all, I think this is a fun thought to consider for a bit, but not = really something worth spending people=E2=80=99s time on, IMO: big = buffers are bad, period. All else are corner cases. I=E2=80=99ll use the opportunity to tell folks that I was also pretty = impressed with Toke=E2=80=99s thesis as well as his performance at the = defense. Among the many cool things he=E2=80=99s developed (or = contributed to), my personal favorite is the airtime fairness scheduler. = But, there were many more. Really good stuff. With that, I wish all the best to all you bloaters out there - thanks = for reducing our queues! Cheers, Michael > On Nov 26, 2018, at 8:08 PM, Pete Heist wrote: >=20 > In Toke=E2=80=99s thesis defense, there was an interesting exchange = with examination committee member Michael (apologies for not catching = the last name) regarding how the CoDel part of fq_codel helps in the = real world: >=20 > https://www.youtube.com/watch?v=3Dupvx6rpSLSw&t=3D2h16m20s = >=20 > My attempt at a transcript is at the end of this message. (I probably = won=E2=80=99t attempt a full defense transcript, but if someone wants = more of a particular section I can try. :) >=20 > So I just thought to continue the discussion- when does the CoDel part = of fq_codel actually help in the real world? I=E2=80=99ll speculate with = a few possibilities: >=20 > 1) Multiplexed HTTP/2.0 requests containing both a saturating stream = and interactive traffic. For example, a game that uses HTTP/2.0 to = download new map data while position updates or chat happen at the same = time. Standalone programs could use HTTP/2.0 this way, or for web apps, = the browser may multiplex concurrent uses of XHR over a single TCP = connection. I don=E2=80=99t know of any examples. >=20 > 2) SSH with port forwarding while using an interactive terminal = together with a bulk transfer? >=20 > 3) Does CoDel help the TCP protocol itself somehow? For example, does = it speed up the round-trip time when acknowledging data segments, = improving behavior on lossy links? Similarly, does it speed up the TCP = close sequence for saturating flows? >=20 > Pete >=20 > --- >=20 > M: In fq_codel what is really the point of CoDel? > T: Yeah, uh, a bit better intra-flow latency... > M: Right, who cares about that? > T: Apparently some people do. > M: No I mean specifically, what types of flows care about that? > T: Yeah, so, um, flows that are TCP based or have some kind of- like, = elastic flows that still want low latency. > M: Elastic flows that are TCP based that want low latency... > T: Things where you want to discover the- like, you want to utilize = the full link and sort of probe the bandwidth, but you still want low = latency. > M: Can you be more concrete what kind of application is that? > T: I, yeah, I=E2=80=A6 > M: Give me any application example that=E2=80=99s gonna benefit from = the CoDel part- CoDel bits in fq_codel? Because I have problems with = this. > T: I, I do too... So like, you can implement things this way but = equivalently if you have something like fq_codel you could, like, if you = have a video streaming application that interleaves control=E2=80=A6 > M: that runs on UDP often. > T: Yeah, but I, Netflix=E2=80=A6 > M: Ok that=E2=80=99s a long way=E2=80=A6 > T: No, I tend to agree with you that, um=E2=80=A6 > M: Because the biggest issue in my opinion is, is web traffic- for web = traffic, just giving it a huge queue makes the chance bigger that uh, = so you may end up with a = (higher) faster completion time by buffering a lot. Uh, you=E2=80=99re = not benefitting at all by keeping the queue very small, you are simply = Right, you=E2=80=99re benefitting altogether by just = which is what the queue does with this nice sparse flow, = uh=E2=80=A6 > T: You have the infinite buffers in the for that to work, = right. One benefit you get from CoDel is that - you screw with things = like - you have to drop eventually. > M: You should at some point. The chances are bigger that the small = flow succeeds (if given a huge queue). And, in web surfing, why does = that, uh(?) > T: Yeah, mmm... > M: Because that would be an example of something where I care about = latency but I care about low completion. Other things where I care about = latency they often don=E2=80=99t send very much. bursts, = you have to accommodate them basically. Or you have interactive traffic = which is UDP and tries to, often react from queueing delay . = I=E2=80=99m beginning to suspect that fq minus CoDel is really the best = out there. > T: But if, yeah, if you have enough buffer. > M: Well, the more the better. > T: Yeah, well. > M: Haha, I got you to say yes. [laughter :] That goes in history. I = said the more the better and you said yeah. > T: No but like, it goes back to good-queue bad-queue, like, buffering = in itself has value, you just need to manage it. > M: Ok. > T: Which is also the reason why just having a small queue doesn=E2=80=99= t help in itself. > M: Right yeah. Uh, I have a silly question about fq_codel, a very = silly one and there may be something I missed in the papers, probably I = did, but I'm I was just wondering I mean first of all this is also a bit = silly in that it=E2=80=99s a security thing, and I think = that=E2=80=99s kind of a package by itself silly because fq_codel often = probably just in principle, is that something I could easily = attack by creating new flows for every packet? > T: No because, they, you will=E2=80=A6 > M: With the sparse flows, and it=E2=80=99s gonna=E2=80=A6 > T: Yeah, but at some point you=E2=80=99re going to go over the = threshold, I, you could, there there=E2=80=99s this thing where the flow = goes in, it=E2=80=99s sparse, it empties out and then you put it on the = normal round robin implementation before you queue And if = you don=E2=80=99t do that than you can have, you could time packets so = that they get priority just at the right time and you could have = lockout. > M: Yes. > T: But now you will just fall back to fq. > M: Ok, it was just a curiousity, it=E2=80=99s probably in the paper. = > T: I think we added that in the RFC, um, you really need to, like, = this part is important. >=20 > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat --Apple-Mail=_F22D3651-76DC-421A-8052-A6066CB70AB6 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi = folks,

That = =E2=80=9CMichael=E2=80=9D dude was me  :)

About the stuff below, a few comments. = First, an impressive effort to dig all of this up - I also thought that = this was an interesting conversation to have!

However, I would like to point out that = thesis defense conversations are meant to be provocative, by design - = when I said that CoDel doesn=E2=80=99t usually help and long queues = would be the right thing for all applications, I certainly didn=E2=80=99t = REALLY REALLY mean that.  The idea was just to be thought provoking = - and indeed I found this interesting: e.g., if you think about a short = HTTP/1 connection, a large buffer just gives it a greater chance to get = all packets across, and the perceived latency from the reduced = round-trips after not dropping anything may in fact be less than with a = smaller (or CoDel=E2=80=99ed) buffer.

But corner cases aside, in fact I very = much agree with the answers to my question Pete gives below, and also = with the points others have made in answering this thread. Jonathan = Morton even mentioned ECN - after Dave=E2=80=99s recent over-reaction to = ECN I made a point of not bringing up ECN *yet* again, but=E2=80=A6 yes = indeed, being able to use ECN to tell an application to back off instead = of requiring to drop a packet is also one of the benefits.
(I think people easily miss the latency benefit of not = dropping a packet, and thereby eliminating head-of-line blocking - = packet drops require an extra RTT for retransmission, which can be quite = a long time. This is about measuring latency at the right = layer...)
BTW, Anna Brunstrom was also very quick = to also give me the HTTP/2.0 example in the break after the defense. = Also, TCP will generally not work very well when queues get very long=E2=80= =A6 the RTT estimate gets way off.

All in all, I think this is a fun = thought to consider for a bit, but not really something worth spending = people=E2=80=99s time on, IMO: big buffers are bad, period. All else are = corner cases.

I=E2=80=99ll use the opportunity to tell folks that I was = also pretty impressed with Toke=E2=80=99s thesis as well as his = performance at the defense. Among the many cool things he=E2=80=99s = developed (or contributed to), my personal favorite is the airtime = fairness scheduler. But, there were many more. Really good = stuff.

With = that, I wish all the best to all you bloaters out there - thanks for = reducing our queues!

Cheers,
Michael


On Nov 26, 2018, at 8:08 PM, Pete Heist <pete@heistp.net> = wrote:

In = Toke=E2=80=99s thesis defense, there was an interesting exchange with = examination committee member Michael (apologies for not catching the = last name) regarding how the CoDel part of fq_codel helps in the real = world:

https://www.youtube.com/watch?v=3Dupvx6rpSLSw&t=3D2h16m20s<= /a>

My attempt at a = transcript is at the end of this message. (I probably won=E2=80=99t = attempt a full defense transcript, but if someone wants more of a = particular section I can try. :)

So I just thought to continue the = discussion- when does the CoDel part of fq_codel actually help in the = real world? I=E2=80=99ll speculate with a few possibilities:

1) Multiplexed HTTP/2.0 = requests containing both a saturating stream and interactive traffic. = For example, a game that uses HTTP/2.0 to download new map data while = position updates or chat happen at the same time. Standalone programs = could use HTTP/2.0 this way, or for web apps, the browser may multiplex = concurrent uses of XHR over a single TCP connection. I don=E2=80=99t = know of any examples.

2) SSH with port forwarding while using an interactive = terminal together with a bulk transfer?

3) Does CoDel help the TCP protocol = itself somehow? For example, does it speed up the round-trip time when = acknowledging data segments, improving behavior on lossy links? = Similarly, does it speed up the TCP close sequence for saturating = flows?

Pete

---

M: In fq_codel what is really the point of CoDel?
T: Yeah, uh, a bit better intra-flow latency...
M: Right, who cares about that?
T: = Apparently some people do.
M: No I mean = specifically, what types of flows care about that?
T:= Yeah, so, um, flows that are TCP based or have some kind of- like, = elastic flows that still want low latency.
M: = Elastic flows that are TCP based that want low latency...
T: Things where you want to discover the- like, you want to = utilize the full link and sort of probe the bandwidth, but you still = want low latency.
M: Can you be more concrete what = kind of application is that?
T: I, yeah, = I=E2=80=A6
M: Give me any application example = that=E2=80=99s gonna benefit from the CoDel part- CoDel bits in = fq_codel? Because I have problems with this.
T: I, = I do too... So like, you can implement things this way but equivalently = if you have something like fq_codel you could, like, if you have a video = streaming application that interleaves control=E2=80=A6
M: <inaudible> that runs on UDP often.
T: Yeah, but I, Netflix=E2=80=A6
M: Ok = that=E2=80=99s a long way=E2=80=A6 <inaudible>
T: No, I tend to agree with you that, um=E2=80=A6
M: Because the biggest issue in my opinion is, is web = traffic- for web traffic, just giving it a huge queue makes the chance = bigger that uh, <inaudible, ed: because of the slow start> so you = may end up with a (higher) faster completion time by buffering a lot. = Uh, you=E2=80=99re not benefitting at all by keeping the queue very = small, you are simply <inaudible> Right, you=E2=80=99re = benefitting altogether by just <inaudible> which is what the queue = does with this nice sparse flow, uh=E2=80=A6 <inaudible>
T: You have the infinite buffers in the <inaudible> for = that to work, right. One benefit you get from CoDel is that - you screw = with things like - you have to drop eventually.
M: = You should at some point. The chances are bigger that the small flow = succeeds (if given a huge queue). And, in web surfing, why does that, = uh(?)
T: Yeah, mmm...
M: = Because that would be an example of something where I care about latency = but I care about low completion. Other things where I care about latency = they often don=E2=80=99t send very much. <inaudible...> bursts, = you have to accommodate them basically. Or you have interactive traffic = which is UDP and tries to, often react from queueing delay = <inaudible>. I=E2=80=99m beginning to suspect that fq minus CoDel = is really the best <inaudible> out there.
T: = But if, yeah, if you have enough buffer.
M: Well, = the more the better.
T: Yeah, well.
M: Haha, I got you to say yes. [laughter :] That goes in = history. I said the more the better and you said yeah.
T: No but like, it goes back to good-queue bad-queue, like, = buffering in itself has value, you just need to manage it.
M: Ok.
T: Which is also the reason why = just having a small queue doesn=E2=80=99t help in itself.
M: Right yeah. Uh, I have a silly question about fq_codel, a = very silly one and there may be something I missed in the papers, = probably I did, but I'm I was just wondering I mean first of all this is = also a bit silly in that <inaudible> it=E2=80=99s a security = thing, and I think that=E2=80=99s kind of a package by itself silly = because fq_codel often probably <inaudible> just in principle, is = that something I could easily attack by creating new flows for every = packet?
T: No because, they, you will=E2=80=A6
<= div class=3D"">M: With the sparse flows, and it=E2=80=99s = gonna=E2=80=A6
T: Yeah, but at some point you=E2=80=99= re going to go over the threshold, I, you could, there there=E2=80=99s = this thing where the flow goes in, it=E2=80=99s sparse, it empties out = and then you put it on the normal round robin implementation before you = queue <inaudible> And if you don=E2=80=99t do that than you can = have, you could time packets so that they get priority just at the right = time and you could have lockout.
M: Yes.
T: But now you will just fall back to fq.
M: Ok, it was just a curiousity, it=E2=80=99s probably in the = paper. <inaudible>
T: I think we added that = in the RFC, um, you really need to, like, this part is = important.

_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

= --Apple-Mail=_F22D3651-76DC-421A-8052-A6066CB70AB6--