From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.perfora.net (mout.perfora.net [74.208.4.196]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mout.perfora.net", Issuer "Thawte SSL CA" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 3651B21F54B for ; Thu, 28 Aug 2014 06:20:00 -0700 (PDT) Received: from J4 (c-68-50-226-187.hsd1.md.comcast.net [68.50.226.187]) by mrelay.perfora.net (node=mreueus001) with ESMTP (Nemesis) id 0LyCFX-1WHo9v2W29-015bZR; Thu, 28 Aug 2014 15:19:50 +0200 From: "Jerry Jongerius" To: "'Greg White'" , "'Sebastian Moeller'" References: <000001cfbefe$69194c70$3b4be550$@duckware.com> In-Reply-To: Date: Thu, 28 Aug 2014 09:19:50 -0400 Message-ID: <000901cfc2c2$c21ae460$4650ad20$@duckware.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_000A_01CFC2A1.3B0DFF50" X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQFFVy+FAO2HJAXNGbNNrLY/R/b/6gGg5QYjATtzUJuc4+ugwA== Content-Language: en-us X-Provags-ID: V02:K0:3tdEeIaBgfHOEVq7Sp2Ilk0aiEKr4GbXtRrmiWpdHct TQASPEKQisQ51hESjuo3zqhG8sDQdtbEcS64zsPverE4uPSCIH vkcepL0IDx29BGOc1rQsJFQ+BgssVdVutyiVDFwCJaFpnzJtAx 0tL7LiCPixMR9Xx3KZYziY+hRiVRnXsWDwN7iccMlehmV4PwDT DhqcHbXJBo63KM3/8OsvHEi+iKMuV12FUPdOkHUhu1iqk8sI1I GFHsWRNCqJlgaBpdsI74A//V7aTHzmms/E6xGmKSivFfpikarY f0+78pyjwSIpITU3tim0bGbcW4EiOtcnqkIasogGfISi0bprNd biZPE0ZXZjDbyNikciEsDu5uE/mtSUXUPYzF67Aqa X-UI-Out-Filterresults: notjunk:1; Cc: bloat@lists.bufferbloat.net Subject: Re: [Bloat] The Dark Problem with AQM in the Internet? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Aug 2014 13:20:00 -0000 This is a multipart message in MIME format. ------=_NextPart_000_000A_01CFC2A1.3B0DFF50 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Mr. White, =20 AQM is a great solution for bufferbloat. End of story. But if you want = to track down which device in the network intentionally dropped a packet = (when many devices in the network path will be running AQM), how are you going = to do that? Or how do you propose to do that? =20 The graph presented is caused the interaction of a single dropped = packet, bufferbloat, and the Westwood+ congestion control algorithm =96 and not = power boost. =20 - Jerry =20 =20 =20 -----Original Message----- From: Greg White [mailto:g.white@CableLabs.com]=20 Sent: Monday, August 25, 2014 1:14 PM To: Sebastian Moeller; Jerry Jongerius Cc: bloat@lists.bufferbloat.net Subject: Re: [Bloat] The Dark Problem with AQM in the Internet? =20 As far as I know there are no deployments of AQM in DOCSIS networks yet. So, the effect you are seeing is unlikely to be due to AQM. =20 As Sebastian indicated, it looks like an interaction between power = boost, a drop tail buffer and the tcp congestion window getting reset to = slow-start. =20 I ran a quick simulation of a simple network with power boost and basic (bloated) drop tail buffer (no AQM) this morning in an attempt to = understand what is going on here. You didn't give me a lot to go on in the text of = your blog post, but nonetheless after playing around with parameters a bit, I = was able to get a result that was close to what you are seeing (attached). = Let me know if you disagree. =20 I'm a bit concerned with the tone of your article, making AQM out to be = the bad guy here ("weapon against end users", etc.). The folks on this list = and those who participate in the IETF AQM WG are working on AQM and packet scheduling algorithms in an attempt to "fix the Internet". At this = point AQM/PS is the best known solution, let's not create negative perceptions unnecessarily. =20 -Greg =20 On 8/23/14, 2:01 PM, "Sebastian Moeller" < moeller0@gmx.de> wrote: =20 >Hi Jerry, >=20 >On Aug 23, 2014, at 20:16 , Jerry Jongerius < = jerryj@duckware.com> wrote: >=20 >> Request for comments on: www.duckware.com/darkaqm >>=20 >> The bottom line: How do you know which AQM device in a network=20 >>intentionally drops a packet, without cooperation from AQM? >>=20 >> Or is this in AQM somewhere and I just missed it? >=20 >=20 >I am sure you will get more expert responses later, but let me try to=20 >comment. >=20 >Paragraph 1: >=20 >I think you hit the nail on the head with your observation: >=20 >The average user can not figure out what AQM device intentionally=20 >dropped packets >=20 >Only, I might add, this does not depend on AQM, the user can not figure = >out where packets where dropped in the case that not all involved=20 >network hops are under said user=B9s control ;) So move on, nothing to=20 >see here ;) >=20 >Paragraph 2: >=20 >There is no guarantee that any network equipment responds to ICMP=20 >requests at all (for example my DSLAM does not). What about pinging a=20 >host further away and look at that hosts RTT development over time? >(Minor clarification: its the load dependent increase of ping RTT to=20 >the CMTS that would be diagnostic of a queue, not the RTT per se). No=20 >increase of ICMP RTT could also mean there is no AQM involved ;) >=20 > I used to think along similar lines, but reading=20 > = = https://www.nanog.org/meetings/nanog47/presentations/Sunday/RAS_Tracero >ute _N47_Sun.pdf made me realize that my assumptions about ping and=20 >trace route were not really backed up by reality. Notably traceroute=20 >will not necessarily show the real data's path and latencies or drop=20 >probability. >=20 >Paragraph 3 >=20 >What is the advertised bandwidth of your link? To my naive eye this=20 >looks a bit like power boosting (the cable company allowing you higher=20 >than advertised bandwidth for a short time that is later reduced to the = >advertised speed). Your plot needs a better legend, BTW, what is the=20 >blue line showing? When you say that neither ping nor trace route=20 >showed anything, I assumed that you measured concurrently to your=20 >download. It would be really great if you could netperf-wrapper to get=20 >comparable data (see the link on=20 > = = http://www.bufferbloat.net/projects/cerowrt/wiki/Quick_Test_for_Bufferb >loa t ) There the latency is not only assessed by ICMP echo requests=20 >but also by UDP packets, and it is very unlikely that your ISP can=20 >special case these in any tricky way, short of giving priority to=20 >sparse flows (which is pretty much what you would like your ISP to do=20 >in the first place ;) ) >=20 > Here is where I reveal that I am just a layman, but you complain about=20 >the loss of one packet, but how do you assume does a (TCP) settle on=20 >its transfer speed? Exactly it keeps increasing until it looses a=20 >packet, then reduces its speed to 50% or so and slowly ramps up again=20 >until the next packet loss. So unless your test data is not TCP I see=20 >no way to avoid packet loss (and no reason why it is harmful). Now if=20 >my power boost intuition should prove right I can explain the massive=20 >drop quite well, TCP had ramped up to above the long-term stable and=20 >suffers several packet losses in a short time, basically resetting it=20 >to 0 or so, therefore the new ramping to 40Mbps looks pretty similar to = >the initial ramping to 110Mbps... >=20 >Paragraph 4: >=20 >I guess, ECN, explicit congestion notification is the best you can=20 >expect, or routers will initially set a mark on a packet to notify the=20 >TCP endpoints that they need to throttle the speed unless that want to=20 >risk packet loss. But not all routers are configured to use it (plus=20 >you need to configure your endpoints correctly, see: > http://www.bufferbloat.net/projects/cerowrt/wiki/Enable_ECN ). But this=20 >will not tell you where along the path congestion occurred, only that=20 >it occurred (and if push comes to shove your packets still get = dropped.) > Also, I believe, a congested router is going to drop = packets to be=20 >able to =B3survive=B2 the current load, it is not going to send = additional=20 >packets to inform you that it is overloaded... > =20 >=20 >Best Regards > Sebastian >=20 >=20 >>=20 >>=20 >> _______________________________________________ >> Bloat mailing list >> Bloat@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/bloat >=20 >_______________________________________________ >Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat =20 ------=_NextPart_000_000A_01CFC2A1.3B0DFF50 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Mr. = White,

 

AQM is a great solution for bufferbloat.=A0 = End of story.=A0 But if you want to track down which device in the = network intentionally dropped a packet (when many devices in the network = path will be running AQM), how are you going to do that?=A0 Or = how do you propose to do that?

 

The = graph presented is caused the interaction of a single dropped packet, = bufferbloat, and the Westwood+ congestion control algorithm – and = not power boost.

 

- = Jerry

 

 

 

-----Original Message-----
From: Greg White = [mailto:g.white@CableLabs.com]
Sent: Monday, August 25, 2014 1:14 = PM
To: Sebastian Moeller; Jerry Jongerius
Cc: = bloat@lists.bufferbloat.net
Subject: Re: [Bloat] The Dark Problem = with AQM in the Internet?

 

As far = as I know there are no deployments of AQM in DOCSIS networks = yet.

So, the effect you are seeing = is unlikely to be due to AQM.

 

As = Sebastian indicated, it looks like an interaction between power boost, a = drop tail buffer and the tcp congestion window getting reset to = slow-start.

 

I ran = a quick simulation of a simple network with power boost and = basic

(bloated) drop tail buffer = (no AQM) this morning in an attempt to understand what is going on here. = You didn't give me a lot to go on in the text of your blog post, but = nonetheless after playing around with parameters a bit, I was able to = get a result that was close to what you are seeing (attached).=A0 Let me = know if you disagree.

 

I'm a = bit concerned with the tone of your article, making AQM out to be the = bad guy here ("weapon against end users", etc.).=A0 The folks = on this list and those who participate in the IETF AQM WG are working on = AQM and packet scheduling algorithms in an attempt to "fix the = Internet".=A0 At this point AQM/PS is the best known solution, = let's not create negative perceptions unnecessarily.

 

-Greg

 

On = 8/23/14, 2:01 PM, "Sebastian Moeller" <moeller0@gmx.de> wrote:

 

>Hi = Jerry,

> 

>On Aug 23, 2014, at 20:16 , Jerry Jongerius = <jerryj@duckware.com> wrote:

> 

>> Request for comments on: www.duckware.com/darkaqm<= /span>

>> =

>> The bottom line: How do = you know which AQM device in a network

>>intentionally=A0 drops a packet, without = cooperation from AQM?

>> =

>> Or is this in AQM = somewhere and I just missed it?

> 

> 

>I am sure you will get more expert responses = later, but let me try to

>comment.

> 

>Paragraph 1:

> 

>I think you hit the nail on the head with your = observation:

> 

>The average user can not figure out what AQM = device intentionally

>dropped = packets

> 

>Only, I might add, this does not depend on AQM, = the user can not figure

>out = where packets where dropped in the case that not all involved =

>network hops are under said = user=B9s control ;) So move on, nothing to

>see here ;)

> 

>Paragraph 2:

> 

>There is no guarantee that any network = equipment responds to ICMP

>requests at all (for example my DSLAM does = not). What about pinging a

>host further away and look at that hosts RTT = development over time?

>(Minor = clarification: its the load dependent increase of ping RTT to =

>the CMTS that would be = diagnostic of a queue, not the RTT per se). No

>increase of ICMP RTT could also mean there is = no AQM involved ;)

> 

>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 I used to = think along similar lines, but reading

>https://www.nanog.org/mee= tings/nanog47/presentations/Sunday/RAS_Tracero

<= p class=3DMsoPlainText>>ute _N47_Sun.pdf made me realize that my = assumptions about ping and

>trace route were not really backed up by = reality. Notably traceroute

>will not necessarily show the real data's path = and latencies or drop

>probability.

> 

>Paragraph 3

> 

>What is the advertised bandwidth of your link? = To my naive eye this

>looks a = bit like power boosting (the cable company allowing you higher =

>than advertised bandwidth for = a short time that is later reduced to the

>advertised speed). Your plot needs a better = legend, BTW, what is the

>blue = line showing? When you say that neither ping nor trace route =

>showed anything, I assumed = that you measured concurrently to your

>download. It would be really great if you could = netperf-wrapper to get

>comparable data (see the link on =

>http://www.bufferbloat.ne= t/projects/cerowrt/wiki/Quick_Test_for_Bufferb

<= p class=3DMsoPlainText>>loa t ) There the latency is not only = assessed by ICMP echo requests

>but also by UDP packets, and it is very = unlikely that your ISP can

>special case these in any tricky way, short of = giving priority to

>sparse = flows (which is pretty much what you would like your ISP to do =

>in the first place ;) = )

> 

>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Here is = where I reveal that I am just a layman, but you complain about =

>the loss of one packet, but = how do you assume does a (TCP) settle on

>its transfer speed? Exactly it keeps increasing = until it looses a

>packet, = then reduces its speed to 50% or so and slowly ramps up again =

>until the next packet loss. = So unless your test data is not TCP I see

>no way to avoid packet loss (and no reason why = it is harmful). Now if

>my = power boost intuition should prove right I can explain the massive =

>drop quite well, TCP had = ramped up to above the long-term stable and

>suffers several packet losses in a short time, = basically resetting it

>to 0 = or so, therefore the new ramping to 40Mbps looks pretty similar to =

>the initial ramping to = 110Mbps...

> 

>Paragraph 4:

> 

>I guess, ECN, explicit congestion notification = is the best you can

>expect, = or routers will initially set a mark on a packet to notify the =

>TCP endpoints that they need = to throttle the speed unless that want to

>risk packet loss. But not all routers are = configured to use it (plus

>you need to configure your endpoints correctly, = see:

>http://www.bufferbloat.ne= t/projects/cerowrt/wiki/Enable_ECN ). But this =

>will not tell you where along = the path congestion occurred, only that

>it occurred (and if push comes to shove your = packets still get dropped.)

>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Also, I = believe, a congested router is going to drop packets to be =

>able to =B3survive=B2 the = current load, it is not going to send additional

>packets to inform you that it is = overloaded...

>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =

> 

>Best Regards

>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 = Sebastian

> 

> 

>>

>>

>> = _______________________________________________

>> Bloat mailing list

>> Bloat@lists.bufferbloat.n= et

>> https://lists.bufferbloat= .net/listinfo/bloat

> 

>_______________________________________________<= o:p>

>Bloat mailing = list

>Bloat@lists.bufferbloat.n= et

>https://lists.bufferbloat= .net/listinfo/bloat

 

------=_NextPart_000_000A_01CFC2A1.3B0DFF50--