From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 8754521F3B5; Sun, 10 May 2015 10:00:54 -0700 (PDT) Received: from hms-beagle-5.home.lan ([217.247.216.138]) by mail.gmx.com (mrgmx003) with ESMTPSA (Nemesis) id 0MXZbS-1YlYZg3t70-00WXjK; Sun, 10 May 2015 19:00:51 +0200 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Sebastian Moeller In-Reply-To: <88693200-1BB3-4E79-9CCF-788041A5713C@gmail.com> Date: Sun, 10 May 2015 19:00:54 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <0D8C5861-3549-4A41-94A7-66A12E522A48@gmx.de> References: <88693200-1BB3-4E79-9CCF-788041A5713C@gmail.com> To: Jonathan Morton X-Mailer: Apple Mail (2.1878.6) X-Provags-ID: V03:K0:dZ1J0647O9LjAJ2Z7JT408nLEIJuks1WnTyC1+34z5Ay4anogiv ehiaKZ6t0XS36JA4FeKunZKUEjyhXEUglNzVOYkHW46fUDpFno8q86hBeTUotYBcwaR6vJT 9Y5hRSh69qr0TnKedg4YY4KM4r1/6K2l2pu+CdcrvEPqqNWjbcvq9tKacX0lGNgyClDbPii 9AeZ5oHqIq0pNuVK5oN1Q== X-UI-Out-Filterresults: notjunk:1; Cc: cake@lists.bufferbloat.net, "codel@lists.bufferbloat.net" , bloat Subject: Re: [Bloat] [Codel] [Cake] Control theory and congestion control X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 May 2015 17:01:26 -0000 Hi Jonathan, On May 10, 2015, at 08:55 , Jonathan Morton = wrote: >=20 >> On 10 May, 2015, at 06:35, Dave Taht wrote: >>=20 >> On Sat, May 9, 2015 at 12:02 PM, Jonathan Morton = wrote: >>>> The "right" amount of buffering is *1* packet, all the time (the = goal is >>>> nearly 0 latency with 100% utilization). We are quite far from = achieving >>>> that on anything... >>>=20 >>> And control theory shows, I think, that we never will unless the = mechanisms >>> available to us for signalling congestion improve. ECN is good, but = it's not >>> sufficient to achieve that ultimate goal. I'll try to explain why. >>=20 >> The conex and dctcp work explored using ecn for multi-bit signalling. >=20 > A quick glance at those indicates that they=92re focusing on the echo = path - getting the data back from the receiver to the sender. That=92s = the *easy* part; all you need is a small TCP option, which can be = slotted into the padding left by TCP Timestamps and/or SACK, so it = doesn=92t even take any extra space. >=20 > But they do nothing to address the problem of allowing routers to = provide a =93hold=94 signal. Even a single ECN mark has to be taken to = mean =93back off=94; being able to signal that more than one ECN mark = happened in one RTT simply means that you now have a way to say =93back = off harder=94. >=20 > The problem is that we need a three-bit signal (five new-style = signalling states, two states indicating legacy ECN support, and one = =93ECN unsupported=94 state) at the IP layer to do it properly, and = we=92re basically out of bits there, at least in IPv4. The best = solution I can think of right now is to use both of the ECT states = somehow, but we=92d have to make sure that doesn=92t conflict too badly = with existing uses of ECT(1), such as the =93nonce sum=94. Backwards = and forwards compatibility here is essential. On the danger of sounding like I had a tin of snark for = breakfast; what about re-dedicating 3 of the 6 TOS bits for this ;) (if = I understand correctly ethernet and MPLS transports only allow 3 bits = anyway, so the 6 bits are fiction anyway, outside of l3-routers) And the = BCP still is to re-color the TOS bits in ingress, so I guess 3 bits = should be plenty. Best Regards Sebastian >=20 > I=92m thinking about the problem. >=20 >>> Bufferbloat is fundamentally about having insufficient information = at the >>> endpoints about conditions in the network. >>=20 >> Well said. >>=20 >>> We've done a lot to improve that, >>> by moving from zero information to one bit per RTT. But to achieve = that holy >>> grail, we need more information still. >>=20 >> context being aqm + ecn, fq, fq+aqm, fq+aqm+ecn, dctcp, conex, etc. >>=20 >>> Specifically, we need to know when we're at the correct BDP, not = just when >>> it's too high. And it'd be nice if we also knew if we were close to = it. But >>> there is currently no way to provide that information from the = network to >>> the endpoints. >>=20 >> This is where I was pointing out that FQ and the behavior of multiple >> flows in their two phases (slow start and congestion avoidance) >> provides a few pieces of useful information that could possibly be >> used to get closer to the ideal. >=20 > There certainly is enough information available in fq_codel and cake = to derive a five-state congestion signal, rather than a two-state one, = with very little extra effort. >=20 > Flow is sparse -> =93Fast up=94 > Flow is saturating, but no standing queue -> =93Slow up=94 > Flow is saturating, with small standing queue -> =93Hold=94 > Flow is saturating, with large standing queue -> =93Slow down=94 > Flow is saturating, with large, *persistent* standing queue -> =93Fast = down=94 >=20 > In simple terms, =93fast=94 here means =93multiplicative=94 and =93slow=94= means =93additive=94, in the sense of AIMD being the current standard = for TCP behaviour. AIMD itself is a result of the two-state =93bang-bang=94= control model introduced back in the 1980s. >=20 > It=92s worth remembering that the Great Internet Congestion Collapse = Event was 30 years ago, and ECN was specified 15 years ago. >=20 >> A control theory-ish issue with codel is that it depends on an = arbitrary ideal (5ms) as a definition for "good queue", where "a >> gooder queue=94 is, in my definition at the moment, "1 packet = outstanding ever closer to 100% of the time while there is 100% = utilization=94. >=20 > As the above table shows, Codel reacts (by design) only to the most = extreme situation that we would want to plug into an improved = congestion-control model. It=92s really quite remarkable, in that = context, that it works as well as it does. I don=92t think we can hope = to do significantly better until a better signalling mechanism is = available. >=20 > But it does highlight that the correct meaning of an ECN mark is =93back= off hard, now=94. That=92s how it=92s currently interpreted by TCPs, = in accordance with the ECN RFCs, and Codel relies on that behaviour too. = We have to use some other, deliberately softer signal to give a =93hold=94= or even a =93slow down=94 indication. >=20 > - Jonathan Morton >=20 > _______________________________________________ > Codel mailing list > Codel@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/codel