From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 6AC6E21F5CC for ; Mon, 16 Nov 2015 05:34:13 -0800 (PST) Received: from hms-beagle.am28.uni-tuebingen.de ([134.2.92.109]) by mail.gmx.com (mrgmx003) with ESMTPSA (Nemesis) id 0Lkx9B-1aYD2K15IT-00ai4a; Mon, 16 Nov 2015 14:34:09 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Sebastian Moeller In-Reply-To: <0B75A2D1-80FC-4499-8DCD-FC2C4E150448@gmail.com> Date: Mon, 16 Nov 2015 14:34:08 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5638A4F4.2010701@darbyshire-bryant.me.uk> <87si4nntcg.fsf@toke.dk> <0196FDEC-50A7-4ECA-9973-1FD23FF2945A@gmx.de> <877flznq3f.fsf@toke.dk> <848953E5-8571-4B81-B67F-D4A7BA4A1F96@darbyshire-bryant.me.uk> <8737wnnpco.fsf@toke.dk> <563B86D4.6030704@darbyshire-bryant.me.uk> <563F21AE.5040506@darbyshire-bryant.me.uk> <56408DAB.6080106@darbyshire-bryant.me.uk> <0B75A2D1-80FC-4499-8DCD-FC2C4E150448@gmail.com> To: Jonathan Morton X-Mailer: Apple Mail (2.1878.6) X-Provags-ID: V03:K0:XPUmB3G+jCn3qc37dSje7Cb6hAMBZXesiapjY7Z8kkLrOr8YOSl owL5kGiOoRM+Sq6KM8urPIvc8yJwX5zquXUZ7mItXk1/uRp1suwr84ZilHcKofUKA6hFTSw W7YHSBID8R3rs5pCDm7Wysi80s1F7atKVdj+zXP1E/tdPUgem0HlpYsO1nZPneaYvj97r4p U+rmlHjdnEZL95Z55IE4g== X-UI-Out-Filterresults: notjunk:1;V01:K0:v7wU4X9AjJ8=:gDNSr9WtQsqdI0cL8CelA1 HuDxIKDGhVx2dtNzlbp6WI9jzBVk2Qo6zJ03T/JwoizZ3kZ7ZgoVO+4kE9Hzk35XvExAj2v6i PyfIqIrHqauN1qVRGzarlAk8hipvnczvd+1Ny4ZQQ2rgN0GD2ySaTf9mlcilOFzEFU3KhnVo9 qaSIViPS6Ol8fb6lW31GD9upohe4/yqX4ozK0ZT5W4TB/N8F8wLNA1MX4iuvW/MQjl7dWsbqJ aC2os3DsZKD3LxiyCgV+vR0+j7ZhADUh49lrFC+Ey1wFi7rjEhYraLmIJs+WzM2G9+ReNA5ZO PR65HiOBQeNwGxPMoueUglhO3tHVWnFoghU4bdmm5FzlBagwxC8UhA6ZvYwLjdYNg+1OjLZK3 5rOR13UFh46FFgShTpe6L9L2e+T+VY36e6GGZze5YhWnqxRU501eQ542iVjsKFNKsGWDovv9F dqTTuQh4DgKU5KzASG2LYv9C1R+IVHOJiDKujIQqVHzhhZIu4iXmGbHwduhVIRotXwtKqT30M Pi19vWMoWNuOlYqHHfKJkx1fDWWIbFHfZ03Tdb8RAe41PRwJJ/arcMMYtWIwAf1/YcORiASg5 ZSGamr7851ynsLvJ5OQDmqwJ7S25CUjfshuApUvkDHkx4wBYttwBj8+xdX1pGA8iq94DzU4RS 6Tmmac3ujmOtnijt807kI8M4/ajyvS3I9bMw72zTFJrc/luMsvgw9ltmwFc52efwg4c5OiIgI dilG1CtQBHVyM7hdBmPqJXDw661MVy3UeVK1VL02pm3oW+4jtVlWr3Q3sDXz2rylTIQJWwD+y JbtwMxA Cc: cake@lists.bufferbloat.net Subject: Re: [Cake] More on 'target' corner cases - rate/target/interval confusion? X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2015 13:34:37 -0000 On Nov 9, 2015, at 16:07 , Jonathan Morton = wrote: >=20 >> On 9 Nov, 2015, at 14:12, Kevin Darbyshire-Bryant = wrote: >>=20 >> In the presence of a full link, that link having competing =91full' = flows in all 'tins', then how should cake split the link in terms of = bandwidth? >=20 > That=92s a good question, and one that I think becomes more critical = at low bandwidths. I=92ve tended towards generous allocations for that = reason, so as to avoid causing trouble to low-latency applications. >=20 > The main requirement I keep in mind is that an application should not = be able to guarantee itself an excessive bandwidth share simply by = selecting a particular DSCP. At the same time, there are applications = for which a relatively large, consistent bandwidth is a requirement for = satisfactory performance (consider streaming video), such that = best-effort traffic should defer to them. These are conflicting = requirements, so a compromise has to be established somehow. Well, or this can be turned into a policy decision that the user = has to make; as always cake should offer a reasonable default ;) >=20 > The current thresholds are at 100%, 15/16, =BE and =BC. Under = saturated conditions, this gives throughputs of 1/16, 3/16, =BD and =BC. = The =93video=94 class (Tin 2) can usurp =BE of the bandwidth when = competing against any mixture of best-effort and bulk traffic. This, = admittedly, might turn out to be too much, so I could consider setting = Tin 2=92s threshold at =BD instead of =BE. >=20 > And yes, I have long noticed that Flent=92s standard RRUL test doesn=92t= use Tin 2 at all. >=20 >> With each increasing priority of tini[0-3], we decrease the = 'expected' bandwidth (good) but also as a result increase the target and = interval. >=20 > Yes, there are a number of counter-intuitive things happening here. >=20 > Most of Cake=92s latency-reducing power comes from the liberal = application of flow isolation, *not* from AQM itself. Diffserv = prioritisation plays a lesser role, and mainly has to do with restoring = the desired allocations of bandwidth, replacing the reliance on measure = queue fill level that some protocols presently use to stay out from = underfoot. Both these mechanisms primarily control the effects that one = flow can have on another, and say little about the latency that a flow = causes to itself. >=20 > This latter is the domain of AQM, specifically Codel, which is what = we=92re talking about when we mention =93interval" and "target=94. As I = mentioned elsewhere recently, Codel is designed specifically to give = congestion signals to TCP-like flows, and deals rather less efficiently = with unresponsive and anti-responsive flows, which as a result tend to = spend some time bouncing off the queue=92s hard limit until Codel finds = the correct operating point to control them. There are other AQMs which = are designed with unresponsive flows more in mind, but which somehow = perform less well with TCP-like flows. >=20 > A key design principle of Codel is that no packet whose sojourn time = is below target will be signalled. However, if the sojourn time is = consistently above target, signalling begins and increases steadily in = frequency. It is also a fundamental truth that if it takes longer than = target to transmit the previous packet, the following packet can have a = =93congested=94 sojourn time even if there is consistently precisely one = packet in the queue (which is the ideal state). This is why I constrain = =93target=94 to be at least 1.5 packet times at MTU; the difference can = be substantial at low bandwidths. >=20 > But there are subtleties here too. >=20 > If there are multiple flows, isolated into multiple queues, then the = effective packet-to-packet time for each queue will be increased = proportionately. Early versions of Codel refused to signal on the last = packet in the queue, to account for this. However, if there were a = large number of occupied queues, this meant that the minimum queue fill = could be rather high, and this seemed to lead to high induced latencies = in fq_codel under heavy load. Due to the statistical multiplexing = effect, it turned out to be sufficient to tune =93target=94 as above for = the final output bandwidth (even though this is unknown to fq_codel) and = to remove the special status of the last remaining packet. >=20 > The same logic could naively be applied to traffic in separate tins. = However, unlike queues for flow isolation, bandwidth is not shared = evenly between tins. More subtly, traffic characteristics also differ - = low-latency traffic tends to be unresponsive to TCP-style congestion = signalling, and dropping any of it tends to reduce service quality in = some way. Note that network-control traffic (most relevantly NTP) falls = into the =93voice=94 category. Since unresponsive flows aren=92t what = Codel is meant to deal with, the mere fact that =93target=94 is higher = is not meaningful - and in any case this has no effect on the primary = flow-isolation mechanism. Well, but not everything in the higher priority tiers is = not-TCP, I believe even DNS can be operated over TCP, no? And not all = non-TCP flows are unresponsive, so there s still some relevance to = target of the sub tiers. I have bee arguing for the extend interval to a = fixed multiple of target and adapt target to the minimal transfer time = in the past, but I am not certain whether that yields better results = than just letting target grow higher as long as target < interval. (In = the end interval is en estimate of the reaction time window we allow the = flows to receive the congestion signals, and the window does just = increase as MTU_transfer_time + interval) >=20 > The tin-specific tuning of target and interval was introduced when = Cake had a separate hard shaper per tin. It was the obvious design at = the time. Now that Cake uses soft shaping between tins (allowing any = tin to use the full link bandwidth if uncontended), it=92s possible that = choosing identical target and interval for all tins might suffice. = Alternatively, we might choose an even more conservative strategy. >=20 > But which - and how do we decide? >=20 > As a final point, I haven=92t even mentioned =93rtt=94 as the = user-specified input to this mayhem. That parameter must be understood = to be *separate* from both =93target=94 and =93interval=94, even though = Codel specifies the latter to be related to expected RTT. Simply put, = the user tells us what the expected RTT is (on the understanding that an = order of magnitude variation either way is typical), and we calculate = =93target=94 and =93interval=94 to be as consistent with that estimate = as is practical, given the link bandwidth and other constraints that he = has also specified. So there is a firm conceptual distinction between = the user=92s intent and the implementation details. Now, I believe the symbolic arguments like interplanetary are a = great and obvious way to signal intent, numerically specified parameters = however should be honored as well as possible, and changes made to these = by cake should be signaled back to the user, preferably with stating the = reason=85. Best Regards Sebastian >=20 > - Jonathan Morton >=20 > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake