From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 10FEC21F5D7 for ; Mon, 16 Nov 2015 05:50:05 -0800 (PST) Received: from hms-beagle.am28.uni-tuebingen.de ([134.2.92.109]) by mail.gmx.com (mrgmx101) with ESMTPSA (Nemesis) id 0MSZ6u-1ZqaQN3gaS-00Rcmg; Mon, 16 Nov 2015 14:49:59 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Sebastian Moeller In-Reply-To: <5649CA7A.5060706@darbyshire-bryant.me.uk> Date: Mon, 16 Nov 2015 14:50:00 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5638A4F4.2010701@darbyshire-bryant.me.uk> <87si4nntcg.fsf@toke.dk> <0196FDEC-50A7-4ECA-9973-1FD23FF2945A@gmx.de> <877flznq3f.fsf@toke.dk> <848953E5-8571-4B81-B67F-D4A7BA4A1F96@darbyshire-bryant.me.uk> <8737wnnpco.fsf@toke.dk> <563B86D4.6030704@darbyshire-bryant.me.uk> <563F21AE.5040506@darbyshire-bryant.me.uk> <56408DAB.6080106@darbyshire-bryant.me.uk> <0B75A2D1-80FC-4499-8DCD-FC2C4E150448@gmail.com> <5649CA7A.5060706@darbyshire-bryant.me.uk> To: Kevin Darbyshire-Bryant X-Mailer: Apple Mail (2.1878.6) X-Provags-ID: V03:K0:YHT0FEX7wYtQ6BNBnnQbVKt1JXXmx0Dhy0PUQb9x2SGblzkE+Rd HcoGBTHVynMR5gZij8kW3x2OCyM+a4Aeyk1gc0ouqZ4KAUWaxjyH761rcd6asJz0Z+bmoHr fH5wWwt7m6nmyShAUJz4xJeexx0bh70FEWvu1MPLxUXIZzFg4j7mGe+CC8/Lbp8SV016A2Z zd35aNMPN79GNwf2VKGCw== X-UI-Out-Filterresults: notjunk:1;V01:K0:pPez+MHH4Yc=:EvyEmkZxUfWEiiJDlVcafu zCIZ083zrWKeFq6bQWHwgHgSsugzAK+FrTl9/3JYQ4H7s3BRrAP2lboYk4g04KUZHWevvCQPJ zSl1hzvsmAVGnAqbfhueP3GMgm7XSTpVICGJ9HqdK9eONujOI7epnhNrtFvkpm5hWlzhl/Xcr roLm7m0yjXncbuixELbhARWepw9Dp8agrTtjdZhgktPuy9iMNC4t4QfIca7pqKhgdAh9JyZrj Z0dJ7b9jB8rb9Bo6D/T4WArUF0RG6EJBEaqq5odmRw8VfAJlWLxEt6+20bG8L0OXZt0B/LB0S B42uT9xs4nfhaD2x1f9lJiqbcbcZyMU+8Yzo8jPiLgjtehTI76hfCgJP2bxQLpIujrAdZ16X9 NGy8tGXEFY918EvYeasnredBk7f74osQi/ktjYLos/iF9ouMKgB5mrrB9EYnpZ70lgQSPZC/+ OdrsblSWP6Bwvvd6lWY8qExPE9eZdjF1ofhGam9Zc7hxXN/R2kyzI0r1qAZHMkcqJ2oZZJa9L ZTEMtcstgnKsv6sCdTk+VyZOI7KhrElVel0bRAXYIOdrWFRFGMpQE2RUQTIROGR4qyEtAX+RZ csiW+mJRAeGKB+eR0bnv5IgrRuzC0ds+vOIEUQxllDLwO7f7Xmrgz/08tn0W8Is0ZlZFQmmFL lYHJA3YiUnY2KsXNBrBAbftxa3xlfgNQOP+i8YKts9vPg/IGmUBvhAvQHXkn9ftLDjgrqRQ0J eIQt8zLMOjYOB8oQ6ES6uCOUOKkdI4skS1wV3bwdUHtdStwaSrqtBnErJWGhHdh/ytsAuqifr snnWS9Q Cc: cake@lists.bufferbloat.net Subject: Re: [Cake] More on 'target' corner cases - rate/target/interval confusion? X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2015 13:50:28 -0000 Hi Kevin, hi Jonathan, On Nov 16, 2015, at 13:22 , Kevin Darbyshire-Bryant = wrote: > I've had a long think :-) >=20 > On 09/11/15 15:07, Jonathan Morton wrote: >>> On 9 Nov, 2015, at 14:12, Kevin Darbyshire-Bryant = wrote: >>>=20 >>> In the presence of a full link, that link having competing =91full' = flows in all 'tins', then how should cake split the link in terms of = bandwidth? >> That=92s a good question, and one that I think becomes more critical = at low bandwidths. I=92ve tended towards generous allocations for that = reason, so as to avoid causing trouble to low-latency applications. >>=20 >> The main requirement I keep in mind is that an application should not = be able to guarantee itself an excessive bandwidth share simply by = selecting a particular DSCP. At the same time, there are applications = for which a relatively large, consistent bandwidth is a requirement for = satisfactory performance (consider streaming video), such that = best-effort traffic should defer to them. These are conflicting = requirements, so a compromise has to be established somehow. >>=20 >> The current thresholds are at 100%, 15/16, =BE and =BC. Under = saturated conditions, this gives throughputs of 1/16, 3/16, =BD and =BC. = The =93video=94 class (Tin 2) can usurp =BE of the bandwidth when = competing against any mixture of best-effort and bulk traffic. This, = admittedly, might turn out to be too much, so I could consider setting = Tin 2=92s threshold at =BD instead of =BE. > Indeed it is a compromise :-) Personally I think 3/4 for video is too > much. Experimentally I found I could significantly reduce bulk & best > effort (down to 1/16th ish) by having video & voice tins fully active.=20= > So I tweaked my cake thresholds to set video to 1/2 rather than 3/4 as > you suggested. Video & Voice can gang together to claim 3/4 bandwidth > still leaving 1/4ish to be battled between bulk & best effort, this is > in my opinion is much better than before. Without a full voice tin, > video could only claim 1/2 in the presence of best effort traffic.=20 > There comes a point where the link bandwidth is so low you have to ask > if you're being realistic in trying to run video over it :-) I just want to note that the rule of thumb for VoIP is 100Kbps = per conversation and direction; if we assume that VoIP will be inside = the VO tier than we should make sure that we have at least 100Kbps = available in both directions, no? I am not opposed to ample bandwidth in = the VI tier, that will allow hoisting all relevant stuff there while = leaving the rest including bit torrent in CS0 or CS1 country. (As far as = I can tell, which is not that far admittedly, no bit torrent client = allows setting DSCP marlins or defaults to CS1, so waiting for them to = move themselves to the BK tier is right up there with waiting for Godot=85= So arguably it seems more worthwhile to move everything else up into a = better tier, offering nicer bandwidth might be an incentive to make apps = look at the problem space=85) Best Regards Sebastian >> And yes, I have long noticed that Flent=92s standard RRUL test = doesn=92t use Tin 2 at all. >>=20 >>> With each increasing priority of tini[0-3], we decrease the = 'expected' bandwidth (good) but also as a result increase the target and = interval. >> Yes, there are a number of counter-intuitive things happening here. >>=20 >> Most of Cake=92s latency-reducing power comes from the liberal = application of flow isolation, *not* from AQM itself. Diffserv = prioritisation plays a lesser role, and mainly has to do with restoring = the desired allocations of bandwidth, replacing the reliance on measure = queue fill level that some protocols presently use to stay out from = underfoot. Both these mechanisms primarily control the effects that one = flow can have on another, and say little about the latency that a flow = causes to itself. >>=20 >> This latter is the domain of AQM, specifically Codel, which is what = we=92re talking about when we mention =93interval" and "target=94. As I = mentioned elsewhere recently, Codel is designed specifically to give = congestion signals to TCP-like flows, and deals rather less efficiently = with unresponsive and anti-responsive flows, which as a result tend to = spend some time bouncing off the queue=92s hard limit until Codel finds = the correct operating point to control them. There are other AQMs which = are designed with unresponsive flows more in mind, but which somehow = perform less well with TCP-like flows. >>=20 >> A key design principle of Codel is that no packet whose sojourn time = is below target will be signalled. However, if the sojourn time is = consistently above target, signalling begins and increases steadily in = frequency. It is also a fundamental truth that if it takes longer than = target to transmit the previous packet, the following packet can have a = =93congested=94 sojourn time even if there is consistently precisely one = packet in the queue (which is the ideal state). This is why I constrain = =93target=94 to be at least 1.5 packet times at MTU; the difference can = be substantial at low bandwidths. >>=20 >> But there are subtleties here too. >>=20 >> If there are multiple flows, isolated into multiple queues, then the = effective packet-to-packet time for each queue will be increased = proportionately. Early versions of Codel refused to signal on the last = packet in the queue, to account for this. However, if there were a = large number of occupied queues, this meant that the minimum queue fill = could be rather high, and this seemed to lead to high induced latencies = in fq_codel under heavy load. Due to the statistical multiplexing = effect, it turned out to be sufficient to tune =93target=94 as above for = the final output bandwidth (even though this is unknown to fq_codel) and = to remove the special status of the last remaining packet. >>=20 >> The same logic could naively be applied to traffic in separate tins. = However, unlike queues for flow isolation, bandwidth is not shared = evenly between tins. More subtly, traffic characteristics also differ - = low-latency traffic tends to be unresponsive to TCP-style congestion = signalling, and dropping any of it tends to reduce service quality in = some way. Note that network-control traffic (most relevantly NTP) falls = into the =93voice=94 category. Since unresponsive flows aren=92t what = Codel is meant to deal with, the mere fact that =93target=94 is higher = is not meaningful - and in any case this has no effect on the primary = flow-isolation mechanism. >>=20 >> The tin-specific tuning of target and interval was introduced when = Cake had a separate hard shaper per tin. It was the obvious design at = the time. Now that Cake uses soft shaping between tins (allowing any = tin to use the full link bandwidth if uncontended), it=92s possible that = choosing identical target and interval for all tins might suffice. = Alternatively, we might choose an even more conservative strategy. >>=20 >> But which - and how do we decide? > I've no idea! I tried the following experiment: set ingress & egress > b/widths to 2mbit & 448kbit respectively (like my parents adsl...on a > good day) Ran a 'rrul8_splittin' and noted ping box_plot for each tin = - > EF was about 30ms higher (120ms) than CS3 (90ms) Thinking I'd found a > smoking gun and that 120ms was quite close to the EF target of 140ms I > tweaked code to copy tin 0 target across all tins, uploaded,flashed = etc > etc, re-ran test......and........ ..... ..... it barely made any > difference at all! Conclusion 1 - more testing required. conclusion = 2 > - my testing strategy is flawed. conclusion 3 - as you said, flow > isolation is more important than excessively messing with targets and > intervals ;-) >=20 >=20 >>=20 >> As a final point, I haven=92t even mentioned =93rtt=94 as the = user-specified input to this mayhem. That parameter must be understood = to be *separate* from both =93target=94 and =93interval=94, even though = Codel specifies the latter to be related to expected RTT. Simply put, = the user tells us what the expected RTT is (on the understanding that an = order of magnitude variation either way is typical), and we calculate = =93target=94 and =93interval=94 to be as consistent with that estimate = as is practical, given the link bandwidth and other constraints that he = has also specified. So there is a firm conceptual distinction between = the user=92s intent and the implementation details. >>=20 >> - Jonathan Morton >>=20 >=20 >=20 > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake