From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt1-x833.google.com (mail-qt1-x833.google.com [IPv6:2607:f8b0:4864:20::833]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 966B13B2A4 for ; Tue, 23 Jul 2019 01:00:57 -0400 (EDT) Received: by mail-qt1-x833.google.com with SMTP id n11so40701670qtl.5 for ; Mon, 22 Jul 2019 22:00:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=nsYYgTCWOFIXyZ+WBEu7OorW722aGxiNfgwBfxzn3CM=; b=gAqRpqajKecV/WxaBonBg9ouSmZsGuPUkZw6C4kqcvdDLMhbY/C868OETqXFWhzZ1W cSAfDWJ1T4sJfn5HAriXLvBuOPSQu2aqmj97Rk3Z1IJHRTNNxqdRwbEfa/PzXU0bna/j iVAs/FXzdUN32UhSXmlq39/EbPbXpFh9mofKCoXTmtGdIMtXetNr5miMovP697mh36BX hyVCjLy3grspVGhbyVnA2fk6094SATlkXgzimO/0wMj7ca5RQAvGmsLS0UWiLG0f9/hT XWs4bDAzhrag4SZWWwSACGXVQGw5sQEVDfSEAE0/tGqtYI8ofipgY3bOcP29BPxshlhC v5Xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=nsYYgTCWOFIXyZ+WBEu7OorW722aGxiNfgwBfxzn3CM=; b=iABHBoqy3HcZt0qE9pfOClsYToeVnvvqi8yyJSNEdyaZpRLL1ISZ7ZGrWujc7+DpbV 2bR4qjSglZxLVGCF77n6Wn2DLYG2KNQP2o6sPo51nse6o2OjTSdvwTgjOV+gjqm2G8vf dXMcgKTQX4w/Yx6abBsn9mEPg0s2cojm3tzvWiblnnFeGBdaSEWUZzIgyjXj6fA0WAHP 8kwaoxa/0gjQLnUzEElJioojG449qIpuaHPzOqoAOPvAR9d3yb+CpuL/nFbduhWsQOlL LkZIUgcltJEo93GTMIOiThsDQmmA5h5QJpwuIhHK473jwQNciMOQXI+IqShMZ2hQ3kBF icHQ== X-Gm-Message-State: APjAAAVoxgOUYjz8ejhYoQI2c41m5yOzyzpMwbpdO1kRuucwD+ht9cfX m/eeov0fpYVV3WHIRTRF5AWKKjZREKU= X-Google-Smtp-Source: APXvYqxTI9VcQygt2ygu6wXkRe9zYuutDV1FVsidG2uhER/xptmUUlZYZGRZq9oJZFTedDFU++2qew== X-Received: by 2002:aed:254c:: with SMTP id w12mr53944265qtc.127.1563858056960; Mon, 22 Jul 2019 22:00:56 -0700 (PDT) Received: from jonathartonsmbp.lan (173-246-8-242.qc.cable.ebox.net. [173.246.8.242]) by smtp.gmail.com with ESMTPSA id h4sm19084552qkk.39.2019.07.22.22.00.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Jul 2019 22:00:56 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Jonathan Morton In-Reply-To: Date: Tue, 23 Jul 2019 01:00:54 -0400 Cc: "David P. Reed" , "ecn-sane@lists.bufferbloat.net" , tsvwg IETF list Content-Transfer-Encoding: quoted-printable Message-Id: <8741152A-EBF0-48F4-A123-B34A638322A1@gmail.com> References: <350f8dd5-65d4-d2f3-4d65-784c0379f58c@bobbriscoe.net> <40605F1F-A6F5-4402-9944-238F92926EA6@gmx.de> <1563401917.00951412@apps.rackspace.com> To: Bob Briscoe X-Mailer: Apple Mail (2.3445.9.1) Subject: Re: [Ecn-sane] per-flow scheduling X-BeenThere: ecn-sane@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of explicit congestion notification's impact on the Internet List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jul 2019 05:00:57 -0000 > On 22 Jul, 2019, at 9:44 am, Bob Briscoe wrote: >=20 > As promised, I've pulled together and uploaded the main architectural = arguments about per-flow scheduling that cause concern: > Per-Flow Scheduling and the End-to-End Argument >=20 > It runs to 6 pages of reading. But I tried to make the time readers = will have to spend worth it. Thanks for posting this. Upon reading, there is much to disagree with, = but at least some points of view can be better understood in their = context. However, there is one thing I must take issue with up front - your = repeated assertion that SCE cannot work without FQ. As some of the = plots we showed you this weekend (and maybe even earlier) demonstrate, = we *do* have SCE working in a single-queue environment, even with = non-SCE traffic mixed in; this is a development since your initial view = of SCE several months ago. SCE will also work just fine (as it always did) with plain RFC-3168 = compliant single-queue AQMs, as might reasonably be deployed to improve = the short-term overload performance of core or access networks, or as a = low-cost bufferbloat mitigation mechanism in switching, head-end or CPE = devices. In that case SCE's behaviour is RFC-3168 compliant and = TCP-friendly, and specifically does not require special treatment in the = network. I would therefore appreciate a revision of your paper which removes that = false assertion from the several places it is made. > Finally, I want to emphasize that the purpose is for advocates of = per-flow scheduling to understand that there is a coherent world view = that agrees with the arguments in this paper. If you have a different = set of assumptions and perspectives that leads you to advocate per-flow = scheduling and disagree with some of these arguments, that's to be = expected.=20 >=20 > The purpose is to explain why some people don't want FQ, and therefore = why it's important to leave the choice open between FQ and DualQ. That = has always been the intent of L4S, which supports both.=20 A central theme of your paper is a distinction of needs between times of = "plenty" and "famine" in terms of network capacity. However, the = dividing line between these states is not defined. This is a crucial = deficiency when the argument made from the start is that FQ can be = helpful during "famine" but is detrimental during "plenty". One reasonable definition of such a dividing line is that "plenty" = exists whenever the link is not fully utilised. But in that case, any = modern FQ algorithm will deliver packets in order of arrival, allowing = all flows to use as much capacity as they desire. Only when the packet = arrival rate begins to exceed that of draining, when a "famine" state = could be said to exist, does FQ apply any management at all. Even then, = if capacity is left over after metering out each flow's fair share, = traffic is free to make use of it. Thus under this definition, FQ is = strictly harmless. Reading between the lines, I suspect that what you mean is that if, = after subtracting the capacity used by inelastic flows (such as the VR = streams mentioned in one scenario), there is "sufficient" capacity left = for elastic flows to experience good performance, then you would say = there is still a state of "plenty". But this is a highly subjective = definition and liable to change over time; would 1.5Mbps leftover = capacity, equivalent to a T1 line of old, be acceptable today? I think = most modern users would classify it as "famine", but it was considered = quite a luxury 20 years ago. Regardless, my assertion is not that FQ is required for ultra-low = latency, but that flows requiring ultra-low latency must be isolated = from general traffic. FQ is a convenient, generic way to do that, and = DualQ is another way; other ways exist, such as through Diffserv. If = both traffic types are fed through a single queue, they will also share = fates with respect to latency performance, such that every little = misbehaviour of the general traffic will be reflected in the latency = seen by the more sensitive flows. It is true that SCE doesn't inherently carry a label distinguishing its = traffic from the general set, and thus DualQ cannot be directly applied = to it. But there is a straightforward way to perform this labelling if = required, right next door in the Diffserv field. The recently proposed = NQB DSCP would likely be suitable. I don't think that the majority of = potential SCE users would need or even want this distinction (the = primary benefit of SCE being better link utilisation by eliminating the = traditional deep sawtooth), but the mechanism exists, orthogonally to = SCE itself. I have also drawn up, as a straw-man proposal, CNQ - Cheap Nasty = Queuing: = https://tools.ietf.org/html/draft-morton-tsvwg-cheap-nasty-queueing-00 This is a single-queue AQM, plus a side channel for prioritising sparse = flows, although the definition of "sparse" is much stricter than for a = true FQ implementation (even LFQ). In essence, CNQ treats a flow as = sparse if its inter-packet gap is greater than the sojourn time of the = main queue, and does not attempt to enforce throughput fairness. This = is probably adequate to assist some common latency-sensitive protocols, = such as ARP, SSH, NTP and DNS, as well as the initial handshake of = longer-lived bulk flows. You will also notice that there is support for = SCE in the application of AQM, though the AQM algorithm itself is only = generically specified. In common with a plain single-queue AQM, CNQ will converge to = approximate fairness between TCP-friendly flows, while keeping typical = latencies under reasonable control. Aggressive or meek flows will also = behave as expected for a single queue, up to a limit where an extremely = meek flow might fall into the sparse queue and thus limit its ability to = give way. This limit will relatively depend on the latency maintained = in the main queue, and will probably be several times less than the fair = share. I hope this serves to illustrate that I'm not against single-queue AQMs = in an appropriate context, but that their performance limitations need = to be borne in mind. In particular, I consider a single-queue AQM (or = CNQ) to be a marked improvement over a dumb FIFO at any bottleneck. - Jonathan Morton