[Ecn-sane] per-flow scheduling

Tue Jul 23 01:00:54 EDT 2019

> On 22 Jul, 2019, at 9:44 am, Bob Briscoe <ietf at bobbriscoe.net> wrote:
> 
> As promised, I've pulled together and uploaded the main architectural arguments about per-flow scheduling that cause concern:
> Per-Flow Scheduling and the End-to-End Argument
> 
> It runs to 6 pages of reading. But I tried to make the time readers will have to spend worth it.

Thanks for posting this.  Upon reading, there is much to disagree with, but at least some points of view can be better understood in their context.

However, there is one thing I must take issue with up front - your repeated assertion that SCE cannot work without FQ.  As some of the plots we showed you this weekend (and maybe even earlier) demonstrate, we *do* have SCE working in a single-queue environment, even with non-SCE traffic mixed in; this is a development since your initial view of SCE several months ago.

SCE will also work just fine (as it always did) with plain RFC-3168 compliant single-queue AQMs, as might reasonably be deployed to improve the short-term overload performance of core or access networks, or as a low-cost bufferbloat mitigation mechanism in switching, head-end or CPE devices.  In that case SCE's behaviour is RFC-3168 compliant and TCP-friendly, and specifically does not require special treatment in the network.

I would therefore appreciate a revision of your paper which removes that false assertion from the several places it is made.

> Finally, I want to emphasize that the purpose is for advocates of per-flow scheduling to understand that there is a coherent world view that agrees with the arguments in this paper. If you have a different set of assumptions and perspectives that leads you to advocate per-flow scheduling and disagree with some of these arguments, that's to be expected. 
> 
> The purpose is to explain why some people don't want FQ, and therefore why it's important to leave the choice open between FQ and DualQ. That has always been the intent of L4S, which supports both. 

A central theme of your paper is a distinction of needs between times of "plenty" and "famine" in terms of network capacity.  However, the dividing line between these states is not defined.  This is a crucial deficiency when the argument made from the start is that FQ can be helpful during "famine" but is detrimental during "plenty".

One reasonable definition of such a dividing line is that "plenty" exists whenever the link is not fully utilised.  But in that case, any modern FQ algorithm will deliver packets in order of arrival, allowing all flows to use as much capacity as they desire.  Only when the packet arrival rate begins to exceed that of draining, when a "famine" state could be said to exist, does FQ apply any management at all.  Even then, if capacity is left over after metering out each flow's fair share, traffic is free to make use of it.  Thus under this definition, FQ is strictly harmless.

Reading between the lines, I suspect that what you mean is that if, after subtracting the capacity used by inelastic flows (such as the VR streams mentioned in one scenario), there is "sufficient" capacity left for elastic flows to experience good performance, then you would say there is still a state of "plenty".  But this is a highly subjective definition and liable to change over time; would 1.5Mbps leftover capacity, equivalent to a T1 line of old, be acceptable today?  I think most modern users would classify it as "famine", but it was considered quite a luxury 20 years ago.

Regardless, my assertion is not that FQ is required for ultra-low latency, but that flows requiring ultra-low latency must be isolated from general traffic.  FQ is a convenient, generic way to do that, and DualQ is another way; other ways exist, such as through Diffserv.  If both traffic types are fed through a single queue, they will also share fates with respect to latency performance, such that every little misbehaviour of the general traffic will be reflected in the latency seen by the more sensitive flows.

It is true that SCE doesn't inherently carry a label distinguishing its traffic from the general set, and thus DualQ cannot be directly applied to it.  But there is a straightforward way to perform this labelling if required, right next door in the Diffserv field.  The recently proposed NQB DSCP would likely be suitable.  I don't think that the majority of potential SCE users would need or even want this distinction (the primary benefit of SCE being better link utilisation by eliminating the traditional deep sawtooth), but the mechanism exists, orthogonally to SCE itself.

I have also drawn up, as a straw-man proposal, CNQ - Cheap Nasty Queuing:

	https://tools.ietf.org/html/draft-morton-tsvwg-cheap-nasty-queueing-00

This is a single-queue AQM, plus a side channel for prioritising sparse flows, although the definition of "sparse" is much stricter than for a true FQ implementation (even LFQ).  In essence, CNQ treats a flow as sparse if its inter-packet gap is greater than the sojourn time of the main queue, and does not attempt to enforce throughput fairness.  This is probably adequate to assist some common latency-sensitive protocols, such as ARP, SSH, NTP and DNS, as well as the initial handshake of longer-lived bulk flows.  You will also notice that there is support for SCE in the application of AQM, though the AQM algorithm itself is only generically specified.

In common with a plain single-queue AQM, CNQ will converge to approximate fairness between TCP-friendly flows, while keeping typical latencies under reasonable control.  Aggressive or meek flows will also behave as expected for a single queue, up to a limit where an extremely meek flow might fall into the sparse queue and thus limit its ability to give way.  This limit will relatively depend on the latency maintained in the main queue, and will probably be several times less than the fair share.

I hope this serves to illustrate that I'm not against single-queue AQMs in an appropriate context, but that their performance limitations need to be borne in mind.  In particular, I consider a single-queue AQM (or CNQ) to be a marked improvement over a dumb FIFO at any bottleneck.

 - Jonathan Morton