[Ecn-sane] per-flow scheduling

[Bob Briscoe]

Jonathan


On 23/07/2019 01:00, Jonathan Morton wrote:
>> On 22 Jul, 2019, at 9:44 am, Bob Briscoe <ietf at bobbriscoe.net> wrote:
>>
>> As promised, I've pulled together and uploaded the main architectural arguments about per-flow scheduling that cause concern:
>> Per-Flow Scheduling and the End-to-End Argument
>>
>> It runs to 6 pages of reading. But I tried to make the time readers will have to spend worth it.
> Thanks for posting this.  Upon reading, there is much to disagree with, but at least some points of view can be better understood in their context.
>
> However, there is one thing I must take issue with up front - your repeated assertion that SCE cannot work without FQ.  As some of the plots we showed you this weekend (and maybe even earlier) demonstrate, we *do* have SCE working in a single-queue environment, even with non-SCE traffic mixed in; this is a development since your initial view of SCE several months ago.
>
> SCE will also work just fine (as it always did) with plain RFC-3168 compliant single-queue AQMs, as might reasonably be deployed to improve the short-term overload performance of core or access networks, or as a low-cost bufferbloat mitigation mechanism in switching, head-end or CPE devices.  In that case SCE's behaviour is RFC-3168 compliant and TCP-friendly, and specifically does not require special treatment in the network.
>
> I would therefore appreciate a revision of your paper which removes that false assertion from the several places it is made.
[BB] I was careful not to say "does not work", because I know your 
definition of "works" is not the same as the community working on 
ultra-low latency. I said it needs FQ "in order to provide benefit". 
I'll explain why SCE doesn't provide benefit in a response to your 
posting on LFQ, to keep both this thread and that one on topic.

>> Finally, I want to emphasize that the purpose is for advocates of per-flow scheduling to understand that there is a coherent world view that agrees with the arguments in this paper. If you have a different set of assumptions and perspectives that leads you to advocate per-flow scheduling and disagree with some of these arguments, that's to be expected.
>>
>> The purpose is to explain why some people don't want FQ, and therefore why it's important to leave the choice open between FQ and DualQ. That has always been the intent of L4S, which supports both.
> A central theme of your paper is a distinction of needs between times of "plenty" and "famine" in terms of network capacity.  However, the dividing line between these states is not defined.  This is a crucial deficiency when the argument made from the start is that FQ can be helpful during "famine" but is detrimental during "plenty".
>
> One reasonable definition of such a dividing line is that "plenty" exists whenever the link is not fully utilised.  But in that case, any modern FQ algorithm will deliver packets in order of arrival, allowing all flows to use as much capacity as they desire.  Only when the packet arrival rate begins to exceed that of draining, when a "famine" state could be said to exist, does FQ apply any management at all.  Even then, if capacity is left over after metering out each flow's fair share, traffic is free to make use of it.  Thus under this definition, FQ is strictly harmless.
>
> Reading between the lines, I suspect that what you mean is that if, after subtracting the capacity used by inelastic flows (such as the VR streams mentioned in one scenario), there is "sufficient" capacity left for elastic flows to experience good performance, then you would say there is still a state of "plenty".  But this is a highly subjective definition and liable to change over time; would 1.5Mbps leftover capacity, equivalent to a T1 line of old, be acceptable today?  I think most modern users would classify it as "famine", but it was considered quite a luxury 20 years ago.
[BB] The objective measure of famine/plenty that all congestion controls 
use is a combination of the loss level, ECN level, queue delay, etc. In 
the paper, I referred to BBRv2's distinction between famine and plenty 
(which is at 1% loss) and in a footnote I expressed a preference for a 
more gradual transition.

Many real-time congestion controls (e.g. the algos used in production 
video chat and conferencing products etc) include such a gradual 
transition from fast responsiveness at high congestion levels (loss in 
single digit percentage area) to being virtually unresponsive at much 
lower levels of congestion.

There's lots of diversity, and this ecosystem is healthy and nowhere 
near threatening Internet collapse. So I think introducing a "standard" 
transition would just create a chilling effect on innovation.

The use of the two words famine and plenty wasn't intended to imply only 
two states. It's a continuum (like the spectrum between famine and plenty).

> Regardless, my assertion is not that FQ is required for ultra-low latency, but that flows requiring ultra-low latency must be isolated from general traffic.  FQ is a convenient, generic way to do that, and DualQ is another way; other ways exist, such as through Diffserv.  If both traffic types are fed through a single queue, they will also share fates with respect to latency performance, such that every little misbehaviour of the general traffic will be reflected in the latency seen by the more sensitive flows.
>
> It is true that SCE doesn't inherently carry a label distinguishing its traffic from the general set, and thus DualQ cannot be directly applied to it.  But there is a straightforward way to perform this labelling if required, right next door in the Diffserv field.  The recently proposed NQB DSCP would likely be suitable.  I don't think that the majority of potential SCE users would need or even want this distinction (the primary benefit of SCE being better link utilisation by eliminating the traditional deep sawtooth), but the mechanism exists, orthogonally to SCE itself.
To enable SCE and RFC3168 in two queues rather than per-flow queues, if 
you required SCE packets to be identified by a DSCP, if the DSCP got 
wiped (which it often does), your SCE traffic would mix with 3168 
traffic and starve itself.


>
> I have also drawn up, as a straw-man proposal, CNQ - Cheap Nasty Queuing:
>
> 	https://tools.ietf.org/html/draft-morton-tsvwg-cheap-nasty-queueing-00
>
> This is a single-queue AQM, plus a side channel for prioritising sparse flows, although the definition of "sparse" is much stricter than for a true FQ implementation (even LFQ).  In essence, CNQ treats a flow as sparse if its inter-packet gap is greater than the sojourn time of the main queue, and does not attempt to enforce throughput fairness.  This is probably adequate to assist some common latency-sensitive protocols, such as ARP, SSH, NTP and DNS, as well as the initial handshake of longer-lived bulk flows.  You will also notice that there is support for SCE in the application of AQM, though the AQM algorithm itself is only generically specified.
>
> In common with a plain single-queue AQM, CNQ will converge to approximate fairness between TCP-friendly flows, while keeping typical latencies under reasonable control.  Aggressive or meek flows will also behave as expected for a single queue, up to a limit where an extremely meek flow might fall into the sparse queue and thus limit its ability to give way.  This limit will relatively depend on the latency maintained in the main queue, and will probably be several times less than the fair share.
>
> I hope this serves to illustrate that I'm not against single-queue AQMs in an appropriate context, but that their performance limitations need to be borne in mind.  In particular, I consider a single-queue AQM (or CNQ) to be a marked improvement over a dumb FIFO at any bottleneck.
OK. Can you say whether you've tested this exhausitvely? Need to know 
before we all spend time reading it in too much depth.



Bob

>
>   - Jonathan Morton
>
> _______________________________________________
> Ecn-sane mailing list
> Ecn-sane at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/ecn-sane

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/