[Ecn-sane] per-flow scheduling

[David P. Reed]

On Thursday, July 18, 2019 12:31am, "Jonathan Morton" <chromatix99 at gmail.com> said:

>> On 18 Jul, 2019, at 1:18 am, David P. Reed <dpreed at deepplum.com> wrote:
>>
>> So what are we talking about here (ignoring the fine points of SCE, some of which
>> I think are debatable - especially the focus on TCP alone, since much traffic
>> will likely move away from TCP in the near future.
> 
> As a point of order, SCE is not specific to TCP.  TCP is merely the most
> convenient congestion-aware protocol to experiment with, and therefore the one we
> have adapted first.  Other protocols which already are (or aspire to be) TCP
> friendly, especially QUIC, should also be straightforward to adapt to SCE.
>
I agree that it is important to show that SCE in the IP layer can be interpreted in each congestion management system, and TCP is a major one. Ideally there would be a general theory that is protocol and use case agnostic, so that the functions in IP are helpful both in particular protocols and also in the very important case of interacti0ns between different coexisting protocols. I believe that SCE can be structured so that it serves that purpose as more and more protocols that are based on UDP generate more and more traffic - 
when we designed UDP we decided NOT to try to put congestion management in UDP deliberately, for two reasons:
1) we didn't yet have a good congestion management approach for TCP, and
2) major use cases that argued for UDP (packet speech, in particular, but lots of computer-computer interactions on LANs, such as Remote Procedure Calls, etc.) were known to require different approaches to congestion management beyond the basic packet-drop (such as rate management via compression variation).
UDP was part of the design choice to allow end-to-end agreement about congestion management implementation.
We now have a very complex protocol due to the WWW, that imperfectly works on TCP. Thus, a new UDP based protocol framework is proposed, and will be quite heavily used in access networks that need congestion management, both at the server end and the client end.
And we have heavy use of media streaming (though it matches TCP adequately well, being file-transfer-like due to buffering at the receiving end). 

Google and others are working hard to transition entirely away from HTTP/TCP to HTTP/QUIC/UDP. This transition will be concurrent, if not prior, to SCE integration into IP. I would hope that QUIC could use SCE to great advantage, especially in helping the co-existence of two competing uses for the same bottleneck paths without queueing delay.

That's the case that matters to me, along with RTP and other uses. From browser-level monitoring, we already see many landing web pages open up HTTP requests to 100's of different server hosts concurrently. Yes, that is hundreds for one, count them, one click.

This is not a bad thing. The designers of the Internet should not be allowed to say that it is wrong. Because it isn't wrong - it's exactly what the Internet is supposed to be able to do! However, the browser or its host must have the information to avoid queue overflow in this new protocol. That means a useful means like SCE

It also, I believe, means that arbitration based on "flows" matter. So per-flow interactions matter. I don't know, but I believe that when lots of browsers end up sharing a bottleneck link/queue, per-flow scheduling may help a reasonable amount, primarily by preventing starvation of resources. (In scheduling of parallel computing, we call that "prevention of livelock". And when you have a hundred processors on a computer - which is what my day job supports, you get livelock ALL the time if you don't guarantee that all contenders on a resource get a chance.)  What does NOT matter is some complex (intserv/diffserv) differentiation at the router level, or at least not much.


> I should also note that TCP is the de-facto gold standard, by which all other
> congestion control is measured, for better or worse.  SCE is included in this,
> insofar as competing reasonably with standard TCP flows under all reasonable
> network conditions is necessary to introduce a new congestion control paradigm. 
> This, I think, is also part of the end-to-end principle.
> 
>  - Jonathan Morton