[Ecn-sane] [tsvwg] Comments on L4S drafts

Fri Jul 19 05:06:38 EDT 2019

Hi Sebastian,

To avoid people to read through the long mail, I think the main point I want to make is:
 "Indeed, having common-Qs supported is one of my requirements. That's why I want to keep the discussion on that level: is there consensus that low latency is only needed for a per flow FQ system with an AQM per flow?"

If there is this consensus, this means that we can use SCE and that from now on, all network nodes have to implement per flow queuing with an AQM per flow.
If there is no consensus, we cannot use SCE and need to use L4S.

For all the other detailed discussion topics, see [K] inline:

Regards,
Koen.

-----Original Message-----
From: Sebastian Moeller <moeller0 at gmx.de> 
Sent: Thursday, July 18, 2019 12:40 AM
To: De Schepper, Koen (Nokia - BE/Antwerp) <koen.de_schepper at nokia-bell-labs.com>
Cc: Holland, Jake <jholland at akamai.com>; Jonathan Morton <chromatix99 at gmail.com>; ecn-sane at lists.bufferbloat.net; tsvwg at ietf.org
Subject: Re: [Ecn-sane] [tsvwg] Comments on L4S drafts

Dear Koen,

> On Jul 10, 2019, at 11:00, De Schepper, Koen (Nokia - BE/Antwerp) <koen.de_schepper at nokia-bell-labs.com> wrote:
[...]
>>> Are you saying that even if a scalable FQ can be implemented in high-volume aggregated links at the same cost and difficulty as dualq, there's a reason not to use FQ?
> 
> FQ for "per-user" isolation in access equipment has clearly an extra cost, not? If we need to implement FQ "per-flow" on top, we need 2 levels of FQ (per-user and per-user-flow, so from thousands to millions of queues). Also, I haven’t seen DC switches coming with an FQ AQM...

	I believe there is work available demonstrating that a) millions of concurrently active flows might be overly pessimistic (even for peering routers) and b) IMHO it is far from settled that these bid transit/peering routers will employ any of the the schemes we are cooking up here. For b) I argue that both L4S "linear" CE-marking and SCE linear ECT(1) marking will give a clear signal of overload that an big ISP might not want to explicitly tell its customers...

[K] a) indeed, if queues can be dynamically allocated you could settle with less, not sure if dynamic allocation is compatible with high speed implementations. Anyway, any additional complexity is additional cost (and cycles, energy, heat dissipation, ...). Of course everything can be done...
b) I don't agree that ECN is a signal of overload. It is a natural part of feedback to tell greedy TCP that it reached its full capacity. Excessive drop/latency is the signal of overload and an ECN-capable AQM switches from ECN to drop anyway in overload conditions.  Excessive drop and latency can also be measured today, not? Running a few probes can tell customers the same with or without ECN, and capacity is measured simply with speedtests.

> 
>>> Is there a use case where it's necessary to avoid strict isolation if strict isolation can be accomplished as cheaply?
> 
> Even if as cheaply, as long as there is no reliable flow identification, it clearly has side effects. Many homeworkers are using a VPN tunnel, which is only one flow encapsulating maybe dozens.

	Fair enough, but why do you see a problem of treating this multiplexed flow as different from any other flow, after all it was the end-points conscious decision to masquerade as a single flow so why assume special treatment; it is not that intermediate hops have any insight into the multiplexing, so why expect them to cater for this?

[K] Because the design of VPN tunnels had as a main goal to maintain a secure/encrypted connection between clients and servers, trying to minimize the overhead on clients and servers by using a single TCP/UDP connection. I don't think the single flow was chosen to get treated as one flow's budget of throughput. This "feature" didn't exist at that (pre-FQ) time.

> Drop and ECN (if implemented correctly) are tunnel agnostic.

	Exactly, and that is true for each identified flow as well, so fq does not diminish this, but rather builds on top of it.

[K] True for flows within a tunnel, but the point was that FQ treats the aggregated tunnel as a single flow compared to other single flows.

> Also how flows are identified might evolve (new transport protocols, encapsulations, ...?).

	You are jesting surely, new protocols? We are in this kefuffle, because you claim that a new protocol to signal linear CE-marking response to be made of unobtaininum so you want to abuse an underused EVN code point as a classifier. If new protocols are an option, just bite the bullet and give tcp-reno a new protocol number and use this for your L4S classifier; problem solved in a nice and clean fashion.

[K] Indeed, it is hardly impossible to deploy new protocols in practice, but I hope we can make it more possible in the future, not less possible... Maybe utopic, but at least we should try to learn from past mistakes.

> Also if strict flow isolation could be done correctly, it has additional issues related to missed scheduling opportunities,

	Please elaborate, how an intermediate hop would know about the desires of the endpoints here. As far as I can tell such hops have their own ideas about optimal scheduling that they will enforce independent of the what the endpoints deem optimal (by ncessity as most endpoints will desire highest priority for their packets).

[K] That network nodes cannot know what the end-systems want is exactly the point. FQ just assumes everybody should have the same throughput, and makes an exception for single packets (to undo the most flagrant disadvantage of this strategy). But again, I don't want to let the discussion get distracted by arguing pro or con FQ. I think we have to live with both now.

[...]

>>> Anyway, to me this discussion is about the tradeoffs between the 2 proposals.  It seems to me SCE has some safety advantages that should not be thrown away lightly, 
> 
> I appreciate the efforts of trying to improve L4S, but nobody working on L4S for years now see a way that SCE can work on a non-FQ system.

	That is a rather peculiar argument, especially given that both you and Bob, major forces in the L4S approach, seemm to have philosophical issues with fq?

[K] I think I am realistic to accept pro's and con's and existence of both. I think wanting only FQ is as philosophical as wanting no FQ at all.

> For me (and I think many others) it is a no-go to only support FQ. Unfortunately we only have half a bit free, 

	??? Again you elaborately state the options in the L4S RFC and just converge on the one which is most convenient, but also not the best match for your requirements.

[K] Indeed, having common-Qs supported is one of my requirements. That's why I want to keep the discussion on that level: is there consensus that low latency is only needed for a per flow FQ system with an AQM per flow?

> and we need to choose how to use it. Would you choose for the existing ECN switches that cannot be upgraded (are there any?) or for all future non-FQ systems.
> 
>>> so if the performance can be made equivalent, it would be good to know about it before committing the codepoint.
> 
> The performance in FQ is clearly equivalent, but for a common-Q behavior, only L4S can work.
> As far as I understood the SCE-LFQ proposal is actually a slower FQ implementation (an FQ in DualQ disguise 😉), so I think not really a better alternative than pure FQ. Also its single AQM on the bulk queue will undo any isolation, as a coupled AQM is stronger than any scheduler, including FQ.

	But how would the bulk queue actually care, being dedicated to bulk flows? This basically just uses a single codel instance for all flows in the bulk queue, exactly the situation codel was designed for, if I recall correctly. Sure this will run into problems with unrepsonsive flows, but not any more than DualQ with or without  queue protection (you can steer misbehaving flows into the the "classic" queue, but this will just change which flows will suffer most of the collateral damage of that unresponsive flow, IMHO).

[K] As far as I recall, CoDel works best for a single flow. For a stateless AQM like a step using only per packet sojourn time, a common AQM over FQs is indeed working as an FQ with an AQM per queue. Applying an stateless AQM for Classic traffic (like sojourn-time RED without smoothing) will have impact on its performance. Adding common state for all bulk queue AQMs will disable the FQ effect. Anyway, the sequential scan at dequeue is the main reason why LFQ will be hard to get traction in high-speed equipment.

Best Regards
	Sebastian Moeller