[Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104

[Jonathan Morton]

> On 21 Mar, 2019, at 8:04 am, Bob Briscoe <ietf at bobbriscoe.net> wrote:

> Congestion controls are tricky to get stable in all situations. So it is important to separate ideas and research from engineering of more mature approaches that are ready for more widespread experimentation on the public Internet. Our goal with L4S was to use proven algorithms, and put in place mechanism to allow those algorithms to evolve.

I hope that from my example, you can see how to adapt a more flexible and "mature" version of DCTCP to use SCE.  You should be able to use the same algorithms that you've worked so hard on; only the signalling method changes, and the trigger for falling back to Classic ECN behaviour is explicit (a plain old CE mark).

As for "proven algorithms", it was conclusively proven that DCTCP was *not* compatible with Classic ECN middleboxes, and had only been proven to work in tightly controlled environments.  I am told that TCP Prague has a failsafe, but I do not yet understand how that failsafe works, and what I have been told sounds fragile.  I am honestly perplexed that no explanation of this is forthcoming.

SCE works transparently with every deployed and proven congestion control algorithm out there, which simply ignores the information SCE provides.  Adaptations of some of those algorithms to incorporate SCE information seem to be straightforward to implement, especially since ns-3 now supports AccECN, so initial full-system experiments should be forthcoming quite soon.  We should even be able to rehabilitate DCTCP without resorting to failsafe workarounds - which *should* have you guys jumping for joy, in theory.

> As regards the desire to use SCE instead of the L4S approach of using a classifier, please answer all the reasons I gave for why that won't work, which I sent in response to your draft some days ago.

I'm afraid that must have got lost in the noise.  There *was* a lot of noise; it gave me a headache.

Regardless, I haven't seen any real claims that SCE won't work, except for some quibbles about RTT-fair convergence with single queues, which I subsequently found an elegant way to address.  We do have a bit of a publication bottleneck over here at the moment; limited manpower.

I have mainly seen claims that SCE isn't a one-for-one replacement for L4S using exactly the same mechanisms and infrastructure as L4S does.  Which is true, but unhelpful, because that would make SCE literally identical to L4S with no advantages of its own.  I'm willing to point out ways to implement L4S' goals using SCE; see below.

> The main one is incremental deployment: the source does not identify its packets as distinct from others, so the source needs the network to use some other identifier if it wants the network to put it in a queue with latency that is isolated from packets not using the scheme. The only way I can see to so this would be to use per-flow-queuing. I think that is an unstated assumption of SCE.

Strictly minimising latency for the individual flow, in the face of competing non-SCE traffic sharing a single queue, is not a goal of SCE per se; I consider it an orthogonal problem which is better addressed by existing solutions.  Coexisting with existing endpoints, existing traffic and existing middleboxes is paramount, and forms our main argument for incremental deployability.

Solutions already available include FQ and Diffserv.  I'll grant you that FQ is easier to implement at lowish speeds, where a cheap CPU can be loaded with flexible software to do the job.  You appear to be more focused on relatively high link capacities, as that is your main argument against FQ.  I'll just note in passing that good FQ can extract a lot of responsiveness from relatively low-capacity links.

Diffserv is widely deployed (in terms of hardware capabilities) and should be a natural fit for distinguishing classes of traffic from each other.  It is rarely used by applications because the networks tend to corrupt it in transit, and rarely make good use of the information into the bargain.  It strikes me that the cable industry may have more influence over that than I do.

> The SCE way round does not allow the ECN field to be used as a classifier…

The ECN field was never intended to be used as a classifier, except to distinguish Not-ECT flows from ECT flows (which a middlebox does need to know, to choose between mark and drop behaviours).  It was intended to be used to convey congestion information from the network to the receiver.  SCE adheres to that ideal.

There is a perfectly good and under-utilised 6-bit field for carrying classifier information, right there in the same byte as the ECN field.  You might want to ask the LE PHB guys for advice on making good use of it.

> You also don't get the benefit of being able to relax resequencing in the network, because the network has no classifier to look at.

My position is that the network is already free to relax resequencing semantics, regardless of the traffic carried.  IP does not guarantee anything about packet ordering, and protocols built on top of it have always had to cope with that, one way or another.

Wifi's head-of-line blocking while performing link-level retries can induce inter-flow coupled delays of many seconds in extreme cases, destroying reliability completely.  Recent work already reduces the effort the Linux wifi stack puts into link-level retries, given that most transports and protocols can survive some level of random loss.  This is done without relying on any classifier, because it benefits all traffic.

On high-capacity bonded links, the likelihood that two packets sent near-simultaneously on different component links, and consequently reordered, both belong to the same flow and will trigger a spurious retransmission, seems to be low enough to not care about, even with existing 3-dupack sensitive TCPs.  Therefore, relaxing resequencing requirements on these links should already be safe.  Perhaps you have hard data showing otherwise?

> …the SCE codepoint would need to be combined with a DSCP, and I assume you don't want to do that.

The SCE codepoint does not need to be combined with a DSCP.  Whether or not a DSCP assignment fits a given application is completely orthogonal to SCE.

You could quite reasonably implement something that looks very like DualQ using a trivial DSCP classifier instead of an ECN-based classifier, and that would be absolutely fine, and it would work with SCE.  It's just not necessary to make SCE work in the first place.

Meanwhile, I still have not seen a detailed answer as to how, precisely, TCP Prague reliably distinguishes a Classic ECN middlebox from an L4S one, in order to activate its failsafe mechanism.  Without that, I'm afraid I must assume that TCP Prague is not incrementally deployable.

Indeed, I was under the impression that DualQ and the use of ECT(1) as a classifier stemmed from this incompatibility, rather than being considered features in their own right.

 - Jonathan Morton