[Bloat] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104

[Sebastian Moeller]

Hi Jonathan,


> On Mar 15, 2019, at 15:27, Jonathan Morton <chromatix99 at gmail.com> wrote:
> 
>> On 15 Mar, 2019, at 3:01 pm, Sebastian Moeller <moeller0 at gmx.de> wrote:
>> 
>> That said, having read through the L4S architecture description and the related appendices of draft-ietf-tsvwg-ecn-l4s-id-05 I came to the conclusion, that this is a mess.
>> 
>> The L4S project proposes a really wide-ranging change of basically the internet (but allow a concurrent operation with legacy probably to cater to the fact that an internet-wide flag-day seems daunting to organize). But then they chicken out when figuring out how to differentiate between their new and the old by proposing to use ECT(1)…
> 
> I think I would be less annoyed by L4S if it proposed to assign a DSCP or three to identify its traffic.  In fact, I'd be interested to hear a defence of why DSCP identification of L4S flows was *not* adopted.  I suspect it would revolve around the widespread practice of re-marking DSCPs by ISPs, which renders DSCP too unreliable for L4S' purposes.

https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-05 contains a discussion of the alternatives, specifically to your point it argues:
"Appendix B.  Alternative Identifiers
[...]
B.2.  ECN Plus a Diffserv Codepoint (DSCP)


   Definition:

      For packets with a defined DSCP, all codepoints of the ECN field
      (except Not-ECT) would signify alternative L4S semantics to those
      for classic ECN [RFC3168], specifically:

      *  The L4S DSCP would signifiy that the packet came from an L4S-
         capable sender;

      *  ECT(0) and ECT(1) would both signify that the packet was
         travelling between transport endpoints that were both ECN-
         capable;

      *  CE would signify that the packet had been marked by an AQM
         implementing the L4S service.

   Use of a DSCP is the only approach for alternative ECN semantics
   given as an example in [RFC4774].  However, it was perhaps considered
   more for controlled environments than new end-to-end services;

   Cons:

   Consumes DSCP pairs:  A DSCP is obviously not orthogonal to Diffserv.
      Therefore, wherever the L4S service is applied to multiple
      Diffserv scheduling behaviours, it would be necessary to replace
      each DSCP with a pair of DSCPs.

   Uses critical lower-layer header space:  The resulting increased
      number of DSCPs might be hard to support for some lower layer
      technologies, e.g. 802.1p and MPLS both offer only 3-bits for a
      maximum of 8 traffic class identifiers.  Although L4S should
      reduce and possibly remove the need for some DSCPs intended for
      differentiated queuing delay, it will not remove the need for
      Diffserv entirely, because Diffserv is also used to allocate
      bandwidth, e.g. by prioritising some classes of traffic over
      others when traffic exceeds available capacity.

   Not end-to-end (host-network):  Very few networks honour a DSCP set
      by a host.  Typically a network will zero (bleach) the Diffserv
      field from all hosts.  Sometimes networks will attempt to identify
      applications by some form of packet inspection and, based on
      network policy, they will set the DSCP considered appropriate for
      the identified application.  Network-based application
      identification might use some combination of protocol ID, port
      numbers(s), application layer protocol headers, IP address(es),
      VLAN ID(s) and even packet timing.

   Not end-to-end (network-network):  Very few networks honour a DSCP
      received from a neighbouring network.  Typically a network will
      zero (bleach) the Diffserv field from all neighbouring networks at
      an interconnection point.  Sometimes bilateral arrangements are
      made between networks, such that the receiving network remarks
      some DSCPs to those it uses for roughly equivalent services.  The
      likelihood that a DSCP will be bleached or ignored depends on the
      type of DSCP:

      Local-use DSCP:  These tend to be used to implement application-
         specific network policies, but a bilateral arrangement to
         remark certain DSCPs is often applied to DSCPs in the local-use
         range simply because it is easier not to change all of a
         network's internal configurations when a new arrangement is
         made with a neighbour;

      Global-use DSCP:  These do not tend to be honoured across network
         interconnections more than local-use DSCPs.  However, if two
         networks decide to honour certain of each other's DSCPs, the
         reconfiguration is a little easier if both of their globally
         recognised services are already represented by the relevant
         global-use DSCPs.

         Note that today a global-use DSCP gives little more assurance
         of end-to-end service than a local-use DSCP.  In future the
         global-use range might give more assurance of end-to-end
         service than local-use, but it is unlikely that either
         assurance will be high, particularly given the hosts are
         included in the end-to-end path.

   Not all tunnels:  Diffserv codepoints are often not propagated to the
      outer header when a packet is encapsulated by a tunnel header.
      DSCPs are propagated to the outer of uniform mode tunnels, but not
      pipe mode [RFC2983], and pipe mode is fairly common.

   ECN hard in some lower layers::  Because this approach uses both the
      Diffserv and ECN fields, an AQM wil only work at a lower layer if
      both can be supported.  If individual network operators wished to
      deploy an AQM at a lower layer, they would usually propagate an IP
      Diffserv codepoint to the lower layer, using for example IEEE
      802.1p.  However, the ECN capability is harder to propagate down
      to lower layers because few lower layers support it.

   Pros:

   Could migrate to e2e:  If all usage of classic ECN migrates to usage
      of L4S, the DSCP would become redundant, and the ECN capability
      alone could eventually identify L4S packets without the
      interconnection problems of Diffserv detailed above, and without
      having permanently consumed more than one codepoint in the IP
      header.  Although the DSCP does not generally function as an end-
      to-end identifier (see above), it could be used initially by
      individual ISPs to introduce the L4S service for their own locally
      generated traffic;"


In essence, they do not want to deal with the diffserv mess under the end-to-end perspective and making it reliable.


> 
> But using the ECN field for this purpose is *also* unreliable, because it is *intended* to be altered on route (in prescribed ways).  In particular, both ECT codepoints can be replaced by CE, erasing the distinction between them when it comes to choosing which half of a "dual queue" AQM to pass it through.  I'm not convinced they've spent very much of the past several years thinking about that.

	There is also a discussion of this in Appendix B.  Alternative Identifiers, which I will not copy here ;)


> 
> Fortunately, L4S flows should work with flow-isolating AQMs (including Cake) without modifying the latter, and without needing to specifically identify which flows are L4S or otherwise.  That really says more about the robustness of proper flow isolation than anything else.  But Codel-type AQMs don't provide the ideal ECN signal for L4S (and nor do RED-type AQMs without specific tuning for L4S), and any single-queue AQM is going to end up with L4S flows outcompeting standard ones quite seriously.

	Except L$S flows are required to "degrade" to classic congestion contro once they have proof of not being handled by a l4s aware AQM, like real packet drops or ECT(0) + CE...

> 
> Since very little "big iron" hardware supports flow-isolating AQM so far, that puts a damper on any "incremental deployment" argument to be made for L4S, even if they claim their dual-queue AQM is easier to implement at high link rates than full flow isolation.  The fact is that either dual-queue or flow-isolation is required in middleboxes *before* endpoint deployment of L4S can begin.

	Sure, the argument there seems to be that the typical bottleneck seems to be the access link and there the ISP who could controll this might have an interest of making that happen. In that light the involvement of cablelabs actually seems like a good thing, except for the part were they presumably took development underground/out of sight...

> 
> SCE explicitly avoids these problems, as both SCE-aware endpoints and SCE-aware middleboxes can safely be deployed without waiting for each other.  Work remains on figuring out precisely how SCE information should be produced and consumed, and verifying that the resulting system behaves as intended when fully deployed - but its interaction with existing deployed hardware and software is explicitly defined to be benign.

	Sure, I am confident that should SCE prevail here, L4S would find uses for SCE, but that still faces the same roll-out issue...

Best Regards
	Sebastian

> 
> - Jonathan Morton
>