[Bloat] Packet reordering and RACK (was The "Some Congestion Experienced" ECN codepoint)

[Sebastian Moeller]

Hi Greg,


> On Mar 14, 2019, at 20:23, Greg White <g.white at CableLabs.com> wrote:
> 
> Sebastian,
> 
> The latency benefit of eliminating the in-order delivery assumption comes from the fact that L2 links aren't reordering within each microflow.  They are reordering on the link as a whole.  So, (e.g.) packets from microflows x, y and z can be held up in a resequencing buffer waiting for a packet from microflow w.  So, it is a definite latency benefit for flows x, y & z if this can be shut off.  

	I see, that I can understand; it is also sad, that nobody thought about doing the reordering per flow, which would reduce one of the cited costs for doing re-sorting, namely increased memory requirement for worst-case queueing, no?

> 
> Philosophically, since protocols and applications can vary in their need for in-order delivery, why does it not make sense to rely on the applications/protocols that need in-order delivery to implement their own resequencing? 

	As I tried to convey, if the local ARQ is considerably faster than the bottleneck link for each flow, local re-ordering comes at a considerably lower intra-flow latency cost than re-ordering past the bottleneck link. And one could argue, if a specific link technology is prone to introduce reordering due to retransmit it might as well try to clean up after itself... To my knowledge most traffic is currently TCP, and TCP has strict ordering requirements, and even if clever techniques like RACK can make it more robust against reordering, the applications will see the full latency hit incurred by re-sorting re-ordered packets in the receivers TCP stack, no?

Anyway, thanks for helping me understand the issues here, as always reality trumps theoretical musings...

> 
> -Greg
> 
> 
> On 3/14/19, 2:43 AM, "Bloat on behalf of Sebastian Moeller" <bloat-bounces at lists.bufferbloat.net on behalf of moeller0 at gmx.de> wrote:
> 
>    Hi,
> 
> 
>> On Mar 14, 2019, at 09:26, Ingemar Johansson S <ingemar.s.johansson at ericsson.com> wrote:
>> 
>> Hi
>> 
>> In addition to the below, in NR (=New Radio, a part of 5G) the RLC layer will 
>> no longer ensure
>> in sequence delivery to higher layers. Packet reordering can occur on the MAC 
>> layer as several HARQ (Hybrid ARQ) run simultaneously to transmit packets, 
>> some processes need to retransmit and there you get packet reordering.
> 
>    	Unfortunate...
> 
>> The PDCP layer can however (optionally) enforce in sequence delivery, 
>> personally I am sceptic about the benefits of this as it adds extra HoL 
>> blocking to solve a problem that RACK can solve.
> 
>    	In the context of the L4S (over-) promises it can not IMHO, unless the lossy link is slower than the internet access link. My rationale is that direct retransmits of packets that did not pass the current "physical" connection and sorting at the remote end of the link before passing the packets on, is only going to introduce more (sorting-)delay then sending out of order and expect the receiver to put things in sequence again, IFF the retransmit process takes the order of time as the transfer of the un-ordered packets over the bottleneck access link. As far as I understand L$S is all about reducing application visible latency, and RACK, in my layman's understanding, is also not going to help much here as it basically introduces another timeout (aka potential delay). I am not trying to pass a judgment on RACK here (and as far as I understand it solves a different problem, by making TCP more tolerant against mild reordering it will increase bandwidth utilization and reduce delays from having to slow down, but it will do nothing for the perceived latency from re-odered packets), all I want to understand how this is going to work in the context of "ultra-low latency" (a term from the L4S RFCs that I believe to be a tad too much)?
> 
> 
>> In addition it costs more 
>> memory in nodes that potentially need to transmit 10s of GByte of data.
> 
>    	Sure, but such is life, this reminds me a bit on debates about cache coherency and un-aligned memory accesses, the hardware people seem to argue life would be simpler/faster if these could be traded-in, but that simply pushes cost and complexity on the software side ;)
> 
>    Best Regards
>            Sebastian
> 
>    P.S.: What is the best place to discuss L4S? Certainly not this mailing list, or?
> 
> 
> 
>> 
>> /Ingemar
>> ======
>> 
>> Date: Tue, 12 Mar 2019 21:39:42 -0700 (PDT)
>> From: David Lang <david at lang.hm>
>> To: Sebastian Moeller <moeller0 at gmx.de>
>> Cc: Mikael Abrahamsson <swmike at swm.pp.se>,  "Holland, Jake"
>> 	<jholland at akamai.com>,  Cake List <cake at lists.bufferbloat.net>,
>> 	"codel at lists.bufferbloat.net" <codel at lists.bufferbloat.net>,  bloat
>> 	<bloat at lists.bufferbloat.net>,  "ecn-sane at lists.bufferbloat.net"
>> 	<ecn-sane at lists.bufferbloat.net>
>> Subject: Re: [Bloat] [Cake] The "Some Congestion Experienced" ECN
>> 	codepoint - a new internet draft -
>> Message-ID: <nycvar.QRO.7.76.6.1903122137430.6242 at qynat-yncgbc>
>> Content-Type: text/plain; charset=US-ASCII; format=flowed
>> 
>> On Mon, 11 Mar 2019, Sebastian Moeller wrote:
>> 
>>> How is packet reordering for anybody but the folks responsible for
>>> operating the "conduits" in any way attractive?
>> 
>> It's more that not worrying about maintaining the order, and just moving the
>> packets as fast as possible reduces the overhead.
>> 
>> The majority of the time, packets will be in order, but race conditions and
>> corner cases are allowed to forward packets out of order rather than having
>> the delay some packets to maintain the order.
>> 
>> David Lang
>> 
>> 
>> 
>> _______________________________________________
>> Bloat mailing list
>> Bloat at lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
> 
>    _______________________________________________
>    Bloat mailing list
>    Bloat at lists.bufferbloat.net
>    https://lists.bufferbloat.net/listinfo/bloat
> 
>