Hi Michael -- glad we are mostly of like mind! Let me try to explain where I was coming from with the super-delayed E2E ACKs below...

On Thu, Oct 21, 2021 at 1:42 AM Michael Welzl <michawe@ifi.uio.no> wrote:
Hi Keith !
  • Protocol 1: The endpoints can now decide what to do with this extra info. For example, they could decide that the endpoint receiver should switch to super delayed ACKs itself (maybe one every N seconds or M megabytes), and the sender will trust the Protocol 2 ACKs for congestion-control and retransmission purposes, while still using the Protocol 1 ACKs for ultimate reliability. (I.e. the sender won't discard outstanding data from a reliable stream until it's been ACKed by the endpoint.)
Not sure I see the point of super delayed ACKs, but …. either way, I think we agree on “The endpoints can now decide what to do with this extra info.”  There can be many variations.

 My thinking here was that ACKs serve at least three different purposes in an ARQ-style transport protocol:
  1. congestion control
  2. loss detection and triggering retransmission of previously sent data (via timeout, dupacks, sender-side timestamp schemes (Sprout/RACK), etc.)
  3. guaranteeing the abstraction of a reliable byte stream, by making the sender retain a copy of the outstanding data (ready for possible retransmission) until ACKed by the receiver
I think most endpoints might be willing to delegate #1 and #2 to an untrusted intermediary that's along the network path and eager to help (it's no worse than a transparent TCP accelerator today), especially if there's some ability for the endpoints to double-check the intermediary's work and say, "you know what, no thanks" if unsatisfied.

But #3 feels different -- if an untrusted intermediary's false ACK causes the sender to delete its copy of data that never makes it to the receiver, that threatens the core service model of the protocol. It's not possible to recover from that without escalating to the application. So the sender probably shouldn't discard data irreversibly until it hears a cryptographically authentic ACK from the E2E receiver, but that could come much later and less frequently.

-Keith