[Bloat] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104

Fri Mar 15 10:06:50 EDT 2019

I would really prefer to move this discussion to the ecn-sane mailing
list, as IMHO, ecn is generally such a tiny thing needed for good
congestion control compared to better transports like pacing + bbr,
and things like bql, fq, and aqm using drop.

I'm going to keep cc-ing that list in the hope that we can keep better
track of the discussion here.

On Fri, Mar 15, 2019 at 6:01 AM Sebastian Moeller <moeller0 at gmx.de> wrote:
>
> Hi Dave,
>
> I pruned the CC list as I am out of my league here and want to restrict the radius of my embarrassment to those that already know my level of incompetence before hand.

IMHO, your work on educating the OpenWrt community over the years on
how to use sqm, makes you much more than "only a grasshopper". You
have a firm grip on what can be achieved in the real world.

>
> That said, having read through the L4S architecture description and the related appendices of draft-ietf-tsvwg-ecn-l4s-id-05 I came to the conclusion, that this is a mess.

I am so glad someone other than I has now read it.

> The L4S project proposes a really wide-ranging change of basically the internet (but allow a concurrent operation with legacy probably to cater to the fact that an internet-wide flag-day seems daunting to organize). But then they chicken out when figuring out how to differentiate between their new and the old by proposing to use ECT(1) for a purpose outside of its nominal purpose namely explicit congestion notification, why not think bolder? If the plan is to change everything the feasibility can not hinge upon the ability to re-using one old legacy bit, can it...
> Conceptually it seems much cleaner to use ECT(1) for congestion notification directly (there are already proposals out there and SCE just added another fully back-ward compatible one) and find another way to mark l4s traffic, sure that is going to be hard and inconvenient, but if you set out to change the internet that is par for the course.
> IMHO they would do more good if they actually fought for a better use of the 6 DSCP bits instead. (say by splitting into two groups of 3 bits, one group for reduced diffserv and one group for new features, that would even

The existing diffserv deployment is a failure. I have another ID
cooking that suggests a better way, going forward, to use them, but I
do not feel at this time it would be useful to present. One big battle
at a time. That ID is very simple, it basically proposes that all
diffserv codepoints be passed through ISPs and transit providers
unchanged, and if any given codepoint must be changed, the only
permissible change is to 0 (BE), and MUST be not be remarked to
anything else, especially not CS1.

>allow for concurrent use of the inevitable L5S and L6S ;) ). Especially since as far as I can understand l4s actually would like to have a more gradual congestion information stream than classic ECN, and since they need to modify DCTCP anyway to make it save for the wider internet, replacing its ECN response should be well inside the scope of work they already have on their list.

Next up for sce was going to be to find if dctcp could be modified to
use it. Also, bittorrent.

> If I sound a bit miffed, it is because after reading https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-03 I do not have the feeling they are trying to build abetter internet, but rather that they are building an internet where I can be a better "product" and customer of newfangled services, and I do not look forward to such a facebooky future with much enthusiasm.

I have rigorously tried to keep the network neutrality debate out of
this. dualpi is just another aqm that needs the same thorough
technical and public evaluation done to it that pie, codel, fq_codel
were subjected to.The use of ect_1 in dualpi for it is nuts IMHO, and
I'd vastly prefer that another L4S codepoint be added to make it work,
but any attempt to do so would also require industry consolidation on
that ID and that would be distracting at this time.

It appears, also, ironically, (I have confirmation from several
sources now) that cake, fq_codel and dualpi are now illegal for an ISP
to use in their provided equipment under california law. The idea of
one day having to appear in court to defend our key algorithms reminds
me of the famous john fogerty case where he demonstrated how blues
music was made.

I wish I knew a lawyer willing to take on "bufferbloat.net vs the
state of california", though, as it may come down to that.

I blew up on this part of the issue here:
http://blog.cerowrt.org/post/net_neutrality_customers/
>
> I hope that the discussion in Prague go well and a compromise/consense can be hashed out as I see different implementations duking it out here, but the overall goal of the competitors seems quite compatible, improving the internet by focussing on latency.
>
> Best Regards
>         Sebastian
>
> > On Mar 15, 2019, at 11:46, Dave Taht <dave.taht at gmail.com> wrote:
> >
> > Bufferbloat.net's ecn-sane working group members have a co-ordinated response to your efforts brewing but it's not ready yet. We have a worldwide team of linux and freebsd developers co-ordinating on landing code for our competing proposal "Some Congestion Experienced", which we submitted to tsvwg last sunday.
> >
> > That draft is under continuous revision, here:
> >
> > https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-tsvwg-sce.txt
> >
> > Our Linux and FreeBSD team is also flying into prague for SCE presentations at netdevconf and ietf.
> >
> > Some background to this: after the L4S/TCP Prague/and dualpi experiments appeared stalled out indefinitely in the IETF, and with our own frustration with IETF processes, bufferbloat.net project members publicly formed our own working group to look into the problems with ecn, back in august of last year.
> >
> > Its charter is here: https://www.bufferbloat.net/projects/ecn-sane/wiki/
> >
> > We were unaware, until last month, that the cable industry had 16 months back gone and formed its own private working group also, and was intending to turn the tcp prague/l4s/dualpi IETF "experiments" into an actual DOCSIS standard.
> >
> > Our SCE proposal appears to be backward compatible with the existing 10s-100s of millions of ecn-enabled fq_codel[1] and sch_cake[2] deployments, and doesn't require any changes to any RFC3168 tcps (or any tcp-friendly congestion control) at all in order to basically work. tcp-prague is subtly incompatible with that, and dualpi, more so. Our proposal is different also, it proposes some receiver side changes in order to get the full benefit of SCE while remaining backward compatible with the existing meaning of the CE codepoint.
> >
> > In either case, either approach essentially permanently redefines the ECT_1 codepoint incompatibly, once and for all, and for all time. This is a final battle over the meaning of a single bit in IP, and I expect the debates to be as difficult as the ones described in https://www.ietf.org/rfc/ien/ien137.txt - I would really, really, really prefer that they stay technical and not veer into politics, but I have little hope for that.
> >
> > The members of the ecn-sane working group are delighted to finally hear that running code for tcp-prague might land this ietf, and look forward to finally testing the whole l4s/tcpprague/dualpi architecture in conjunction with the flent.org 's and irtt's exhaustive suite of tests and servers in the cloud in the coming months, both against our existing, deployed, fq_codel, fq_pie, cake and pie derived solutions and our new SCE proposal. We hope to finally be able to write new tests for flent in particular, that can show tcpprague off in the ways that are important to those developing it. Flent has some basic dctcp tests, but nothing that can get down below a 20ms sample rate on modern hardware. (currently. It's easy to add tests, flent is written in python)
> >
> > We also hope that more test tools and implementations in ns2 and ns3 show up for tcpprauge  and dualpi show up soon also, from members of those projects.
> >
> > Note: sunday's dual-pi linux submission was kicked back from the linux networking developers due to some technical and legal problems with linux net-next HEAD ( https://patchwork.ozlabs.org/patch/1054521/  ) , and I do hope that a corrected version lands soon, so we can safely test it with current versions of OpenWrt, etc.
> >
> > Finally, running code. Will we find consensus?
> >
> > Thx!
> >
> >
> > [1] https://tools.ietf.org/html/rfc8290
> > [2] sch_cake was available for 3 years out of tree and was mainlined last august, in linux 4.19. It is partially described by "Piece of CAKE: A Comprehensive Queue Management Solution for Home Gateways" "https://arxiv.org/pdf/1804.07617.pdf
> >
> > A second paper describing its fq_codel-derived "cobalt" AQM algorithm is awaiting publication in a peer reviewed journal. It has been part of openwrt and the related sqm-scripts for many years and is widely deployed on multiple commercial products, such as those from eero and evenroute.
> >
> > Cake has a docsis specific mode which we longed for cablelabs to evaluate.
> >
> >
> > On Fri, Mar 15, 2019 at 2:33 AM Bob Briscoe <ietf at bobbriscoe.net> wrote:
> > Forwarding to tcpm & iccrg - apologies if you were already on one of the lists that received this.
> >
> > Olivier has been working hard on integrating the pieces of a Linux implementation of TCP Prague, and is close to having a version ported against the tip of the Linux mainline tree. This is his request for more people to get involved.
> >
> >
> > Bob
> >
> >
> > -------- Forwarded Message --------
> > Subject:      [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
> > Date: Wed, 6 Mar 2019 10:26:05 +0000
> > From: Tilmans, Olivier (Nokia - BE/Antwerp) <olivier.tilmans at nokia-bell-labs.com>
> > To:   hackathon at ietf.org <hackathon at ietf.org>, tcpprague at ietf.org <tcpprague at ietf.org>
> > CC:   dlebrun at google.com <dlebrun at google.com>, Joakim Misund <joakim.misund at gmail.com>, Bob Briscoe <research at bobbriscoe.net>, Quentin De Coninck <quentin.deconinck at uclouvain.be>, François Michel <francois.michel at uclouvain.be>, Mirja Kuehlewind <mirja.kuehlewind at tik.ee.ethz.ch>, Maxime Piraux <maxime.piraux at uclouvain.be>, Olga Albisser <olga at albisser.org>, Fabien Duchêne <fabien.duchene at uclouvain.be>, De Schepper, Koen (Nokia - BE/Antwerp) <koen.de_schepper at nokia-bell-labs.com>
> >
> > Hi all,
> >
> > We'll be working on the "TCP Prague" congestion control/L4S architecture during the IETF-104 hackaton.
> > This topics aims at accelerating the work that started during the IETF93 (coincidentally also in Prague), in order to get TCP Prague to an 'usable' state—i.e., meet the safety requirements and have supporting materials (e.g., VMs, labs) to let people experiment with it. Depending on people's interest, prototyping something similar for QUIC is another possible output.
> >
> > Details and links to resources/supporting drafts are available at https://trac.ietf.org/trac/ietf/meeting/wiki/104hackathon#tcp-prague and copied below.
> > Additionally, few topics will presented during netdev 0x13 the week before.
> >
> > See you in Prague.
> >
> > Best,
> > Olivier
> >
> >
> > Implementation and experimentation of TCP Prague/L4S
> >
> > * Champion
> > * Olivier Tilmans <olivier.tilmans at nokia-bell-labs.com>
> > * Projects
> > * Prototype the "TCP Prague" congestion control on Linux
> > * Finalize the implementation of accurate ECN (draft conformance), and port it on Linux v5.x * Build tooling around L4S to let people experiment with the technology (e.g., virtual machine, or mininet labs)
> > * Work towards "QUIC Prague"
> > * Resources
> > * TCP Prague
> > * Repository — https://github.com/L4STeam/tcp-prague
> > * Requirements — https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-05#page-21
> > * Upcoming netdev talk — https://netdevconf.org/0x13/session.html?talk-tcp-prague-l4s
> > * Accurate ECN
> > * Specs — https://tools.ietf.org/html/draft-ietf-tcpm-accurate-ecn-07
> > * Implementation for Linux v4.17 — https://github.com/mirjak/linux-accecn
> > * Past netdev talk — https://www.netdevconf.org/2.2/session.html?kuhlewind-accecn-talk
> > * Paced Chirping * Repository — https://github.com/JoakimMisund/PacedChirping
> > * Upcoming netdev talk — https://netdevconf.org/0x13/session.html?talk-chirp
> > * L4S architecture
> > * Specs — https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-03
> > * DualPI2 AQM
> > * Specs — https://tools.ietf.org/html/draft-ietf-tsvwg-aqm-dualq-coupled-08
> > * Repository — https://github.com/L4STeam/sch_dualpi2_upstream
> > * Upcoming netdev talk — https://netdevconf.org/0x13/session.html?talk-DUALPI2-AQM
> > * RITE Project — https://riteproject.eu/dctth/#code
> > _______________________________________________
> > tcpPrague mailing list
> > tcpPrague at ietf.org
> > https://www.ietf.org/mailman/listinfo/tcpprague
> > _______________________________________________
> > iccrg mailing list
> > iccrg at irtf.org
> > https://www.irtf.org/mailman/listinfo/iccrg
> >
> >
> > --
> >
> > Dave Täht
> > CTO, TekLibre, LLC
> > http://www.teklibre.com
> > Tel: 1-831-205-9740
> > _______________________________________________
> > Bloat mailing list
> > Bloat at lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat
>

--

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740