[Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104

Fri Mar 15 13:01:23 EDT 2019

The absolute fundamental issue with diffserv, IMO, is that the carriers cannot agree on an actual specification of what routers and gateways are supposed to do, while being compatible with the core principle of the IP layer: do not hold packets if they cause increasing queueing delay. (this is in the Best Practices RFC)

And it's not for lack of trying. Dave Clark led a working group at the MIT communications futures program (where I was a principle) that included most major backbone providers' key folks that was specifically focused on getting a widespread agreement on what any of the code points might mean, not as bullshit English descriptions of what kind of traffic each one represented, but as an operational description of what should be done by a router to manage its queues.

They couldn't even agree (after about 18 months of meetings) about what the bullshit English intent was, much less what operational semantics in the packet forwarding network had to be.

So if the responsible network engineers in the carriers cannot agree on anything, IETF is wasting its time.

-----Original Message-----
From: "Sebastian Moeller" <moeller0 at gmx.de>
Sent: Friday, March 15, 2019 11:52am
To: "Dave Täht" <dave.taht at gmail.com>
Cc: ecn-sane at lists.bufferbloat.net, "bloat" <bloat at lists.bufferbloat.net>
Subject: Re: [Ecn-sane] [Bloat] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104

Hi Dave,

> On Mar 15, 2019, at 15:06, Dave Taht <dave.taht at gmail.com> wrote:
> 
> I would really prefer to move this discussion to the ecn-sane mailing
> list, as IMHO, ecn is generally such a tiny thing needed for good
> congestion control compared to better transports like pacing + bbr,
> and things like bql, fq, and aqm using drop.
> 
> I'm going to keep cc-ing that list in the hope that we can keep better
> track of the discussion here.

 Ah, sure, I basically wanted to avoid annoying the ietf lists as I feel out of place there, having only had a cursory read of the relevant RFCs.

> 
> On Fri, Mar 15, 2019 at 6:01 AM Sebastian Moeller <moeller0 at gmx.de> wrote:
>> 
>> Hi Dave,
>> 
>> I pruned the CC list as I am out of my league here and want to restrict the radius of my embarrassment to those that already know my level of incompetence before hand.
> 
> IMHO, your work on educating the OpenWrt community over the years on
> how to use sqm, makes you much more than "only a grasshopper". You
> have a firm grip on what can be achieved in the real world.
> 
>> 
>> That said, having read through the L4S architecture description and the related appendices of draft-ietf-tsvwg-ecn-l4s-id-05 I came to the conclusion, that this is a mess.
> 
> I am so glad someone other than I has now read it.
> 
>> The L4S project proposes a really wide-ranging change of basically the internet (but allow a concurrent operation with legacy probably to cater to the fact that an internet-wide flag-day seems daunting to organize). But then they chicken out when figuring out how to differentiate between their new and the old by proposing to use ECT(1) for a purpose outside of its nominal purpose namely explicit congestion notification, why not think bolder? If the plan is to change everything the feasibility can not hinge upon the ability to re-using one old legacy bit, can it...
>> Conceptually it seems much cleaner to use ECT(1) for congestion notification directly (there are already proposals out there and SCE just added another fully back-ward compatible one) and find another way to mark l4s traffic, sure that is going to be hard and inconvenient, but if you set out to change the internet that is par for the course.
>> IMHO they would do more good if they actually fought for a better use of the 6 DSCP bits instead. (say by splitting into two groups of 3 bits, one group for reduced diffserv and one group for new features, that would even
> 
> The existing diffserv deployment is a failure.

 Which is a shame, but one RFC's failure is another one's opportunity.

> I have another ID
> cooking that suggests a better way, going forward, to use them, but I
> do not feel at this time it would be useful to present. One big battle
> at a time.

:)

> That ID is very simple, it basically proposes that all
> diffserv codepoints be passed through ISPs and transit providers
> unchanged, and if any given codepoint must be changed, the only
> permissible change is to 0 (BE), and MUST be not be remarked to
> anything else, especially not CS1.

 I like the simplicity, but I also like the split into two groups of 3 bits, say "active bit pattern" (for transport purposes) and "intenden bit pattern" for the sender to transmit intent, which than can be conveyed lossless to the receiver; my gut feeling is that throwing the transport people a bone here might work better, as in the end they are the one that will carry the "burden" of the change. IMHO this has the additional advantage that it will make "tabula rasa" of the existing distict set of bit patterns used in the real world. Which in turn probably is this ideas downfall...

> 
>> allow for concurrent use of the inevitable L5S and L6S ;) ). Especially since as far as I can understand l4s actually would like to have a more gradual congestion information stream than classic ECN, and since they need to modify DCTCP anyway to make it save for the wider internet, replacing its ECN response should be well inside the scope of work they already have on their list.
> 
> Next up for sce was going to be to find if dctcp could be modified to
> use it. Also, bittorrent.

 YES! I endorse this message.

> 
>> If I sound a bit miffed, it is because after reading https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-03 I do not have the feeling they are trying to build abetter internet, but rather that they are building an internet where I can be a better "product" and customer of newfangled services, and I do not look forward to such a facebooky future with much enthusiasm.
> 
> I have rigorously tried to keep the network neutrality debate out of
> this.

 And I really do not want to start this here, as I agree that this is about a technical question. That said, I had hoped more out of the l4s promises from an end-user perspective. Then again, it if this is going to happen it needs the ISPs buy-in so it might be politically wise to emphasize the advantages for ISPs.

> dualpi is just another aqm that needs the same thorough
> technical and public evaluation done to it that pie, codel, fq_codel
> were subjected to.

+1, I note that the main argument against fq is "we can do it with two" without a convincing argument why two is better than the few 100s you realistically will never need for fq even on an busy core router.

> The use of ect_1 in dualpi for it is nuts IMHO,

 At least it is unclean...

> and
> I'd vastly prefer that another L4S codepoint be added to make it work,
> but any attempt to do so would also require industry consolidation on
> that ID and that would be distracting at this time.

 But as I said, L4S aims to change everything so why stop there ;)

> 
> It appears, also, ironically, (I have confirmation from several
> sources now) that cake, fq_codel and dualpi are now illegal for an ISP
> to use in their provided equipment under california law.

 I will happily look at specific sections of code if pointed to, but I will not actively search for. At least the European net neutrality rules do not make any of these illegal, as the rules allow for traffic management (and special services at extra cost as long as these are not built be restricting the best-effort net, no idea how that should be poiced) (and in the end the rules only apply for ISPs in one's own network one can DPI to a far greater extent if one is such inclined).

> The idea of
> one day having to appear in court to defend our key algorithms reminds
> me of the famous john fogerty case where he demonstrated how blues
> music was made.

 I see a nice art-house movie coming up.

> 
> I wish I knew a lawyer willing to take on "bufferbloat.net vs the
> state of california", though, as it may come down to that.
> 
> I blew up on this part of the issue here:
> http://blog.cerowrt.org/post/net_neutrality_customers/

 I read that, but it does not reflect to well on the situation this side of the pond. We had situations where ISPs worked hard (but without success) to switch from their flat-rate access to introduce volume caps, that served a dual purpose, a) extracting more revenue from customers and b) allowing to make "zero-rating" deals with content-providers (which in the end are also payed for by the end-users, indirectly). Sure, this is all normal in business, but that does not necessarily mean that I do need to like being a pawn in a business transaction between global corporations. (I consider this to be at least somewhat related to the net neutrality debate).

Best Regards
 Sebastian

>> 
>> I hope that the discussion in Prague go well and a compromise/consense can be hashed out as I see different implementations duking it out here, but the overall goal of the competitors seems quite compatible, improving the internet by focussing on latency.
>> 
>> Best Regards
>> Sebastian
>> 
>>> On Mar 15, 2019, at 11:46, Dave Taht <dave.taht at gmail.com> wrote:
>>> 
>>> Bufferbloat.net's ecn-sane working group members have a co-ordinated response to your efforts brewing but it's not ready yet. We have a worldwide team of linux and freebsd developers co-ordinating on landing code for our competing proposal "Some Congestion Experienced", which we submitted to tsvwg last sunday.
>>> 
>>> That draft is under continuous revision, here:
>>> 
>>> https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-tsvwg-sce.txt
>>> 
>>> Our Linux and FreeBSD team is also flying into prague for SCE presentations at netdevconf and ietf.
>>> 
>>> Some background to this: after the L4S/TCP Prague/and dualpi experiments appeared stalled out indefinitely in the IETF, and with our own frustration with IETF processes, bufferbloat.net project members publicly formed our own working group to look into the problems with ecn, back in august of last year.
>>> 
>>> Its charter is here: https://www.bufferbloat.net/projects/ecn-sane/wiki/
>>> 
>>> We were unaware, until last month, that the cable industry had 16 months back gone and formed its own private working group also, and was intending to turn the tcp prague/l4s/dualpi IETF "experiments" into an actual DOCSIS standard.
>>> 
>>> Our SCE proposal appears to be backward compatible with the existing 10s-100s of millions of ecn-enabled fq_codel[1] and sch_cake[2] deployments, and doesn't require any changes to any RFC3168 tcps (or any tcp-friendly congestion control) at all in order to basically work. tcp-prague is subtly incompatible with that, and dualpi, more so. Our proposal is different also, it proposes some receiver side changes in order to get the full benefit of SCE while remaining backward compatible with the existing meaning of the CE codepoint.
>>> 
>>> In either case, either approach essentially permanently redefines the ECT_1 codepoint incompatibly, once and for all, and for all time. This is a final battle over the meaning of a single bit in IP, and I expect the debates to be as difficult as the ones described in https://www.ietf.org/rfc/ien/ien137.txt - I would really, really, really prefer that they stay technical and not veer into politics, but I have little hope for that.
>>> 
>>> The members of the ecn-sane working group are delighted to finally hear that running code for tcp-prague might land this ietf, and look forward to finally testing the whole l4s/tcpprague/dualpi architecture in conjunction with the flent.org 's and irtt's exhaustive suite of tests and servers in the cloud in the coming months, both against our existing, deployed, fq_codel, fq_pie, cake and pie derived solutions and our new SCE proposal. We hope to finally be able to write new tests for flent in particular, that can show tcpprague off in the ways that are important to those developing it. Flent has some basic dctcp tests, but nothing that can get down below a 20ms sample rate on modern hardware. (currently. It's easy to add tests, flent is written in python)
>>> 
>>> We also hope that more test tools and implementations in ns2 and ns3 show up for tcpprauge and dualpi show up soon also, from members of those projects.
>>> 
>>> Note: sunday's dual-pi linux submission was kicked back from the linux networking developers due to some technical and legal problems with linux net-next HEAD ( https://patchwork.ozlabs.org/patch/1054521/ ) , and I do hope that a corrected version lands soon, so we can safely test it with current versions of OpenWrt, etc.
>>> 
>>> Finally, running code. Will we find consensus?
>>> 
>>> Thx!
>>> 
>>> 
>>> [1] https://tools.ietf.org/html/rfc8290
>>> [2] sch_cake was available for 3 years out of tree and was mainlined last august, in linux 4.19. It is partially described by "Piece of CAKE: A Comprehensive Queue Management Solution for Home Gateways" "https://arxiv.org/pdf/1804.07617.pdf
>>> 
>>> A second paper describing its fq_codel-derived "cobalt" AQM algorithm is awaiting publication in a peer reviewed journal. It has been part of openwrt and the related sqm-scripts for many years and is widely deployed on multiple commercial products, such as those from eero and evenroute.
>>> 
>>> Cake has a docsis specific mode which we longed for cablelabs to evaluate.
>>> 
>>> 
>>> On Fri, Mar 15, 2019 at 2:33 AM Bob Briscoe <ietf at bobbriscoe.net> wrote:
>>> Forwarding to tcpm & iccrg - apologies if you were already on one of the lists that received this.
>>> 
>>> Olivier has been working hard on integrating the pieces of a Linux implementation of TCP Prague, and is close to having a version ported against the tip of the Linux mainline tree. This is his request for more people to get involved.
>>> 
>>> 
>>> Bob
>>> 
>>> 
>>> -------- Forwarded Message --------
>>> Subject: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
>>> Date: Wed, 6 Mar 2019 10:26:05 +0000
>>> From: Tilmans, Olivier (Nokia - BE/Antwerp) <olivier.tilmans at nokia-bell-labs.com>
>>> To: hackathon at ietf.org <hackathon at ietf.org>, tcpprague at ietf.org <tcpprague at ietf.org>
>>> CC: dlebrun at google.com <dlebrun at google.com>, Joakim Misund <joakim.misund at gmail.com>, Bob Briscoe <research at bobbriscoe.net>, Quentin De Coninck <quentin.deconinck at uclouvain.be>, François Michel <francois.michel at uclouvain.be>, Mirja Kuehlewind <mirja.kuehlewind at tik.ee.ethz.ch>, Maxime Piraux <maxime.piraux at uclouvain.be>, Olga Albisser <olga at albisser.org>, Fabien Duchêne <fabien.duchene at uclouvain.be>, De Schepper, Koen (Nokia - BE/Antwerp) <koen.de_schepper at nokia-bell-labs.com>
>>> 
>>> Hi all,
>>> 
>>> We'll be working on the "TCP Prague" congestion control/L4S architecture during the IETF-104 hackaton.
>>> This topics aims at accelerating the work that started during the IETF93 (coincidentally also in Prague), in order to get TCP Prague to an 'usable' state—i.e., meet the safety requirements and have supporting materials (e.g., VMs, labs) to let people experiment with it. Depending on people's interest, prototyping something similar for QUIC is another possible output.
>>> 
>>> Details and links to resources/supporting drafts are available at https://trac.ietf.org/trac/ietf/meeting/wiki/104hackathon#tcp-prague and copied below.
>>> Additionally, few topics will presented during netdev 0x13 the week before.
>>> 
>>> See you in Prague.
>>> 
>>> Best,
>>> Olivier
>>> 
>>> 
>>> Implementation and experimentation of TCP Prague/L4S
>>> 
>>> * Champion
>>> * Olivier Tilmans <olivier.tilmans at nokia-bell-labs.com>
>>> * Projects
>>> * Prototype the "TCP Prague" congestion control on Linux
>>> * Finalize the implementation of accurate ECN (draft conformance), and port it on Linux v5.x * Build tooling around L4S to let people experiment with the technology (e.g., virtual machine, or mininet labs)
>>> * Work towards "QUIC Prague"
>>> * Resources
>>> * TCP Prague
>>> * Repository — https://github.com/L4STeam/tcp-prague
>>> * Requirements — https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-05#page-21
>>> * Upcoming netdev talk — https://netdevconf.org/0x13/session.html?talk-tcp-prague-l4s
>>> * Accurate ECN
>>> * Specs — https://tools.ietf.org/html/draft-ietf-tcpm-accurate-ecn-07
>>> * Implementation for Linux v4.17 — https://github.com/mirjak/linux-accecn
>>> * Past netdev talk — https://www.netdevconf.org/2.2/session.html?kuhlewind-accecn-talk
>>> * Paced Chirping * Repository — https://github.com/JoakimMisund/PacedChirping
>>> * Upcoming netdev talk — https://netdevconf.org/0x13/session.html?talk-chirp
>>> * L4S architecture
>>> * Specs — https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-03
>>> * DualPI2 AQM
>>> * Specs — https://tools.ietf.org/html/draft-ietf-tsvwg-aqm-dualq-coupled-08
>>> * Repository — https://github.com/L4STeam/sch_dualpi2_upstream
>>> * Upcoming netdev talk — https://netdevconf.org/0x13/session.html?talk-DUALPI2-AQM
>>> * RITE Project — https://riteproject.eu/dctth/#code
>>> _______________________________________________
>>> tcpPrague mailing list
>>> tcpPrague at ietf.org
>>> https://www.ietf.org/mailman/listinfo/tcpprague
>>> _______________________________________________
>>> iccrg mailing list
>>> iccrg at irtf.org
>>> https://www.irtf.org/mailman/listinfo/iccrg
>>> 
>>> 
>>> --
>>> 
>>> Dave Täht
>>> CTO, TekLibre, LLC
>>> http://www.teklibre.com
>>> Tel: 1-831-205-9740
>>> _______________________________________________
>>> Bloat mailing list
>>> Bloat at lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/bloat
>> 
> 
> 
> --
> 
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-205-9740

_______________________________________________
Ecn-sane mailing list
Ecn-sane at lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/ecn-sane
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/bloat/attachments/20190315/c48bcc88/attachment-0002.html>