From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 6CE273B29E for ; Thu, 1 Dec 2022 10:53:15 -0500 (EST) Received: by mail-wr1-x430.google.com with SMTP id m14so3365262wrh.7 for ; Thu, 01 Dec 2022 07:53:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=psFrld2GMSkMWrRGLWOJjUfXlLuruo+oPlzjTK3on3Y=; b=Nu9ft2ixiiACBOnHtY8U82flsGkUL3vZWCk1OY4ny0kNkvajSEe/+4U7Znr44WJ2Ac RWU2A2tuQmRVuwVsPASZas2R2ihkRflRT31ubssNkt6aTJdOJqo5udfAVYYD09JATzjB ndBC2k6s7BmOZC9wYcv/U+QJXDLm7zYN/ZZ/+WnjrK2wNd/W6/IbWGLw2v/0J0BF0iEM JklymMeXDAS7VwhoynSkCAxRBW9RQsbJ7PkM9sbP2b/vN+6+6oS5kjb8bN3IIgP+YG26 2lkxXiJi8khhU9Yh1sgiQ6Pwcjyt+xSK87hCEBJuMa/q/LPmB+idPb+zfvGTv9cBAieP xehw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=psFrld2GMSkMWrRGLWOJjUfXlLuruo+oPlzjTK3on3Y=; b=li3l+8Or9jm3xEi0WiIGVKG81prshZeMKPRAOaarOtLP4cTmaSKclSw/YKGGECh5g/ 8VpUxXTpYHzdSrG8DO59ZRtqxQzDnZ7h2sjm75V89x7CBEHOWsaWEImIzzBaCHUMW4Ki fIzX+v3O7ScwbCom0bIGfpk0475e0OHOjbe8qfcBGACCH9sUptasogCRy1Im7L6Z2OrG jBcN3B8iXBXXM9pPUOPfZAtejoPH71lzVzJAL3x8TbSri/Vfnirjt0fiWqfvDoS4gyd7 phdveLP583Xp2BPHj2J1tfGAtWsSvZ8ve6BkWnTMAXYsHRwlu7BKu5DNH9cfc6k1YcMR nWKg== X-Gm-Message-State: ANoB5pms1l+6eg7nVIulVyAolehltRXfJN4DutgpXTcLNBt2lleC6Qvb 5qcs9FobN8ITD5Y/RWpXcrwpkkxbHIQ/FFH+7uI= X-Google-Smtp-Source: AA0mqf7mKBHQILkRgFS+R50ppLllX7I2usPXHytDETEPKh6ZWEKLyN0SwnRcpruArMR5G4JRIGGddMijMxTpVbwH7Cg= X-Received: by 2002:adf:ead2:0:b0:23a:ce24:1bf0 with SMTP id o18-20020adfead2000000b0023ace241bf0mr9814818wrn.383.1669909994202; Thu, 01 Dec 2022 07:53:14 -0800 (PST) MIME-Version: 1.0 References: <2628090B-4AB7-425E-9C00-DECB8378F839@cable.comcast.com> <6edea155-0f44-aea1-28b1-03bbec9b259f@bobbriscoe.net> In-Reply-To: From: Dave Taht Date: Thu, 1 Dec 2022 07:53:01 -0800 Message-ID: To: Yiannis Yiakoumis Cc: tsvwg@ietf.org, "Livingood, Jason" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Thu, 01 Dec 2022 10:55:23 -0500 Subject: Re: [LibreQoS] [tsvwg] FYI: draft-livingood-low-latency-deployment-00.txt X-BeenThere: libreqos@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Many ISPs need the kinds of quality shaping cake can do List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2022 15:53:15 -0000 The below framing is extremely good. The caveats are key phrases, like L4S "implemented right", and the conflation of NQB's behaviors with L4S's other behaviors. I don't have any real issue with the NQB codepoint! but... L4S has been shown to be extremely RTT unfair, with issues in slow start, path jitter, latecomer disadvantages, the need for "policers", and so on. The only implementation I've seen of an L4S AQM allocates 9/10s of the bandwidth to it... Over here, in the network neutrality section, is a pretty good set of links to existing law in the USA: https://libreqos.io/features/ It's a mess! A big legal problem is there doesn't seem to be agreed upon definitions for what prioritization OR deprioritization mean when it comes to packets vs a vs applications. On Thu, Dec 1, 2022 at 3:36 AM Yiannis Yiakoumis wro= te: > > Hi Jason, > > Some comments below. > > Section 2: ... NQB flows tend to be limited in their capacity needs - for= example a DNS lookup will not need to consume the full capacity of an end = user's connection - but the end user is highly sensitive to any delays. > > > There is *at least* one popular application where this is not the case: i= nteractive video streaming of content rendered at the edge/server (eventual= ly a high-quality, 60FPS video stream is transmitted). > > This can be read as that there is a significant trade-off between throug= hput and latency, and therefore other apps won=E2=80=99t be incentivized to= use a low-latency service. See below comment on user-centric access contro= l for more context on this. > > Maybe remove it all together? > > > Section 3 : Network Neutrality and Low Latency Networking > > > The document frames the discussion on net neutrality as =E2=80=9CL4S is n= ot a fast-lane=E2=80=9D, and =E2=80=9Cit doesn=E2=80=99t affect other apps = throughput=E2=80=9D. This is risky and somewhat flawed. Zero-Rating service= s had the exact same characteristics (new capability, orthogonal to through= put) and ended-up regulated/banned both in US and Europe. > > > An alternative is =E2=80=9Cimplemented right, L4S is fully-aligned with N= N and has no impact on user choice and competition=E2=80=9D. In case you fi= nd it useful, here is a rewrite of Section 3 that roughly captures this. > > > Network Neutrality (a.k.a. Net Neutrality, and NN hereafter) is a concep= t that can mean a variety of things within a country, as well as between di= fferent countries, based on different opinions, market structures, business= practices, laws, and regulations. Generally speaking, NN means that Inter= net Service Providers (ISPs) should not limit user choice or affect competi= tion between application providers. In the context of the United States' ma= rketplace, it has come to mean that ISPs should not block, throttle, or dep= rioritize lawful application traffic, and should not engage in paid priorit= ization, among other things. The meaning of NN can be complex and ever cha= nging, so the specific details are out of scope for this document. Despite= that, NN concerns certainly bear on the deployment of new technologies by = ISPs, at least in the US, and so should be taken into account in making dep= loyment design decisions. > > > The principles below describe guidelines for a user-centric, application-= agnostic, and monetizable L4S that we believe is aligned with NN frameworks= and interpretations, at least in the U.S. and Europe. > > > Application-Agnostic L4S > > > NN demands that all applications should be treated the same by ISPs. As s= uch, any application should be able to request access to an L4S service usi= ng the available marking techniques, and the network should forward packets= through an L4S queue only based on such markings, without inferring or tak= ing into consideration which application packets originate from. > > > User-Centric Monetization > > > To incentivize L4S deployments, ISPs should be able to monetize it. This = can be controversial and perceived as paid prioritization. To avoid such co= nflicts, providers should charge users (and not application providers) for = access to L4S, and follow common charging regimes used for best-effort serv= ices. For example, different price-points may be achieved by adjusting the= throughput, monthly data allowance, or maximum latency of an L4S service. = Providers should not limit the number or types of applications that can acc= ess L4S, as this would eventually conflict with the application-agnostic re= quirement described above. > > > User-Centric Access Control > > > In some cases, an end-user might want to control which applications have = access to their L4S service. For example, they might have a 1GB monthly dat= a cap for L4S, and opt to use it only for a gaming application instead of v= ideo calls. Or they may want to prevent a legacy device that uses DSCP mark= ings from accessing L4S, as this could possibly degrade its operation. When= needed, access control should be user-centric, and respect the application= -agnostic requirement described above. > > > Access control that depends on application signature detection from a net= work router would violate the application-agnostic requirement, as it will = be coarse, inaccurate, and will only support a limited number of popular ap= plications supported by the network vendor (as explained in Section X.X). I= n contrast, access control that uses an Operating System=E2=80=99s permissi= on capabilities is compatible with the application-agnostic requirement, as= any application can trivially request access to such a permission and user= s can manage the decision. Similarly, a home WiFi router that allows users = to control which devices can potentially access L4S provides an application= -agnostic mechanism to deal with legacy devices. > > > On Thu, Dec 1, 2022 at 3:26 AM Yiannis Yiakoumis wrote: >> >> >> >> ---------- Forwarded message --------- >> From: Bob Briscoe >> Date: Tue, Nov 8, 2022 at 4:19 AM >> Subject: Re: [tsvwg] FYI: draft-livingood-low-latency-deployment-00.txt >> To: Livingood, Jason , tsvwg@ietf.org >> >> >> Jason, >> >> Indeed, I didn't qualify my earlier statement properly; sry about that. >> Latency done the L4S way is not a zero sum game. >> But Latency done the 'traditional' way by prioritizing bandwidth (e.g. a= s with Diffserv) is zero-sum. >> >> >> Bob >> >> >> On 08/11/2022 10:42, Livingood, Jason wrote: >> >> Thanks for the feedback, Bob! I will put it in my queue to consider for = the next update. I did want to say, however, that the sentence to which you= object below was taken nearly verbatim from your email to me on August 18t= h =E2=80=9CUnlike bandwidth priority, low latency is not a zero sum game - = everyone can have lower latency at no-one else's expense.=E2=80=9D =E2=80= =93 so you are essentially objecting to your own text. ;-) But I=E2=80=99ll= go back and re-read that email and this one below, as perhaps I misunderst= ood your August suggestion (it=E2=80=99s been known to happen!). >> >> >> >> Thanks as always! >> Jason >> >> >> >> From: tsvwg on behalf of Bob Briscoe >> Date: Monday, November 7, 2022 at 17:39 >> To: "Livingood, Jason" ,= "tsvwg@ietf.org" >> Subject: Re: [tsvwg] FYI: draft-livingood-low-latency-deployment-00.txt >> >> >> >> Jason, >> >> From my experience, many experienced network engineers (and public polic= y people who have made the effort to learn how QoS really works) disagree w= ith statements like: >> >> unlike with >> >> bandwidth priority on a highly/fully utilized link, low latency is >> >> not a zero sum game - everyone can potentially have lower latency at >> >> no one else's expense. >> >> This is because, until L4S, low latency was generally provided by giving= priority to a partition of the bandwidth of a link (and policing the rate = into that partition). >> >> With such experts (and therefore in this draft), I have found it's neces= sary to start an explanation with a strongly negative heads up: "L4S does /= not/ provide low latency in the same way as previous technologies like Diff= serv." Otherwise, they have no reason to believe they need to understand so= mething new here, and they assume it's just working like they think Diffser= v works. Then, when they read statements like that quoted above, they assum= e it's just political word-games, and walk away muttering "Move along, noth= ing new here - same old technology; same old word-games". >> >> This is particularly hard to understand, because the L4S DualQ /does/ us= e prioritization internally. The difference is that the prioritization only= applies over sub-RTT timescales, while on longer timescales, it is balance= d by an equal and opposite force: congestion signals that cause the sender = to yield to other traffic as if it had no priority. >> >> Then, that gets further confused, because that back-pressure relies on c= ongestion control behaviour that is ultimately voluntary. Then one gets int= o discussions about the necessity of policing, and most people have forgott= en what the original question was by this stage. >> >> And it's even more confused, because NQB and L4S are often lumped togeth= er, and NQB /does/ work like Diffserv (because it lacks the L4S congestion = signals). However, NQB is intended not to be asserted by a network operator= , because it's conditional on the sender's intended behaviour, which only t= he sender knows (altho the network could police it). >> >> In summary, it's all very subtle. But I believe it will help for this dr= aft to explain how low latency has been done in the past (in a zero-sum way= ), and what is different here. >> >> >> Bob >> >> On 24/10/2022 17:13, Livingood, Jason wrote: >> >> FYI - may be of interest to this group. Right now, this is an individual= submission. Feel free to email me 1:1 if you have feedback or wish to sugg= est changes/additions. >> >> >> >> Jason >> >> >> >> On 10/24/22, 12:09, "internet-drafts@ietf.org" wrote: >> >> >> >> >> >> A new version of I-D, draft-livingood-low-latency-deployment-00.txt >> >> has been successfully submitted by Jason Livingood and posted to the >> >> IETF repository. >> >> >> >> Name: draft-livingood-low-latency-deployment >> >> Revision: 00 >> >> Title: Comcast ISP Low Latency Deployment Design Recom= mendations >> >> Document date: 2022-10-24 >> >> Group: Individual Submission >> >> Pages: 10 >> >> URL: https://datatracker.ietf.org/doc/draft-livingood-low= -latency-deployment/ >> >> >> >> Abstract: >> >> The IETF's Transport Area Working Group (TSVWG) has finalized >> >> experimental RFCs for Low Latency, Low Loss, Scalable Throughput >> >> (L4S) and new Non-Queue-Building (NQB) per hop behavior. These >> >> documents do a good job of describing a new architecture and prot= ocol >> >> for deploying low latency networking. But as is normal for many = such >> >> standards, especially those in experimental status, certain desig= n >> >> decisions are ultimately left to implementers. This document >> >> explores the potential implications of key deployment design >> >> decisions and makes recommendations for those decisions that may = help >> >> drive adoption. >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> ________________________________________________________________ >> >> Bob Briscoe http://bobbriscoe.net/ >> >> >> -- >> ________________________________________________________________ >> Bob Briscoe http://bobbriscoe.net/ >> >> --=20 This song goes out to all the folk that thought Stadia would work: https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-69813666656= 07352320-FXtz Dave T=C3=A4ht CEO, TekLibre, LLC