From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id D53913B29D; Thu, 28 Sep 2023 16:18:24 -0400 (EDT) Received: by mail-pj1-x102d.google.com with SMTP id 98e67ed59e1d1-27734d76e1bso7363167a91.2; Thu, 28 Sep 2023 13:18:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695932304; x=1696537104; darn=lists.bufferbloat.net; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=6f+h9WwqafIvlrQUjNWLf3IekyoPqTL8zhzZlmR3XeM=; b=nJzz8g6Wtp8G4MUgfW7lC3knJ5pdmaTH+VUPe7QEFNlFtweuIKMrrF3BVkKfbskkjg Op8FQxoFQlyaY9wIgnhmczrdnlEYd0kgYiabI1R4qyywRYs45n9yOKSUOAj7/tXLiC02 tFv0rBywlZ1q4zLZmJJNVX30PUUtf0wNZDfCzdw1ZfTw9t+s4IPFIfeUJPbcO/wG6qBO C/Alu/XtaQYIdxexUioUHnmssf0/G+c6tkdLAIbBxewEK9a/K6r09GlrI+IR7F5f3irY 8jsaDTfUVNql62nW4lvkMSrrx8ba7eECfQmDuqYoCUbqSU1LplawOmMNdS9w/cDlUhUN E23w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695932304; x=1696537104; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6f+h9WwqafIvlrQUjNWLf3IekyoPqTL8zhzZlmR3XeM=; b=ZXKudOoVMBQsZfdgIALIQI5YWUWxQb6VTMS9sVKt1vmkO9UCOtH6XYVOcmX2D7dvxj 8B0ChL13OjCJNLuj3VH1zqww9IYP5TvOvfZUpkAUL92LLB8TfL04N9+dUPwY0re7iZAs MPp/5+YPwkCnrZir/UHbOtl8BK552Ivtcrp1z3rWUE9Qrb7HR80YI5LwPeqP4FlmPb2x zEH96RfKqsPrc0/WgeEk84B16eY2P1nqnYxFMSy/D5Dt/kSrnlNbcMQmueTfrOuU+JEY 6ZWw3hBfHJVVgSDfjCF0dNO9Jyd4quZo+8kzS7xm76tGcYkdL71+YfyDJhFoijIUxgUg QXVw== X-Gm-Message-State: AOJu0Yz0eaivZICwitkMIfARBpWIgRd3fdAa8T0M7KmIA8mUX1NN7XX/ NJ7Hg1556IPhuerigOqhceV19SsRgalF8ueDXV8= X-Google-Smtp-Source: AGHT+IEb0cbuxnC8w6DvmolKjeD0STnwDeIFD5yvcqHA+693dwxeLgTkZzXmCbLvkXfa3W7v+OzW1xNhYdTzoGX+QRg= X-Received: by 2002:a17:90b:1e0b:b0:267:faba:705 with SMTP id pg11-20020a17090b1e0b00b00267faba0705mr2153302pjb.10.1695932303561; Thu, 28 Sep 2023 13:18:23 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dave Taht Date: Thu, 28 Sep 2023 13:18:10 -0700 Message-ID: To: dan Cc: Sebastian Moeller , Dave Taht via Starlink , Rpm , libreqos , Jamal Hadi Salim , bloat Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [LibreQoS] [Rpm] net neutrality back in the news X-BeenThere: libreqos@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Many ISPs need the kinds of quality shaping cake can do List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2023 20:18:25 -0000 I love that there are oh, 700+ people on these mailing lists, but we have zero visibility due to google not indexing them, where hackernews does. This is going to be an issue dominating the web (again, sadly) for a few weeks at least, and it would really help to be doing it there, rather than here: https://news.ycombinator.com/item?id=3D37694306#37694307 As for me, up until tuesday I was trying to finish some technical stuff, and I have a proposal due sept 30 that I have not started yet. I regret now, even mentioning it on these lists. Please go forth and find a forum with new folk to debate with? I do look forward to whatever is supposed to be announced today(?), and reading it thoroughly, and I hope that instead of starting with decade old positions, that you assemble your thoughts, as to whatever the FCC is actually proposing moving forward. Ideally, in order to participate in the legal process, we would need to file a NPRM, which I am willing to co-ordinate. On Thu, Sep 28, 2023 at 1:09=E2=80=AFPM dan wrote: > > "(I assume most ISPs want happy customers)." > made me laugh a little. 'Most' by quantity of businesses maybe, but 'mos= t' in terms of customers being served by puts the Spectrums and Comcasts in= the mix (in the US) and they don't care about happy customers they care ab= out defacto monopolies in markets so that they don't have to care about hap= py customers. That might not be their motivation, but 30 years of their be= havior makes it clear that it's the case. > > > From my perspective, there should be a clear separation between carriers = and last mile delivery.. Even if it's the same company, the rules defining= each really should be different. > > Common Carriers or rather, carrier class services for 'internet', should = be completely neutral. Packets are packets. However, I think it's importa= nt to carve out methods to have dedicated links for real time flows at the = carrier level. I don't know what that model looks like exactly, but being = too stubborn about purist NN principals could really hurt VoIP services if = there aren't methods to handle that. I guess I really am describing 'inter= net fast lanes' for certain classes of services that we deem important enou= gh as a whole. not individual ISPs deciding, but rather 'the will of the p= eople' saying VoIP is more important than netflix, you can carve out dedica= ted capacity for that. > > For the last mile, I'm actually less concerned with pure NN and more conc= erned with no-blocking or 'brand' prioritization and required/label transpa= rency. ie, if an ISP wants to sell a DPI's QoE'd service that says 'videos= at 1080p' and that is on the product label clearly and not in small print,= then that's fine so long as 'videos' is agnostic of the vendor INCLUDING t= he ISP's own video products. "100Mbps internet services with 1080P video $= 65" for example. "Upgrade to our 100M with 4k video for +$10/m". With tha= t level of transparency I think that consumer protections are well handled.= And, because of that transparency, normal market capitalism mechanisms wo= rk. I can say "100Mbps for $65 and full resolution video" for example. O= r "100Mbps Net Neutral service". The secret DPI's QoE shaping is my main = concern here, and where I think consumer protection needs to be pursued. > > Again however, I think that ISPs should be able to offer dedicated paths = or bandwidth for specific services like VoIP. Services listed in a publicl= y determined products list. > > > On Thu, Sep 28, 2023 at 1:40=E2=80=AFPM Dave Taht via LibreQoS wrote: >> >> @Sebastian: This is a really great list of what the issues were in the >> EU, and if y'all can repost there, rather than here, perhaps more >> light will be generated outside our circles. >> >> https://news.ycombinator.com/item?id=3D37694306#37694307 >> >> On Thu, Sep 28, 2023 at 12:31=E2=80=AFPM Sebastian Moeller wrote: >> > >> > >> > >> > > On Sep 28, 2023, at 18:38, Dave Taht wrote: >> > > >> > > On Wed, Sep 27, 2023 at 11:25=E2=80=AFPM Sebastian Moeller wrote: >> > >> >> > >> Hi Dave, >> > >> >> > >> please excuse a number of tangents below ;) >> > > >> > > It would be nice, if as a (dis)organisation... the bufferbloat team >> > > could focus on somehow getting both sides of the network neutrality >> > > debate deeplying understanding the technological problem their >> > > pre-conceptions face, and the (now readily available and inexpensive= ) >> > > solutions that could be deployed, by most ISPs, over a weekend. We a= re >> > > regularly bringing up a few thousand people a week on libreqos (that >> > > we know of), and then of course, there are all the home routers and >> > > CPE that are increasingly capable of doing the right thing. >> > > >> > > The time to try and shift the memes in in the coming days, and weeks= . >> > > >> > > So ya'all being distracting below... aggh... ok, I'll bite. >> > >> > [SM] Sorry to drag you into the weeds... >> > >> > >> > > >> > >> >> > >> >> > >>> On Sep 27, 2023, at 20:21, Dave Taht via Rpm wrote: >> > >>> >> > >>> Jason just did a beautiful thread as to what was the original sour= ce >> > >>> of the network neutrality >> > >>> bittorrent vs voip bufferbloat blowup. >> > >>> >> > >>> https://twitter.com/jlivingood/status/1707078242857849244 >> > >> >> > >> But the core issue IMHO really was an economic one, the over= -subscription ratios that worked before torrenting simply did not cut it an= y more in an environment when customers actually tried to use their contrac= ted access rates "quantitatively". In my outsider perspective an ISP owes i= ts customers the contracted rates (with some slack*) and any sort of over-s= ubscription is a valid economic optimization an ISP might take, IFF that IS= P is prepared to rapidly increase segment capacity (or down-grade customer = plans)) if the deployed over-subscription rate proves to have been too opti= mistic. Mind you, most ISPs market plans by access speed (and charge more f= or higher speeds) and hence are somewhat responsible to actually deliver (a= gain with some slack) these speeds. >> > > >> > > The average use at peak hours for home broadband is below 5mbits per >> > > subscriber regardless of on a 25mbit or gbit plan. Remarkably, >> > > business plans are less (but more bursty). Oversubscription within a >> > > set of parameters is needed because, in part because upstream >> > > bandwidth to the internet remains expensive (although I hope that co= st >> > > continues to drop rapidly), and in part, because it is needlessly >> > > expensive to provision exactly the contracted rate to the customer. = As >> > > one example, I use the inverse square law as a rule of thumb for >> > > wireless deployments - the range you can get from a 25/10 deployment >> > > is geometrically better than 100/25, and the average bandwidth usage >> > > nearly the same. >> > >> > [SM] +1; hence my argument is not oversubscription per se is b= ad, but that oversubscription needs to be managed and the level of oversubs= cription needs to to be adapted ot he actual usage patterns. I am sure howe= ver that ISPs already actively do that (I assume most ISPs want happy custo= mers). >> > >> > >> > > Agreeing on industry standards for what the "slack" should be, might= be of help. >> > >> > [SM] Not being part of that industry at all I would love to se= e that as well, but can not contribute to defining that in any way. >> > >> > >> > > >> > >> >> > >> >> > >> *) Claiming "Up to", only carries that far, if you sell me X and I = mostly get Y (with Y close to X) and occasionally Z (with Z << X), that is = acceptable unless occasionally is "at every late afternoon and evening" pri= me-time. >> > > >> > > We have discussed elsewhere, the idea of a minimum contracted rate >> > > (down to), which is easier to reason about. >> > >> > [SM] Indeed, the German regulator (and I am not saying this is= the only or best option) decided to require ISPs to give 6 numbers pre-con= tract signing: 3 for up- and 3 for downstream: a maximal (net) rate, a mini= mal (net) rate, and a usually achievable (net) rate, all rates were defined= as IP/TCP goodput to make verification easier on end-users. The regulator = also defined a measurement regime that end-users can follow to check whethe= r the ISP is fulfilling the contract and the law gives remedies if ISPs do = not deliver (either the right to immediately cancel the contract and/or the= right to adapt the monthly payments to the actually delivered ratio of the= contracted rates*). I think I need to add, that ISPs can set these numbers= freely, but are only allowed to advertise with the maximum rate. >> > But if we talk about a single number per direction, minimal ra= te is probably easiest, or something like a "usually achievable rate" (that= needs to be met or exceeded in say 90% of >=3D 20 measurements or so). >> > >> > >> > *) In a ironic twist however the regulator so far has not explained ho= w deviations in each of the 6 numbers above should be aggregated to get one= single contract rate achievement ratio..., most ISPs took measures into th= eir own hands and simply offer customers typically a permanent rebate of 5 = EUR or immediate cancelation what ever the customer prefers.... >> > >> > >> > > >> > >> >> > >>> >> > >>> Seeing all the political activity tied onto it since (and now agai= n) >> > >>> reminds of two families at war about an incident that had happened >> > >>> generations and generations before, where the two sides no longer >> > >>> remembered why they hated each other so, but just went on hating, = and >> > >>> not forgiving, and not moving on. >> > >>> >> > >>> Yes, there are entirely separate and additional NN issues, but the >> > >>> technical problem of providing common carriage between two very >> > >>> different network application types (voip/gaming vs file transfer)= is >> > >>> thoroughly solved now, >> > >> >> > >> I am not sure this was at the core of the problem, my take i= s really that "always-on" and relative upload-heavy torrent simply demonstr= ated painfully for all involved that the old oversubscription ratios were n= ot suitable for the changed traffic profiles. I have some sympathy for the = ISPs here, they certainly did not try to sell their customers short (good I= SPs try to retain their customers and that works best when customers are ha= ppy with the service) and having this problem appear on many segments at th= e same time is not a fun place to be, and upload was/is often (too) low in = DOCSIS segments anyway; but this is why e.g. my bit torrent could affect yo= ur VoIP, simply by pushing the whole segment into upload capacity congestio= n... (ISPs in theory could fix this by plain old QoS engineering, but the i= ssue at hand was with a non-ISP VoIP/SIP service and there QoS becomes tric= ky if the ISP as these packets need to be identified in a way that is not g= ame-able**) >> > >> > [SM] See my later mail to Jason, I will not claim I know Comca= st's issues better than him and will accept that self-congestion also playe= d a major role in the initial problem. Over here in Europe the net neutrali= ty debate was kicked of less by an unfortunate confluence of new usage prof= iles and traditional oversibscription ratios and less than ideal packet sch= eduling, but rather by a series of pretty apparent attempts at restricting = consumer choice, see eg. https://www.accessnow.org/wp-content/uploads/archi= ve/docs/Net_Neutrality_Ending_Network_Discrimination_in_Europe_Access_v3.pd= f for an admitted slightly biased position: >> > >> > " =E2=80=A2 Blocking of applications and services: In order to m= aximise profits, some ISPs =E2=80=93 that also offer their own services and= applications online =E2=80=93 exclude certain services and applications of= competing market players. The most prominent case of this form of network = discrimination is European mobile providers (like Deutsche Telekom) blockin= g or restricting the use of Voice over IP (VoIP) services (like Skype and V= iber) for their customers.20 >> > >> > =E2=80=A2 Slowing or =E2=80=9Cthrottling=E2=80=9D internet spe= eds: Some ISPs slow down specific services (like YouTube) and applications = (like Skype), or ask users to pay an extra fee to have access to these inte= rnet platforms. Given the high latency (delay) sensitivity of many applicat= ions, ISPs are able to compromise the correct functioning of these services= by slowing them down, preventing the services from running properly. Often= ISPs =E2=80=93 especially telecommunication companies =E2=80=93 do this to= favour their own voice calling services over VoIP services, thereby crushi= ng competition. >> > >> > >> > =E2=80=A2 Blocking websites: ISPs often block websites for a n= umber of reasons =E2=80=93 to secure their network, or to avoid competition= , and sometimes for social, public relations or political reasons. In the U= K, for instance, Orange Telecom blocked the French digital rights advocacy = group, La Quadrature du Net=E2=80=99s website on pre-paid mobile accounts.2= 1 >> > >> > =E2=80=A2 Preferential treatment of services and platforms: IS= Ps can also impose data caps on internet access contracts while granting da= ta allowance exceptions to a company=E2=80=99s own proprietary streaming se= rvices (like Deutsche Telekom to its own =E2=80=9CT-Entertain=E2=80=9D).22 = They can (and do) also grant preferential treatment to select services =E2= =80=93 such as Orange France with the popular music streaming service Deeze= r23 =E2=80=93 ahead of other competitors, effectively imposing anti-competi= tive limitations on markets such as those for legal online music. Moreover,= generally only large, well-established companies can afford this preferent= ial treatment, resulting in a further stifling of innovation." >> > >> > These look like clearer attempts to monetize the ISP position as gate-= keeper to his customer's eye-balls (for content providers) as well as gate-= keeper to the internet for for paying customers. >> > I find it much harder to have sympathy for the listed examples of ISP = behavior than the situation of rapidly changing usage pattern posed where t= echnical changes needed to be made, but where no attempt at unfair monetiza= tion was evident, but maybe I am overly sensitive. These are all examples t= hat make me personally applaud the EU for its intervention to make clear ru= les what "internet access providers" can and can not do. (The EU also was q= uite flexible in applying/interpreting its rules during the covid isolation= periods, when it made it clear that e.g. treating certain traffic classes = e.g. streaming media to lower priority than video conferences was within th= e neutrality framework as long as it was based on traffic class and not on = specific service providers IIRC). >> > >> > >> > >> > > >> > > Torrent problem thoroughly solved with FQ on the path and backpressu= re >> > > from the mac scheduler. >> > > >> > > https://perso.telecom-paristech.fr/drossi/paper/rossi14comnet-b.pdf >> > >> > [SM] Thanks for the link. This is mainly for the self-congesri= on case? >> > >> > >> > > >> > >> I agree that on a single link we mostly solved the problem (= a few corner cases are left on links with very low capacity, where essentia= lly we can only manage the pain, not remove it)... >> > >> >> > >> >> > >> **) Which is not rocket science either, a VoIP stream takes ~100 Kb= ps, so in theory an ISP might simply allow each customer say 5 VoIP stream = equivalents by allowing up to 500Kbps od traffic marked with a specific DSC= P as higher priority (with higher access probability for the shared medium)= . I am not sayng this is my preferred solution, just saying this is a solut= ion that would have been available at the time if I memorize my docsis capa= bilities correctly. >> > >> >> > >> >> > >>> and if only all sides recognised at least this >> > >>> much, and made peace over it, and worked together to deploy those >> > >>> solutions, maybe, just maybe, we could find mutually satisfactory >> > >>> solutions to the other problems that plague the internet today, li= ke >> > >>> security, and the ipv6 rollout. >> > >> >> > >> +1. IPv6 is IMHO a prime example where the regulators should= stop talking softly and start showing the big stick they carry. >> > > >> > > I really would like to see a push for IPv6 mandated as a part of the >> > > BEAD programs. >> > >> > [SM] +1; I am not the biggest IPv6 fan, but that's what we hav= e and it is mostly serviceable so let's get this "party started" finally. >> > >> > > >> > >> Heck in Germany we have ISPs that still only supply CG-NATed IPv4 a= ddresses only.. (most mass market ISPs do much better in that regard, but f= or the stragglers it would help if the regulator would demand IPv6 with PD*= **). Regarding security, the easiest way to achieve that would be to put so= me heavy requirements on IoT manufacturers and vendors (like do what you pl= ease as long as you are local LAN only, but once you reach out into the clo= ud you need to fulfill the following list of requirements, with timely secu= rity updates over a reasonable long usage period), however that might not b= e very attractive for politicians/regulators to tackle (active regulatory a= cts tends to get bad press unless something bad happened, but even then the= press often complains about the acts coming too late, but I digress****) >> > > >> > > There is a separate NRPM going on for the new cybersecurity label at >> > > the FCC. If I had time, and a partner, >> > > we could rework the doc we did a few years ago. It is due oct 6. >> > > >> > >> >> > >> >> > >> ***) Strictly speaking IPv6 is required, since "internet access" is= defined as reaching all of the internet (as far as in the ISPs power) and = IPv6-only sites are not reachable for the CG-NAT-only customers. But so far= the local regulator does not seem to enforce that requirement, or hopefull= y is working on this quietly behind the curtains. >> > >> >> > >> ****) This is not to diss the press, they are doing what they are s= upposed to do, but it just shows that active regulation is a tricky busines= s, and a light touch typically "looks better" (even though I see no real ev= idence it actually works better). >> > >> >> > >>> If anyone here knows anyone more political, still vibrating with 1= 0+ >> > >>> years of outrage about NN on this fronts, on one side or the other= , if >> > >>> you could sit them down, over a beer, and try to explain that at t= he >> > >>> start it was a technical problem nobody understood at the time, ma= ybe >> > >>> that would help. >> > >> >> > >> So in the EU that debate is essentially settled, there is a = EU regulation that essentially spills out what ISPs owe their customers and= that has become the law of the land. The rationale for required un-biased = service and freedom to select terminal devices is well justified by the mar= ket ideals of the EU, allowing ISPs to discriminate packets or terminal dev= ices restricts the market and will lead to undesired outcomes. (Fun fact mo= st big players in capitalist societies argue for "free markets" but at the = same time act to work-around the market mechanism by trying to move the mar= ket into an oligo- or even monopoly condition, which is why strong regulati= on is required*****). >> > > >> > > Governments make safer markets feasible. >> > >> > [SM] Yes, I agree, both in markets are a pretty decent tool, b= ut need constant maintenance. >> > >> > Regards & Sorry for the tangent >> > Sebastian >> > >> > > >> > >> >> > >> *****) This is akin to professional sports where the audience gener= ally accepts that referees are necessary and occasionally need to make "pai= nful" calls, as the alternative would be anarchy, but I digress. >> > >> >> > >> Regards >> > >> Sebastian >> > >> >> > >> >> > >>> >> > >>> -- >> > >>> Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-musi= c-bof.html >> > >>> Dave T=C3=A4ht CSO, LibreQos >> > >>> _______________________________________________ >> > >>> Rpm mailing list >> > >>> Rpm@lists.bufferbloat.net >> > >>> https://lists.bufferbloat.net/listinfo/rpm >> > >> >> > > >> > > >> > > -- >> > > Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-= bof.html >> > > Dave T=C3=A4ht CSO, LibreQos >> > > <2510.jpeg> >> > >> >> >> -- >> Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.= html >> Dave T=C3=A4ht CSO, LibreQos >> _______________________________________________ >> LibreQoS mailing list >> LibreQoS@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/libreqos --=20 Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.htm= l Dave T=C3=A4ht CSO, LibreQos