From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id ED2763B29D; Thu, 28 Sep 2023 15:40:00 -0400 (EDT) Received: by mail-pg1-x52c.google.com with SMTP id 41be03b00d2f7-577fb90bb76so7309529a12.2; Thu, 28 Sep 2023 12:40:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695930000; x=1696534800; darn=lists.bufferbloat.net; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=RnsAZTwlWGJ3yLWiyBeGsHjRHFNFwOYP2vmT2eEAlSw=; b=EjwNNxud9V1MtaWKd9m+WLueY/ef9Fef5TF85C+YEh8Rh+Eh437IGS+8IkmD8z2YOt kOcdy7gxSSc0Og/3svWeOQL49Fv0iibNKa3fXXCdd60aYQiG0OcA5oD6VX1SJHmc5aMt 9R/1426ll8EcJRmIXA2DgxfoabJi1u/kvHRty0DzkFknCUX/1utnpDoz3MK/E3qzXBGO ATxOEnmCMe8R0ucgO1FGp2NYH2Ifv4yofAxUN/Ll1cnG+mRV7MYsNa+TSPNYiZU3hpax Mpt1jHwLGc01sPyRCO46T85WdP9hrYFJUrny1Mtdo/Fc+Pc6RzeF4r7EXejg7u22Ty5T xluQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695930000; x=1696534800; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RnsAZTwlWGJ3yLWiyBeGsHjRHFNFwOYP2vmT2eEAlSw=; b=xMYyjfvH1llAn3GBu64Mzi7ukbSO7tZwZBUqomM9mT6q8N6A2gmmItfJ9YYeLJqflh Mj9x2pZ9qcC9k9joUlGQvIUtdrYhQkB+DkFV1gmPA5Vu59SqgvcRA228lvIiebp5/e1p Q7qko8RtW7CoUq2E40CioysxlqvDfOVujDgMFdxxWRggiCBgIK37tefw2pTd9nvjyiBF DwhvhGw3yB0pJoutf78cseevnCALNXUSejjePduULfDgtsVhb9mIIVm7/MHr4kPMREGg HN2C0UZZUtMcD8Z/d5s9UHvLYdHoVz0lK7GytxJeX71z2Oc2RQ/5OP+ybX0bf9GabT+9 Mq0w== X-Gm-Message-State: AOJu0YxXmp4ZbNZONqY4Idgc8y1bFmaxz9O7fTc4XUhlHvw856lOdv3p H09638bXs2wgz9fuQuKzO6Qs1YtVSh1dHNdF3WIbaSAAFJc= X-Google-Smtp-Source: AGHT+IEJ3/9GjDeeZu5SrPPqp9q3HmAFhSQAFhRtuM84EPeIj/Xpm7uhqHjwzVMw+HKX5Rc/aY+8n4JRyR9A5KNIEJM= X-Received: by 2002:a17:90a:3884:b0:26b:49de:13bd with SMTP id x4-20020a17090a388400b0026b49de13bdmr2028840pjb.36.1695929999557; Thu, 28 Sep 2023 12:39:59 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dave Taht Date: Thu, 28 Sep 2023 12:39:47 -0700 Message-ID: To: Sebastian Moeller Cc: libreqos , Dave Taht via Starlink , bloat , Rpm , Jamal Hadi Salim Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Rpm] net neutrality back in the news X-BeenThere: rpm@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: revolutions per minute - a new metric for measuring responsiveness List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2023 19:40:01 -0000 @Sebastian: This is a really great list of what the issues were in the EU, and if y'all can repost there, rather than here, perhaps more light will be generated outside our circles. https://news.ycombinator.com/item?id=3D37694306#37694307 On Thu, Sep 28, 2023 at 12:31=E2=80=AFPM Sebastian Moeller wrote: > > > > > On Sep 28, 2023, at 18:38, Dave Taht wrote: > > > > On Wed, Sep 27, 2023 at 11:25=E2=80=AFPM Sebastian Moeller wrote: > >> > >> Hi Dave, > >> > >> please excuse a number of tangents below ;) > > > > It would be nice, if as a (dis)organisation... the bufferbloat team > > could focus on somehow getting both sides of the network neutrality > > debate deeplying understanding the technological problem their > > pre-conceptions face, and the (now readily available and inexpensive) > > solutions that could be deployed, by most ISPs, over a weekend. We are > > regularly bringing up a few thousand people a week on libreqos (that > > we know of), and then of course, there are all the home routers and > > CPE that are increasingly capable of doing the right thing. > > > > The time to try and shift the memes in in the coming days, and weeks. > > > > So ya'all being distracting below... aggh... ok, I'll bite. > > [SM] Sorry to drag you into the weeds... > > > > > >> > >> > >>> On Sep 27, 2023, at 20:21, Dave Taht via Rpm wrote: > >>> > >>> Jason just did a beautiful thread as to what was the original source > >>> of the network neutrality > >>> bittorrent vs voip bufferbloat blowup. > >>> > >>> https://twitter.com/jlivingood/status/1707078242857849244 > >> > >> But the core issue IMHO really was an economic one, the over-su= bscription ratios that worked before torrenting simply did not cut it any m= ore in an environment when customers actually tried to use their contracted= access rates "quantitatively". In my outsider perspective an ISP owes its = customers the contracted rates (with some slack*) and any sort of over-subs= cription is a valid economic optimization an ISP might take, IFF that ISP i= s prepared to rapidly increase segment capacity (or down-grade customer pla= ns)) if the deployed over-subscription rate proves to have been too optimis= tic. Mind you, most ISPs market plans by access speed (and charge more for = higher speeds) and hence are somewhat responsible to actually deliver (agai= n with some slack) these speeds. > > > > The average use at peak hours for home broadband is below 5mbits per > > subscriber regardless of on a 25mbit or gbit plan. Remarkably, > > business plans are less (but more bursty). Oversubscription within a > > set of parameters is needed because, in part because upstream > > bandwidth to the internet remains expensive (although I hope that cost > > continues to drop rapidly), and in part, because it is needlessly > > expensive to provision exactly the contracted rate to the customer. As > > one example, I use the inverse square law as a rule of thumb for > > wireless deployments - the range you can get from a 25/10 deployment > > is geometrically better than 100/25, and the average bandwidth usage > > nearly the same. > > [SM] +1; hence my argument is not oversubscription per se is bad,= but that oversubscription needs to be managed and the level of oversubscri= ption needs to to be adapted ot he actual usage patterns. I am sure however= that ISPs already actively do that (I assume most ISPs want happy customer= s). > > > > Agreeing on industry standards for what the "slack" should be, might be= of help. > > [SM] Not being part of that industry at all I would love to see t= hat as well, but can not contribute to defining that in any way. > > > > > >> > >> > >> *) Claiming "Up to", only carries that far, if you sell me X and I mos= tly get Y (with Y close to X) and occasionally Z (with Z << X), that is acc= eptable unless occasionally is "at every late afternoon and evening" prime-= time. > > > > We have discussed elsewhere, the idea of a minimum contracted rate > > (down to), which is easier to reason about. > > [SM] Indeed, the German regulator (and I am not saying this is th= e only or best option) decided to require ISPs to give 6 numbers pre-contra= ct signing: 3 for up- and 3 for downstream: a maximal (net) rate, a minimal= (net) rate, and a usually achievable (net) rate, all rates were defined as= IP/TCP goodput to make verification easier on end-users. The regulator als= o defined a measurement regime that end-users can follow to check whether t= he ISP is fulfilling the contract and the law gives remedies if ISPs do not= deliver (either the right to immediately cancel the contract and/or the ri= ght to adapt the monthly payments to the actually delivered ratio of the co= ntracted rates*). I think I need to add, that ISPs can set these numbers fr= eely, but are only allowed to advertise with the maximum rate. > But if we talk about a single number per direction, minimal rate = is probably easiest, or something like a "usually achievable rate" (that ne= eds to be met or exceeded in say 90% of >=3D 20 measurements or so). > > > *) In a ironic twist however the regulator so far has not explained how d= eviations in each of the 6 numbers above should be aggregated to get one si= ngle contract rate achievement ratio..., most ISPs took measures into their= own hands and simply offer customers typically a permanent rebate of 5 EUR= or immediate cancelation what ever the customer prefers.... > > > > > >> > >>> > >>> Seeing all the political activity tied onto it since (and now again) > >>> reminds of two families at war about an incident that had happened > >>> generations and generations before, where the two sides no longer > >>> remembered why they hated each other so, but just went on hating, and > >>> not forgiving, and not moving on. > >>> > >>> Yes, there are entirely separate and additional NN issues, but the > >>> technical problem of providing common carriage between two very > >>> different network application types (voip/gaming vs file transfer) is > >>> thoroughly solved now, > >> > >> I am not sure this was at the core of the problem, my take is r= eally that "always-on" and relative upload-heavy torrent simply demonstrate= d painfully for all involved that the old oversubscription ratios were not = suitable for the changed traffic profiles. I have some sympathy for the ISP= s here, they certainly did not try to sell their customers short (good ISPs= try to retain their customers and that works best when customers are happy= with the service) and having this problem appear on many segments at the s= ame time is not a fun place to be, and upload was/is often (too) low in DOC= SIS segments anyway; but this is why e.g. my bit torrent could affect your = VoIP, simply by pushing the whole segment into upload capacity congestion..= . (ISPs in theory could fix this by plain old QoS engineering, but the issu= e at hand was with a non-ISP VoIP/SIP service and there QoS becomes tricky = if the ISP as these packets need to be identified in a way that is not game= -able**) > > [SM] See my later mail to Jason, I will not claim I know Comcast'= s issues better than him and will accept that self-congestion also played a= major role in the initial problem. Over here in Europe the net neutrality = debate was kicked of less by an unfortunate confluence of new usage profile= s and traditional oversibscription ratios and less than ideal packet schedu= ling, but rather by a series of pretty apparent attempts at restricting con= sumer choice, see eg. https://www.accessnow.org/wp-content/uploads/archive/= docs/Net_Neutrality_Ending_Network_Discrimination_in_Europe_Access_v3.pdf f= or an admitted slightly biased position: > > " =E2=80=A2 Blocking of applications and services: In order to maxi= mise profits, some ISPs =E2=80=93 that also offer their own services and ap= plications online =E2=80=93 exclude certain services and applications of co= mpeting market players. The most prominent case of this form of network dis= crimination is European mobile providers (like Deutsche Telekom) blocking o= r restricting the use of Voice over IP (VoIP) services (like Skype and Vibe= r) for their customers.20 > > =E2=80=A2 Slowing or =E2=80=9Cthrottling=E2=80=9D internet speeds= : Some ISPs slow down specific services (like YouTube) and applications (li= ke Skype), or ask users to pay an extra fee to have access to these interne= t platforms. Given the high latency (delay) sensitivity of many application= s, ISPs are able to compromise the correct functioning of these services by= slowing them down, preventing the services from running properly. Often IS= Ps =E2=80=93 especially telecommunication companies =E2=80=93 do this to fa= vour their own voice calling services over VoIP services, thereby crushing = competition. > > > =E2=80=A2 Blocking websites: ISPs often block websites for a numb= er of reasons =E2=80=93 to secure their network, or to avoid competition, a= nd sometimes for social, public relations or political reasons. In the UK, = for instance, Orange Telecom blocked the French digital rights advocacy gro= up, La Quadrature du Net=E2=80=99s website on pre-paid mobile accounts.21 > > =E2=80=A2 Preferential treatment of services and platforms: ISPs = can also impose data caps on internet access contracts while granting data = allowance exceptions to a company=E2=80=99s own proprietary streaming servi= ces (like Deutsche Telekom to its own =E2=80=9CT-Entertain=E2=80=9D).22 The= y can (and do) also grant preferential treatment to select services =E2=80= =93 such as Orange France with the popular music streaming service Deezer23= =E2=80=93 ahead of other competitors, effectively imposing anti-competitiv= e limitations on markets such as those for legal online music. Moreover, ge= nerally only large, well-established companies can afford this preferential= treatment, resulting in a further stifling of innovation." > > These look like clearer attempts to monetize the ISP position as gate-kee= per to his customer's eye-balls (for content providers) as well as gate-kee= per to the internet for for paying customers. > I find it much harder to have sympathy for the listed examples of ISP beh= avior than the situation of rapidly changing usage pattern posed where tech= nical changes needed to be made, but where no attempt at unfair monetizatio= n was evident, but maybe I am overly sensitive. These are all examples that= make me personally applaud the EU for its intervention to make clear rules= what "internet access providers" can and can not do. (The EU also was quit= e flexible in applying/interpreting its rules during the covid isolation pe= riods, when it made it clear that e.g. treating certain traffic classes e.g= . streaming media to lower priority than video conferences was within the n= eutrality framework as long as it was based on traffic class and not on spe= cific service providers IIRC). > > > > > > > Torrent problem thoroughly solved with FQ on the path and backpressure > > from the mac scheduler. > > > > https://perso.telecom-paristech.fr/drossi/paper/rossi14comnet-b.pdf > > [SM] Thanks for the link. This is mainly for the self-congesrion = case? > > > > > >> I agree that on a single link we mostly solved the problem (a f= ew corner cases are left on links with very low capacity, where essentially= we can only manage the pain, not remove it)... > >> > >> > >> **) Which is not rocket science either, a VoIP stream takes ~100 Kbps,= so in theory an ISP might simply allow each customer say 5 VoIP stream equ= ivalents by allowing up to 500Kbps od traffic marked with a specific DSCP a= s higher priority (with higher access probability for the shared medium). I= am not sayng this is my preferred solution, just saying this is a solution= that would have been available at the time if I memorize my docsis capabil= ities correctly. > >> > >> > >>> and if only all sides recognised at least this > >>> much, and made peace over it, and worked together to deploy those > >>> solutions, maybe, just maybe, we could find mutually satisfactory > >>> solutions to the other problems that plague the internet today, like > >>> security, and the ipv6 rollout. > >> > >> +1. IPv6 is IMHO a prime example where the regulators should st= op talking softly and start showing the big stick they carry. > > > > I really would like to see a push for IPv6 mandated as a part of the > > BEAD programs. > > [SM] +1; I am not the biggest IPv6 fan, but that's what we have a= nd it is mostly serviceable so let's get this "party started" finally. > > > > >> Heck in Germany we have ISPs that still only supply CG-NATed IPv4 addr= esses only.. (most mass market ISPs do much better in that regard, but for = the stragglers it would help if the regulator would demand IPv6 with PD***)= . Regarding security, the easiest way to achieve that would be to put some = heavy requirements on IoT manufacturers and vendors (like do what you pleas= e as long as you are local LAN only, but once you reach out into the cloud = you need to fulfill the following list of requirements, with timely securit= y updates over a reasonable long usage period), however that might not be v= ery attractive for politicians/regulators to tackle (active regulatory acts= tends to get bad press unless something bad happened, but even then the pr= ess often complains about the acts coming too late, but I digress****) > > > > There is a separate NRPM going on for the new cybersecurity label at > > the FCC. If I had time, and a partner, > > we could rework the doc we did a few years ago. It is due oct 6. > > > >> > >> > >> ***) Strictly speaking IPv6 is required, since "internet access" is de= fined as reaching all of the internet (as far as in the ISPs power) and IPv= 6-only sites are not reachable for the CG-NAT-only customers. But so far th= e local regulator does not seem to enforce that requirement, or hopefully i= s working on this quietly behind the curtains. > >> > >> ****) This is not to diss the press, they are doing what they are supp= osed to do, but it just shows that active regulation is a tricky business, = and a light touch typically "looks better" (even though I see no real evide= nce it actually works better). > >> > >>> If anyone here knows anyone more political, still vibrating with 10+ > >>> years of outrage about NN on this fronts, on one side or the other, i= f > >>> you could sit them down, over a beer, and try to explain that at the > >>> start it was a technical problem nobody understood at the time, maybe > >>> that would help. > >> > >> So in the EU that debate is essentially settled, there is a EU = regulation that essentially spills out what ISPs owe their customers and th= at has become the law of the land. The rationale for required un-biased ser= vice and freedom to select terminal devices is well justified by the market= ideals of the EU, allowing ISPs to discriminate packets or terminal device= s restricts the market and will lead to undesired outcomes. (Fun fact most = big players in capitalist societies argue for "free markets" but at the sam= e time act to work-around the market mechanism by trying to move the market= into an oligo- or even monopoly condition, which is why strong regulation = is required*****). > > > > Governments make safer markets feasible. > > [SM] Yes, I agree, both in markets are a pretty decent tool, but = need constant maintenance. > > Regards & Sorry for the tangent > Sebastian > > > > >> > >> *****) This is akin to professional sports where the audience generall= y accepts that referees are necessary and occasionally need to make "painfu= l" calls, as the alternative would be anarchy, but I digress. > >> > >> Regards > >> Sebastian > >> > >> > >>> > >>> -- > >>> Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-b= of.html > >>> Dave T=C3=A4ht CSO, LibreQos > >>> _______________________________________________ > >>> Rpm mailing list > >>> Rpm@lists.bufferbloat.net > >>> https://lists.bufferbloat.net/listinfo/rpm > >> > > > > > > -- > > Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof= .html > > Dave T=C3=A4ht CSO, LibreQos > > <2510.jpeg> > --=20 Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.htm= l Dave T=C3=A4ht CSO, LibreQos