From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi1-x235.google.com (mail-oi1-x235.google.com [IPv6:2607:f8b0:4864:20::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 952573B29E; Fri, 29 Sep 2023 12:15:56 -0400 (EDT) Received: by mail-oi1-x235.google.com with SMTP id 5614622812f47-3af609c3e74so1944413b6e.2; Fri, 29 Sep 2023 09:15:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696004156; x=1696608956; darn=lists.bufferbloat.net; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=dcf2+PoGrPIMUs1V0U2Ey/cFML7imRJ273RAs1QHMFE=; b=DC/utnSAzWTI9Jg7ZqdF7/4AZBUYI/V7rBbRz7fxzftroBiplP0HHoLIL1f/h20sB2 ShTWIYsGpfLvno/Oxpmk+1VCgdvRvUBFsChaS/XLs5PrGbzJob1xsnR3QeRPdyyI/40L DVVZH4kOFFY3Za1nY5aVR+aQyTjRIcSCqp5470Se28SnBtvj2AkpwPl/AUE4c3IuOphp jOlrdLECryHQGnpnt964uC8fLdbFJOA01mLy6jx1BaU19MLan0x9pc2TqI2rHiYEj9DV BC93ugH/ETx44x70JSZ8M3MZ1Th8/yHhP30JVq/cDQXQhTMyLPafCF08wthbSqvWIqtm RkZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696004156; x=1696608956; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dcf2+PoGrPIMUs1V0U2Ey/cFML7imRJ273RAs1QHMFE=; b=moa+J+8U2rXgE9TqK9wNPT3i91BARdnqJe3BIPttPS1LH7mWe+7it527xMCcEU2Kme TAh5YYRJGZU8Vn0yrWRxI6JDi12spSYlyPiR+H7wC6GYNTKFRMArUyk8auAMd4yTtfmU amAReln3qch6a7UGH0L+ZgsUpc+js+aeU8pBQJTzOIrlI7iJmkGi2ZMtqRQnjlSghTeB 3RW+gL6HDbI9Oxo4Emz16kfCeDDXcBCWrB9joWMjxKqYUSF67omr5oCB2Dtl7yEuUkFA auG9kHWWYpXXOnSEDG3UUDzfKytbGTEdte2sLbqPb+saxbeRHT5PATTJwovtFBSyyPM1 gPAw== X-Gm-Message-State: AOJu0Yz+WN6oAfjRAXCVLorG3T61Xn45mk7D6iNmiC3i9Cf8GvB++y6M pYnzDbsNIYqIGeAw802CWUkWzEMjxekV/p6gHss= X-Google-Smtp-Source: AGHT+IGmMMZFBIm1tZy3nMORVUufxi+8ah/IwUX2V+3tIS01AFE7T3ym8/XDSLpeosxe4OMUdfXfp/hFgCUVffNyrKk= X-Received: by 2002:a05:6358:7e84:b0:13a:4855:d885 with SMTP id o4-20020a0563587e8400b0013a4855d885mr5260327rwn.10.1696004155533; Fri, 29 Sep 2023 09:15:55 -0700 (PDT) MIME-Version: 1.0 References: <5so3r00n-31pn-14s7-7775-08731s3s551r@ynat.uz> <7508FE73-A154-4CBA-984C-748A80C5FFEC@gmail.com> <039490DA-48A7-4AE2-B00F-AA2A260FB747@gmail.com> In-Reply-To: <039490DA-48A7-4AE2-B00F-AA2A260FB747@gmail.com> From: dan Date: Fri, 29 Sep 2023 10:15:44 -0600 Message-ID: To: Rich Brown Cc: Jonathan Morton , David Lang , Rpm , Jamal Hadi Salim , libreqos , Dave Taht via Starlink , "Livingood, Jason" , bloat Content-Type: multipart/alternative; boundary="0000000000003a6c8b060681bdb1" Subject: Re: [Rpm] [Bloat] [Starlink] [LibreQoS] net neutrality back in the news X-BeenThere: rpm@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: revolutions per minute - a new metric for measuring responsiveness List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2023 16:15:56 -0000 --0000000000003a6c8b060681bdb1 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable ok, lots and lots of great comments here for sure. bandwidth abundance: Not for most people and ISPs. The 'carriers' aren't carrying to many places at affordable rates. I've pulled quotes from Lumen and Zayo at over $5k/month/gig. We typically pay 900-1400 for a gig of service. This isn't abundance and it's widespread and it leaves only major providers that can afford/amortize out 100G transits etc. My answer to this is one that Dave and I have bounced back and forth is the idea of micro IXs in every municipality and having that somehow tied into access to the ROW in counties etc. Not fully hashed out, but the fiber is in the ground, it could be sold, but the carriers are not well incentivised to sell it. It takes the better part of a year to get a DIA within 100ft of a Lumen hut sometimes... Heck, it could even be a government program to get an =CE=BC*IX* with x feet of every school, city hall, and library. I d= on't care how it's done but this would get abundance NEAR end users and open up essentially every town to competition. monopoly. This is a historical thing for most cable and DSL incumbents. They have enjoyed virtual monopolies with cable owning population centers and DSL owning the outskirts and there is no product darwinism here where customer satisfaction is a pressure. That may not be the future but it definitely is the past. These companies may have to shift into customer satisfaction as a major part instead of a minor part of their corporate culture to fend off fttx and ultra-modern wisps. Starlink is not offering significant competition to major carriers. Starlink's 1.5 million customers are at LEAST 90% pulled from other satellite services and small ISPs. Spectrum and Comcast's losses to starlink are measured in decimal points. Only fttx and ultra-modern wireless tech really threatens these incumbents. Typical wisps aren't putting a dent in these guys, just scraping the paint off their bumper. We're pulling customers at the scale of 'dozens' for example. Spectrum's management doesn't know we exist we're such a small threat to them. But to further the point here, these fttx and ultra-modern wisps can only exist in places where there is adequate carrier services to start with. In areas where they spend the money and do the build but there aren't good carrier services, those fiber services suck and the wISPs start to claw back even with inferior technology. We've pulled quite a few customers off fttx deployments because of this sort of situation. On Fri, Sep 29, 2023 at 7:28=E2=80=AFAM Rich Brown wrote: > Thank you Jonathan for this clear description of the issues and their > history. I wonder if there's a fourth one - privacy. > > Rosenworcel's talk > https://docs.fcc.gov/public/attachments/DOC-397257A1.pdf also points out > that ISPs might want to monetize our traffic patterns and location data. > (This is less of an issue in the EU, but the US remains a Wild West in th= is > regard.) > > I am hopeful that the FCC will include this in their NPRM (which must be > available by now but I haven't looked...) > > - Rich Brown > > > On Sep 29, 2023, at 12:54 AM, Jonathan Morton via Rpm < > rpm@lists.bufferbloat.net> wrote: > > > >> On 29 Sep, 2023, at 1:19 am, David Lang via Bloat < > bloat@lists.bufferbloat.net> wrote: > >> > >> Dave T called out earlier that the rise of bittorrent was a large part > of the inital NN discussion here in the US. But a second large portion wa= s > a money grab from ISPs thinking that they could hold up large paid websit= es > (netflix for example) for additional fees by threatening to make their > service less useful to their users (viewing their users as an asset to be > marketed to the websites rather than customers to be satisfied by providi= ng > them access to the websites) > >> > >> I don't know if a new round of "it's not fair that Netflix doesn't pay > us for the bandwidth to service them" would fall flat at this point or no= t. > > > > I think there were three more-or-less separate concerns which have, ove= r > time, fallen under the same umbrella: > > > > > > 1: Capacity-seeking flows tend to interfere with latency-sensitive > flows, and the "induced demand" phenomenon means that increases in link > rate do not in themselves solve this problem, even though they may be sol= d > as doing so. > > > > This is directly addressed by properly-sized buffers and/or AQM, and > even better by FQ and SQM. It's a solved problem, so long as the solutio= ns > are deployed. It's not usually necessary, for example, to specifically > enhance service for latency-sensitive traffic, if FQ does a sufficiently > good job. An increased link rate *does* enhance service quality for both > latency-sensitive and capacity-seeking traffic, provided FQ is in use. > > > > > > 2: Swarm traffic tends to drown out conventional traffic, due to > congestion control algorithms which try to be more-or-less fair on a > per-flow basis, and the substantially larger number of parallel flows use= d > by swarm traffic. This also caused subscribers using swarm traffic to > impair the service of subscribers who had nothing to do with it. > > > > FQ on a per-flow basis (see problem 1) actually amplifies this effect, > and I think it was occasionally used as an argument for *not* deploying > FQ. ISPs' initial response was to outright block swarm traffic where the= y > could identify it, which was then softened to merely throttling it heavil= y, > before NN regulations intervened. Usage quotas also showed up around thi= s > time, and were probably related to this problem. > > > > This has since been addressed by several means. ISPs may use FQ on a > per-subscriber basis to prevent one subscriber's heavy traffic from > degrading service for another. Swarm applications nowadays tend to emplo= y > altruistic congestion control which deliberately compensates for the larg= e > number of flows, and/or mark them with one or more of the Least Effort > class DSCPs. Hence, swarm applications are no longer as damaging to > service quality as they used to be. Usage quotas, however, still remain = in > use as a profit centre, to the point where an "unlimited" service is a ra= re > and precious specimen in many jurisdictions. > > > > > > 3: ISPs merged with media distribution companies, creating a conflict > of interest in which the media side of the business wanted the internet > side to actively favour "their own" media traffic at the expense of "the > competition". Some ISPs began to actively degrade Netflix traffic, in > particular by refusing to provision adequate peering capacity at the node= s > through which Netflix traffic predominated, or by zero-rating (for the > purpose of usage quotas) traffic from their own media empire while refusi= ng > to do the same for Netflix traffic. > > > > **THIS** was the true core of Net Neutrality. NN regulations forced > ISPs to carry Netflix traffic with reasonable levels of service, even > though they didn't want to for purely selfish and greedy commercial > reasons. NN succeeded in curbing an anti-competitive and consumer-hostil= e > practice, which I am perfectly sure would resume just as soon as NN > regulations were repealed. > > > > And this type of practice is just the sort of thing that technologies > like L4S are designed to support. The ISPs behind L4S actively do not wa= nt > a technology that works end-to-end over the general Internet. They want > something that can provide a domination service within their own walled > gardens. That's why L4S is a NN hazard, and why they actively resisted a= ll > attempts to displace it with SCE. > > > > > > All of the above were made more difficult to solve by the monopolistic > nature of the Internet service industry. It is actively difficult for > Internet users to move to a truly different service, especially one based > on a different link technology. When attempts are made to increase > competition, for example by deploying a publicly-funded network, the > incumbents actively sabotage those attempts by any means they can. > Monopolies are inherently customer-hostile, and arguments based on market > forces fail in their presence. > > > > - Jonathan Morton > > > > _______________________________________________ > > Rpm mailing list > > Rpm@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/rpm > > --0000000000003a6c8b060681bdb1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
ok, lots and lots of great comments here for sure.=C2=A0= =C2=A0

bandwidth abundance:=C2=A0 Not for most people and ISPs.=C2= =A0 The 'carriers' aren't carrying to many places at affordable= rates.=C2=A0 I've pulled quotes from Lumen and Zayo at over $5k/month/= gig.=C2=A0 We typically pay 900-1400 for a gig of service.=C2=A0 This isn&#= 39;t abundance and it's widespread and it leaves only major=C2=A0provid= ers that can afford/amortize out 100G transits etc.
My answer to this is= one that Dave and I have bounced back and forth is the idea of micro IXs i= n every municipality and having that somehow tied into access to the ROW in= counties etc.=C2=A0 Not fully hashed out, but the fiber is in the ground, = it could be sold, but the carriers are not well incentivised to sell it.=C2= =A0 It takes the better part of a year to get a DIA within 100ft of a Lumen= hut sometimes...=C2=A0 Heck, it could even be a government program to get = an=C2=A0=CE=BCIX with x feet of= every school, city hall, and library.=C2=A0 I don't care how it's = done but this would get abundance NEAR end users and open up essentially ev= ery town to competition.

monopoly.=C2=A0 This is a historical thing = for most cable and DSL incumbents.=C2=A0 They have enjoyed virtual monopoli= es with cable owning population centers and DSL owning the outskirts and th= ere is no product darwinism here where customer satisfaction is a pressure.= =C2=A0 That may not be the future but it definitely is the past.=C2=A0 Thes= e companies may have to shift into customer satisfaction as a major part in= stead of a minor part of their corporate culture to fend off fttx and ultra= -modern wisps.

Starlink is not offering significant competition to m= ajor carriers.=C2=A0 =C2=A0 Starlink's 1.5 million customers are at LEA= ST 90% pulled from other satellite services and small ISPs.=C2=A0 Spectrum = and Comcast's losses to starlink are measured in decimal points.
Only fttx and ultra-modern wireless tech really threatens these incumbents= .=C2=A0 Typical wisps aren't putting a dent in these guys, just scrapin= g the paint off their bumper.=C2=A0 We're pulling customers at the scal= e of 'dozens' for example.=C2=A0 Spectrum's management doesn= 9;t know we exist we're such a small threat to them.=C2=A0 =C2=A0
But to further the point here, these fttx and ultra-modern wisps can only= exist in places where there is adequate carrier services to start with.=C2= =A0 In areas where they spend the money and do the build but there aren'= ;t good carrier services, those fiber services suck and the wISPs start to = claw back even with inferior technology.=C2=A0 We've pulled quite a few= customers off fttx deployments because of this sort of situation.

<= /div>
O= n Fri, Sep 29, 2023 at 7:28=E2=80=AFAM Rich Brown <richb.hanover@gmail.com> wrote:
Thank you Jonathan for this cl= ear description of the issues and their history. I wonder if there's a = fourth one - privacy.

Rosenworcel's talk https://docs.fcc.gov/= public/attachments/DOC-397257A1.pdf also points out that ISPs might wan= t to monetize our traffic patterns and location data. (This is less of an i= ssue in the EU, but the US remains a Wild West in this regard.)

I am hopeful that the FCC will include this in their NPRM (which must be av= ailable by now but I haven't looked...)

- Rich Brown

> On Sep 29, 2023, at 12:54 AM, Jonathan Morton via Rpm <rpm@lists.bufferbloat.ne= t> wrote:
>
>> On 29 Sep, 2023, at 1:19 am, David Lang via Bloat <bloat@lists.bufferbloa= t.net> wrote:
>>
>> Dave T called out earlier that the rise of bittorrent was a large = part of the inital NN discussion here in the US. But a second large portion= was a money grab from ISPs thinking that they could hold up large paid web= sites (netflix for example) for additional fees by threatening to make thei= r service less useful to their users (viewing their users as an asset to be= marketed to the websites rather than customers to be satisfied by providin= g them access to the websites)
>>
>> I don't know if a new round of "it's not fair that Ne= tflix doesn't pay us for the bandwidth to service them" would fall= flat at this point or not.
>
> I think there were three more-or-less separate concerns which have, ov= er time, fallen under the same umbrella:
>
>
> 1:=C2=A0 Capacity-seeking flows tend to interfere with latency-sensiti= ve flows, and the "induced demand" phenomenon means that increase= s in link rate do not in themselves solve this problem, even though they ma= y be sold as doing so.
>
> This is directly addressed by properly-sized buffers and/or AQM, and e= ven better by FQ and SQM.=C2=A0 It's a solved problem, so long as the s= olutions are deployed.=C2=A0 It's not usually necessary, for example, t= o specifically enhance service for latency-sensitive traffic, if FQ does a = sufficiently good job.=C2=A0 An increased link rate *does* enhance service = quality for both latency-sensitive and capacity-seeking traffic, provided F= Q is in use.
>
>
> 2:=C2=A0 Swarm traffic tends to drown out conventional traffic, due to= congestion control algorithms which try to be more-or-less fair on a per-f= low basis, and the substantially larger number of parallel flows used by sw= arm traffic.=C2=A0 This also caused subscribers using swarm traffic to impa= ir the service of subscribers who had nothing to do with it.
>
> FQ on a per-flow basis (see problem 1) actually amplifies this effect,= and I think it was occasionally used as an argument for *not* deploying FQ= .=C2=A0 ISPs' initial response was to outright block swarm traffic wher= e they could identify it, which was then softened to merely throttling it h= eavily, before NN regulations intervened.=C2=A0 Usage quotas also showed up= around this time, and were probably related to this problem.
>
> This has since been addressed by several means.=C2=A0 ISPs may use FQ = on a per-subscriber basis to prevent one subscriber's heavy traffic fro= m degrading service for another.=C2=A0 Swarm applications nowadays tend to = employ altruistic congestion control which deliberately compensates for the= large number of flows, and/or mark them with one or more of the Least Effo= rt class DSCPs.=C2=A0 Hence, swarm applications are no longer as damaging t= o service quality as they used to be.=C2=A0 Usage quotas, however, still re= main in use as a profit centre, to the point where an "unlimited"= service is a rare and precious specimen in many jurisdictions.
>
>
> 3:=C2=A0 ISPs merged with media distribution companies, creating a con= flict of interest in which the media side of the business wanted the intern= et side to actively favour "their own" media traffic at the expen= se of "the competition".=C2=A0 Some ISPs began to actively degrad= e Netflix traffic, in particular by refusing to provision adequate peering = capacity at the nodes through which Netflix traffic predominated, or by zer= o-rating (for the purpose of usage quotas) traffic from their own media emp= ire while refusing to do the same for Netflix traffic.
>
> **THIS** was the true core of Net Neutrality.=C2=A0 NN regulations for= ced ISPs to carry Netflix traffic with reasonable levels of service, even t= hough they didn't want to for purely selfish and greedy commercial reas= ons.=C2=A0 NN succeeded in curbing an anti-competitive and consumer-hostile= practice, which I am perfectly sure would resume just as soon as NN regula= tions were repealed.
>
> And this type of practice is just the sort of thing that technologies = like L4S are designed to support.=C2=A0 The ISPs behind L4S actively do not= want a technology that works end-to-end over the general Internet.=C2=A0 T= hey want something that can provide a domination service within their own w= alled gardens.=C2=A0 That's why L4S is a NN hazard, and why they active= ly resisted all attempts to displace it with SCE.
>
>
> All of the above were made more difficult to solve by the monopolistic= nature of the Internet service industry.=C2=A0 It is actively difficult fo= r Internet users to move to a truly different service, especially one based= on a different link technology.=C2=A0 When attempts are made to increase c= ompetition, for example by deploying a publicly-funded network, the incumbe= nts actively sabotage those attempts by any means they can.=C2=A0 Monopolie= s are inherently customer-hostile, and arguments based on market forces fai= l in their presence.
>
> - Jonathan Morton
>
> _______________________________________________
> Rpm mailing list
> Rpm@lis= ts.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/rpm

--0000000000003a6c8b060681bdb1--