[NNagain] On "Throttling" behaviors

Mon Oct 2 03:28:00 EDT 2023

Hi Dan,

> On Oct 2, 2023, at 03:34, dan via Nnagain <nnagain at lists.bufferbloat.net> wrote:
> 
> I think a big problem with Net Neutrality is it's sort of a topic for providers and not for end users.

	[SM] I respectfully disagree. If my ISP sells me access to the internet, I expect being able to access the whole internet, at least as far it is with the reasonable power of my ISP. So I would argue it is a topic for both parties.

>  Most end users have no concept of what it is, why it's good or bad, and if they have a compliant service or not.  They confuse 'speed' with packet loss and latency.

	[SM] I would agree that the majority of users might not have actionable concepts of how the internet works, let alone what to expect precisely, but I predict if we ask them whether they think it a good idea for their ISP to pick winners and losers of content providers I expect most of them to say no.

> This is why my arguments are primarily from a consumer protection perspective of forcing providers to be transparent in their NN or non-NN activities.

	[SM] Transparency is indded an important factor, but IMHO not sufficient.

>  If they are going to shape streaming video down, they should have to put that in the big print on the plan.  In doing this, customers would get some practical information that doesn't rely on heavy technical details and various opinions.  Consumers will understand the 100x20 w/ 1080p streams.  This could be on the broadband nutrition label *but* I would say you need to have that up-to-date for the times with key information on there and with strikethrough text.  ie, Download [ 100 ], Upload [ 20 ], streaming [ 720 1080p 4k ] and that list can be updated yearly during the BDC process or whatever.  

	[SM] Here we get into the weeds:
a) for customers only having access to a single (broadband) ISP this information would hardly be actionable
b) personally I could agree to this, but e.g. Netflix [720], Amazon [4K], GoogleTV [720] (or what ever google calls this service right now) would IMHO be problematic.
c) I think that this gets really hard to enforce unless the ISP also throttled VPN/encrypted traffic and there I see a line crossed, no amount of transparency would make it palatable if an ISP would heavily throttle all encrypted traffic.

> 
> I'm really a proponent of creating a competitive environment, not controlling the technical details of those products.

	[SM] +1; but that only ever helps in a competitive market environment and that means there need to be enough buyers and sellers on both sides that no single one gets an undue influence over the market. At least in Germany we at best have an oligopoly market for internet access, so the "free market" by itself does not solve the discrimination problem. That might well be different in other parts of the world.

>  And making sure key parts of the rules require transparency, because that can help drive competition.  If a local competitor is running a DPI's QoE box shaping down streaming, I think they should have to share that conspicuously with the end user.  I can then market directly against that if I want.

	[SM] Again I fully endorse that, but feel that by itself would not be enough.

> I would say that I think that any government money should require NN plans.  If a company is taking government bribes for 100x20 speed, that should be a NN 100x20.  They can offer 100x20 w/ 1080p streaming also, but that gov money should have NN baked into what they/we are paying for.

	[SM] Interesting approach that I agree with, but again do not think it to be enough.

> 
> I know that one of the initial ideas of NN was to say that there are no fast lanes, but that's ridiculous because just having different speed plans IS having fast lanes for premium pricing.

	[SM] I think this is a misunderstanding. NN is not against ISPs offering different access capacity tiers at different prices, but that within the transparent customer-known "capacity-limit" the ISP does not pick winners or losers amount flows and especially not based on financial considerations. If an ISPs sells internet access they better do so, and they better not try to sell the same traffic a second time to the content provider. So the fast lane "fast lane" argument really means that the ISP does not built an unfair fast-lane for content provider A (capA) over content provider B (capB), and even that is subtle, if cabA puts caching nodes with the ISPs network, but capB does not a result in accessibility is not "on the ISP" however if the ISP would additionally slow down capB traffic that would be a problem. In a sense the thing that is problematic is building fast-lanes by slowing down all the rest.

>  The only possible way around this is the allow or even enforce a price per Mbps and make it like a utility which I don't think anyone wants.  Any other model eliminates the sane enforcement of a 'no fast lanes' policy.  

	[SM] I think this is an attempt at reductio ad absurdum, that does not really cover the NN position well enough to be useful here. Think about it that wat, ISPs are not supposed to pick winners and losers (or rather tilt the table to create winners and losers).

> I believe that transparency is the key here.

	[SM] THe EU actually agrees., an impoertant part of the NN-relevant EU regulation covers transparency.

>  Transparency in how bits are delivered and managed to the end user allows the consumer to make choices without demanding unrealistic education of those consumers.

	[SM] That however seems quite a challenge... though it is not unheard of to offer multiple parallel explanations at different levels of abstraction/complexity...

>  They know if they want more than 1080p streaming.  They know if they want a gamer or home work focused service that does that by shaping streaming down to 1080p.  There are consumer benefits to a non-NN service.  

	[SM] Really only if the "unfairness" of that service are exactly tailored to a specific users needs and desires, not sure that any single such a policy would be desirable for the majority of end-users, but I might be insufficiently creative here.

> Those benefits dont outweigh the harm from secret shaping, but if all of this is clearly shown in a conspicuous way like the broadband nutrition label and there were teeth in that label, then NN is a feature that can be advertised and DPI shaped is another feature that must be advertised.

	[SM] The only way I see DPI shaping ever to be in the end-users interesst is if that feature is under the end-users control. E.g. on a very slow link the administrator might decide that interactive VCs are more important than streaming video and might desire ways t achieve this policy and might appreciate if the ISPs offers such de-prioritization as a self-serve option

> Now for my ISP version of this.

	[SM] And now it gets interesting ;)

> 
> I WANT to compete against the companies that have to put 1080p and no 4k in their advertisements.  I want to offer full NN packages for the same price as their 'only 1080p' package.  I want the competitive environment and I want to know if the service area next to mine with just 1 or 2 providers doesn't offer NN packages so I can move in on them and offer them.  

	[SM] I wish all ISPs would have such a "bring it on" let's duke it out on the merits attitude! Yest the big mass-market ISPs in my home market do not really qualify for that all they compete about are nominal "speed"/price (really capacity/price) and some additional features like flat rate for telephony to fixed line and/or mobile networks. No details given about specific properties... for these one needs to scour rge internet fora to figure out what a specific ISPs customers mainly kvetch about (and then one needs to abstract over the fact that not all complaints are created equal and some say more about a user's deficiency in expectation than about the ISP's service delivery).

> 
> So from my perspective, transparency in shaping is all I really think should be done.  And teeth in resolving a lie, ie false advertising, with a rulebook already in place for that so no new legal theory needs worked out.  "I bought the 1G NN package and I'm running the NN test suite and it says I'm shaped to 1080p video", rectify this ISP or I'm going to throw a legal tantrum to FTC, FCC, and maybe my own lawyer and get some penalty claims.  And that test suite is pretty darn simple.  speed test, 1G.  Netflix, 1080p.  DPI shaped.

	[SM] Alas, that NN test suit is currently made of unobtainium. Though some individual parts exist already. For example kudos to netflix for fast.com, which as far as I understand uses essentially the same of its CDN nodes for speedtests it would likely pick for content delivery so these are pretty "realistic" testing conditions. Alas the other big streaming platform have as far as I know not followed suit.
As I tried to brain storm in a different context, what we would need at the very least would be speedtests that houses its servers in at least the networks of the big transit providers and runs comparative tests against these (either all or random pairs selected for each test). And while this would already be pretty ambitious to implement this would only scratch the surface of what would be required. However one redeeming fact is IMHO that NN-violations that achieve financial goals, by necessity need to be perceivable by the eye-balls so the thing we are looking for is likely not subtle but "in your face".

> 
> End rant.

	[SM] I rally wish more ISPs would have your spirit (and that we had more ISPs competing for eye-balls)! Also I think that your stance let the market settle this is conceptually in-line with our current macroeconomic system and theory (I am just not sure whether yje internet access market generally is healthy enough to relat on this mechanism to sort out bad apples).

Regards
	Sebastian

> 
> On Sun, Oct 1, 2023 at 4:01 PM Patrick Maupin via Nnagain <nnagain at lists.bufferbloat.net> wrote:
> > “neutrality” is framed by the experience of end users
> 
> Assuming, arguendo, that's true, then the issue becomes one of ensuring that bad actors can't slowly strangle the competition until eventually the antitrust laws have to be invoked.
> 
> Because that's always too late.
> 
> Dave Taht and others have correctly pointed out that many of the issues that have cropped up have purely technical solutions.
> 
> Maybe everything devolves to a technical issue?  I'm not sure, but it may be worthwhile to view current protocols, norms, and regulations as a system that is under security threats from bad actors*, and to enumerate some of the potential exploits and consider potential mitigations.
> 
> Bearing in mind, of course, that one of the tools available to bad actors may, in fact, be current or new regulations that impede the self-help capabilities of ISPs and other ecosystem participants, but also bearing in mind that the history of the internet as a whole (replete with cooperative endeavors) is completely different from the history of many of the rent-seeking businesses that are now aggressively monetizing it.
> 
> In any case, it's easy these days to sit out negotiations between Comcast and Netflix.  Or Comcast and Disney over football, or whatever.  Because they're all big boys.
> 
> The holy grail is, as always, to guarantee that the _next_ netflix can't be easily strangled in its crib.
> 
>  - Pat Maupin
> 
> [*] If there are no would-be bad actors, no regulation is needed.  But if there are would-be bad actors, then they will attempt to be instrumental in shaping any new regulations, and many of the presumed would-be bad actors have time, money, and histories of securing significant face time with regulatory agencies such as the FCC.
> 
> On Sun, Oct 1, 2023 at 3:51 PM Dave Cohen via Nnagain <nnagain at lists.bufferbloat.net> wrote:
> It’s useful for us to remember here that, politics aside, “neutrality” is framed by the experience of end users, not by the experience of operators. While much of Martin’s argument that malicious treatment and incapable design cannot often be distinguished from one another is correct, I believe it to be beside the point; from the perspective of most end users, “I can’t watch Netflix because Comcast is throttling it” and “I can’t watch Netflix because Comcast doesn’t have enough capacity where Netflix traffic ingresses” may be distinguishable arguments but they’re also equally egregious. (With apologies to Jason, Comcast was my home ISP during that era of NN debates, so my specific argument here is reflexive.) Most users will be sympathetic enough to tolerate an occasional service hiccup but will not tolerate repeated issues, regardless of the cause of those issues or their personal ability to differentiate between those causes. 
> 
> Of course this is a two way street. Anyone inclined to believe that large companies are going to conspiratorially behave maliciously took a big giant victory lap when Cogent press released that they had successfully throttled Netflix traffic in a controlled experiment, when of course that commanded no such thing. 
> 
> I recall getting into a Twitter fight with Martin around this time on that point. My argument was effectively that end user perception wasn’t going to change and an ISP trying to nuance its way through how it presents its traffic management (in effect, the “Quality Floor” framework) was missing the point with its customers; throwing more bandwidth at the problem was the only way out. I’d like to think that time has proven my perspective correct, although the growth in utilization of CDN and edge networking topologies also subverted the need for bandwidth growth, at least over the public Internet, to be quite so rapid. 
> 
> Dave Cohen
> craetdave at gmail.com
> 
> > On Oct 1, 2023, at 3:52 PM, Dave Taht via Nnagain <nnagain at lists.bufferbloat.net> wrote:
> > 
> > I kind of expect many, many forks of conversations here, and for those
> > introducing a new topic,
> > I would like to encourage using a relevant subject line, so I have
> > changed this one to suit.
> > 
> >> On Sun, Oct 1, 2023 at 11:57 AM Frantisek Borsik
> >> <frantisek.borsik at gmail.com> wrote:
> >> 
> >> OK, so I will bite the bullet! I have invited Ajit Pai and Martin Geddes to join us here and let's see if they still have some time and/or even stomach for current round of NN discussion.
> > 
> > Honestly I was hoping for some time to setup, and even perhaps, have
> > enough of us here
> > to agree on one of the definitions of NN, to start with!
> > 
> >> Anyway, here is my bullet. I will argue with Martin, that - Net Neutrality CAN'T be implemented:
> > 
> > You meant "argue along with" rather than "with". I know you are not a
> > native english speaker, but in the way you said it, it meant you were
> > arguing against what he described.
> > 
> >> 
> >>> Whilst people argue over the virtues of net neutrality as a regulatory policy, computer science tells us regulatory implementation is a fool’s errand.
> >>> Suppose for a moment that you are the victim of a wicked ISP that engages in disallowed “throttling” under a “neutral” regime for Internet access. You like to access streaming media from a particular “over the top” service provider. By coincidence, the performance of your favoured application drops at the same time your ISP launches a rival content service of its own.
> >>> You then complain to the regulator, who investigates. She finds that your ISP did indeed change their traffic management settings right at the point that the “throttling” began. A swathe of routes, including the one to your preferred “over the top” application, have been given a different packet scheduling and routing treatment.
> >>> It seems like an open-and-shut case of “throttling” resulting in a disallowed “neutrality violation”. Or is it?
> >>> Here’s why the regulator’s enforcement order will never survive the resulting court case and expert witness scrutiny:
> >> 
> >> 
> >> https://www.martingeddes.com/one-reason-net-neutrality-cant-implemented/
> > 
> > Throttling, using DPI or other methods, is indeed feasible. It is very
> > straightforward to limit flows to or from a given set of IP addresses.
> > However, there are also technical limitations, based on for example,
> > the underlying connectivity of a path be it one gbit or 10, which
> > would also show a customer problem in unwinding the difference between
> > intentionally throttling and merely being out of bandwidth across that
> > link. A lot of the netflix controversy was generated because netflix
> > suddenly ate far far more bandwidth that anyone had provisioned, and
> > was ultimately addressed by them developing and making easily
> > available a caching architecture that could be located within an ISPs
> > borders, saving an enormous amount on transit costs.  The rest of the
> > computing universe followed with enormous numbers of CDNs from the
> > bigger services being built out in the years following.
> > 
> > In part I kind of reject a few older arguments here in the face of
> > subsequent, technical improvements on how the internet works.
> > 
> > I had many discussions with martin back in the day. The reasoning in
> > this piece is pretty sound, except that "fairness" can be achieved via
> > various means (notably flow (fair) queueing), and it has always been a
> > (imperfectly implemented) goal of our e2e congestion control
> > algorithms to ultimately converge to a equal amount of bandwidth at
> > differing RTTs to different services.
> > 
> > My principal kvetch with his work was that every blog entry during
> > this period making these points then ended up linking to a "Quality
> > Floor", called "delta-something", a mathematical method that was
> > ill-documented, not available as open source and impossible, for me,
> > at least to understand. The link to that solution is broken in that
> > link, for example. That quasi-mystical jump to "the solution", I was
> > never able to make.
> > 
> > I believe something like this method lives on in Domos´s work today,
> > and despite them filing an internet draft on the subject (
> > https://datatracker.ietf.org/doc/draft-olden-ippm-qoo/ ) I remain
> > mostly in the dark, without being able to evaluate their methods
> > against tools I already understand.
> > 
> > I like the idea of what I think a "quality floor" might provide, which
> > is something that fq-everywhere can provide, and no known e2e
> > congestion control can guarantee.
> > 
> > I would like it if instead of ISPs selling "up to X bandwidth" they
> > sold a minimum guarantee of at least Y bandwidth, which is easier to
> > reason and provision for, but harder to sell.
> > 
> > Instead:
> > 
> > In the last 14 years I focused on restoring correct behavior of the
> > well-defined congestion controls of the internet, first documented
> > here:  https://ee.lbl.gov/papers/congavoid.pdf and built on the
> > decades since, while many others made enormous advances on what was
> > possible - packet pacing, for example, is a genuine breakthrough in
> > how multiple services from multiple providers can leave sufficient
> > space for other flows to co-exist and eventually use up their fair
> > share.
> > 
> >> 
> >> I hope you will read the link ^^ before jumping to Martin's conclusion, but still, here it is:
> >> 
> >>> 
> >>> So if not “neutrality”, then what else?
> > 
> > This is the phase where his arguments began to fall into the weeds.
> > 
> >>> The only option is to focus on the end-to-end service quality.
> > 
> > I agree that achieving quality on various good metrics, especially
> > under load, is needed. The popular MOS metric for voip could use some
> > improvement, and we lack any coherent framework for measuring
> > videoconferencing well.
> > 
> >>> The local traffic management is an irrelevance and complete distraction.
> > 
> > I am not sure how to tie this to the rest of the argument. The local
> > traffic management can be as simple as short buffers, or inordinately
> > complex, as you will find complex organisations internally trying to
> > balance the needs for throughput and latency, and for example, the
> > CAKE qdisc not only does active queue management, and FQ, but
> > optionally allows additional means of differentiation for
> > voice/videoconferencing, best effort, and "bulk" traffic, in an
> > effort, ultimately to balance goals to achieve the "service quality"
> > the user desires. Then there are the side-effects of various layer 2
> > protocols - wifi tends to be batchy, 5G tends towards being hugely
> > bufferbloated - PON has a 250us lower limit, cable
> > 
> >> Terms like “throttling” are technically meaningless. The lawgeneers who have written articles and books saying otherwise are unconsciously incompetent at computer science.
> > 
> > There are use cases, both good and bad, for "throttling". It is and
> > has always been technically feasible to rate limit flows from anyone
> > to anyone. Advisable, sometimes! DDOS attacks are one case where
> > throttling is needed.
> > 
> > Breaking the user perception of being intentionally throttled vs the
> > fate of the the rest of the network would be a goodness. The side
> > effects of one service, living on a slow network, becoming suddenly
> > popular, is known as the "slashdot effect", and is mostly mediated by
> > leveraging CDN and cloud technologies, and totally out of the control
> > of the local ISP.
> > 
> >>> We computer scientists call this viable alternative “end-to-end” approach a “quality floor”.
> > 
> > In googling I have thus far been unable to find a definition of
> > "Quality floor". Cite, please?
> > 
> >> The good news is that we now have a practical means to measure it and hard science to model it.
> > 
> > Weeds, here.
> > 
> >>> Maybe we should consciously and competently try it?
> > 
> > ... if only we had running code and rough consensus.
> > 
> >> 
> >> 
> >> 
> >> All the best,
> >> 
> >> Frank
> >> 
> >> Frantisek (Frank) Borsik
> >> 
> >> 
> >> 
> >> https://www.linkedin.com/in/frantisekborsik
> >> 
> >> Signal, Telegram, WhatsApp: +421919416714
> >> 
> >> iMessage, mobile: +420775230885
> >> 
> >> Skype: casioa5302ca
> >> 
> >> frantisek.borsik at gmail.com
> >> 
> >> 
> >> 
> >>> On Sun, Oct 1, 2023 at 7:15 PM Dave Taht via Nnagain <nnagain at lists.bufferbloat.net> wrote:
> >>> 
> >>> I am pleased to see over 100 people have signed up for this list
> >>> already. I am not really planning on "activating" this list until
> >>> tuesday or so, after a few more people I have reached out to sign up
> >>> (or not).
> >>> 
> >>> I would like y´all to seek out people with differing opinions and
> >>> background, in the hope that one day, we can shed more light than heat
> >>> about the science and technologies that "govern" the internet, to
> >>> those that wish to regulate it. In the short term, I would like enough
> >>> of us to agree on an open letter, or NPRM filing,and to put out a
> >>> press release(s), in the hope that this time, the nn and title ii
> >>> discussion is more about real, than imagined, internet issues. [1]
> >>> 
> >>> I am basically planning to move the enormous discussion from over
> >>> here, titled "network neutrality back in the news":
> >>> 
> >>> https://lists.bufferbloat.net/pipermail/starlink/2023-September/thread.html
> >>> 
> >>> to here. I expect that we are going to be doing this discussion for a
> >>> long time, and many more issues besides my short term ones will be
> >>> discussed. I hope that we can cleanly isolate technical issues from
> >>> political ones, in particular, and remain civil, and factual, and
> >>> avoid hyperbole.
> >>> 
> >>> Since the FCC announcement of a proposed NPRM as of Oct 19th... my own
> >>> initial impetus was to establish why the NN debate first started in
> >>> 2005, and the conflict between the legal idea of "common carriage" vs
> >>> what the internet was actually capable of in mixing voip and
> >>> bittorrent, in
> >>> "The Bufferbloat vs Bittorrent vs Voip" phase. Jim Gettys, myself, and
> >>> Jason Livinggood have weighed in on their stories on linkedin,
> >>> twitter, and elsewhere.
> >>> 
> >>> There was a second phase, somewhat triggered by netflix, that Jonathan
> >>> Morton summarized in that thread, ending in the first establishment of
> >>> some title ii rules in 2015.
> >>> 
> >>> The third phase was when title ii was rescinded... and all that has
> >>> happened since.
> >>> 
> >>> I, for one, am fiercely proud about how our tech community rose to
> >>> meet the challenge of covid, and how, for example, videoconferencing
> >>> mostly just worked for so many, after a postage stamp sized start in
> >>> 2012[2]. The oh-too-faint-praise for that magnificent effort from
> >>> higher levels rankles me greatly, but I will try to get it under
> >>> control.
> >>> 
> >>> And this fourth phase, opening in a few weeks, is more, I think about
> >>> privacy and power than all the other phases, and harmonization with EU
> >>> legislation, perhaps. What is on the table for the industry and
> >>> internet is presently unknown.
> >>> 
> >>> So here we "NN-again". Lay your issues out!
> >>> 
> >>> 
> >>> 
> >>> [1] I have only had one fight with the FCC. Won it handily:
> >>> https://www.computerworld.com/article/2993112/vint-cerf-and-260-experts-give-fcc-a-plan-to-secure-wi-fi-routers.html
> >>> In this case this is not so much a fight, I hope, but a collaborative
> >>> effort towards a better, faster, lower latency, and more secure,
> >>> internet for everyone.
> >>> 
> >>> [2] https://archive.org/details/video1_20191129
> >>> --
> >>> Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.html
> >>> Dave Täht CSO, LibreQos
> >>> _______________________________________________
> >>> Nnagain mailing list
> >>> Nnagain at lists.bufferbloat.net
> >>> https://lists.bufferbloat.net/listinfo/nnagain
> > 
> > 
> > 
> > -- 
> > Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.html
> > Dave Täht CSO, LibreQos
> > _______________________________________________
> > Nnagain mailing list
> > Nnagain at lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/nnagain
> _______________________________________________
> Nnagain mailing list
> Nnagain at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/nnagain
> _______________________________________________
> Nnagain mailing list
> Nnagain at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/nnagain
> _______________________________________________
> Nnagain mailing list
> Nnagain at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/nnagain