Re: [Bloat] On metrics

Re: [Bloat] On metrics

[Rich Brown]

> On Mar 19, 2023, at 11:03 PM, bloat-request@lists.bufferbloat.net wrote:
> 
>> Consumers really need things like published performance specs so they can
>> assemble their needs like an a la carte menu.  What do you do, what’s
>> important to you, what details support that need, and they need that in a
>> simple way.   Like a little app that says “how many 1080p TVs or 4K TVs,
>> how many gaming consoles, do you take zoom calls or VoIP/phone calls.  Do
>> you send large emails, videos, or pictures.”
> 
> The problem is that these needs really are not that heavy. Among my ISP 
> connections, I have a 8/1 dsl connection, even when I fail over to that, I can 
> run my 4k tv + a couple other HD TVs + email (although it's at the ragged edge, 
> trying to play 4k at 2x speed can hiccup, and zoom calls can stutter when large 
> emails/downloads flow)

I want to second David Lang's comment. I live in a small rural NH town that was stuck at DSL prior to a local company raising the money to install fiber to all premises. 

Before the fiber came in, I had 7mbps/768kbps service. If I wanted to bond two circuits, I could get 15/1mbps. But many neighbors had 3mbps/768kbps - or worse - so they were basically unserved. We frequently saw people parked outside our public library after hours to get internet. (And yes, a good router improved things. I told a lot of people about the IQrouter that turned the unusable service into merely slow.)

But there is a huge swath of rural US that is in the same situation, with zero or one provider of dreadful service.

What's the value of a "nutrition label"?

a) It's meaningless for those rural customers. They have no choice beyond "take it or leave it." 

b) For the lucky ones where alternative providers compete, the proposed label does provide a standardized format that lays out purported speeds and the the pricing tiers (including overage charges). I don't think I'd ever believe the latency numbers.

c) Coming back to metrics: we can't look to a federally-agreed-to Nutrition Label to give guidance for which provider offers the right choice for your mix of gadgets. The most important advice I can imagine is "get a router that manages your latency", and your problems will go away, or at least be *much* better.

Rich

Re: [Bloat] On metrics

[Sebastian Moeller]

So over here in Germany we already have something similar.

ISPs are required by law and regulation to supply potential customers with the following standardized information before closing a contract:

example "Product information sheet"
h++ps://www.bundesnetzagentur.de/SharedDocs/Downloads/DE/Sachgebiete/Telekommunikation/Unternehmen_Institutionen/Anbieterpflichten/Kundenschutz/Transparenzmaßnahmen/templates_for_information_sheets.pdf;jsessionid=475EBAEFFD5ADDD7D3E30FDBF9CA29C2?__blob=publicationFile&v=1

instructions how to create a "Product information sheet"
h++ps://www.bundesnetzagentur.de/SharedDocs/Downloads/DE/Sachgebiete/Telekommunikation/Unternehmen_Institutionen/Anbieterpflichten/Kundenschutz/Transparenzmaßnahmen/Instruction_for_drawing_up_PIS.pdf;jsessionid=475EBAEFFD5ADDD7D3E30FDBF9CA29C2?__blob=publicationFile&v=1

The national regulatory agency also established a (somewhat complicated and time-intensive) process to check whether ISPs actually deliver the contractually promised throughput (and they still ignore latency which clearly should be improved).

Customers who show that ISPs doe not deliver can opt either:
a) get an immediate right to cancel the contract
b) opt for getting the price reduced proportional to the amount of contractual fulfillment (for the duration of the existing contract, after that the ISP can opt not to offer only lower speedgrades)
c) sue the ISP in court (as before)


This will not help in conditions like Rich's, but it generally helps in getting the whole market get an understanding that contracts do matter. (ISPs are free to only promise those numbers they see fit, they are only held accountable to actually fulfill their commitments).

I would expect that for a plan-by-plan information something that gives reliable information about generally achievable capacity (and preferably latency under load) would be helpful. Especially when combined with an official web-site that step-by-step explains what kind of capacity and (worst case latency) different use-cases require.

That is have the label not try to explain everything but have it refer to a well-written web site that helps to put the numbers into perspective.

But I might be biased the the method I know and there might be better ways.

Regards
	Sebastian

P.S.: I know some here operate ISPS and hence see this from a different angle than end-customers, but the measurement procedure is pretty fair for ISPs and customers (and arduous enough that folks are unlikely to run a measurement campaign just for fun or to pester an ISP).






> On Mar 20, 2023, at 12:30, Rich Brown via Bloat <bloat@lists.bufferbloat.net> wrote:
> 
> 
> 
>> On Mar 19, 2023, at 11:03 PM, bloat-request@lists.bufferbloat.net wrote:
>> 
>>> Consumers really need things like published performance specs so they can
>>> assemble their needs like an a la carte menu.  What do you do, what’s
>>> important to you, what details support that need, and they need that in a
>>> simple way.   Like a little app that says “how many 1080p TVs or 4K TVs,
>>> how many gaming consoles, do you take zoom calls or VoIP/phone calls.  Do
>>> you send large emails, videos, or pictures.”
>> 
>> The problem is that these needs really are not that heavy. Among my ISP 
>> connections, I have a 8/1 dsl connection, even when I fail over to that, I can 
>> run my 4k tv + a couple other HD TVs + email (although it's at the ragged edge, 
>> trying to play 4k at 2x speed can hiccup, and zoom calls can stutter when large 
>> emails/downloads flow)
> 
> I want to second David Lang's comment. I live in a small rural NH town that was stuck at DSL prior to a local company raising the money to install fiber to all premises. 
> 
> Before the fiber came in, I had 7mbps/768kbps service. If I wanted to bond two circuits, I could get 15/1mbps. But many neighbors had 3mbps/768kbps - or worse - so they were basically unserved. We frequently saw people parked outside our public library after hours to get internet. (And yes, a good router improved things. I told a lot of people about the IQrouter that turned the unusable service into merely slow.)
> 
> But there is a huge swath of rural US that is in the same situation, with zero or one provider of dreadful service.
> 
> What's the value of a "nutrition label"?
> 
> a) It's meaningless for those rural customers. They have no choice beyond "take it or leave it." 
> 
> b) For the lucky ones where alternative providers compete, the proposed label does provide a standardized format that lays out purported speeds and the the pricing tiers (including overage charges). I don't think I'd ever believe the latency numbers.
> 
> c) Coming back to metrics: we can't look to a federally-agreed-to Nutrition Label to give guidance for which provider offers the right choice for your mix of gadgets. The most important advice I can imagine is "get a router that manages your latency", and your problems will go away, or at least be *much* better.
> 
> Rich
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] On metrics

[dan]

[-- Attachment #1: Type: text/plain, Size: 4607 bytes --]

On Mar 19, 2023 at 3:00:35 PM, rjmcmahon <rjmcmahon@rjmcmahon.com> wrote:

> Hi All,
>
> It seems getting the metrics right is critical. Our industry can't be
> reporting things that mislead or misassign blame. The medical community
> doesn't treat people for cancer without having a high degree they've
> gotten the diagnostics correct as an example.
>
> An initial metric, per this group, would be geared towards
> responsiveness or the speed of causality. Here, we may need to include
> linear distance, the power required to achieve a responsiveness and to
> take account of Pareto efficiencies, where one device's better
> responsiveness can't make another's worse.
>
> An example per a possible FiWi new & comprehensive metric: A rating
> could be something like 10K responses per second at 1Km terrestrial
> (fiber) cable / 6m radius free space range / 5W total / 0-impact to
> others. If consumers can learn to read nutrition labels they can also
> learn to read these.
>
> Maybe a device produces a scan code qr based upon its e3e measurement
> and the scan code qr loads a page with human interpretable analysis?
> Similar to how we now pull up menus on our mobile phones listing the
> food items and the nutrition information that's available to seat at a
> table. Then, in a perfect world, there is a rating per each link hop or
> better, network jurisdiction. Each jurisdiction could decide if they
> want to participate or not, similar to connecting up an autonomous
> system or not. I think measurements of network jurisdictions without
> prior agreements are unfair. The lack of measurement capability is
> likely enough pressure needed to motivate actions.
>
> Bob
>
> PS. As a side note, and a shameless plug, iperf 2 now supports
> bounceback and a big issue has been clock sync for one way delays (OWD.)
> Per a comment from Jean Tourrhiles
> https://sourceforge.net/p/iperf2/tickets/242/ I added some unsync
> detections in the bounceback measurements. Contact me directly if your
> engineering team needs more information on iperf 2.
>

A food nutrition label is actually a great example of bad information in
consumer hands.  Since adding those, Americans weights have ballooned.  I’m
not saying they are in direct correlation, but that information has
definitely not caused an improvement in health by any measure at all.
Definitely not a model to pursue.

There needs to be a clear distinction between what’s valuable to the
consumer and what’s valuable to the ISP to improve services. These are
dramatically different pieces of data.   For the consumer, information that
directions their choice of product is important.  Details about various
points in the process are useless to them.  How many hops has no value to
them, only the latenc, jitter, throughput, and probably some rating on slow
start or other things that are part of the ISP equation but entirely for
the purpose of making better choices for their needs.  10k responses in x
seconds is meaningless to a consumer.  I have never, and I’m not
exaggerating, EVER had a home user or IT guy or point person for an MSP
ever ask about packet rates or any of the stats that keep getting brought
up.  This is a solution looking for a problem.

Consumers really need things like published performance specs so they can
assemble their needs like an a la carte menu.  What do you do, what’s
important to you, what details support that need, and they need that in a
simple way.   Like a little app that says “how many 1080p TVs or 4K TVs,
how many gaming consoles, do you take zoom calls or VoIP/phone calls.  Do
you send large emails, videos, or pictures.”

Put another way, all the specs are like telling a soccer mom the torque
curve of their minivan.

If ‘we’ the industry make a nutrition type label that has numbers on it
that are not useful in the context of a consumer decision making process in
a direct x gives you y way, it creates data that will get misinterpreted.

These stats should be made available for the providers pushing data so that
they can make sure they are meeting the human readable and useful data.  I
care about the AS path and the latency on my upstream services to various
providers, I can try to make better choices on buying lumen or hurricane
and how I will route those things because I understand how they affect the
service.  Those are really useful numbers for me and any ISP that wants to
have higher “A+ for gaming because latency and jitter are low and bandwidth
is adequate” ratings.

[-- Attachment #2: Type: text/html, Size: 5723 bytes --]

[Bloat] On metrics

[rjmcmahon]

Hi All,

It seems getting the metrics right is critical. Our industry can't be 
reporting things that mislead or misassign blame. The medical community 
doesn't treat people for cancer without having a high degree they've 
gotten the diagnostics correct as an example.

An initial metric, per this group, would be geared towards 
responsiveness or the speed of causality. Here, we may need to include 
linear distance, the power required to achieve a responsiveness and to 
take account of Pareto efficiencies, where one device's better 
responsiveness can't make another's worse.

An example per a possible FiWi new & comprehensive metric: A rating 
could be something like 10K responses per second at 1Km terrestrial 
(fiber) cable / 6m radius free space range / 5W total / 0-impact to 
others. If consumers can learn to read nutrition labels they can also 
learn to read these.

Maybe a device produces a scan code qr based upon its e3e measurement 
and the scan code qr loads a page with human interpretable analysis? 
Similar to how we now pull up menus on our mobile phones listing the 
food items and the nutrition information that's available to seat at a 
table. Then, in a perfect world, there is a rating per each link hop or 
better, network jurisdiction. Each jurisdiction could decide if they 
want to participate or not, similar to connecting up an autonomous 
system or not. I think measurements of network jurisdictions without 
prior agreements are unfair. The lack of measurement capability is 
likely enough pressure needed to motivate actions.

Bob

PS. As a side note, and a shameless plug, iperf 2 now supports 
bounceback and a big issue has been clock sync for one way delays (OWD.) 
Per a comment from Jean Tourrhiles 
https://sourceforge.net/p/iperf2/tickets/242/ I added some unsync 
detections in the bounceback measurements. Contact me directly if your 
engineering team needs more information on iperf 2.