On top of all that, the latency responses tend to be non parametric and 
may need full pdfs/cdfs along with non-parametric statistical process 
controls. Attached is an example from many years ago which was a 
firmware bug that sometimes delayed packet processing, creating a second 
node in the pdf.

Engineers and their algorithms can be this way it seems.

Bob
> I didn't study the whole report, but I didn't notice any metrics
> associated with *variance* of latency or bandwidth.  It's common for
> vendors to play games ("Lies, damn lies, and statistics!") to make
> their metrics look good.   A metric of latency that says something
> like "99% less than N milliseconds" doesn't necessarily translate into
> an acceptable user performance.
> 
> It's also important to look at the specific techniques used for taking
> measurements.  For example, if a measurement is performed every
> fifteen minutes, extrapolating the metric as representative of all the
> time between measurements can also lead to a metric judgement which
> doesn't reflect the reality of what the user actually experiences.
> 
> In addition, there's a lot of mechanism between the ISPs' handling of
> datagrams and the end-user.   The users' experience is affected by how
> all of that mechanism interacts as underlying network behavior
> changes.  When a TCP running in some host decides it needs to
> retransmit, or an interactive audio/video session discards datagrams
> because they arrive too late to be useful, the user sees unacceptable
> performance even though the network operators may think everything is
> running fine.   Measurements from the end-users' perspective might
> indicate performance is quite different from what measurements at the
> ISP level suggest.
> 
> Gamers are especially sensitive to variance, but it will also apply to
> interactive uses such as might occur in telemedicine or remote
> operations.  A few years ago I helped a friend do some tests for a
> gaming situation and we discovered that the average latency was
> reasonably low, but occasionally, perhaps a few times per hour,
> latency would increase to 10s of seconds.
> 
> In a game, that often means the player loses.  In a remote surgery it
> may mean horrendous outcomes.  As more functionality is performed "in
> the cloud" such situations will become increasingly common.
> 
> Jack Haverty
> 
> On 2/26/24 12:02, rjmcmahon via Nnagain wrote:
> 
>> Thanks for sharing this. I'm trying to find out what are the key
>> metrics that will be used for this monitoring. I want to make sure
>> iperf 2 can cover the technical, traffic related ones that make
>> sense to a skilled network operator, including a WiFi BSS manager. I
>> didn't read all 327 pages though, from what I did read, I didn't see
>> anything obvious. I assume these types of KPIs may be in reference
>> docs or something.
>> 
>> Thanks in advance for any help on this.
>> Bob
>> 
>>> And...
>>> 
>>> Our bufferbloat.net submittal was cited multiple times! Thank you
>>> all
>>> for participating in that process!
>>> 
>>> https://docs.fcc.gov/public/attachments/DOC-400675A1.pdf
>>> 
>>> It is a long read, and does still start off on the wrong feet
>>> (IMHO),
>>> in particular not understanding the difference between idle and
>>> working latency.
>>> 
>>> It is my hope that by widening awareness of more of the real
>>> problems
>>> with latency under load to policymakers and other submitters
>>> downstream from this new FCC document, and more reading what we
>>> had to
>>> say, that we will begin to make serious progress towards finally
>>> fixing bufferbloat in the USA.
>>> 
>>> I do keep hoping that somewhere along the way in the future, the
>>> costs
>>> of IPv4 address exhaustion and the IPv6 transition, will also get
>>> raised to the national level. [1]
>>> 
>>> We are still collecting signatures for what the bufferbloat
>>> project
>>> members wrote, and have 1200 bucks in the kitty for further
>>> articles
>>> and/or publicity. Thoughts appreciated as to where we can go next
>>> with
>>> shifting the national debate about bandwidth in a better
>>> direction!
>>> Next up would be trying to get a meeting, and to do an ex-parte
>>> filing, I think, and I wish we could do a live demonstration on
>>> television about it as good as feynman did here:
>>> 
>>> https://www.youtube.com/watch?v=raMmRKGkGD4
>>> 
>>> Our original posting is here:
>>> 
>> 
> https://docs.google.com/document/d/19ADByjakzQXCj9Re_pUvrb5Qe5OK-QmhlYRLMBY4vH4/edit
>>> 
>>> 
>>> Larry's wonderful post is here:
>>> https://circleid.com/posts/20231211-its-the-latency-fcc
>>> 
>>> [1] How can we get more talking about IPv4 and IPv6, too? Will we
>>> have
>>> to wait another year?
>>> 
>>> 
>> 
> https://hackaday.com/2024/02/14/floss-weekly-episode-769-10-more-internet/
>>> 
>>> 
>>> --
>>> https://blog.cerowrt.org/post/2024_predictions/
>>> Dave Täht CSO, LibreQos
>> _______________________________________________
>> Nnagain mailing list
>> Nnagain@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/nnagain
> _______________________________________________
> Nnagain mailing list
> Nnagain@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/nnagain