<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    Yes, latency is complicated....  Back when I was involved in the
    early Internet (early 1980s), we knew that latency was an issue
    requiring much further research, but we figured that meanwhile
    problems could be avoided by keeping traffic loads well below
    capacity while the appropriate algorithms could be discovered by the
    engineers (I was one...).  Forty years later, it seems like it's
    still a research topic.<br>
    <br>
    Years later in the 90s I was involved in operating an international
    corporate intranet.  We quickly learned that keeping the human users
    happy required looking at more than the routers and circuits between
    them.  With much of the "reliability mechanisms" of TCP et al now
    located in the users' computers rather than the network switches,
    evaluating users' experience with "the net" required measurements
    from the users' perspective.   <br>
    <br>
    To do that, we created a policy whereby every LAN attached to the
    long-haul backbone had to have a computer on that LAN to which we
    had remote access.   That enabled us to perform "ping" tests and
    also collect data about TCP behavior (duplicates, retransmissions,
    etc.) using SNMP, etherwatch, et al.   It was not unusual for the
    users' data to indicate that "the net", as they saw it, was
    misbehaving while the network data, as seen by the operators,
    indicated that all the routers and circuits were working just fine.<br>
    <br>
    If the government regulators want to keep the users happy, IMHO they
    need to understand this.<br>
    <br>
    Jack Haverty<br>
    <br>
    <br>
    <div class="moz-cite-prefix">On 2/26/24 16:25, rjmcmahon wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:09af7b4f2ff73657a1d1b24db3d23fac@rjmcmahon.com">On top
      of all that, the latency responses tend to be non parametric and
      may need full pdfs/cdfs along with non-parametric statistical
      process controls. Attached is an example from many years ago which
      was a firmware bug that sometimes delayed packet processing,
      creating a second node in the pdf.
      <br>
      <br>
      Engineers and their algorithms can be this way it seems.
      <br>
      <br>
      Bob
      <br>
      <blockquote type="cite">I didn't study the whole report, but I
        didn't notice any metrics
        <br>
        associated with *variance* of latency or bandwidth.  It's common
        for
        <br>
        vendors to play games ("Lies, damn lies, and statistics!") to
        make
        <br>
        their metrics look good.   A metric of latency that says
        something
        <br>
        like "99% less than N milliseconds" doesn't necessarily
        translate into
        <br>
        an acceptable user performance.
        <br>
        <br>
        It's also important to look at the specific techniques used for
        taking
        <br>
        measurements.  For example, if a measurement is performed every
        <br>
        fifteen minutes, extrapolating the metric as representative of
        all the
        <br>
        time between measurements can also lead to a metric judgement
        which
        <br>
        doesn't reflect the reality of what the user actually
        experiences.
        <br>
        <br>
        In addition, there's a lot of mechanism between the ISPs'
        handling of
        <br>
        datagrams and the end-user.   The users' experience is affected
        by how
        <br>
        all of that mechanism interacts as underlying network behavior
        <br>
        changes.  When a TCP running in some host decides it needs to
        <br>
        retransmit, or an interactive audio/video session discards
        datagrams
        <br>
        because they arrive too late to be useful, the user sees
        unacceptable
        <br>
        performance even though the network operators may think
        everything is
        <br>
        running fine.   Measurements from the end-users' perspective
        might
        <br>
        indicate performance is quite different from what measurements
        at the
        <br>
        ISP level suggest.
        <br>
        <br>
        Gamers are especially sensitive to variance, but it will also
        apply to
        <br>
        interactive uses such as might occur in telemedicine or remote
        <br>
        operations.  A few years ago I helped a friend do some tests for
        a
        <br>
        gaming situation and we discovered that the average latency was
        <br>
        reasonably low, but occasionally, perhaps a few times per hour,
        <br>
        latency would increase to 10s of seconds.
        <br>
        <br>
        In a game, that often means the player loses.  In a remote
        surgery it
        <br>
        may mean horrendous outcomes.  As more functionality is
        performed "in
        <br>
        the cloud" such situations will become increasingly common.
        <br>
        <br>
        Jack Haverty
        <br>
        <br>
        On 2/26/24 12:02, rjmcmahon via Nnagain wrote:
        <br>
        <br>
        <blockquote type="cite">Thanks for sharing this. I'm trying to
          find out what are the key
          <br>
          metrics that will be used for this monitoring. I want to make
          sure
          <br>
          iperf 2 can cover the technical, traffic related ones that
          make
          <br>
          sense to a skilled network operator, including a WiFi BSS
          manager. I
          <br>
          didn't read all 327 pages though, from what I did read, I
          didn't see
          <br>
          anything obvious. I assume these types of KPIs may be in
          reference
          <br>
          docs or something.
          <br>
          <br>
          Thanks in advance for any help on this.
          <br>
          Bob
          <br>
          <br>
          <blockquote type="cite">And...
            <br>
            <br>
            Our bufferbloat.net submittal was cited multiple times!
            Thank you
            <br>
            all
            <br>
            for participating in that process!
            <br>
            <br>
            <a class="moz-txt-link-freetext" href="https://docs.fcc.gov/public/attachments/DOC-400675A1.pdf">https://docs.fcc.gov/public/attachments/DOC-400675A1.pdf</a>
            <br>
            <br>
            It is a long read, and does still start off on the wrong
            feet
            <br>
            (IMHO),
            <br>
            in particular not understanding the difference between idle
            and
            <br>
            working latency.
            <br>
            <br>
            It is my hope that by widening awareness of more of the real
            <br>
            problems
            <br>
            with latency under load to policymakers and other submitters
            <br>
            downstream from this new FCC document, and more reading what
            we
            <br>
            had to
            <br>
            say, that we will begin to make serious progress towards
            finally
            <br>
            fixing bufferbloat in the USA.
            <br>
            <br>
            I do keep hoping that somewhere along the way in the future,
            the
            <br>
            costs
            <br>
            of IPv4 address exhaustion and the IPv6 transition, will
            also get
            <br>
            raised to the national level. [1]
            <br>
            <br>
            We are still collecting signatures for what the bufferbloat
            <br>
            project
            <br>
            members wrote, and have 1200 bucks in the kitty for further
            <br>
            articles
            <br>
            and/or publicity. Thoughts appreciated as to where we can go
            next
            <br>
            with
            <br>
            shifting the national debate about bandwidth in a better
            <br>
            direction!
            <br>
            Next up would be trying to get a meeting, and to do an
            ex-parte
            <br>
            filing, I think, and I wish we could do a live demonstration
            on
            <br>
            television about it as good as feynman did here:
            <br>
            <br>
            <a class="moz-txt-link-freetext" href="https://www.youtube.com/watch?v=raMmRKGkGD4">https://www.youtube.com/watch?v=raMmRKGkGD4</a>
            <br>
            <br>
            Our original posting is here:
            <br>
            <br>
          </blockquote>
          <br>
        </blockquote>
<a class="moz-txt-link-freetext" href="https://docs.google.com/document/d/19ADByjakzQXCj9Re_pUvrb5Qe5OK-QmhlYRLMBY4vH4/edit">https://docs.google.com/document/d/19ADByjakzQXCj9Re_pUvrb5Qe5OK-QmhlYRLMBY4vH4/edit</a>
        <br>
        <blockquote type="cite">
          <blockquote type="cite">
            <br>
            <br>
            Larry's wonderful post is here:
            <br>
            <a class="moz-txt-link-freetext" href="https://circleid.com/posts/20231211-its-the-latency-fcc">https://circleid.com/posts/20231211-its-the-latency-fcc</a>
            <br>
            <br>
            [1] How can we get more talking about IPv4 and IPv6, too?
            Will we
            <br>
            have
            <br>
            to wait another year?
            <br>
            <br>
            <br>
          </blockquote>
          <br>
        </blockquote>
<a class="moz-txt-link-freetext" href="https://hackaday.com/2024/02/14/floss-weekly-episode-769-10-more-internet/">https://hackaday.com/2024/02/14/floss-weekly-episode-769-10-more-internet/</a>
        <br>
        <blockquote type="cite">
          <blockquote type="cite">
            <br>
            <br>
            --
            <br>
            <a class="moz-txt-link-freetext" href="https://blog.cerowrt.org/post/2024_predictions/">https://blog.cerowrt.org/post/2024_predictions/</a>
            <br>
            Dave Täht CSO, LibreQos
            <br>
          </blockquote>
          _______________________________________________
          <br>
          Nnagain mailing list
          <br>
          <a class="moz-txt-link-abbreviated" href="mailto:Nnagain@lists.bufferbloat.net">Nnagain@lists.bufferbloat.net</a>
          <br>
          <a class="moz-txt-link-freetext" href="https://lists.bufferbloat.net/listinfo/nnagain">https://lists.bufferbloat.net/listinfo/nnagain</a>
          <br>
        </blockquote>
        _______________________________________________
        <br>
        Nnagain mailing list
        <br>
        <a class="moz-txt-link-abbreviated" href="mailto:Nnagain@lists.bufferbloat.net">Nnagain@lists.bufferbloat.net</a>
        <br>
        <a class="moz-txt-link-freetext" href="https://lists.bufferbloat.net/listinfo/nnagain">https://lists.bufferbloat.net/listinfo/nnagain</a>
        <br>
      </blockquote>
    </blockquote>
    <br>
  </body>
</html>