<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
During the 90s, I was part of a small team responsible for operating
a corporate small-i internet. We used cisco equipment, in offices
in 100 or so countries, all interconnected with circuits from
various "phone companies". No fiber or wifi around yet. Our
internet was not connected to The Internet, but used all the same
equipment and software.<br>
<br>
To figure out how to respond to user complaints (e.g., "The network
is really slow today!"), we discovered that we needed good
instrumentation not only in the various routers but also in the
computers in the user environments. TCP, implemented in the user
computers, did a good job at hiding problems. For example, TCPs
could get into a semi-stable state where every datagram was getting
transmitted twice with the second copy being discarded at the
destination. That caused "network throughput" (datagrams/second) to
go way up, but user performance to go way down.<br>
<br>
To get the data, we used SNMP and simple scripts to probe the
various devices from our NOC, stuff all the data into a database,
and use the available data analysis tools to figure out what was
happening. At the time, SNMP "MIB"s were defined for both routers
and computers. So we could collect data from the TCPs involved in
a problem, as well as data from router operations. When routers
reported N datagrams/time, the associated TCPs might report N/2
retransmissions from the sender, and N/2 discards from the receiver.<br>
<br>
With today's ability to synchronize clocks across the network, it
should now also be possible to report data about latency.
Interactive operations - (A/V conferencing, gaming, telepresence,
etc.) probably have lots of metrics that could be useful much as we
found TCP's to be.<br>
<br>
I'm way out of touch with network operations now. Do current
network managers look at metrics from the enduser computers'
perspective? Can latency be measured between two user devices
involved in a problem report?<br>
<br>
Jack<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 11/14/23 13:45, rjmcmahon via
Nnagain wrote:<br>
</div>
<blockquote type="cite"
cite="mid:fd64c09e8ad39660d44216a9eedd0862@rjmcmahon.com">I think
the missing metrics & test vectors are around latency more
than bandwidth.
<br>
<br>
I've attached a WiFi low latency table. Feel free to comment.
<br>
<br>
Good metrics could allow for a comprehensive analysis, at least
from a WiFi perspective.
<br>
<br>
Latency under load is a good start, but likely not enough.
<br>
<br>
Bob
<br>
<br>
PS. Agreed, the digital transition with storage has many engineers
who have contributed over decades. I'm very grateful to all of
them.
<br>
<br>
<blockquote type="cite">I am glad you are reaching out, but it may
be difficult for us to do a
<br>
joint filing.
<br>
<br>
In particular I question the seeming assumption that more wifi
devices
<br>
will drive demand for more bandwidth,
<br>
and extrapolating from 18 devices forward may also well be a
trend
<br>
that will reverse completely in favor of more bluetooth and
thread
<br>
implementations from phone to device.
<br>
<br>
Of those 20 wifi devices today are probably
<br>
<br>
1 or more laptops
<br>
1 or more tablets
<br>
1 or more phones
<br>
1 or more tvs
<br>
<br>
and of those usually only one will be active per person, while
they
<br>
are in the home, and even then....as one semi-hard number, even
at
<br>
peak hours (with the libreqos data I have), only 1/6th of
households
<br>
are watching video, and very, very few, more than one stream at
the
<br>
same time.
<br>
<br>
The steady upload bandwidth pumpers are primarily video
surveillance
<br>
devices (which as a personal preference I would prefer remain in
the
<br>
home unless otherwise activated). I do not presently know much
about
<br>
the frame rates or real bandwidth requirements of popular
devices like
<br>
ring, etc. Similarly I am biased towards "Babycams" sending
video
<br>
from up to downstairs only and not into the cloud. I know I am
bucking
<br>
the trend on this, but it will make me skeptical of much "data"
that
<br>
exists today on it.
<br>
<br>
Then you have loads of extremely low bandwidth devices - alexa
and
<br>
other automation is measured in bits/ms, light switches, a
couple bits
<br>
a day, audio streaming 128kbit/s (when you use it). Automatic
updates
<br>
to phones and tablets, etc, take place entirely asynchronously
<br>
nowadays and do not need much bandwidth. A small business just
needs
<br>
to
<br>
*reliably* clear credit card transactions every few minutes.
<br>
<br>
Perhaps the biggest steady-state bandwidth suck is home gaming
<br>
updates, but while a big market, if you haven't noticed
birthrates are
<br>
down, and immigration being canceled.
<br>
<br>
Thus I feel that the opposite number of 70-80% two people or
less per
<br>
household that you are not optimizing for, dominates the
curves.
<br>
<br>
Looking at the actual useage disparity (delta) between fiber'd
cities
<br>
and rural, uptake of passive video streaming services vs
spotify,
<br>
would give me a more pessimistic projection than most.
Regrettably I
<br>
lack the time and as few fund accurate scenarios, I would merely
be
<br>
willing to write down my estimate and find some sort of online
<br>
"futures" market to place puts on.
<br>
<br>
Lastly, a goodly percentage of the people I know just need food,
<br>
shelter, a job and a phone, and with broadband costs
skyrocketing,
<br>
aside from the gaming market and business, that is all they can
<br>
afford, even with ACP. And all that they need. Nobody has a
landline
<br>
anymore, and if it weren't for "Tv", few would want broadband at
even
<br>
25/10.
<br>
<br>
<br>
On Tue, Nov 14, 2023 at 2:40 PM rjmcmahon via Nnagain
<br>
<a class="moz-txt-link-rfc2396E" href="mailto:nnagain@lists.bufferbloat.net"><nnagain@lists.bufferbloat.net></a> wrote:
<br>
<blockquote type="cite">
<br>
Thanks for sharing this. I agree this works for researchers.
<br>
<br>
I think we're at a different state and economic returns matter
too.
<br>
<br>
I sent the following to our engineers in hopes we can all
better
<br>
understand what we're all trying to accomplish.
<br>
<br>
Hi All,
<br>
<br>
The attached Notice of Inquiry by the FCC shows how much our
work
<br>
matters to most everyone in our country (and, by inference,
worldwide.)
<br>
Broadband networks are no longer entertainment or social
networks but
<br>
they are critical to all regardless of gender, age, race,
ethnic group,
<br>
etc. People's health, learning, and ability to earn for their
families
<br>
all depend on us providing world class engineering to our
customers who
<br>
in turn provide these networks for each and all of us, our
friends &
<br>
families, our neighbors, and most everyone else.
<br>
<br>
Early in my career, I worked at Cisco and had the privilege to
work on
<br>
some of the first BGP routers that enabled the commercial
build out of
<br>
the internet, and I'm very thankful we did that way ahead of
the 2019
<br>
pandemic. There was no "pandemic use case" that drove us - we
just
<br>
wanted to build the best products that engineers could build.
A
<br>
worldwide pandemic w/o the internet could have been disastrous
- so that
<br>
work by many in the mid 1990s seems to have paid off well.
<br>
<br>
I hope you each realize, today, what you've accomplished since
then and
<br>
continue to be a part of. It's truly significant. It's been a
high honor
<br>
to work with so many of you over the last 14+ years.
<br>
</blockquote>
<br>
This is beautiful, btw. I feel much the same way about linux
being now
<br>
so used heavily in the space program,
<br>
and all our code, and hardware, that will propagate across the
solar
<br>
system, and of the millions of people, that contributed to it.
<br>
<br>
<blockquote type="cite">To the FCC report:
<br>
<br>
We begin this annual inquiry in the wake of the COVID-19
pandemic during
<br>
which time Americans increasingly turned to their broadband
connections
<br>
to conduct their lives online by using telemedicine to access
<br>
healthcare, working from home, attending classes remotely,
connecting by
<br>
video with out-of-town family and friends, and streaming
entertainment.
<br>
Our experiences with the pandemic made it clear that broadband
is no
<br>
longer a luxury but a necessity that will only become more
important
<br>
with time. Never before has the critical importance of
ensuring that all
<br>
Americans have access to high-speed, affordable broadband been
more
<br>
evident.
<br>
<br>
Also note, we have more work to do. We need to increase
resiliency as an
<br>
example. Also, the thing I'm most passionate about is low
latency. The
<br>
FCC is now recognizing the importance of that. People are
slowly
<br>
learning why latency is becoming equally important to capacity
when it
<br>
comes to quality of service.
<br>
<br>
Bob
<br>
<br>
PS. The rest is TLDR but I thought I post some snippets for
those
<br>
interested
<br>
<br>
We believe that in examining household use cases, a simple
summation of
<br>
required speeds for individual activities may provide a
misleading
<br>
picture of actual broadband needs for at least three reasons.
First, we
<br>
believe it is appropriate to take into account at least
occasional
<br>
downloads of very large files which can be
bandwidth-intensive. Second,
<br>
it is important to account for larger households; in 2022,
approximately
<br>
21% of all U.S. households had four or more people, and the
number of
<br>
families seeking out multigenerational homes to live with
additional
<br>
relatives rose.57 Households of all sizes must have sufficient
bandwidth
<br>
to satisfy their needs. In addition, the number of connected
devices per
<br>
household continues to grow, from 18.6 in the average
household in 2021
<br>
to 20.2 in the first half of 2022.58 Taking these factors into
account
<br>
suggests that fixed broadband download/upload needs could
easily exceed
<br>
100/20 Mbps.
<br>
<br>
...
<br>
<br>
Service Quality. We recognize that other factors, besides the
speed of a
<br>
broadband connection, can affect consumers’ ability to use the
services
<br>
effectively. Chief among these factors is latency, which is
the measure
<br>
of the time it takes a packet of data to travel from one point
in the
<br>
network to another, and which is typically measured by
round-trip time
<br>
in milliseconds (ms). As a measurement of advanced
telecommunications
<br>
capability, latency can be critical because it affects a
consumer’s
<br>
ability to use real-time applications, including voice over
Internet
<br>
Protocol, video calling, distance learningapplications, and
online
<br>
gaming. Actual (as opposed to advertised) speed received,
consistency of
<br>
speed, and data allowances are also important. Such factors
are not
<br>
simply a matter of service interruptions and consumer
satisfaction—they
<br>
have a real and significant effect on Americans’ ability to
use critical
<br>
web-based applications, including those that facilitate
telehealth,
<br>
telework, and virtual learning.
<br>
<br>
<br>
<br>
> In the beginning days of the Arpanet, circa early 1970s,
ARPA made a
<br>
> policy decision about use of the Arpanet. First, Arpa
Program
<br>
> Managers, located on the East Coast of the US, were
assigned computer
<br>
> accounts on USC-ISIA, located on the West Coast in LA.
Thus to do
<br>
> their work, exchanging email, editting documents, and
such, they had
<br>
> to *use* the Arpanet to connect their terminals in
Washington to the
<br>
> PDP-10 in California - 3000 miles away.
<br>
>
<br>
> Second, ARPA began requiring all of their contractors
(researchers at
<br>
> Universities etc.) to interact with Arpa using email and
FTP. If
<br>
> your site was "on the Arpanet", you had to use the
Arpanet. If you
<br>
> wanted your proposal for next year's research to be
funded, you had to
<br>
> submit your proposal using the net.
<br>
>
<br>
> This policy caused a profound attention, by everyone
involved, to
<br>
> making the Arpanet work and be useful as a collaboration
tool.
<br>
>
<br>
> JCR Licklider (aka Lick) was my advisor at MIT, and then
my boss when
<br>
> I joined the Research Staff. Lick had been at ARPA for
a while,
<br>
> promoting his vision of a "Galactic Network" that
resulted in the
<br>
> Arpanet as a first step. At MIT, Lick still had need for
lots of
<br>
> interactions with others. My assignment was to build
and operate the
<br>
> email system for Lick's group at MIT on our own PDP-10.
Lick had a
<br>
> terminal in his office and was online a lot. If email
didn't work, I
<br>
> heard about it. If the Arpanet didn't work, BBN heard
about it.
<br>
>
<br>
> This pressure was part of Arpa policy. Sometimes it's
referred to as
<br>
> "eating your own dog food" -- i.e., making sure your
"dog" will get
<br>
> the same kind of nutrition you enjoy. IMHO, that
pressure policy was
<br>
> important, perhaps crucial, to the success of the
Arpanet.
<br>
>
<br>
> In the 70s, meetings still occurred, but a lot of
progress was made
<br>
> through the use of the Arpanet. You can only do so much
with email
<br>
> and file interactions. Today, the possibilities for far
richer
<br>
> interactions are much more prevalent. But IMHO they are
held back,
<br>
> possibly because no one is feeling the pressure to "make
it work".
<br>
> Gigabit throughputs are common, but why does my video and
audio still
<br>
> break up...?
<br>
>
<br>
> It's important to have face-to-face meetings, but perhaps
if the IETF
<br>
> scheduled a future meeting to be online only, whatever
needs to happen
<br>
> to make it work would happen? Perhaps...
<br>
>
<br>
> Even a "game" might drive progress. At Interop '92, we
resurrected
<br>
> the old "MazeWars" game using computers scattered across
the show
<br>
> exhibit halls. The engineers in the control room above
the floor felt
<br>
> the pressure to make sure the Game continued to run. At
the time, the
<br>
> Internet itself was too slow for enjoyable gameplay at
any distance.
<br>
> Will the Internet 30 years later work?
<br>
>
<br>
> Or perhaps the IETF, or ISOC, or someone could take on a
highly
<br>
> visible demo involving non-techie end users. An online
meeting of
<br>
> the UN General Assembly? Or some government bodies - US
Congress,
<br>
> British Parliament, etc.
<br>
>
<br>
> Such an event would surface the issues, both technical
and policy, to
<br>
> the engineers, corporations, policy-makers, and others
who might have
<br>
> the ability and interest to "make it work".
<br>
>
<br>
> Jack
<br>
>
<br>
> On 11/14/23 10:10, Sebastian Moeller wrote:
<br>
>
<br>
>> Hi Jack,
<br>
>>
<br>
>>> On Nov 14, 2023, at 13:02, Jack Haverty via
Nnagain
<br>
>>> <a class="moz-txt-link-rfc2396E" href="mailto:nnagain@lists.bufferbloat.net"><nnagain@lists.bufferbloat.net></a> wrote:
<br>
>>>
<br>
>>> If video conferencing worked well enough, they
would not have to
<br>
>>> all get together in one place and would instead
hold IETF meetings
<br>
>>> online ...?
<br>
>>
<br>
>> [SM] Turns out that humans are social creatures, and
some things
<br>
>> work better face-to-face and in the hallway (and if
that is only
<br>
>> building trust and sympathy) than over any remote
technology.
<br>
>>
<br>
>>> Did anyone measure latency? Does anyone measure
throughput of
<br>
>>> "useful" traffic - e.g., excluding video/audio
data that didn't
<br>
>>> arrive in time to be actually used on the screen
or speaker?
<br>
>>
<br>
>> [SM] Utility is in the eye of the beholder, no?
<br>
>>
<br>
>> Jack Haverty
<br>
>>
<br>
>> On 11/14/23 09:25, Vint Cerf via Nnagain wrote:
<br>
>>
<br>
>> if they had not been all together they would have
been consuming
<br>
>> tons of video capacity doing video conference
calls....
<br>
>>
<br>
>> :-))
<br>
>> v
<br>
>>
<br>
>> On Tue, Nov 14, 2023 at 10:46 AM Livingood, Jason via
Nnagain
<br>
>> <a class="moz-txt-link-rfc2396E" href="mailto:nnagain@lists.bufferbloat.net"><nnagain@lists.bufferbloat.net></a> wrote:
<br>
>> On the subject of how much bandwidth does one
household need, here's
<br>
>> a fun stat for you.
<br>
>>
<br>
>> At the IETF’s 118th meeting last week (Nov 4 – 10,
2023), there
<br>
>> were over 1,000 engineers in attendance. At peak
there were 870
<br>
>> devices connected to the WiFi network. Peak bandwidth
usage:
<br>
>>
<br>
>> • Downstream peak ~750 Mbps
<br>
>> • Upstream ~250 Mbps
<br>
>>
<br>
>> From my pre-meeting Twitter poll
<br>
>>
(<a class="moz-txt-link-freetext" href="https://twitter.com/jlivingood/status/1720060429311901873">https://twitter.com/jlivingood/status/1720060429311901873</a>):
<br>
>>
<br>
>> <image001.png>
<br>
>>
<br>
>> _______________________________________________
<br>
>> Nnagain mailing list
<br>
>> <a class="moz-txt-link-abbreviated" href="mailto:Nnagain@lists.bufferbloat.net">Nnagain@lists.bufferbloat.net</a>
<br>
>> <a class="moz-txt-link-freetext" href="https://lists.bufferbloat.net/listinfo/nnagain">https://lists.bufferbloat.net/listinfo/nnagain</a>
<br>
>>
<br>
>> --
<br>
>> Please send any postal/overnight deliveries to:
<br>
>> Vint Cerf
<br>
>> Google, LLC
<br>
>> 1900 Reston Metro Plaza, 16th Floor
<br>
>> Reston, VA 20190
<br>
>> +1 (571) 213 1346
<br>
>>
<br>
>> until further notice
<br>
>>
<br>
>> _______________________________________________
<br>
>> Nnagain mailing list
<br>
>>
<br>
>> <a class="moz-txt-link-abbreviated" href="mailto:Nnagain@lists.bufferbloat.net">Nnagain@lists.bufferbloat.net</a>
<br>
>> <a class="moz-txt-link-freetext" href="https://lists.bufferbloat.net/listinfo/nnagain">https://lists.bufferbloat.net/listinfo/nnagain</a>
<br>
>>
<br>
>> _______________________________________________
<br>
>> Nnagain mailing list
<br>
>> <a class="moz-txt-link-abbreviated" href="mailto:Nnagain@lists.bufferbloat.net">Nnagain@lists.bufferbloat.net</a>
<br>
>> <a class="moz-txt-link-freetext" href="https://lists.bufferbloat.net/listinfo/nnagain">https://lists.bufferbloat.net/listinfo/nnagain</a>
<br>
> _______________________________________________
<br>
> Nnagain mailing list
<br>
> <a class="moz-txt-link-abbreviated" href="mailto:Nnagain@lists.bufferbloat.net">Nnagain@lists.bufferbloat.net</a>
<br>
> <a class="moz-txt-link-freetext" href="https://lists.bufferbloat.net/listinfo/nnagain">https://lists.bufferbloat.net/listinfo/nnagain</a>
<br>
_______________________________________________
<br>
Nnagain mailing list
<br>
<a class="moz-txt-link-abbreviated" href="mailto:Nnagain@lists.bufferbloat.net">Nnagain@lists.bufferbloat.net</a>
<br>
<a class="moz-txt-link-freetext" href="https://lists.bufferbloat.net/listinfo/nnagain">https://lists.bufferbloat.net/listinfo/nnagain</a>
<br>
</blockquote>
</blockquote>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Nnagain mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Nnagain@lists.bufferbloat.net">Nnagain@lists.bufferbloat.net</a>
<a class="moz-txt-link-freetext" href="https://lists.bufferbloat.net/listinfo/nnagain">https://lists.bufferbloat.net/listinfo/nnagain</a>
</pre>
</blockquote>
<br>
</body>
</html>