<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
When I introduced the VGV acronym, I used Gaming as the G, but that
was probably misleading. There are many "use cases" other than
gaming. A better term might be Interactive, and a VIV acronym. <br>
<br>
Personally, I've been experimenting with "home automation", and have
found the cloud-based schemes to be intolerable. When you flip a
light switch, you expect the light to come on almost instantly, and
cloud-based mechanisms are disturbingly inconsistent, possibly due
to variability in latency. Lights turning on 20 seconds after you
flip the switch is unacceptable. <br>
<br>
There are other such "serious" Interactive use cases not related to
Gaming. Consider, for example, a "driverless" vehicle which relies
on the ability for a human at some remote control center to "take
over" control of a vehicle when necessary. Or a power plant, or
medical device, or ...<br>
<br>
IMHO, "Use Cases" are the foundation to setting good policy and
regulation. What should Users expect to be able to do when using
the Internet? <br>
<br>
Back in the early days, the design of Internet mechanisms was driven
by several "use cases". Since the project was funded by the
military, the use cases were also military. But humans need
similar kinds of communications, whether they are exchanging
commands to the troops or tweets to the forums.<br>
<br>
One such "use case" motivated the concern for latency. The scenario
was simple. An army is engaged in some action, and an online
conference is being held amongst all the players, from the generals
in HQs to the field commanders in tents or even jeeps or airplanes.
The conference supports multimedia interactions, using something
like a "shared whiteboard" that everyone can see (video was just a
dream in 1980) and change using a mouse or some kind of pointer
device.<br>
<br>
In such a scenario, the participants might be viewing a map, and
exchanging information while pointing on the map. "The enemy HQ is
here." "Our patrol is here." "Send the battalion here." "We'll
order artillery to strike here." It's very important that the
pointing is synchronized with the speech, and that the graphics and
speech are delivered intact.<br>
<br>
Those scenarios motivated the inclusion of Internet mechanisms such
as SQ, TTL, TOS et al.<br>
<tt> </tt><br>
Such scenarios are used today, quite frequently, in normal everyday
life by humans in all sorts of activities. They're no longer just
military situations.<br>
<br>
Considering "freedom of speech", what are the Use Cases that the
Internet must support? Setting aside the question of what can be
said, are there other aspects of freedom of speech? <br>
<br>
One example - assuming people should have the right to speak
anonymously, should they also have the right to speak
non-anonymously? Should someone have the right to know that speech
attributed to some person actually was spoken by that person?<br>
<br>
Today, regulators are dealing with this question in other venues,
e.g., by adding, and enforcing, the notion of "verified" telephone
numbers, in response to the tsunami of robocalls, phishing, and such
activities. Should the Internet also provide a way to verify the
sources of emails, blog posts, etc.?<br>
<br>
I've "digitally signed" this message, using the technology which I
found in my email program. Maybe that will enable you to believe
that I actually wrote this message. But I rarely receive any
"signed" email, except occasionally from members of the Technorati.
Perhaps the requirement to be anonymous is just part of the Internet
now? Or is that a Use Case that simply hasn't been developed,
propagated, and enforced yet?<br>
<br>
What are the "Use Cases" that the Internet must support, and that
regulators should enforce?<br>
<br>
Jack Haverty<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 10/17/23 10:26, Spencer Sevilla via
Nnagain wrote:<br>
</div>
<blockquote type="cite"
cite="mid:BFCFC49C-24D2-4243-9602-596A7599DB64@gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
I know this is a small side note but I felt compelled to speak up
in defense of online gaming. I’m not a gamer at all and up till a
year or two ago, would have agreed with Dick’s take about benefit
to “society as a whole.” However, lately I’ve started hearing some
research on the benefits of groups of friends using online games
to socialize together, effectively using the game primarily as a
group call.
<div><br>
</div>
<div>There’s also this project, where people have collected
banned/censored books into a library in Minecraft. Specifically
as a solution to contexts where regulators/censors ban and
monitor content through other channels (websites etc) but don’t
surveil Minecraft... Presumably because they share Dick’s
opinion ;-) <a href="https://www.uncensoredlibrary.com/en"
moz-do-not-send="true" class="moz-txt-link-freetext">https://www.uncensoredlibrary.com/en</a>
<div>
<div><br>
<blockquote type="cite">
<div>On Oct 17, 2023, at 03:26, Sebastian Moeller via
Nnagain <a class="moz-txt-link-rfc2396E" href="mailto:nnagain@lists.bufferbloat.net"><nnagain@lists.bufferbloat.net></a> wrote:</div>
<br class="Apple-interchange-newline">
<div>
<meta charset="UTF-8">
<span
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Hi
Richard,</span><br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<blockquote type="cite"
style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">On
Oct 16, 2023, at 20:04, Dick Roy
<a class="moz-txt-link-rfc2396E" href="mailto:dickroy@alum.mit.edu"><dickroy@alum.mit.edu></a> wrote:<br>
<br>
Good points all, Sebastien. How to "trade-off" a
fixed capacity amongst many users is ultimately a game
theoretic problem when users are allowed to make
choices, which is certainly the case here. Secondly,
any network that can and does generate "more traffic"
(aka overhead such as ACKs NACKs and retries) reduces
the capacity of the network, and ultimately can lead
to the "user" capacity going to zero! Such is life in
the fast lane (aka the internet).<br>
<br>
Lastly, on the issue of low-latency real-time
experience, there are many applications that need/want
such capabilities that actually have a net benefit to
the individuals involved AND to society as a whole.
IMO, interactive gaming is NOT one of those.<br>
</blockquote>
<br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<span class="Apple-tab-span"
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: pre; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"> </span><span
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">[SM]
Yes, gaming is one obvious example of a class of uses
that work best with low latency and low jitter, not
necessarily an example for a use-case worthy enough to
justify the work required to increase the
responsiveness of the internet. Other examples are
video conferences, VoIP, in extension of both musical
collaboration over the internet, and surprising to
some even plain old web browsing (it often needs to
first read a page before being able to follow links
and load resources, and every read takes at best a
single RTT). None of these are inherently beneficial
or detrimental to individuals or society, but most can
be used to improve the status quo... I would argue
that in the last 4 years the relevance of interactive
use-cases has been made quite clear to a lot of
folks...</span><br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<blockquote type="cite"
style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">OK,
so now you know I don't engage in these time sinks
with no redeeming social value.:)<br>
</blockquote>
<br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<span class="Apple-tab-span"
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: pre; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"> </span><span
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">[SM]
Duly noted ;)</span><br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<blockquote type="cite"
style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">Since
it is not hard to argue that just like power
distribution, information exchange/dissemination is
"in the public interest", the question becomes "Do we
allow any and all forms of information
exchange/dissemination over what is becoming something
akin to a public utility?" FWIW, I don't know the
answer to this question! :)<br>
</blockquote>
<br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<span class="Apple-tab-span"
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: pre; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"> </span><span
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">[SM]
This is an interesting question and one (only)
tangentially related to network neutrality... it is
more related to freedom of speech and limits thereof.
Maybe a question for another mailing list? Certainly
one meriting a topic change...</span><br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<span
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Regards</span><br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<span class="Apple-tab-span"
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: pre; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"> </span><span
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Sebastian</span><br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<blockquote type="cite"
style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br>
Cheers,<br>
<br>
RR<br>
<br>
-----Original Message-----<br>
From: Sebastian Moeller [<a class="moz-txt-link-freetext" href="mailto:moeller0@gmx.de">mailto:moeller0@gmx.de</a>]<span
class="Apple-converted-space"> </span><br>
Sent: Monday, October 16, 2023 10:36 AM<br>
To: <a class="moz-txt-link-abbreviated" href="mailto:dickroy@alum.mit.edu">dickroy@alum.mit.edu</a>; Network Neutrality is back!
Let´s make the technical aspects heard this time!<br>
Subject: Re: [NNagain] transit and peering costs
projections<br>
<br>
Hi Richard,<br>
<br>
<br>
<blockquote type="cite">On Oct 16, 2023, at 19:01,
Dick Roy via Nnagain
<a class="moz-txt-link-rfc2396E" href="mailto:nnagain@lists.bufferbloat.net"><nnagain@lists.bufferbloat.net></a> wrote:<br>
<br>
Just an observation: ANY type of congestion control
that changes application behavior in response to
congestion, or predicted congestion (ENC), begs the
question "How does throttling of application
information exchange rate (aka behavior) affect the
user experience and will the user tolerate it?"<span
class="Apple-converted-space"> </span><br>
</blockquote>
<br>
<span class="Apple-tab-span" style="white-space: pre;"> </span>[SM]
The trade-off here is, if the application does not
respond (or rather if no application would respond) we
would end up with congestion collapse where no
application would gain much of anything as the network
busies itself trying to re-transmit dropped packets
without making much head way... Simplistic game theory
application might imply that individual applications
could try to game this, and generally that seems to be
true, but we have remedies for that available..<br>
<br>
<br>
<blockquote type="cite"><br>
Given any (complex and packet-switched) network
topology of interconnected nodes and links, each
with possible a different capacity and
characteristics, such as the internet today, IMO the
two fundamental questions are:<br>
<br>
1) How can a given network be operated/configured so
as to maximize aggregate throughput (i.e. achieve
its theoretical capacity), and<br>
2) What things in the network need to change to
increase the throughput (aka parameters in the
network with the largest Lagrange multipliers
associated with them)?<br>
</blockquote>
<br>
<span class="Apple-tab-span" style="white-space: pre;"> </span>[SM]
The thing is we generally know how to maximize
(average) throughput, just add (over-)generous amounts
of buffering, the problem is that this screws up the
other important quality axis, latency... We ideally
want low latency and even more low latency variance
(aka jitter) AND high throughput... Turns out though
that above a certain throughput threshold* many users
do not seem to care all that much for more throughput
as long as interactive use cases are sufficiently
responsive... but high responsiveness requires low
latency and low jitter... This is actually a good
thing, as that means we do not necessarily aim for
100% utilization (almost requiring deep buffers and
hence resulting in compromised latency) but can get
away with say 80-90% where shallow buffers will do (or
rather where buffer filling stays shallow, there is
IMHO still value in having deep buffers for rare
events that need it).<br>
<br>
<br>
<br>
*) This is not a hard physical law so the exact
threshold is not set in stone, but unless one has many
parallel users, something in the 20-50 Mbps range is
plenty and that is only needed in the "loaded"
direction, that is for pure consumers the upload can
be thinner, for pure producers the download can be
thinner.<br>
<br>
<br>
<blockquote type="cite"><br>
I am not an expert in this field,<br>
</blockquote>
<br>
[SM] Nor am I, I come from the wet-ware side of
things so not even soft- or hard-ware ;)<br>
<br>
<br>
<blockquote type="cite">however it seems to me that
answers to these questions would be useful, assuming
they are not yet available!<br>
<br>
Cheers,<br>
<br>
RR<br>
<br>
<br>
<br>
-----Original Message-----<br>
From: Nnagain
[<a class="moz-txt-link-freetext" href="mailto:nnagain-bounces@lists.bufferbloat.net">mailto:nnagain-bounces@lists.bufferbloat.net</a>] On
Behalf Of rjmcmahon via Nnagain<br>
Sent: Sunday, October 15, 2023 1:39 PM<br>
To: Network Neutrality is back! Let´s make the
technical aspects heard this time!<br>
Cc: rjmcmahon<br>
Subject: Re: [NNagain] transit and peering costs
projections<br>
<br>
Hi Jack,<br>
<br>
Thanks again for sharing. It's very interesting to
me.<br>
<br>
Today, the networks are shifting from capacity
constrained to latency<span
class="Apple-converted-space"> </span><br>
constrained, as can be seen in the IX discussions
about how the speed of<span
class="Apple-converted-space"> </span><br>
light over fiber is too slow even between Houston
& Dallas.<br>
<br>
The mitigations against standing queues (which cause
bloat today) are:<br>
<br>
o) Shrink the e2e bottleneck queue so it will drop
packets in a flow and<span
class="Apple-converted-space"> </span><br>
TCP will respond to that "signal"<br>
o) Use some form of ECN marking where the network
forwarding plane<span class="Apple-converted-space"> </span><br>
ultimately informs the TCP source state machine so
it can slow down or<span
class="Apple-converted-space"> </span><br>
pace effectively. This can be an earlier feedback
signal and, if done<span
class="Apple-converted-space"> </span><br>
well, can inform the sources to avoid bottleneck
queuing. There are<span
class="Apple-converted-space"> </span><br>
couple of approaches with ECN. Comcast is trialing
L4S now which seems<span
class="Apple-converted-space"> </span><br>
interesting to me as a WiFi test & measurement
engineer. The jury is<span
class="Apple-converted-space"> </span><br>
still out on this and measurements are needed.<br>
o) Mitigate source side bloat via TCP_NOTSENT_LOWAT<br>
<br>
The QoS priority approach per congestion is
orthogonal by my judgment as<span
class="Apple-converted-space"> </span><br>
it's typically not supported e2e, many networks will
bleach DSCP<span class="Apple-converted-space"> </span><br>
markings. And it's really too late by my judgment.<br>
<br>
Also, on clock sync, yes your generation did us both
a service and<span class="Apple-converted-space"> </span><br>
disservice by getting rid of the PSTN TDM clock ;)
So IP networking<span class="Apple-converted-space"> </span><br>
devices kinda ignored clock sync, which makes e2e
one way delay (OWD)<span
class="Apple-converted-space"> </span><br>
measurements impossible. Thankfully, the GPS atomic
clock is now<span class="Apple-converted-space"> </span><br>
available mostly everywhere and many devices use
TCXO oscillators so<span
class="Apple-converted-space"> </span><br>
it's possible to get clock sync and use oscillators
that can minimize<span class="Apple-converted-space"> </span><br>
drift. I pay $14 for a Rpi4 GPS chip with pulse per
second as an<span class="Apple-converted-space"> </span><br>
example.<br>
<br>
It seems silly to me that clocks aren't synced to
the GPS atomic clock<span
class="Apple-converted-space"> </span><br>
even if by a proxy even if only for measurement and
monitoring.<br>
<br>
Note: As Richard Roy will point out, there really is
no such thing as<span class="Apple-converted-space"> </span><br>
synchronized clocks across geographies per general
relativity - so those<span
class="Apple-converted-space"> </span><br>
syncing clocks need to keep those effects in mind. I
limited the iperf 2<span
class="Apple-converted-space"> </span><br>
timestamps to microsecond precision in hopes
avoiding those issues.<br>
<br>
Note: With WiFi, a packet drop can occur because an
intermittent RF<span class="Apple-converted-space"> </span><br>
channel condition. TCP can't tell the difference
between an RF drop vs a<span
class="Apple-converted-space"> </span><br>
congested queue drop. That's another reason ECN
markings from network<span
class="Apple-converted-space"> </span><br>
devices may be better than dropped packets.<br>
<br>
Note: I've added some iperf 2 test support around
pacing as that seems<span
class="Apple-converted-space"> </span><br>
to be the direction the industry is heading as
networks are less and<span
class="Apple-converted-space"> </span><br>
less capacity strained and user quality of
experience is being driven by<span
class="Apple-converted-space"> </span><br>
tail latencies. One can also test with the Prague
CCA for the L4S<span class="Apple-converted-space"> </span><br>
scenarios. (This is a fun project:
<a class="moz-txt-link-freetext" href="https://www.l4sgear.com/">https://www.l4sgear.com/</a> and fairly<span
class="Apple-converted-space"> </span><br>
low cost)<br>
<br>
--fq-rate n[kmgKMG]<br>
Set a rate to be used with fair-queuing based
socket-level pacing, in<span
class="Apple-converted-space"> </span><br>
bytes or bits per second. Only available on
platforms supporting the<span
class="Apple-converted-space"> </span><br>
SO_MAX_PACING_RATE socket option. (Note: Here the
suffixes indicate<span class="Apple-converted-space"> </span><br>
bytes/sec or bits/sec per use of uppercase or
lowercase, respectively)<br>
<br>
--fq-rate-step n[kmgKMG]<br>
Set a step of rate to be used with fair-queuing
based socket-level<span
class="Apple-converted-space"> </span><br>
pacing, in bytes or bits per second. Step occurs
every<span class="Apple-converted-space"> </span><br>
fq-rate-step-interval (defaults to one second)<br>
<br>
--fq-rate-step-interval n<br>
Time in seconds before stepping the fq-rate<br>
<br>
Bob<br>
<br>
PS. Iperf 2 man page
<a class="moz-txt-link-freetext" href="https://iperf2.sourceforge.io/iperf-manpage.html">https://iperf2.sourceforge.io/iperf-manpage.html</a><br>
<br>
<blockquote type="cite">The "VGV User" (Voice,
Gaming, Videoconferencing) cares a lot about<br>
latency. It's not just "rewarding" to have lower
latencies; high<br>
latencies may make VGV unusable. Average (or
"typical") latency as<br>
the FCC label proposes isn't a good metric to
judge usability. A path<br>
which has high variance in latency can be unusable
even if the average<br>
is quite low. Having your voice or video or
gameplay "break up"<br>
every minute or so when latency spikes to 500 msec
makes the "user<br>
experience" intolerable.<br>
<br>
A few years ago, I ran some simple "ping" tests to
help a friend who<br>
was trying to use a gaming app. My data was only
for one specific<br>
path so it's anecdotal. What I saw was surprising
- zero data loss,<br>
every datagram was delivered, but occasionally a
datagram would take<br>
up to 30 seconds to arrive. I didn't have the
ability to poke around<br>
inside, but I suspected it was an experience of
"bufferbloat", enabled<br>
by the dramatic drop in price of memory over the
decades.<br>
<br>
It's been a long time since I was involved in
operating any part of<br>
the Internet, so I don't know much about the inner
workings today.<br>
Apologies for my ignorance....<br>
<br>
There was a scenario in the early days of the
Internet for which we<br>
struggled to find a technical solution. Imagine
some node in the<br>
bowels of the network, with 3 connected "circuits"
to some other<br>
nodes. On two of those inputs, traffic is
arriving to be forwarded<br>
out the third circuit. The incoming flows are
significantly more than<br>
the outgoing path can accept.<br>
<br>
What happens? How is "backpressure" generated so
that the incoming<br>
flows are reduced to the point that the outgoing
circuit can handle<br>
the traffic?<br>
<br>
About 45 years ago, while we were defining TCPV4,
we struggled with<br>
this issue, but didn't find any consensus
solutions. So "placeholder"<br>
mechanisms were defined in TCPV4, to be replaced
as research continued<br>
and found a good solution.<br>
<br>
In that "placeholder" scheme, the "Source Quench"
(SQ) IP message was<br>
defined; it was to be sent by a switching node
back toward the sender<br>
of any datagram that had to be discarded because
there wasn't any<br>
place to put it.<br>
<br>
In addition, the TOS (Type Of Service) and TTL
(Time To Live) fields<br>
were defined in IP.<br>
<br>
TOS would allow the sender to distinguish
datagrams based on their<br>
needs. For example, we thought "Interactive"
service might be needed<br>
for VGV traffic, where timeliness of delivery was
most important.<span class="Apple-converted-space"> </span><br>
"Bulk" service might be useful for activities like
file transfers,<br>
backups, et al. "Normal" service might now mean
activities like<br>
using the Web.<br>
<br>
The TTL field was an attempt to inform each
switching node about the<br>
"expiration date" for a datagram. If a node
somehow knew that a<br>
particular datagram was unlikely to reach its
destination in time to<br>
be useful (such as a video datagram for a frame
that has already been<br>
displayed), the node could, and should, discard
that datagram to free<br>
up resources for useful traffic. Sadly we had no
mechanisms for<br>
measuring delay, either in transit or in queuing,
so TTL was defined<br>
in terms of "hops", which is not an accurate proxy
for time. But<br>
it's all we had.<br>
<br>
Part of the complexity was that the "flow control"
mechanism of the<br>
Internet had put much of the mechanism in the
users' computers' TCP<br>
implementations, rather than the switches which
handle only IP.<br>
Without mechanisms in the users' computers, all a
switch could do is<br>
order more circuits, and add more memory to the
switches for queuing.<span
class="Apple-converted-space"> </span><br>
Perhaps that led to "bufferbloat".<br>
<br>
So TOS, SQ, and TTL were all placeholders, for
some mechanism in a<br>
future release that would introduce a "real" form
of Backpressure and<br>
the ability to handle different types of traffic.
Meanwhile, these<br>
rudimentary mechanisms would provide some flow
control. Hopefully the<br>
users' computers sending the flows would respond
to the SQ<br>
backpressure, and switches would prioritize
traffic using the TTL and<br>
TOS information.<br>
<br>
But, being way out of touch, I don't know what
actually happens<br>
today. Perhaps the current operators and current
government watchers<br>
can answer?:git clone
<a class="moz-txt-link-freetext" href="https://rjmcmahon@git.code.sf.net/p/iperf2/code">https://rjmcmahon@git.code.sf.net/p/iperf2/code</a><span
class="Apple-converted-space"> </span><br>
iperf2-code<br>
<br>
1/ How do current switches exert Backpressure to
reduce competing<br>
traffic flows? Do they still send SQs?<br>
<br>
2/ How do the current and proposed government
regulations treat the<br>
different needs of different types of traffic,
e.g., "Bulk" versus<br>
"Interactive" versus "Normal"? Are Internet
carriers permitted to<br>
treat traffic types differently? Are they
permitted to charge<br>
different amounts for different types of service?<br>
<br>
Jack Haverty<br>
<br>
On 10/15/23 09:45, Dave Taht via Nnagain wrote:<br>
<blockquote type="cite">For starters I would like
to apologize for cc-ing both nanog and my<br>
new nn list. (I will add sender filters)<br>
<br>
A bit more below.<br>
<br>
On Sun, Oct 15, 2023 at 9:32 AM Tom Beecher
<a class="moz-txt-link-rfc2396E" href="mailto:beecher@beecher.cc"><beecher@beecher.cc></a><span
class="Apple-converted-space"> </span><br>
wrote:<br>
<blockquote type="cite">
<blockquote type="cite">So for now, we'll keep
paying for transit to get to the others<span
class="Apple-converted-space"> </span><br>
(since it’s about as much as transporting
IXP from Dallas), and<span
class="Apple-converted-space"> </span><br>
hoping someone at Google finally sees
Houston as more than a third<span
class="Apple-converted-space"> </span><br>
rate city hanging off of Dallas. Or… someone
finally brings a<span
class="Apple-converted-space"> </span><br>
worthwhile IX to Houston that gets us more
than peering to Kansas<span
class="Apple-converted-space"> </span><br>
City. Yeah, I think the former is more
likely. 😊<br>
</blockquote>
<br>
There is often a chicken/egg scenario here
with the economics. As an<span
class="Apple-converted-space"> </span><br>
eyeball network, your costs to build out and
connect to Dallas are<span
class="Apple-converted-space"> </span><br>
greater than your transit cost, so you do
that. Totally fair.<br>
<br>
However think about it from the content side.
Say I want to build<span
class="Apple-converted-space"> </span><br>
into to Houston. I have to put routers in, and
a bunch of cache<span
class="Apple-converted-space"> </span><br>
servers, so I have capital outlay , plus opex
for space, power,<span
class="Apple-converted-space"> </span><br>
IX/backhaul/transit costs. That's not cheap,
so there's a lot of<span
class="Apple-converted-space"> </span><br>
calculations that go into it. Is there enough
total eyeball traffic<span
class="Apple-converted-space"> </span><br>
there to make it worth it? Is saving 8-10ms
enough of a performance<span
class="Apple-converted-space"> </span><br>
boost to justify the spend? What are the long
term trends in that<span
class="Apple-converted-space"> </span><br>
market? These answers are of course different
for a company running<span
class="Apple-converted-space"> </span><br>
their own CDN vs the commercial CDNs.<br>
<br>
I don't work for Google and obviously don't
speak for them, but I<span
class="Apple-converted-space"> </span><br>
would suspect that they're happy to eat a
8-10ms performance hit to<span
class="Apple-converted-space"> </span><br>
serve from Dallas , versus the amount of
capital outlay to build out<span
class="Apple-converted-space"> </span><br>
there right now.<br>
</blockquote>
The three forms of traffic I care most about are
voip, gaming, and<br>
videoconferencing, which are rewarding to have
at lower latencies.<br>
When I was a kid, we had switched phone
networks, and while the sound<br>
quality was poorer than today, the voice latency
cross-town was just<br>
like "being there". Nowadays we see 500+ms
latencies for this kind of<br>
traffic.<br>
<br>
As to how to make calls across town work that
well again, cost-wise, I<br>
do not know, but the volume of traffic that
would be better served by<br>
these interconnects quite low, respective to the
overall gains in<br>
lower latency experiences for them.<br>
<br>
<br>
<br>
<blockquote type="cite">On Sat, Oct 14, 2023 at
11:47 PM Tim Burke <a class="moz-txt-link-rfc2396E" href="mailto:tim@mid.net"><tim@mid.net></a> wrote:<br>
<blockquote type="cite">I would say that a
1Gbit IP transit in a carrier neutral DC can
be<span class="Apple-converted-space"> </span><br>
had for a good bit less than $900 on the
wholesale market.<br>
<br>
Sadly, IXP’s are seemingly turning into a
pay to play game, with<span
class="Apple-converted-space"> </span><br>
rates almost costing as much as transit in
many cases after you<span
class="Apple-converted-space"> </span><br>
factor in loop costs.<br>
<br>
For example, in the Houston market (one of
the largest and fastest<span
class="Apple-converted-space"> </span><br>
growing regions in the US!), we do not have
a major IX, so to get up<span
class="Apple-converted-space"> </span><br>
to Dallas it’s several thousand for a 100g
wave, plus several<span
class="Apple-converted-space"> </span><br>
thousand for a 100g port on one of those
major IXes. Or, a better<span
class="Apple-converted-space"> </span><br>
option, we can get a 100g flat internet
transit for just a little<span
class="Apple-converted-space"> </span><br>
bit more.<br>
<br>
Fortunately, for us as an eyeball network,
there are a good number<span
class="Apple-converted-space"> </span><br>
of major content networks that are allowing
for private peering in<span
class="Apple-converted-space"> </span><br>
markets like Houston for just the cost of a
cross connect and a QSFP<span
class="Apple-converted-space"> </span><br>
if you’re in the right DC, with Google and
some others being the<span
class="Apple-converted-space"> </span><br>
outliers.<br>
<br>
So for now, we'll keep paying for transit to
get to the others<span
class="Apple-converted-space"> </span><br>
(since it’s about as much as transporting
IXP from Dallas), and<span
class="Apple-converted-space"> </span><br>
hoping someone at Google finally sees
Houston as more than a third<span
class="Apple-converted-space"> </span><br>
rate city hanging off of Dallas. Or… someone
finally brings a<span
class="Apple-converted-space"> </span><br>
worthwhile IX to Houston that gets us more
than peering to Kansas<span
class="Apple-converted-space"> </span><br>
City. Yeah, I think the former is more
likely. 😊<br>
<br>
See y’all in San Diego this week,<br>
Tim<br>
<br>
On Oct 14, 2023, at 18:04, Dave Taht
<a class="moz-txt-link-rfc2396E" href="mailto:dave.taht@gmail.com"><dave.taht@gmail.com></a> wrote:<br>
<blockquote type="cite">This set of
trendlines was very interesting.
Unfortunately the<span
class="Apple-converted-space"> </span><br>
data<br>
stops in 2015. Does anyone have more
recent data?<br>
<br>
<a class="moz-txt-link-freetext" href="https://drpeering.net/white-papers/Internet-Transit-Pricing-Historical-And-Projected.php">https://drpeering.net/white-papers/Internet-Transit-Pricing-Historical-And-Projected.php</a><br>
<br>
I believe a gbit circuit that an ISP can
resell still runs at about<br>
$900 - $1.4k (?) in the usa? How about
elsewhere?<br>
<br>
...<br>
<br>
I am under the impression that many IXPs
remain very successful,<br>
states without them suffer, and I also
find the concept of doing<span
class="Apple-converted-space"> </span><br>
micro<br>
IXPs at the city level, appealing, and now
achievable with cheap<span
class="Apple-converted-space"> </span><br>
gear.<br>
Finer grained cross connects between telco
and ISP and IXP would<span
class="Apple-converted-space"> </span><br>
lower<br>
latencies across town quite hugely...<br>
<br>
PS I hear ARIN is planning on dropping the
price for, and bundling<span
class="Apple-converted-space"> </span><br>
3<br>
BGP AS numbers at a time, as of the end of
this year, also.<br>
<br>
<br>
<br>
--<br>
Oct 30:<span class="Apple-converted-space"> </span><br>
<a class="moz-txt-link-freetext" href="https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.html">https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.html</a><br>
Dave Täht CSO, LibreQos<br>
</blockquote>
</blockquote>
</blockquote>
<br>
<br>
</blockquote>
<br>
_______________________________________________<br>
Nnagain mailing list<br>
<a class="moz-txt-link-abbreviated" href="mailto:Nnagain@lists.bufferbloat.net">Nnagain@lists.bufferbloat.net</a><br>
<a class="moz-txt-link-freetext" href="https://lists.bufferbloat.net/listinfo/nnagain">https://lists.bufferbloat.net/listinfo/nnagain</a><br>
</blockquote>
_______________________________________________<br>
Nnagain mailing list<br>
<a class="moz-txt-link-abbreviated" href="mailto:Nnagain@lists.bufferbloat.net">Nnagain@lists.bufferbloat.net</a><br>
<a class="moz-txt-link-freetext" href="https://lists.bufferbloat.net/listinfo/nnagain">https://lists.bufferbloat.net/listinfo/nnagain</a><br>
<br>
_______________________________________________<br>
Nnagain mailing list<br>
<a class="moz-txt-link-abbreviated" href="mailto:Nnagain@lists.bufferbloat.net">Nnagain@lists.bufferbloat.net</a><br>
<a class="moz-txt-link-freetext" href="https://lists.bufferbloat.net/listinfo/nnagain">https://lists.bufferbloat.net/listinfo/nnagain</a><br>
</blockquote>
<br>
</blockquote>
<br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<span
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">_______________________________________________</span><br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<span
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Nnagain
mailing list</span><br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<span
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;"><a class="moz-txt-link-abbreviated" href="mailto:Nnagain@lists.bufferbloat.net">Nnagain@lists.bufferbloat.net</a></span><br
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<span
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;"><a class="moz-txt-link-freetext" href="https://lists.bufferbloat.net/listinfo/nnagain">https://lists.bufferbloat.net/listinfo/nnagain</a></span></div>
</blockquote>
</div>
<br>
</div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Nnagain mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Nnagain@lists.bufferbloat.net">Nnagain@lists.bufferbloat.net</a>
<a class="moz-txt-link-freetext" href="https://lists.bufferbloat.net/listinfo/nnagain">https://lists.bufferbloat.net/listinfo/nnagain</a>
</pre>
</blockquote>
<br>
</body>
</html>