<div dir="ltr"><div>Great points Vint as you're absolutely right - there are multiple modalities here (and in the past it was spam from thousands of postcards, then mimeographs, then faxes, etc.) <br></div><div><br></div><div>The standard historically has been set by the Administrative Conference of the United States: <a href="https://www.acus.gov/about-acus">https://www.acus.gov/about-acus</a></div><div><br></div><div>In 2020 there seemed to be an effort to gave the General Services Administration weigh-in, however they closed that rulemaking attempt without publishing any of the comments they got and no announcement why it was closed. <br></div><div><br></div><div>As for what part of Congress - I believe ACUS was championed by both the Senate and House Judiciary Committees as it has oversight and responsibility for the interpretations of the Administrative Procedure Act of 1946 (which sets out the whole rulemaking procedure). <br></div><div><br></div><div>Sadly there isn't a standard across agencies - which also means there isn't a standard across Administrations. Back in 2018 and 2020, both with this group of 52 people here <a href="https://tinyurl.com/letter-signed-52-people">https://tinyurl.com/letter-signed-52-people</a> - as well as individually - I did my darnest to encourage them to do a standard. <br></div><div><br></div><div>There's also the National Academy of Public Administration which is probably the latest remaining non-partisan forum for discussions like this too. <br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Oct 9, 2023 at 7:46 PM Vint Cerf <<a href="mailto:vint@google.com">vint@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">David, this is a good list.<div>FACA has rules for public participation, for example.</div><div><br></div><div>I think it should be taken into account for any public commenting process that online (and offline such as USPS or fax and phone calls) that spam and artificial inflation of comments are possible. Is there any specific standard for US agency public comment handling? If now, what committees of the US Congress might have jurisdiction?</div><div><br></div><div>v</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Oct 10, 2023 at 8:22 AM David Bray, PhD via Nnagain <<a href="mailto:nnagain@lists.bufferbloat.net" target="_blank">nnagain@lists.bufferbloat.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>I'm all for doing new things to make things better. <br></div><div><br></div><div>At the same time, I used to do bioterrorism preparedness and response from 2000-2005 (and aside from asking myself what kind of crazy world needed counter-bioterrorism efforts... I also realized you don't want to interject something completely new in the middle of an unfolding crisis event). If something were to be injected now, it would have to have consensus from both sides, otherwise at least one side (potentially detractors from both) will claim that whatever form the new approaches take are somehow advantaging "the other side" and disadvantaging them.</div><div><br></div><div>Probably would take a ruling by the Administrative Conference of the United States, at a minimum to answer these five questions - and even then, introducing something completely different in the midst of a political melee might just invite mudslinging unless moderate voices on both sides can reach some consensus. <br></div><div><br></div><div>
<p style="margin-left:40px"><b>1. Does identity matter regarding who files a comment or not — and must one be a U.S. person in order to file?</b></p><p style="margin-left:40px"><b>2.
Should agencies publish real-time counts of the number of comments
received — or is it better to wait until the end of a commenting round
to make all comments available, including counts?</b></p><p style="margin-left:40px"><b>3. Should
third-party groups be able to file on behalf of someone else or not —
and do agencies have the right to remove spam-like comments?</b></p><p style="margin-left:40px"><b>4.
Should the public commenting process permit multiple comments per
individual for a proceeding — and if so, how many comments from a single
individual are too many? 100? 1000? More?</b></p><p style="margin-left:40px"><b>5. Finally, should the
U.S. government itself consider, given public perceptions about
potential conflicts of interest for any agency performing a public
commenting process, whether it would be better to have third-party
groups take responsibility for assembling comments and then filing those
comments via a validated process with the government?</b></p>
</div><div> <br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Oct 7, 2023 at 4:10 PM Jack Haverty <<a href="mailto:jack@3kitty.org" target="_blank">jack@3kitty.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><u></u>
<div>
Hi again David et al,<br>
<br>
Interesting frenzy...lots of questions that need answers and
associated policies. I served 6 years as an elected official (in a
small special district in California), so I have some small
understanding of the government side of things and the constraints
involved. Being in charge doesn't mean you can do what you want.<br>
<br>
I'm thinking here more near-term and incremental steps. You said
"These same questions need pragmatic pilots that involve the public
..."<br>
<br>
So, how about using the current NN situation for a pilot? Keep all
the current ways and emerging AI techniques to continue to flood the
system with comments. But also offer an *optional* way for humans
to "register" as a commenter and then submit their (latest only)
comment into the melee. Will people use it? Will "consumers" (the
lawyers, commissioners, etc.) find it useful?<br>
<br>
I've found it curious, for decades now, that there are (too many)
mechanisms for "secure email", that may help with the flood of
disinformation from anonymous senders, but very very few people use
them. Maybe they don't know how; maybe the available schemes are
too flawed; maybe ...?<br>
<br>
About 30 years ago, I was a speaker in a public meeting orchestrated
by USPS, and recommended that they take a lead role, e.g., by acting
as a national CA - certificate authority. Never happened though.
FCC issues lots of licenses...perhaps they could issue online
credentials too?<br>
<br>
Perhaps a "pilot" where you will also accept comments by email, some
possibly sent by "verified" humans if they understand how to do so,
would be worth trying? Perhaps comments on "technical aspects"
coming from people who demonstrably know how to use technology would
be valuable to the policy makers?<br>
<br>
The Internet, and technology such as TCP, began as an experimental
pilot about 50 years ago. Sometimes pilots become infrastructures.<br>
<br>
FYI, I'm signing this message. Using OpenPGP. I could encrypt it
also, but my email program can't find your public key.<br>
<br>
Jack Haverty<br>
<br>
<br>
<div>On 10/5/23 14:21, David Bray, PhD
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Indeed Jack - a few things to balance - the Administrative
Procedure Act of 1946 (on which the idea of rulemaking is
based) us about raising legal concerns that must be answered
by the agency at the time the rulemaking is done. It's not a
vote nor is it the case that if the agency gets tons of
comments in one direction that they have to go in that
direction. Instead it's only about making sure legal concerns
are considered and responded to before the agency before the
agency acts. (Which is partly why sending "I'm for XYZ" or
"I'm against ABC" really doesn't mean anything to an agency -
not only is that not a legal argument or concern, it's also
not something where they're obligated to follow these comments
- it's not a vote or poll). <br>
</div>
<div><br>
</div>
<div>That said, political folks have spun things to the public
as if it is a poll/vote/chance to act. The raise a valid legal
concern part of the APA of 1946 is omitted. Moreover the fact
that third party law firms and others like to submit comments
on behalf of clients - there will always be a third party
submitting multiple comments for their clients (or "clients")
because that's their business. <br>
</div>
<div><br>
</div>
<div>In the lead up to 2017, the Consumer and Government Affairs
Bureau of the FCC got an inquiry from a firm asking how they
could submit 1 million comments a day on an "upcoming privacy
proceeding" (their words, astute observers will note there was
no privacy proceeding before the FCC in 2017). When the Bureau
asked me, I told them either mail us a CD to upload it or
submit one comment with 1 million signatures. To attempt to
flood us with 1 million comments a day (aside from the fact
who can "predict" having that many daily) would deny resources
to others. In the mess that followed, what was released to the
public was so redacted you couldn't see the legitimate
concerns and better paths that were offered to this entity. <br>
</div>
<div><br>
</div>
<div>And the FCC isn't alone. EPA, FTC, and other regulatory
agencies have had these hijinks for years - and before the
Internet it was faxes, mass mimeographs (remember blue ink?),
and postcards.The Administrative Conference of the United
States (ACUS) - is the body that is supposed to provide
consistent guidance for things like this across the U.S.
government. I've briefed them and tried to raise awareness of
these issues - as I think fundamentally this is a **process**
question that once answered, tech can support. However they're
not technologies and updating the interpretation of the
process isn't something lawyers are apt to do until the
evidence that things are in trouble is overwhelming. <br>
</div>
<div><br>
</div>
<div>52 folks wrote a letter to them - and to GSA - back in
2020. GSA had a rulemaking of its own on how to improve
things, yet oddly never published any of the comments it
received (including ours) and closed the rulemaking quietly.
Here's the letter: <a href="https://tinyurl.com/letter-signed-52-people" target="_blank">https://tinyurl.com/letter-signed-52-people</a></div>
<div><br>
</div>
<div>And here's an article published in OODAloop about this -
and why Generative AI is probably going to make things even
more challenging: <a href="https://www.oodaloop.com/archive/2023/04/18/why-a-pause-on-ai-development-is-not-the-answer-an-insiders-perspective/" target="_blank">https://www.oodaloop.com/archive/2023/04/18/why-a-pause-on-ai-development-is-not-the-answer-an-insiders-perspective/</a></div>
<div><br>
</div>
<div>[snippet of the article] <strong>Now in 2023 and Beyond:
Proactive Approaches to AI and Society</strong></div>
<div>
<p>Looking to the future, to effectively address the
challenges arising from AI, <a href="https://davidbray.medium.com/challenges-and-needed-new-solutions-for-open-societies-to-maintain-civil-discourse-part-1-b5ea95f8c679" target="_blank">we must foster a proactive,
results-oriented, and cooperative approach with the public</a>.
Think tanks and universities can engage the public in
conversations about how to work, live, govern, and co-exist
with modern technologies that impact society. By involving
diverse voices in the decision-making process, we can better
address and resolve the complex challenges AI presents on
local and national levels.</p>
<p>In addition, we must encourage industry and political
leaders to participate in finding non-partisan, multi-sector
solutions if civil societies are to remain stable. By
working together, we can bridge the gap between
technological advancements and their societal implications. </p>
<p>Finally, launching AI pilots across various sectors, such
as work, education, health, law, and civil society, is
essential. We must learn by doing on how we can create
responsible civil environments where AIs can be developed
and deployed responsibly. These initiatives can help us
better understand and integrate AI into our lives, ensuring
its potential is harnessed for the greater good while
mitigating risks. </p>
<p>In 2019 and 2020, a group of <a href="https://tinyurl.com/letter-signed-52-people" target="_blank">fifty-two people asked the
Administrative Conference of the United States </a>(which
helps guide rulemaking procedures for federal agencies),
General Accounting Office, and the General Services
Administration to call attention to the need to address the
challenges of chatbots flooding public commenting procedures
and potentially crowding out or denying services to actual
humans wanting to leave a comment. <a href="https://davidbray.medium.com/challenges-and-needed-new-solutions-for-open-societies-to-maintain-civil-discourse-part-1-b5ea95f8c679" target="_blank">We asked</a>: </p>
<p style="margin-left:40px"><b>1. Does identity matter
regarding who files a comment or not — and must one be a
U.S. person in order to file?</b></p>
<p style="margin-left:40px"><b>2. Should agencies publish
real-time counts of the number of comments received — or
is it better to wait until the end of a commenting round
to make all comments available, including counts?</b></p>
<p style="margin-left:40px"><b>3. Should third-party groups be
able to file on behalf of someone else or not — and do
agencies have the right to remove spam-like comments?</b></p>
<p style="margin-left:40px"><b>4. Should the public commenting
process permit multiple comments per individual for a
proceeding — and if so, how many comments from a single
individual are too many? 100? 1000? More?</b></p>
<p style="margin-left:40px"><b>5. Finally, should the U.S.
government itself consider, given public perceptions about
potential conflicts of interest for any agency performing
a public commenting process, whether it would be better to
have third-party groups take responsibility for assembling
comments and then filing those comments via a validated
process with the government?</b></p>
<p>These same questions need pragmatic pilots that involve the
public to <a href="https://davidbray.medium.com/challenges-and-needed-new-solutions-for-open-societies-to-maintain-civil-discourse-part-2-2f637c472112" target="_blank">co-explore and co-develop how we
operate effectively amid these technological shifts</a>.
As the capabilities of LLMs continue to grow, we need
positive change agents willing to tackle the messy issues at
the intersection of technology and society. The challenges
are immense, but so too are the opportunities for positive
change. Let’s seize this moment to create a better tomorrow
for all. Working together, <a href="https://medium.com/peoplecentered/the-need-for-people-centered-sources-of-hope-for-our-digital-future-ahead-ef491dd2703d" target="_blank">we can co-create a future that
embraces AI’s potential while mitigating its risks</a>,
informed by the hard lessons we have already learned. <br>
</p>
<p>Full article: <a href="https://www.oodaloop.com/archive/2023/04/18/why-a-pause-on-ai-development-is-not-the-answer-an-insiders-perspective/" target="_blank">https://www.oodaloop.com/archive/2023/04/18/why-a-pause-on-ai-development-is-not-the-answer-an-insiders-perspective/</a></p>
<p>Hope this helps.</p>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Oct 5, 2023 at 4:44 PM
Jack Haverty via Nnagain <<a href="mailto:nnagain@lists.bufferbloat.net" target="_blank">nnagain@lists.bufferbloat.net</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> Thanks for all your efforts to keep the "feedback loop"
to the rulemakers functioning! <br>
<br>
I'd like to offer a suggestion for a hopefully politically
acceptable way to handle the deluge, derived from my own
battles with "email" over the years (decades).<br>
<br>
Back in the 1970s, I implemented one of the first email
systems on the Arpanet, under the mentorship of JCR
Licklider, who had been pursuing his vision of a "Galactic
Network" at ARPA and MIT. One of the things we discovered
was the significance of anonymity. At the time, anonymity
was forbidden on the Arpanet; you needed an account on some
computer, protected by passwords, in order to legitimately
use the network. The mechanisms were crude and easily
broken, but the principle applied.<br>
<br>
Over the years, that principle has been forgotten, and the
right to be anonymous has become entrenched. But many uses
of the network, and needs of its users, demand
accountability, so all sorts of mechanisms have been pasted
on top of the network to provide ways to judge user
identity. Banks, medical services, governments, and
businesses all demand some way of proving your identity,
with passwords, various schemes of 2FA, VPNs, or other such
technology, with varying degrees of protection. It is
still possible to be anonymous on the net, but many things
you do require you to prove, to some extent, who you are.<br>
<br>
So, my suggestion for handling the deluge of "comments" is:<br>
<br>
1/ create some mechanism for "registering" your intent to
submit a comment. Make it hard for bots to register.
Perhaps you can leverage the work of various partners, e.g.,
ISPs, retailers, government agencies, financial
institutions, of others who already have some way of
identifying their users.<br>
<br>
2/ Also make registration optional - anyone can still submit
comments anonymously if they choose.<br>
<br>
3/ for "registered commenters", provide a way to "edit" your
previous comment - i.e., advise that your comment is always
the last one you submitted. I.E., whoever you are, you can
only submit one comment, which will be the last one you
submit.<br>
<br>
4/ In the thousands of pages of comments, somehow flag the
ones that are from registered commenters, visible to the
people who read the comments. Even better, provide those
"information consumers" with ways to sort, filter, and
search through the body of comments.<br>
<br>
This may not reduce the deluge of comments, but I'd expect
it to help the lawyers and politicians keep their heads
above the water.<br>
<br>
Anonymity is an important issue for Net Neutrality too, but
I'll opine about that separately.....<br>
<br>
Jack Haverty<br>
<br>
<br>
<div>On 10/2/23 12:38, David Bray, PhD via Nnagain wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>Greetings all and thank you Dave Taht for
that very kind intro... <br>
<br>
</div>
First, I'll open with I'm a gosh-darn
non-partisan, which means I swore an oath to
uphold the Constitution first and serve the United
States - not a specific party, tribe, or ideology.
This often means, especially in today's era of
24/7 news and social media, non-partisans have to
"top cover". <br>
<br>
</div>
Second, I'll share that in what happened in 2017
(which itself was 10x what we saw in 2014) my
biggest concern was and remains that a few actors
attempted to flood the system with
less-than-authentic comments. <br>
<br>
</div>
In some respects this is not new. The whole "notice
and comment" process is a legacy process that goes
back decades. And the FCC (and others) have had
postcard floods of comments, mimeographed letters of
comments, faxed floods of comments, and now this -
which, when combined with generative AI, will be yet
another flood. <br>
<br>
</div>
Which gets me to my biggest concern as a non-partisan in
2023-2024, namely how LLMs might misuse and abuse the
commenting process further. <br>
<br>
Both in 2014 and 2017, I asked FCC General Counsel if I
could use CAPTChA to try to reduce the volume of web
scrapers or bots both filing and pulling info from the
Electronic Comment Filing System.
<div dir="auto"><br>
</div>
<div dir="auto">Both times I was told *no* out of
concerns that they might prevent someone from filing.
I asked if I could block obvious spam, defined as
someone filing a comment >100 times a minute, and
was similarly told no because one of those possible
comments might be genuine and/or it could be an ex
party filing en masse for others.
<div dir="auto"><br>
</div>
<div dir="auto">For 2017 we had to spin up 30x the
number of AWS cloud instances to handle the load -
and this was a flood of comments at 4am, 5am, and
6am ET at night which normally shouldn’t see such
volumes. When I said there was a combination of
actual humans wanting to leave comments and others
who were effectively denying service to others
(especially because if anyone wanted to do a batch
upload of 100,000 comments or more they could submit
a CSV file or a comment with 100,000 signatories) -
both parties said no, that couldn’t be happening. </div>
<div dir="auto"><br>
</div>
<div dir="auto">Until 2021 when the NY Attorney
General proved that was exactly what was happening
with 18m of the 23m apparently from non-authentic
origin with ~9m from one side of the political aisle
(and six companies) and ~9m from the other side of
the political aisle (and one or more teenagers). </div>
<div dir="auto"><br>
</div>
<div dir="auto">So with Net Neutrality back on the
agenda - here’s a simple <span>prediction</span>,
even if the volume of comments is somehow
controlled, 10,000+ pages of comments produced by
ChatGPT or a different LLM is both possible and
probably will be done. The question is if someone
includes a legitimate legal argument on page 6,517 -
will FCC’s lawyers spot it and respond to it as part
of the NPRM? <br>
<br>
</div>
<div>Hope this helps and with highest regards, <br>
<br>
</div>
<div>-d. <br>
<span class="gmail_signature_prefix">-- </span><br>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div dir="ltr">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt;font-family:"Courier New";color:black"><span></span></span></p>
</div>
<div dir="ltr">
<div dir="ltr">
<div>
<p class="MsoNormal"><span style="font-size:10pt;font-family:"Courier New";color:black"><span></span></span></p>
</div>
<span style="font-size:10pt;font-family:"Courier New";color:black">Principal,
<a href="https://www.leaddoadapt.com/" target="_blank"><span>LeadDoAdapt</span>
<span>Ventures</span>, Inc.</a>
& Distinguished Fellow <br>
</span></div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt;font-family:"Courier New";color:black"><a href="https://www.stimson.org/ppl/david-bray/" target="_blank">Henry S.
Stimson Center</a>, <a href="https://bens.org/people/dr-david-bray/" target="_blank">Business
Executives for National Security</a><br>
</span></p>
<p class="MsoNormal"><span style="font-size:10pt;font-family:"Courier New";color:black"><br>
</span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Oct 2, 2023 at
2:15 PM Dave Taht via Nnagain <<a href="mailto:nnagain@lists.bufferbloat.net" target="_blank">nnagain@lists.bufferbloat.net</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">All:<br>
<br>
I have spent the last several days reaching out to as
many people I<br>
know with a deep understanding of the policy and
technical issues<br>
surrounding the internet, to participate on this list.
I encourage you<br>
all to reach out on your own, especially to those that
you can<br>
constructively and civilly disagree with, and
hopefully work with, to<br>
establish technical steps forward. Quite a few have
joined silently!<br>
So far, 168 people have joined!<br>
<br>
Please welcome Dr David Bray[1], a self-described
"human flack jacket"<br>
who, in the last NN debate, stood up for the non
-partisan FCC IT team<br>
that successfully kept the system up 99.4% of the time
despite the<br>
comment floods and network abuses from all sides. He
has shared with<br>
me privately many sad (and some hilarious!) stories of
that era, and I<br>
do kind of hope now, that some of that history
surfaces, and we can<br>
learn from it.<br>
<br>
Thank you very much, David, for putting down your
painful memories[2],<br>
and agreeing to join here. There is a lot to tackle
here, going<br>
forward.<br>
<br>
[1] <a href="https://www.stimson.org/ppl/david-bray/" rel="noreferrer" target="_blank">https://www.stimson.org/ppl/david-bray/</a><br>
[2] "Pain shared is reduced. Joy shared, increased." -
Spider Robinson<br>
<br>
<br>
-- <br>
Oct 30: <a href="https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.html" rel="noreferrer" target="_blank">https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.html</a><br>
Dave Täht CSO, LibreQos<br>
_______________________________________________<br>
Nnagain mailing list<br>
<a href="mailto:Nnagain@lists.bufferbloat.net" target="_blank">Nnagain@lists.bufferbloat.net</a><br>
<a href="https://lists.bufferbloat.net/listinfo/nnagain" rel="noreferrer" target="_blank">https://lists.bufferbloat.net/listinfo/nnagain</a><br>
</blockquote>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
Nnagain mailing list
<a href="mailto:Nnagain@lists.bufferbloat.net" target="_blank">Nnagain@lists.bufferbloat.net</a>
<a href="https://lists.bufferbloat.net/listinfo/nnagain" target="_blank">https://lists.bufferbloat.net/listinfo/nnagain</a>
</pre>
</blockquote>
<br>
</div>
_______________________________________________<br>
Nnagain mailing list<br>
<a href="mailto:Nnagain@lists.bufferbloat.net" target="_blank">Nnagain@lists.bufferbloat.net</a><br>
<a href="https://lists.bufferbloat.net/listinfo/nnagain" rel="noreferrer" target="_blank">https://lists.bufferbloat.net/listinfo/nnagain</a><br>
</blockquote>
</div>
</blockquote>
<br>
</div>
</blockquote></div>
_______________________________________________<br>
Nnagain mailing list<br>
<a href="mailto:Nnagain@lists.bufferbloat.net" target="_blank">Nnagain@lists.bufferbloat.net</a><br>
<a href="https://lists.bufferbloat.net/listinfo/nnagain" rel="noreferrer" target="_blank">https://lists.bufferbloat.net/listinfo/nnagain</a><br>
</blockquote></div><br clear="all"><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div>Please send any postal/overnight deliveries to:</div><div><div>Vint Cerf</div><div>Google, LLC</div><div>1900 Reston Metro Plaza, 16th Floor</div><div>Reston, VA 20190</div><div>+1 (571) 213 1346<br></div><div><br style="color:rgb(34,34,34)"></div></div><div><br></div><div>until further notice</div><div><br></div><div><br></div><div><br></div></div></div>
</blockquote></div>