From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp74.iad3a.emailsrvr.com (smtp74.iad3a.emailsrvr.com [173.203.187.74]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id CC9703B2A4 for ; Tue, 3 Jan 2023 17:58:32 -0500 (EST) Received: from app14.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by smtp26.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 30520424E for ; Tue, 3 Jan 2023 17:58:32 -0500 (EST) Received: from deepplum.com (localhost.localdomain [127.0.0.1]) by app14.wa-webapps.iad3a (Postfix) with ESMTP id 1AE01600BA for ; Tue, 3 Jan 2023 17:58:32 -0500 (EST) Received: by apps.rackspace.com (Authenticated sender: dpreed@deepplum.com, from: dpreed@deepplum.com) with HTTP; Tue, 3 Jan 2023 17:58:32 -0500 (EST) X-Auth-ID: dpreed@deepplum.com Date: Tue, 3 Jan 2023 17:58:32 -0500 (EST) From: "David P. Reed" To: starlink@lists.bufferbloat.net MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20230103175832000000_27390" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: References: X-Client-IP: 209.6.168.128 Message-ID: <1672786712.106922180@apps.rackspace.com> X-Mailer: webmail/19.0.22-RC X-Classification-ID: 8cbbb846-d1fe-4784-ba15-e411859bdc5c-1-1 Subject: Re: [Starlink] Researchers Seeking Probe Volunteers in USA X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jan 2023 22:58:32 -0000 ------=_20230103175832000000_27390 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0AA serious question: =0A =0A=0A> ... The QMap Probe is=0A> preconfigured = to automatically test various network metrics with little burden on=0A> you= r available bandwidth.=0A =0AThose of us here, like me and Dave Taht, who h= ave measured the big elephants in the room (esp. for Starlink) like "lag un= der load" and "fairness with respect to competing traffic on the same " probably were not consulted, if the goal is "little burden on your avail= able bandwidth".=0A =0AI've spent many years reading papers about "low cost= metrics" for lag and fairness in the Internet context, starting with the o= ld "packet-pair" techniques, and basically they are almost impossible to in= terpret and compare between service provider architectures. It's hard enoug= h to compare DOCSIS 3 setups with other DOCSIS 3 CMTS's in other cities, or= LTE data networks between different cities.=0A =0AFrankly, I expect the re= sults will be treated like other "quality metrics" - J.D. Power comes to mi= nd from consulting experience in the automotive industry - and be cherry-pi= cked to distort the results.=0A =0A> This data is used to estimate the qual= ity of internet=0A> service.=0A =0A =0AI have NEVER seen a technical defini= tion of "quality of service" that made sense in terms of how the users expe= rience their use of the Internet. It would be wonderful if such a definitio= n actually existed. So how does relative "estimation" of an undefined conce= pt work?=0A =0AWhat questions really matter to users, in particular? Well, = "availability" might matter, but availability is relative to "momentary nee= d". A network that is 99.9% available, but the 0.1% of the time that the us= er NEEDS it is what matters to the user, not the rest of the 86,400 seconds= each day. =0A =0A[As an aside, in a proceeding I participated under the C= RTC, Canada's "FCC" regulator on Network Management that focused on "qualit= y", we queried that since none of the operators in Canada actually measured= "response time" of their networks in any way, so how could they know that = they were improving service? The response on the record from some of the la= rgest Broadband ISPs in Canada was discouraging. They said I was wrong, and= that they constantly measured "utilization" of the network capacity at eve= ry router, and the *average* utilization was almost always < 85%. They then= invoked Little's Lemma in queueing theory to say that proved that the qual= ity of service was *perfect*.=0AThis was in a legal regulatory proceeding, = *under oath*. I just cannot understand how folks technical enough to invoke= Little's Lemma could be so ignorant. Little's Lemma isn't at all good at c= onverting "average utilization" to "user experienced lag under load", it's = mathematically *invalid*. But what is worse is that they had no idea of wha= t user experienced "quality" was. It's like a software vendor saying they h= ave no bugs, that they know about, when they have no way for users to repor= t bugs at all. OR in the modern context, where reporting bugs is a hassle a= nd there's no "bug bounty" or expectation that the company will fix a repor= ted bug in a timely manner.]=0A =0A =0A =0A =0A> A public report of our fin= dings will be published on our website in 2023.=0A =0ABy all means particip= ate if you want, but I suspect that the "raw data" will not be made availab= le, and looking at the existing reports, it will be hard to extract meaning= ful comparisons relevant to real user experience at the test sites.=0A =0A= =0A> Please check out some of our existing reports to get a better feel of = what we=0A> measure:=0A> https://www.netforecast.com/audit-reports/.=0A> =0A> The QMap Probe requires no regular m= aintenance or monitoring by you. It may=0A> occasionally need rebooting (tu= rning off and on), and we would contact you via=0A> email or text to reques= t this. The device sends various test packets to the=0A> internet and recor= ds their response characteristics. Our device has no knowledge=0A> of -- an= d does not communicate out -- any information about you or any devices in= =0A> your home.=0A> =0A> We will include a prepaid shipping label for retur= ning the QMap Probe in the=0A> original box (please keep this!). Once we re= ceive the device back, we will send=0A> you a $200 Amazon gift card (one pe= r household).=0A> =0A> To volunteer, please fill out this relatively painle= ss survey:=0A> https://www.surveymonkey.com/r/8VZSB3M.=0A> Thank you!=0A> =0A> ///END///=0A> -------------- next part = --------------=0A> An HTML attachment was scrubbed...=0A> URL:=0A> =0A> =0A> ------------------------------=0A> =0A> Message= : 2=0A> Date: Tue, 3 Jan 2023 15:57:30 -0500=0A> From: Vint Cerf =0A> To: "Livingood, Jason" =0A> Cc: Da= ve Taht via Starlink =0A> Subject: Re: [Sta= rlink] Researchers Seeking Probe Volunteers in USA=0A> Message-ID:=0A> =0A> Conten= t-Type: text/plain; charset=3D"utf-8"=0A> =0A> netforecast was started by a= good friend of mine - they are first rate.=0A> =0A> v=0A> =0A> =0A> On Tue= , Jan 3, 2023 at 3:53 PM Livingood, Jason via Starlink <=0A> starlink@lists= .bufferbloat.net> wrote:=0A> =0A> > Forwarding on from a group doing some S= tarlink research. I am aware of at=0A> > least one other researcher that wi= ll soon do the same and will forward that=0A> > later (guessing a week or t= wo).=0A> >=0A> >=0A> >=0A> > Jason=0A> >=0A> > ///FORWARD///=0A> >=0A> > We= need volunteers! NetForecast, a leader in measuring the quality of=0A> > i= nternet service, is conducting a performance study of several types of=0A> = > internet delivery technologies, including low-earth-orbit satellite like= =0A> > Starlink.=0A> >=0A> >=0A> >=0A> > As a volunteer, you will host one = of our proprietary QMap Probes in your=0A> > home network. This will connec= t to the internet via an available ethernet=0A> > port on your gateway/rout= er and plug into a standard power outlet. The QMap=0A> > Probe is preconfig= ured to automatically test various network metrics with=0A> > little burden= on your available bandwidth. This data is used to estimate=0A> > the quali= ty of internet service. A public report of our findings will be=0A> > publi= shed on our website in 2023. Please check out some of our existing=0A> > re= ports to get a better feel of what we measure:=0A> > https://www.netforecas= t.com/audit-reports/=0A> >=0A> =0A> > .= =0A> >=0A> >=0A> >=0A> > The QMap Probe requires no regular maintenance or = monitoring by you. It=0A> > may occasionally need rebooting (turning off an= d on), and we would contact=0A> > you via email or text to request this. Th= e device sends various test=0A> > packets to the internet and records their= response characteristics. Our=0A> > device has no knowledge of -- and does= not communicate out -- any=0A> > information about you or any devices in y= our home.=0A> >=0A> >=0A> >=0A> > We will include a prepaid shipping label = for returning the QMap Probe in=0A> > the original box (please keep this!).= Once we receive the device back, we=0A> > will send you a $200 Amazon gift= card (one per household).=0A> >=0A> >=0A> >=0A> > To volunteer, please fil= l out this relatively painless survey:=0A> > https://www.surveymonkey.com/r= /8VZSB3M=0A> >=0A> .=0A> > Thank you!=0A> >= =0A> >=0A> >=0A> > ///END///=0A> > ________________________________________= _______=0A> > Starlink mailing list=0A> > Starlink@lists.bufferbloat.net=0A= > > https://lists.bufferbloat.net/listinfo/starlink=0A> >=0A> =0A> =0A> --= =0A> Please send any postal/overnight deliveries to:=0A> Vint Cerf=0A> Goog= le, LLC=0A> 1900 Reston Metro Plaza, 16th Floor=0A> Reston, VA 20190=0A> +1= (571) 213 1346=0A> =0A> =0A> until further notice=0A> -------------- next = part --------------=0A> An HTML attachment was scrubbed...=0A> URL:=0A> =0A> -------------- next part --------------=0A> A non-te= xt attachment was scrubbed...=0A> Name: smime.p7s=0A> Type: application/pkc= s7-signature=0A> Size: 3995 bytes=0A> Desc: S/MIME Cryptographic Signature= =0A> URL:=0A> =0A> =0A> ------------------------------= =0A> =0A> Subject: Digest Footer=0A> =0A> _________________________________= ______________=0A> Starlink mailing list=0A> Starlink@lists.bufferbloat.net= =0A> https://lists.bufferbloat.net/listinfo/starlink=0A> =0A> =0A> --------= ----------------------=0A> =0A> End of Starlink Digest, Vol 22, Issue 6=0A>= ***************************************=0A> ------=_20230103175832000000_27390 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

A serious question:&n= bsp;

=0A

 

=0A
=0A

> ... The QMap Probe is> preconfigured to automatically test various network metrics with li= ttle burden on
> your available bandwidth.

=0A

 

=0A

Those of us here, like me and Dave Tah= t, who have measured the big elephants in the room (esp. for Starlink) like= "lag under load" and "fairness with respect to competing traffic on the sa= me <link>" probably were not consulted, if the goal is "little burden= on your available bandwidth".

=0A

 

=0A

I've spent many years reading papers about "low cost metri= cs" for lag and fairness in the Internet context, starting with the old "pa= cket-pair" techniques, and basically they are almost impossible to interpre= t and compare between service provider architectures. It's hard enough to c= ompare DOCSIS 3 setups with other DOCSIS 3 CMTS's in other cities, or LTE d= ata networks between different cities.

=0A

 =0A

Frankly, I expect the results will be treated like= other "quality metrics" - J.D. Power comes to mind from consulting experie= nce in the automotive industry - and be cherry-picked to distort the result= s.

=0A

 

=0A

> This d= ata is used to estimate the quality of internet
> service.

=0A 

=0A

 

=0A

I have NEVER seen a technical definition of "quality of servic= e" that made sense in terms of how the users experience their use of the In= ternet. It would be wonderful if such a definition actually existed. So how= does relative "estimation" of an undefined concept work?

=0A

 

=0A

What questions really matt= er to users, in particular? Well, "availability" might matter, but availabi= lity is relative to "momentary need". A network that is 99.9% available, bu= t the 0.1% of the time that the user NEEDS it is what matters to the user, = not the rest of the 86,400 seconds each day. 

=0A

 

=0A

[As an aside, in a proceeding I partic= ipated under the  CRTC, Canada's "FCC" regulator on Network Management= that focused on "quality", we queried that since none of the operator= s in Canada actually measured "response time" of their networks in any way,= so how could they know that they were improving service? The response on t= he record from some of the largest Broadband ISPs in Canada was discouragin= g. They said I was wrong, and that they constantly measured "utilization" o= f the network capacity at every router, and the *average* utilization was a= lmost always < 85%. They then invoked Little's Lemma in queueing theory = to say that proved that the quality of service was *perfect*.

=0A

This was in a legal regulatory proceeding, *under oath*. I ju= st cannot understand how folks technical enough to invoke Little's Lemma co= uld be so ignorant. Little's Lemma isn't at all good at converting "average= utilization" to "user experienced lag under load", it's mathematically *in= valid*. But what is worse is that they had no idea of what user experienced= "quality" was. It's like a software vendor saying they have no b= ugs, that they know about, when they have no way for users to report bugs a= t all. OR in the modern context, where reporting bugs is a hassle and there= 's no "bug bounty" or expectation that the company will fix a reported bug = in a timely manner.]

=0A

 

=0A

 

=0A

 

=0A

=  

=0A

> A public report of our findings will= be published on our website in 2023.

=0A

 

= =0A

By all means participate if you want, but I suspect= that the "raw data" will not be made available, and looking at the existin= g reports, it will be hard to extract meaningful comparisons relevant to re= al user experience at the test sites.

=0A

 

= =0A


> Please check out some of our existing re= ports to get a better feel of what we
> measure:
> https://= www.netforecast.com/audit-reports/<https://urldefense.com/v3/__https:/ww= w.netforecast.com/audit-reports/__;!!CQl3mcHX2A!FrL2Yijo-63gS4PMToq0adfntj2= fhza8ekyba1EbS8-tCgsQpg5MsIAYAvP5xUzLdDRa667bslUTtw_s0WpvvBpJEFKpAJeQfQ$>= ;.
>
> The QMap Probe requires no regular maintenance or m= onitoring by you. It may
> occasionally need rebooting (turning off= and on), and we would contact you via
> email or text to request t= his. The device sends various test packets to the
> internet and re= cords their response characteristics. Our device has no knowledge
>= of -- and does not communicate out -- any information about you or any dev= ices in
> your home.
>
> We will include a prepaid= shipping label for returning the QMap Probe in the
> original box = (please keep this!). Once we receive the device back, we will send
>= ; you a $200 Amazon gift card (one per household).
>
> To = volunteer, please fill out this relatively painless survey:
> https= ://www.surveymonkey.com/r/8VZSB3M<https://urldefense.com/v3/__https:/www= .surveymonkey.com/r/8VZSB3M__;!!CQl3mcHX2A!FrL2Yijo-63gS4PMToq0adfntj2fhza8= ekyba1EbS8-tCgsQpg5MsIAYAvP5xUzLdDRa667bslUTtw_s0WpvvBpJEFJ_EHoUnA$>.> Thank you!
>
> ///END///
> -------------- = next part --------------
> An HTML attachment was scrubbed...
= > URL:
> <https://lists.bufferbloat.net/pipermail/starlink/at= tachments/20230103/aa1e6019/attachment-0001.html>
>
> -= -----------------------------
>
> Message: 2
> Dat= e: Tue, 3 Jan 2023 15:57:30 -0500
> From: Vint Cerf <vint@google= .com>
> To: "Livingood, Jason" <Jason_Livingood@comcast.com&g= t;
> Cc: Dave Taht via Starlink <starlink@lists.bufferbloat.net&= gt;
> Subject: Re: [Starlink] Researchers Seeking Probe Volunteers = in USA
> Message-ID:
> <CAHxHgge6sTffAqaMLv7z1k0ZnYtWw7s= +OgXBJOBnm5zAwHjR+w@mail.gmail.com>
> Content-Type: text/plain; = charset=3D"utf-8"
>
> netforecast was started by a good fr= iend of mine - they are first rate.
>
> v
>
= >
> On Tue, Jan 3, 2023 at 3:53 PM Livingood, Jason via Starlin= k <
> starlink@lists.bufferbloat.net> wrote:
>
= > > Forwarding on from a group doing some Starlink research. I am awa= re of at
> > least one other researcher that will soon do the sa= me and will forward that
> > later (guessing a week or two).
> >
> >
> >
> > Jason
> &g= t;
> > ///FORWARD///
> >
> > We need volun= teers! NetForecast, a leader in measuring the quality of
> > int= ernet service, is conducting a performance study of several types of
&= gt; > internet delivery technologies, including low-earth-orbit satellit= e like
> > Starlink.
> >
> >
> >= ;
> > As a volunteer, you will host one of our proprietary QMap = Probes in your
> > home network. This will connect to the intern= et via an available ethernet
> > port on your gateway/router and= plug into a standard power outlet. The QMap
> > Probe is precon= figured to automatically test various network metrics with
> > l= ittle burden on your available bandwidth. This data is used to estimate
> > the quality of internet service. A public report of our finding= s will be
> > published on our website in 2023. Please check out= some of our existing
> > reports to get a better feel of what w= e measure:
> > https://www.netforecast.com/audit-reports/
&= gt; >
> <https://urldefense.com/v3/__https:/www.netforecast.c= om/audit-reports/__;!!CQl3mcHX2A!FrL2Yijo-63gS4PMToq0adfntj2fhza8ekyba1EbS8= -tCgsQpg5MsIAYAvP5xUzLdDRa667bslUTtw_s0WpvvBpJEFKpAJeQfQ$>
> >= ; .
> >
> >
> >
> > The QMap Pr= obe requires no regular maintenance or monitoring by you. It
> >= may occasionally need rebooting (turning off and on), and we would contact=
> > you via email or text to request this. The device sends var= ious test
> > packets to the internet and records their response= characteristics. Our
> > device has no knowledge of -- and does= not communicate out -- any
> > information about you or any dev= ices in your home.
> >
> >
> >
> &= gt; We will include a prepaid shipping label for returning the QMap Probe i= n
> > the original box (please keep this!). Once we receive the = device back, we
> > will send you a $200 Amazon gift card (one p= er household).
> >
> >
> >
> > = To volunteer, please fill out this relatively painless survey:
> &g= t; https://www.surveymonkey.com/r/8VZSB3M
> >
> <http= s://urldefense.com/v3/__https:/www.surveymonkey.com/r/8VZSB3M__;!!CQl3mcHX2= A!FrL2Yijo-63gS4PMToq0adfntj2fhza8ekyba1EbS8-tCgsQpg5MsIAYAvP5xUzLdDRa667bs= lUTtw_s0WpvvBpJEFJ_EHoUnA$>.
> > Thank you!
> >> >
> >
> > ///END///
> > _______= ________________________________________
> > Starlink mailing li= st
> > Starlink@lists.bufferbloat.net
> > https://lis= ts.bufferbloat.net/listinfo/starlink
> >
>
> > --
> Please send any postal/overnight deliveries to:
= > Vint Cerf
> Google, LLC
> 1900 Reston Metro Plaza, 16t= h Floor
> Reston, VA 20190
> +1 (571) 213 1346
> >
> until further notice
> -------------- next part= --------------
> An HTML attachment was scrubbed...
> URL:=
> <https://lists.bufferbloat.net/pipermail/starlink/attachments= /20230103/e49cfc9f/attachment.html>
> -------------- next part -= -------------
> A non-text attachment was scrubbed...
> Nam= e: smime.p7s
> Type: application/pkcs7-signature
> Size: 39= 95 bytes
> Desc: S/MIME Cryptographic Signature
> URL:
> <https://lists.bufferbloat.net/pipermail/starlink/attachments/2023= 0103/e49cfc9f/attachment.bin>
>
> ---------------------= ---------
>
> Subject: Digest Footer
>
> = _______________________________________________
> Starlink mailing = list
> Starlink@lists.bufferbloat.net
> https://lists.buffe= rbloat.net/listinfo/starlink
>
>
> --------------= ----------------
>
> End of Starlink Digest, Vol 22, Issue= 6
> ***************************************
>

=0A ------=_20230103175832000000_27390--