[NNagain] ntia explicitly excludes latency under load from bead measurements

[NNagain] ntia explicitly excludes latency under load from bead measurements

[Dave Taht]

from:

https://www.ntia.gov/sites/default/files/2024-12/draft_performance_measures_for_bead_last-mile_networks_policy_notice.pdf

"For latency testing, a provider must conduct a minimum of one test
per minute — 60 tests per hour — for each testing hour. If the
consumer load during a latency test exceeds 200,000 bits per second
downstream,26 the provider may cancel the test and reevaluate whether
the consumer load exceeds 200,000 bits per second downstream before
retrying the test in the next minute."

w/a 100ms baseline. Hilarious. Sad.

-- 
Dave Täht CSO, LibreQos

Re: [NNagain] ntia explicitly excludes latency under load from bead measurements

[Frantisek Borsik]

[-- Attachment #1: Type: text/plain, Size: 1157 bytes --]

Are they retarded, or what? LOL


All the best,

Frank

Frantisek (Frank) Borsik



https://www.linkedin.com/in/frantisekborsik

Signal, Telegram, WhatsApp: +421919416714

iMessage, mobile: +420775230885

Skype: casioa5302ca

frantisek.borsik@gmail.com


On Mon, Dec 9, 2024 at 6:42 PM Dave Taht via Nnagain <
nnagain@lists.bufferbloat.net> wrote:

> from:
>
>
> https://www.ntia.gov/sites/default/files/2024-12/draft_performance_measures_for_bead_last-mile_networks_policy_notice.pdf
>
> "For latency testing, a provider must conduct a minimum of one test
> per minute — 60 tests per hour — for each testing hour. If the
> consumer load during a latency test exceeds 200,000 bits per second
> downstream,26 the provider may cancel the test and reevaluate whether
> the consumer load exceeds 200,000 bits per second downstream before
> retrying the test in the next minute."
>
> w/a 100ms baseline. Hilarious. Sad.
>
> --
> Dave Täht CSO, LibreQos
> _______________________________________________
> Nnagain mailing list
> Nnagain@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/nnagain
>

[-- Attachment #2: Type: text/html, Size: 2991 bytes --]

Re: [NNagain] ntia explicitly excludes latency under load from bead measurements

[Eugene Chang]

[-- Attachment #1: Type: text/plain, Size: 1748 bytes --]

What do the engineers working for the FCC think about this?
I assume there are engineers supporting the commissioners?
Does anyone here know the engineers?

Gene
----------------------------------------------
Eugene Chang,
eugene.chang@ieee.org
m 781-799-0233 (in Honolulu)

On Dec 9, 2024, at 7:46 AM, Frantisek Borsik via Nnagain <nnagain@lists.bufferbloat.net> wrote:

Are they retarded, or what? LOL


All the best,

Frank
Frantisek (Frank) Borsik

https://www.linkedin.com/in/frantisekborsik
Signal, Telegram, WhatsApp: +421919416714
iMessage, mobile: +420775230885
Skype: casioa5302ca
frantisek.borsik@gmail.com<mailto:frantisek.borsik@gmail.com>


On Mon, Dec 9, 2024 at 6:42 PM Dave Taht via Nnagain <nnagain@lists.bufferbloat.net<mailto:nnagain@lists.bufferbloat.net>> wrote:
from:

https://www.ntia.gov/sites/default/files/2024-12/draft_performance_measures_for_bead_last-mile_networks_policy_notice.pdf

"For latency testing, a provider must conduct a minimum of one test
per minute — 60 tests per hour — for each testing hour. If the
consumer load during a latency test exceeds 200,000 bits per second
downstream,26 the provider may cancel the test and reevaluate whether
the consumer load exceeds 200,000 bits per second downstream before
retrying the test in the next minute."

w/a 100ms baseline. Hilarious. Sad.

--
Dave Täht CSO, LibreQos
_______________________________________________
Nnagain mailing list
Nnagain@lists.bufferbloat.net<mailto:Nnagain@lists.bufferbloat.net>
https://lists.bufferbloat.net/listinfo/nnagain
_______________________________________________
Nnagain mailing list
Nnagain@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/nnagain


[-- Attachment #2: Type: text/html, Size: 10336 bytes --]

Re: [NNagain] ntia explicitly excludes latency under load from bead measurements

[Sebastian Moeller]

On 9 December 2024 18:42:01 CET, Dave Taht via Nnagain <nnagain@lists.bufferbloat.net> wrote:
>from:
>
>https://www.ntia.gov/sites/default/files/2024-12/draft_performance_measures_for_bead_last-mile_networks_policy_notice.pdf
>
>"For latency testing, a provider must conduct a minimum of one test
>per minute — 60 tests per hour — for each testing hour. If the
>consumer load during a latency test exceeds 200,000 bits per second
>downstream,26 the provider may cancel the test and reevaluate whether
>the consumer load exceeds 200,000 bits per second downstream before
>retrying the test in the next minute."
>
>w/a 100ms baseline. Hilarious. Sad.

[SM] This is not all that terrible for minimal RTT measurements, it does however do zilch for latency under load. The 100ms is also not that terrible, given that the US is large, think test point in Anchorage reflector in Miami...

(I agree this could be more ambitious, but for reference the German regulator allows up to 150ms OWD or 300ms RTT, with reflectors mostly in the center (Frankfurt), and Germany is small, with largest distance ~1000km or 10ms RTT in fiber... I would take your 100ms every day ;) )


How about courteously applaud them for taking latency serious and advise them to include additional latency under load tests...



>
>-- 
>Dave Täht CSO, LibreQos
>_______________________________________________
>Nnagain mailing list
>Nnagain@lists.bufferbloat.net
>https://lists.bufferbloat.net/listinfo/nnagain

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [NNagain] ntia explicitly excludes latency under load from bead measurements

[Livingood, Jason]

>w/a 100ms baseline. Hilarious. Sad.

[SM] This is not all that terrible for minimal RTT measurements, it does however do zilch for latency under load. The 100ms is also not that terrible, given that the US is large, think test point in Anchorage reflector in Miami...

[JL] The 100ms was initially put there to exclude satellite internet from being eligible (prior to LEO operators coming onto the scene). 

[JL] Some concerns I noticed in the document:

1. They want 10% of homes tested in Sec. 3.2, which seems to be an extremely large percentage - well above typical statistical significance - the FCC MBA only needed >30 to be valid nationally. Amusingly, the example they cite in Sec. 3.3 works out to 5% - so they are not internally consistent.
2. In Sec. 3.4, expecting the ISP to temporarily upgrade subs to the highest tier to run tests and then downgrade them again does not make practical sense – for example they suggest that rather than randomly selecting from users in the highest tier that instead you must select from all tiers and then upgrade those not on the highest tier.
3. In Sec. 3.9, it seems a bit too proscriptive on the (IXP) server locations – could be simplified to a regional peering location of the ISP network to allow for flexibility.
4. In Sec. 3.10, similarly proscriptive for example by defining the duration of a test as 10-15 seconds – what if they test can be completed just as accurately in 9 seconds? ISPs should have latitude to configure these tests & the state of the art is moving fast. 

Re: [NNagain] ntia explicitly excludes latency under load from bead measurements

[Robert McMahon]

[-- Attachment #1.1: Type: text/plain, Size: 2462 bytes --]

We use five for our subgroups and two second traffic runs to obtain single
variate gaussians for control charts.

A gaussian measurement is ok for capacity but not for latency or
responsiveness. There, a kolmogorov-smirnov against an ideal or a
comparison set can be used. Or the 99% per the tail of a non-linear
distribution - though that may not be sufficiently descriptive or
correlated for a QoE metric

Then if the transport has a 3WHS, then that is useful. So are flow
completion timed. And typically the duration of slow start is useful.

Bob

On Mon, Dec 9, 2024, 10:53 AM Livingood, Jason via Nnagain <
nnagain@lists.bufferbloat.net> wrote:

> >w/a 100ms baseline. Hilarious. Sad.
>
> [SM] This is not all that terrible for minimal RTT measurements, it does
> however do zilch for latency under load. The 100ms is also not that
> terrible, given that the US is large, think test point in Anchorage
> reflector in Miami...
>
> [JL] The 100ms was initially put there to exclude satellite internet from
> being eligible (prior to LEO operators coming onto the scene).
>
> [JL] Some concerns I noticed in the document:
>
> 1. They want 10% of homes tested in Sec. 3.2, which seems to be an
> extremely large percentage - well above typical statistical significance -
> the FCC MBA only needed >30 to be valid nationally. Amusingly, the example
> they cite in Sec. 3.3 works out to 5% - so they are not internally
> consistent.
> 2. In Sec. 3.4, expecting the ISP to temporarily upgrade subs to the
> highest tier to run tests and then downgrade them again does not make
> practical sense – for example they suggest that rather than randomly
> selecting from users in the highest tier that instead you must select from
> all tiers and then upgrade those not on the highest tier.
> 3. In Sec. 3.9, it seems a bit too proscriptive on the (IXP) server
> locations – could be simplified to a regional peering location of the ISP
> network to allow for flexibility.
> 4. In Sec. 3.10, similarly proscriptive for example by defining the
> duration of a test as 10-15 seconds – what if they test can be completed
> just as accurately in 9 seconds? ISPs should have latitude to configure
> these tests & the state of the art is moving fast.
>
>
>
>
> _______________________________________________
> Nnagain mailing list
> Nnagain@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/nnagain
>

[-- Attachment #1.2: Type: text/html, Size: 3131 bytes --]

[-- Attachment #2: ControlChartConstantsAndFormulae.pdf --]
[-- Type: application/pdf, Size: 59376 bytes --]