Re: [Bloat] fcc request for standardized speed testing

[David Collier-Brown <davec-b@rogers.com>]

[-- Attachment #1: Type: text/plain, Size: 1868 bytes --]


On 12/19/24 11:42, Stephen Hemminger via Bloat wrote:
> On Thu, 19 Dec 2024 11:31:08 -0500
> David Collier-Brown via Bloat<bloat@lists.bufferbloat.net> wrote:
>
>> On 12/18/24 17:17, David Lang via Bloat wrote:
>>> so, what happens when a standardized test is mandated and then it's
>>> found that that test isn't as good as others?
>>>
>>> I'm leery of any government mandates.
>> Governments in general are good a "policing" things*, such as
>> deficiencies in specifications and persons trying to weasel around them.
>>
>> At the same time, good specifiers write in "or better" clauses so that
>> subsequent standards can be a few lines added to the original work.
>>
>> We can tell that is broken in Canada when the CRTC does a request for
>> comments ... but then rejects all the comments and proposed amendments.
>> Oh, and resists publishing them (:-))
>>
>> Have you seen that in the US?
>>
>> --dave
> And it will just create benchmark cheating...
> Look at any of the standardized database benchmarks as an example.
> The benchmark starts out trying to an express a workload; then the vendors
> discover new and creative ways to get higher numbers.
> _______________________________________________

Hmmn, and that's for non-mandated performance standards!

I'll speculate here that we have a "whack-a-mole" situation. No matter 
how many holes you fix, you can't make a benchmark fair.

The interesting question is if you can make a "not less than" rule 
system monotonically reduce the attack surface, instead of leaving it 
the same size or worse (:-))

I know we do that with case law (I'm a former Quicklaw nerd) but it can 
be /arbitrarily/ hard...

--dave

-- 
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
davecb@spamcop.net           |              -- Mark Twain

[-- Attachment #2: Type: text/html, Size: 2768 bytes --]