On 12/19/24 11:42, Stephen Hemminger via Bloat wrote: > On Thu, 19 Dec 2024 11:31:08 -0500 > David Collier-Brown via Bloat wrote: > >> On 12/18/24 17:17, David Lang via Bloat wrote: >>> so, what happens when a standardized test is mandated and then it's >>> found that that test isn't as good as others? >>> >>> I'm leery of any government mandates. >> Governments in general are good a "policing" things*, such as >> deficiencies in specifications and persons trying to weasel around them. >> >> At the same time, good specifiers write in "or better" clauses so that >> subsequent standards can be a few lines added to the original work. >> >> We can tell that is broken in Canada when the CRTC does a request for >> comments ... but then rejects all the comments and proposed amendments. >> Oh, and resists publishing them (:-)) >> >> Have you seen that in the US? >> >> --dave > And it will just create benchmark cheating... > Look at any of the standardized database benchmarks as an example. > The benchmark starts out trying to an express a workload; then the vendors > discover new and creative ways to get higher numbers. > _______________________________________________ Hmmn, and that's for non-mandated performance standards! I'll speculate here that we have a "whack-a-mole" situation. No matter how many holes you fix, you can't make a benchmark fair. The interesting question is if you can make a "not less than" rule system monotonically reduce the attack surface, instead of leaving it the same size or worse (:-)) I know we do that with case law (I'm a former Quicklaw nerd) but it can be /arbitrarily/ hard... --dave -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest davecb@spamcop.net | -- Mark Twain