SPCC: Testrun of Lc0 42741 finished

[sovaz1997]

you provide w=22 games
A child laughs

You have no understanding whatsoever and yet you criticize,
Someday you may have something useful to say
I do not hold my breath

No, you want to pretend to be noble, and protect the status quo.

And you want to ignore the facts, and misrepresent my testing by suggesting I have not tested NN engines and the best A/B engines in many thousands of games, and other testing like TCEC, and other testers.

You say there is no evidence that NN scale better the A/B engine when you know that this is a lie.

You will not answer how Lc0 dominates all A/B engines when not played at ridiculously fast time controls in a effort to gather 5000 low quality games in the shortest amount of time.

You are a pretender....

Your test does not say anything about Leela's level, because an insufficient number of games.

[Michel]

Leela is best. Any test that shows otherwise is flawed.

[zullil]

... played at ridiculously fast time controls in a effort to gather 5000 low quality games in the shortest amount of time.

I'll probably regret getting into this, but ...

I assume the reason for using fast time controls is to get enough games to reach a certain level of statistical significance. I doubt it's part of some widespread conspiracy to keep neural net engines down.

In the "Maybe I understand your frustration" category, I've long wondered if Stockfish would be even stronger on high-end hardware at long time controls if it were possible to use much longer time controls during Fishtest testing of patches. But it's not; the time required to reach a statistically sound conclusion would simply be too long. So we make due with a Stockfish whose search and evaluation are optimized to play very fast games.

[Modern Times]

I'll probably regret getting into this, but ...

As will I...…

But lets assume that NN engines do scale better than A/B engines (and I do think there is some evidence that that is the case) then is mwyoung suggesting that everyone adjust their testing conditions to show NN engines in the best possible light ? Some might say that is bias and favouritism. If NN engine authors want their engines to be #1 in all test conditions, then they know what they have to do - make improvements. If they don't care, then that is also fine. Focusing on events like TCEC is a good strategy. In a few years or a decade normal consumer hardware will catch up to that level anyway.