Is this SF NN almost like 20 MB book?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, Dann Corbit, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Alayan
Posts: 489
Joined: Tue Nov 19, 2019 7:48 pm
Full name: Alayan Feh

Re: Is this SF NN almost like 20 MB book?

Post by Alayan » Wed Aug 05, 2020 5:18 pm

An interesting test to run would be to measure result spread at different time conditions.

Have a 500 openings book (1000 games match) and run 100 (or more) matches at 10s+0.1, 20s+0.2, 1m+0.6s, 3m+1.8s, 10m+6s. Use a few threads to ensure some SMP variation. Self-play so that the theoretical elo difference should be 0 in each match is probably the soundest way to proceed.

Then analyze the data. Higher draw rate should by itself ensure a lower variance in the results, but I'd expect that even after correcting for the impact of draw rate (an easy trick would be to rerun the ultra-bullet test with a different contempt value that produce a similar draw rate to the rapid test) we'll measure a variance reduction.

From this perspective, while the TCEC superfinal is extremely small sample size (only 100 games) and while the results of individual games could definitely flip if they were run again, the overall results aren't that unreliable.

Post Reply