SPCC: Testrun of Stockfish 240413 finished

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

User avatar
pohl4711
Posts: 2471
Joined: Sat Sep 03, 2011 7:25 am
Location: Berlin, Germany
Full name: Stefan Pohl

SPCC: Testrun of Stockfish 240413 finished

Post by pohl4711 »

My UHO-Top15 Ratinglist is the world's first engine-ratinglist, using UHO-openings, and the world's first ratinglist offering additionally Gamepair-statistics.

Ratinglist-testrun of Stockfish 240413 finished.

https://www.sp-cc.de

Also take a look at the EAS-Ratinglist, the world's first engine-ratinglist not measuring strength of engines but engines's style of play:
https://www.sp-cc.de/eas-ratinglist.htm


(Perhaps you have to clear your browsercache (press STRG+SHIFT+DEL) or reload the website))
Jouni
Posts: 3320
Joined: Wed Mar 08, 2006 8:15 pm

Re: SPCC: Testrun of Stockfish 240413 finished

Post by Jouni »

This bigger net clearly needs couple months of tuning and/or Epyc CPU :) .
Jouni
Jouni
Posts: 3320
Joined: Wed Mar 08, 2006 8:15 pm

Re: SPCC: Testrun of Stockfish 240413 finished

Post by Jouni »

Jouni
User avatar
pohl4711
Posts: 2471
Joined: Sat Sep 03, 2011 7:25 am
Location: Berlin, Germany
Full name: Stefan Pohl

Re: SPCC: Testrun of Stockfish 240413 finished

Post by pohl4711 »

Jouni wrote: Sat Apr 20, 2024 9:37 am https://tests.stockfishchess.org/tests/ ... e4cefc1490

+7 Elo patch :P .
Wow. You are right, this is an impressive patch... Only 12936 games to pass in LTC. This is really, really rare.
But mention, +7 Elo in 60sec+600ms selfplay means only +3 Elo or so in my ratinglist-testrun...

Commit finny-tables (take 2)
Info LTC: take 2
Submitter gabe
TC 60+0.6
SPRT elo0: 0.50 alpha: 0.05 elo1: 2.50 beta: 0.05 (normalized)
LLR 2.95 [-2.94,2.94] (accepted)
Elo 7.04 [4.20,9.89] (95%)
LOS 100.0%
Games 12936 [w:26.2%, l:24.1%, d:49.8%]
Pentanomial [7, 1278, 3633, 1540, 10]


It is patch to speedup the nnue using. It is not new as far as I can read here. But Stockfish did not have finny-tables until now.
https://github.com/official-stockfish/S ... nt-9169650
Uri Blass
Posts: 10418
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: SPCC: Testrun of Stockfish 240413 finished

Post by Uri Blass »

pohl4711 wrote: Sat Apr 20, 2024 9:55 am
Jouni wrote: Sat Apr 20, 2024 9:37 am https://tests.stockfishchess.org/tests/ ... e4cefc1490

+7 Elo patch :P .
Wow. You are right, this is an impressive patch... Only 12936 games to pass in LTC. This is really, really rare.
But mention, +7 Elo in 60sec+600ms selfplay means only +3 Elo or so in my ratinglist-testrun...

Commit finny-tables (take 2)
Info LTC: take 2
Submitter gabe
TC 60+0.6
SPRT elo0: 0.50 alpha: 0.05 elo1: 2.50 beta: 0.05 (normalized)
LLR 2.95 [-2.94,2.94] (accepted)
Elo 7.04 [4.20,9.89] (95%)
LOS 100.0%
Games 12936 [w:26.2%, l:24.1%, d:49.8%]
Pentanomial [7, 1278, 3633, 1540, 10]


It is patch to speedup the nnue using. It is not new as far as I can read here. But Stockfish did not have finny-tables until now.
https://github.com/official-stockfish/S ... nt-9169650
You do not have a good estimate for rating improvement from SPRT tests.
The only way to have a good estimate is to use fixed number of games.
Jouni
Posts: 3320
Joined: Wed Mar 08, 2006 8:15 pm

Re: SPCC: Testrun of Stockfish 240413 finished

Post by Jouni »

Nice speedup also:

Code: Select all

Result of  10 runs
==================
base (./stockfish.master       ) =     905097  +/- 9483
test (./stockfish.patch        ) =     970304  +/- 7068
diff                             =     +65207  +/- 7312

speedup        = +0.0720
P(speedup > 0) =  1.0000
But no pull request yet.
Jouni
Viz
Posts: 81
Joined: Tue Apr 09, 2024 6:24 am
Full name: Michael Chaly

Re: SPCC: Testrun of Stockfish 240413 finished

Post by Viz »

Not "also", this patch is a pure speedup.
Should translate quite well into rating lists ofc.
Jouni
Posts: 3320
Joined: Wed Mar 08, 2006 8:15 pm

Re: SPCC: Testrun of Stockfish 240413 finished

Post by Jouni »

For some reason this was not merged in Stockfish dev-20240421-d47aa639.
Jouni
Viz
Posts: 81
Joined: Tue Apr 09, 2024 6:24 am
Full name: Michael Chaly

Re: SPCC: Testrun of Stockfish 240413 finished

Post by Viz »

Jouni wrote: Mon Apr 22, 2024 1:50 pm For some reason this was not merged in Stockfish dev-20240421-d47aa639.
Had some crashes with windows clang 16 compiler (fixed now) + code style is reformatted to look like everything else, will take some time.