Stockfish 211013 syzy vs default

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

Jouni
Posts: 3279
Joined: Wed Mar 08, 2006 8:15 pm

Stockfish 211013 syzy vs default

Post by Jouni »

I tested effect of 5 piece syzygy tables (on harddrive). 150 positions from EG_MSb.epd with average 1 sec for move. Result:

1 Stockfish 211013 64 syz +83/=161/-56 54.50% 163.5/300
2 Stockfish 211013 64 def +56/=161/-83 45.50% 136.5/300

+31 ELO - great work Ronald!
Jouni
Vinvin
Posts: 5228
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: Stockfish 211013 syzy vs default

Post by Vinvin »

Great !

I think the effect is even bigger with more time.
Could you test with 4 seconds instead of 1 ?
Vinvin
Posts: 5228
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: Stockfish 211013 syzy vs default

Post by Vinvin »

Please, post the games.
ernest
Posts: 2041
Joined: Wed Mar 08, 2006 8:30 pm

Re: Stockfish 211013 syzy vs default

Post by ernest »

Jouni wrote:+31 ELO
± 27.5 (2SD error bar)
Andre
Posts: 98
Joined: Thu Jul 23, 2009 5:40 am

Re: Stockfish 211013 syzy vs default

Post by Andre »

Now an interesting info would be to know the difference between official Stockfish 4.0 vs. Stockfish 211013
Jouni
Posts: 3279
Joined: Wed Mar 08, 2006 8:15 pm

Re: Stockfish 211013 syzy vs default

Post by Jouni »

I repeated test against Houdini 3 with surprise. SF score has been from start 52 % = +16 ELO. But old SF dev version (no bases) got 70-78 % in same test !! Really surprising. Yes I know You need 20k games.
Jouni
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Stockfish 211013 syzy vs default

Post by Laskos »

Jouni wrote:I repeated test against Houdini 3 with surprise. SF score has been from start 52 % = +16 ELO. But old SF dev version (no bases) got 70-78 % in same test !! Really surprising. Yes I know You need 20k games.
Use SPRT with cutechess-cli, it will say how many games you need, for large enough ELO differences, often no more than several hundred.