AlphaZero Chess test - SF8 16 GB vs. 512 MB

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

fastgm
Posts: 818
Joined: Mon Aug 19, 2013 6:57 pm

AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by fastgm »

Conditions:
Dual AMD Opteron 6376, Cutechess-Cli, 100 games, fixed 60 seconds per move, opening book: Noomen_2-move_Testsuite.pgn, no adjudication rules, no tablebases

Stockfish 8, 31 threads, 16384 MB Hash
vs
Stockfish 8, 31 threads, 512 MB Hash

Code: Select all

1   Stockfish 8 T31 16384    +7  +3/=96/-1 51.00%   51.0/100
2   Stockfish 8 T31 512      -7  +1/=96/-3 49.00%   49.0/100
 
 Engine                 Depth       Time   Games     Moves  Average
--------------------------------------------------------------------
 Stockfish 8 T31 16384  42.11  114:02:17     100      7078    58.00
 Stockfish 8 T31 512    42.54  113:58:48     100      7076    57.99

    Program                  Elo    +   -   Games   Score   Av.Op.  Draws
 --------------------------------------------------------------------------
  1 Stockfish 8 T31 16384  : 3003   16  12   100    51.0 %   2997   96.0 %
  2 Stockfish 8 T31 512    : 2997   12  16   100    49.0 %   3003   96.0 %
 
Wins   = 3
Draws  = 96
Losses = 1
Av.Op. Elo = 3000
 
Result     : 51.0/100 (+3,=96,-1)
Perf.      : 51.0 %
Margins    :
 68 %      : (+  1.1,-  0.9 %) -> [ 50.1, 52.1 %]
 95 %      : (+  2.3,-  1.7 %) -> [ 49.3, 53.3 %]
 99.7 %    : (+  3.8,-  2.4 %) -> [ 48.6, 54.8 %]
 
Elo        : 3007
Margins    :
 68 %      : (+  8,-  6) -> [3001,3015]
 95 %      : (+ 16,- 12) -> [2995,3023]
 99.7 %    : (+ 26,- 16) -> [2990,3033]
 
Games        :    100 (finished)
 
White Wins   :      4 ( 4.0 %)
Black Wins   :      0 ( 0.0 %)
Draws        :     96 (96.0 %)
 
White Perf.  : 52.0 %
Black Perf.  : 48.0 %
 
Individual statistics:
 
1 Stockfish 8 T31 16384  : 3003  100 (+  3,= 96,-  1), 51.0 %
2 Stockfish 8 T31 512    : 2997  100 (+  1,= 96,-  3), 49.0 %
All games:
http://www.fastgm.de/schach/SF8-T31-163 ... ec-100.pgn
.
Modern Times
Posts: 3539
Joined: Thu Jun 07, 2012 11:02 pm

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by Modern Times »

I've never been a fan of these huge hash sizes. Perhaps they make sense for infinite analysis where in conjunction with large pages they give a boost. But I think maybe it also varies depending on CPU, motherboard and RAM architecture as well.
elcabesa
Posts: 855
Joined: Sun May 23, 2010 1:32 pm

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by elcabesa »

Can you try to see what happens with 21 core vs 16 core?
syzygy
Posts: 5555
Joined: Tue Feb 28, 2012 11:56 pm

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by syzygy »

So in a 100-game match played at 1 minute/move and with many cores, handicapping SF8 by giving it a small hash table does not "cripple" it.
User avatar
Ozymandias
Posts: 1532
Joined: Sun Oct 25, 2009 2:30 am

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by Ozymandias »

The only one single condition that "crippled" it, was the absence of a book.
Jouni
Posts: 3271
Joined: Wed Mar 08, 2006 8:15 pm

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by Jouni »

Really surprising! 128 GB SF limit is definitely no problem. TCEC should go to 1 GB for optimal play?
Jouni
User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by CMCanavessi »

Ozymandias wrote:The only one single condition that "crippled" it, was the absence of a book.
And TBs.
User avatar
Ozymandias
Posts: 1532
Joined: Sun Oct 25, 2009 2:30 am

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by Ozymandias »

CMCanavessi wrote:
Ozymandias wrote:The only one single condition that "crippled" it, was the absence of a book.
And TBs.
TBs have a bigger impact at ultra fast TCs. IIRC, this was 1min/move on good HW, so not exactly crippling.
User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by CMCanavessi »

Ozymandias wrote:
CMCanavessi wrote:
Ozymandias wrote:The only one single condition that "crippled" it, was the absence of a book.
And TBs.
TBs have a bigger impact at ultra fast TCs. IIRC, this was 1min/move on good HW, so not exactly crippling.
No, it's not crippling but you don't just have 1 cause, you have to add them all up.

1GB hash + no TBs + no book + 1 year old version + weird time control = cripple.

If you separate them, the effect is not that noticeable.
User avatar
Ozymandias
Posts: 1532
Joined: Sun Oct 25, 2009 2:30 am

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by Ozymandias »

CMCanavessi wrote:
Ozymandias wrote:
CMCanavessi wrote:
Ozymandias wrote:The only one single condition that "crippled" it, was the absence of a book.
And TBs.
TBs have a bigger impact at ultra fast TCs. IIRC, this was 1min/move on good HW, so not exactly crippling.
No, it's not crippling but you don't just have 1 cause, you have to add them all up.

1GB hash + no TBs + no book + 1 year old version + weird time control = cripple.

If you separate them, the effect is not that noticeable.
The book alone is noticeable.