AlphaZero Chess test - SF8 16 GB vs. 512 MB

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
fastgm
Posts: 391
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by fastgm » Sun Jan 07, 2018 12:19 pm

Conditions:
Dual AMD Opteron 6376, Cutechess-Cli, 100 games, fixed 60 seconds per move, opening book: Noomen_2-move_Testsuite.pgn, no adjudication rules, no tablebases

Stockfish 8, 31 threads, 16384 MB Hash
vs
Stockfish 8, 31 threads, 512 MB Hash

Code: Select all

1   Stockfish 8 T31 16384    +7  +3/=96/-1 51.00%   51.0/100
2   Stockfish 8 T31 512      -7  +1/=96/-3 49.00%   49.0/100
 
 Engine                 Depth       Time   Games     Moves  Average
--------------------------------------------------------------------
 Stockfish 8 T31 16384  42.11  114:02:17     100      7078    58.00
 Stockfish 8 T31 512    42.54  113:58:48     100      7076    57.99

    Program                  Elo    +   -   Games   Score   Av.Op.  Draws
 --------------------------------------------------------------------------
  1 Stockfish 8 T31 16384  : 3003   16  12   100    51.0 %   2997   96.0 %
  2 Stockfish 8 T31 512    : 2997   12  16   100    49.0 %   3003   96.0 %
 
Wins   = 3
Draws  = 96
Losses = 1
Av.Op. Elo = 3000
 
Result     : 51.0/100 (+3,=96,-1)
Perf.      : 51.0 %
Margins    :
 68 %      : (+  1.1,-  0.9 %) -> [ 50.1, 52.1 %]
 95 %      : (+  2.3,-  1.7 %) -> [ 49.3, 53.3 %]
 99.7 %    : (+  3.8,-  2.4 %) -> [ 48.6, 54.8 %]
 
Elo        : 3007
Margins    :
 68 %      : (+  8,-  6) -> [3001,3015]
 95 %      : (+ 16,- 12) -> [2995,3023]
 99.7 %    : (+ 26,- 16) -> [2990,3033]
 
Games        :    100 (finished)
 
White Wins   :      4 ( 4.0 %)
Black Wins   :      0 ( 0.0 %)
Draws        :     96 (96.0 %)
 
White Perf.  : 52.0 %
Black Perf.  : 48.0 %
 
Individual statistics:
 
1 Stockfish 8 T31 16384  : 3003  100 (+  3,= 96,-  1), 51.0 %
2 Stockfish 8 T31 512    : 2997  100 (+  1,= 96,-  3), 49.0 %
All games:
http://www.fastgm.de/schach/SF8-T31-163 ... ec-100.pgn
.

Modern Times
Posts: 2319
Joined: Thu Jun 07, 2012 9:02 pm

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by Modern Times » Sun Jan 07, 2018 1:08 pm

I've never been a fan of these huge hash sizes. Perhaps they make sense for infinite analysis where in conjunction with large pages they give a boost. But I think maybe it also varies depending on CPU, motherboard and RAM architecture as well.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

elcabesa
Posts: 803
Joined: Sun May 23, 2010 11:32 am
Contact:

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by elcabesa » Sun Jan 07, 2018 1:29 pm

Can you try to see what happens with 21 core vs 16 core?

syzygy
Posts: 4425
Joined: Tue Feb 28, 2012 10:56 pm

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by syzygy » Sun Jan 07, 2018 1:51 pm

So in a 100-game match played at 1 minute/move and with many cores, handicapping SF8 by giving it a small hash table does not "cripple" it.

User avatar
Ozymandias
Posts: 1073
Joined: Sun Oct 25, 2009 12:30 am

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by Ozymandias » Mon Jan 08, 2018 6:55 am

The only one single condition that "crippled" it, was the absence of a book.

Jouni
Posts: 1953
Joined: Wed Mar 08, 2006 7:15 pm

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by Jouni » Mon Jan 08, 2018 1:13 pm

Really surprising! 128 GB SF limit is definitely no problem. TCEC should go to 1 GB for optimal play?
Jouni

User avatar
CMCanavessi
Posts: 832
Joined: Thu Dec 28, 2017 3:06 pm
Location: Argentina

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by CMCanavessi » Mon Jan 08, 2018 3:18 pm

Ozymandias wrote:The only one single condition that "crippled" it, was the absence of a book.
And TBs.

User avatar
Ozymandias
Posts: 1073
Joined: Sun Oct 25, 2009 12:30 am

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by Ozymandias » Mon Jan 08, 2018 3:44 pm

CMCanavessi wrote:
Ozymandias wrote:The only one single condition that "crippled" it, was the absence of a book.
And TBs.
TBs have a bigger impact at ultra fast TCs. IIRC, this was 1min/move on good HW, so not exactly crippling.

User avatar
CMCanavessi
Posts: 832
Joined: Thu Dec 28, 2017 3:06 pm
Location: Argentina

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by CMCanavessi » Mon Jan 08, 2018 4:48 pm

Ozymandias wrote:
CMCanavessi wrote:
Ozymandias wrote:The only one single condition that "crippled" it, was the absence of a book.
And TBs.
TBs have a bigger impact at ultra fast TCs. IIRC, this was 1min/move on good HW, so not exactly crippling.
No, it's not crippling but you don't just have 1 cause, you have to add them all up.

1GB hash + no TBs + no book + 1 year old version + weird time control = cripple.

If you separate them, the effect is not that noticeable.

User avatar
Ozymandias
Posts: 1073
Joined: Sun Oct 25, 2009 12:30 am

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Post by Ozymandias » Mon Jan 08, 2018 6:49 pm

CMCanavessi wrote:
Ozymandias wrote:
CMCanavessi wrote:
Ozymandias wrote:The only one single condition that "crippled" it, was the absence of a book.
And TBs.
TBs have a bigger impact at ultra fast TCs. IIRC, this was 1min/move on good HW, so not exactly crippling.
No, it's not crippling but you don't just have 1 cause, you have to add them all up.

1GB hash + no TBs + no book + 1 year old version + weird time control = cripple.

If you separate them, the effect is not that noticeable.
The book alone is noticeable.

Post Reply