Stockfish 10 64-bit Gauntlet for CCRL 40/40

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
Graham Banks
Posts: 32636
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Stockfish 10 64-bit Gauntlet for CCRL 40/40

Post by Graham Banks » Mon Dec 31, 2018 8:35 pm

My email addresses:
gbanksnz at gmail.com
gbanksnz at yahoo.co.nz

User avatar
Graham Banks
Posts: 32636
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Re: Stockfish 10 64-bit Gauntlet for CCRL 40/40

Post by Graham Banks » Mon Dec 31, 2018 8:40 pm

Code: Select all

CCRL 40/40 Rating List - Custom engine selection
952777 games played by 2318 programs, run by 22 testers
Ponder off, General books (up to 12 moves), 3-4-5 piece EGTB
Time control: Equivalent to 40 moves in 40 minutes on Athlon 64 X2 4600+ (2.4 GHz), about 15 minutes on a modern Intel CPU.
Computed on December 29, 2018 with Bayeselo based on 952'777 games
Tested by CCRL team, 2005-2018, http://computerchess.org.uk/ccrl/4040/

Rank               Engine                Elo   +    -   Score  AvOp  Games
1 Stockfish 10 64-bit                3381  +15  -15  76.7% -183.8  1742
  Stockfish 9 64-bit                 3367  +12  -12  72.2% -155.3  2495
  Stockfish 8 64-bit                 3301  +11  -11  69.9% -131.1  2670
  Stockfish 7 64-bit                 3245  +13  -13  67.9% -112.6  1809
  Stockfish 6 64-bit                 3225  +15  -15  68.7% -117.8  1461
  Stockfish 5 64-bit                 3193  +16  -16  67.5% -107.2  1222
  Stockfish DD 64-bit                3164  +16  -15  63.3%  -79.8  1257
  Stockfish 4 64-bit                 3122  +18  -18  59.7%  -59.4   915
  Stockfish 3 64-bit                 3097  +16  -16  56.5%  -39.4  1186
  Stockfish 2.3.1 64-bit             3078  +17  -17  57.1%  -43.7  1065
  Stockfish 2.2.2 64-bit             3073  +20  -20  54.5%  -31.9   792
  Stockfish 2.2 64-bit               3064  +36  -36  57.2%  -43.9   222
  Stockfish 2.1.1 64-bit             3054  +15  -15  51.2%   -6.9  1258
  Stockfish 2.2.2 32-bit             3051  +14  -14  61.9%  -77.6  1591
  Stockfish 2.3.1 32-bit             3040  +17  -17  57.2%  -44.8  1095
  Stockfish 1.9.1 64-bit             3036  +21  -21  60.6%  -72.0   733
  Stockfish 2.0.1 64-bit             3031  +23  -22  57.7%  -48.6   611
  Stockfish 1.8 64-bit               3028  +22  -22  62.7%  -87.4   703
  Stockfish 2.1.1 32-bit             3028  +16  -16  63.8%  -92.7  1362
  Stockfish 2.0.1 32-bit             3013  +18  -17  66.9% -109.4  1074
  Stockfish 1.7.1 64-bit             3009  +22  -22  61.3%  -74.6   669
  Stockfish 1.9.1 32-bit             3000  +17  -17  67.3% -111.3  1124
  Stockfish 1.8 32-bit               2987  +18  -18  67.5% -110.0  1020
  Stockfish 1.7.1 32-bit             2973  +17  -17  67.1% -112.5  1140
  Stockfish 1.6.3 64-bit             2965  +20  -20  58.6%  -54.9   801
  Stockfish 1.6s 64-bit              2952  +29  -29  52.0%  -14.8   346
  Stockfish 1.6.3 32-bit             2933  +17  -17  61.4%  -72.8  1084
  Stockfish 1.5.1 64-bit             2888  +28  -28  54.7%  -29.6   416
  Stockfish 1.5.1 32-bit             2865  +15  -15  55.4%  -35.5  1486
  Stockfish 1.4 64-bit               2849  +19  -19  53.1%  -17.9   844
  Stockfish 1.4 32-bit               2830  +16  -17  53.9%  -25.6  1120
  Stockfish 1.3.1 32-bit             2793  +25  -25  46.1%  +25.3   512
  Stockfish 1.2 Default              2780  +22  -22  45.5%  +27.8   663
  Stockfish 1.01                     2754  +32  -32  52.5%  -19.7   302
My email addresses:
gbanksnz at gmail.com
gbanksnz at yahoo.co.nz

Chris Formula
Posts: 4
Joined: Sun Aug 21, 2016 5:59 am

Re: Stockfish 10 64-bit Gauntlet for CCRL 40/40

Post by Chris Formula » Wed Jan 09, 2019 5:58 am

Great work Graham! Any plan testing Leela Chess Zero v0.20.1?

User avatar
Graham Banks
Posts: 32636
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Re: Stockfish 10 64-bit Gauntlet for CCRL 40/40

Post by Graham Banks » Wed Jan 09, 2019 6:17 am

Chris Formula wrote:
Wed Jan 09, 2019 5:58 am
Great work Graham! Any plan testing Leela Chess Zero v0.20.1?
It's possible. :)
My email addresses:
gbanksnz at gmail.com
gbanksnz at yahoo.co.nz

Chessqueen
Posts: 502
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: Stockfish 10 64-bit Gauntlet for CCRL 40/40

Post by Chessqueen » Sat Jan 12, 2019 12:17 am

Chris Formula wrote:
Wed Jan 09, 2019 5:58 am
Great work Graham! Any plan testing Leela Chess Zero v0.20.1?
Here is a small match from Youtube
https://www.youtube.com/watch?v=_3n1iwetA2Y

ernest
Posts: 1833
Joined: Wed Mar 08, 2006 7:30 pm

Re: Stockfish 10 64-bit Gauntlet for CCRL 40/40

Post by ernest » Sat Jan 12, 2019 12:44 am

Graham Banks wrote:
Mon Dec 31, 2018 8:40 pm

Code: Select all

CCRL 40/40 Rating List - Custom engine selection
952777 games played by 2318 programs, run by 22 testers
Ponder off, General books (up to 12 moves), 3-4-5 piece EGTB
Time control: Equivalent to 40 moves in 40 minutes on Athlon 64 X2 4600+ (2.4 GHz), about 15 minutes on a modern Intel CPU.
Computed on December 29, 2018 with Bayeselo based on 952'777 games
Tested by CCRL team, 2005-2018, http://computerchess.org.uk/ccrl/4040/

Rank               Engine                Elo   +    -   Score  AvOp  Games
1 Stockfish 10 64-bit                3381  +15  -15  76.7% -183.8  1742
  Stockfish 9 64-bit                 3367  +12  -12  72.2% -155.3  2495
  Stockfish 8 64-bit                 3301  +11  -11  69.9% -131.1  2670
  Stockfish 7 64-bit                 3245  +13  -13  67.9% -112.6  1809
  Stockfish 6 64-bit                 3225  +15  -15  68.7% -117.8  1461
  Stockfish 5 64-bit                 3193  +16  -16  67.5% -107.2  1222
  Stockfish DD 64-bit                3164  +16  -15  63.3%  -79.8  1257
  Stockfish 4 64-bit                 3122  +18  -18  59.7%  -59.4   915
  Stockfish 3 64-bit                 3097  +16  -16  56.5%  -39.4  1186
  Stockfish 2.3.1 64-bit             3078  +17  -17  57.1%  -43.7  1065
  Stockfish 2.2.2 64-bit             3073  +20  -20  54.5%  -31.9   792
  Stockfish 2.2 64-bit               3064  +36  -36  57.2%  -43.9   222
  Stockfish 2.1.1 64-bit             3054  +15  -15  51.2%   -6.9  1258
  Stockfish 2.2.2 32-bit             3051  +14  -14  61.9%  -77.6  1591
  Stockfish 2.3.1 32-bit             3040  +17  -17  57.2%  -44.8  1095
  Stockfish 1.9.1 64-bit             3036  +21  -21  60.6%  -72.0   733
  Stockfish 2.0.1 64-bit             3031  +23  -22  57.7%  -48.6   611
  Stockfish 1.8 64-bit               3028  +22  -22  62.7%  -87.4   703
  Stockfish 2.1.1 32-bit             3028  +16  -16  63.8%  -92.7  1362
  Stockfish 2.0.1 32-bit             3013  +18  -17  66.9% -109.4  1074
  Stockfish 1.7.1 64-bit             3009  +22  -22  61.3%  -74.6   669
  Stockfish 1.9.1 32-bit             3000  +17  -17  67.3% -111.3  1124
  Stockfish 1.8 32-bit               2987  +18  -18  67.5% -110.0  1020
  Stockfish 1.7.1 32-bit             2973  +17  -17  67.1% -112.5  1140
  Stockfish 1.6.3 64-bit             2965  +20  -20  58.6%  -54.9   801
  Stockfish 1.6s 64-bit              2952  +29  -29  52.0%  -14.8   346
  Stockfish 1.6.3 32-bit             2933  +17  -17  61.4%  -72.8  1084
  Stockfish 1.5.1 64-bit             2888  +28  -28  54.7%  -29.6   416
  Stockfish 1.5.1 32-bit             2865  +15  -15  55.4%  -35.5  1486
  Stockfish 1.4 64-bit               2849  +19  -19  53.1%  -17.9   844
  Stockfish 1.4 32-bit               2830  +16  -17  53.9%  -25.6  1120
  Stockfish 1.3.1 32-bit             2793  +25  -25  46.1%  +25.3   512
  Stockfish 1.2 Default              2780  +22  -22  45.5%  +27.8   663
  Stockfish 1.01                     2754  +32  -32  52.5%  -19.7   302
Strange...

Only +14 Elo SF10/SF9 ??? :shock:

tpoppins
Posts: 919
Joined: Tue Nov 24, 2015 8:11 pm
Location: upstate

Re: Stockfish 10 64-bit Gauntlet for CCRL 40/40

Post by tpoppins » Mon Jan 14, 2019 5:38 am

Given the error margins and limited development time (compared to the v8-v9 cycle) hardly strange.
Tirsa Poppins
CCRL

ernest
Posts: 1833
Joined: Wed Mar 08, 2006 7:30 pm

Re: Stockfish 10 64-bit Gauntlet for CCRL 40/40

Post by ernest » Mon Jan 14, 2019 10:03 pm

tpoppins wrote:
Mon Jan 14, 2019 5:38 am
Given the error margins and limited development time (compared to the v8-v9 cycle) hardly strange.
Thanks for commenting ! :D

But precisely, the error margins make it hard to allow the +14 Elo to reach the announced +50 Elo...
(1% to 2% probability, maybe)

And perhaps CCRL is finding that the +50 Elo is simply not true !

Jouni
Posts: 1953
Joined: Wed Mar 08, 2006 7:15 pm

Re: Stockfish 10 64-bit Gauntlet for CCRL 40/40

Post by Jouni » Tue Jan 15, 2019 9:08 am

Here's rating difference SF10 - SF9 from four lists (single core):

FastGM 41/53/47
Pohl 57
CEGT 59/43
CCRL 16/15
Jouni

Modern Times
Posts: 2319
Joined: Thu Jun 07, 2012 9:02 pm

Re: Stockfish 10 64-bit Gauntlet for CCRL 40/40

Post by Modern Times » Tue Jan 15, 2019 11:08 pm

I got +52 Elo at chess960, but it is so seriously strong there is a lack of opponents for it.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

Post Reply