8 engine, 8CPU, octo roundrobin 40/40 tournament

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Dan Cooper
Posts: 184
Joined: Sun Nov 01, 2015 2:15 am

8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by Dan Cooper » Mon Dec 14, 2015 12:20 am

These are approximately the top 8 actively developed engines with Houdini and Gull thrown in as gatekeepers/benchmarks. SF-YBW representative included as well.

Nice result for the latest Andscacs so far.

Intel i7 5960X @ 4.5Ghz
8GB hash
Syzygy 6-man EGTB (Protector with 5-man Nalimov)
Ponder off
Barracuda 2015 book
40 moves in 40 minutes

Default UCI settings except increased move overhead for some engines and set Houdini contempt to 0.

Standings after round 14 of 56

Code: Select all

 RANK   ENGINE                                    GAMES  POINTS   1  2  3  4  5  6  7  8  
-------------------------------------------------------------------------------------------
   1.   KOMODO 9.3 64-BIT                           14     9.0    ** == == == 1= 1= =1 =1 
   2.   STOCKFISH 051215 64 BMI2                    14     8.5    == ** == == == =1 1= 1= 
   3.   STOCKFISH 070915 64 BMI2                    14     8.0    == == ** == == =1 == 1= 
   4.   HOUDINI 4 PRO X64                           14     7.5    == == == ** == == == 1= 
   5.   ANDSCACS 0.84                               14     6.5    0= == == == ** 01 == == 
   6.   PROTECTOR 1.9.0                             14     6.0    0= =0 =0 == 10 ** =1 == 
   7.   GULL 3 X64                                  14     5.5    =0 0= == == == =0 ** == 
   8.   NIRVANACHESS 2.2                            14     5.0    =0 0= 0= 0= == == == ** 

Total games = 56

ordo-win64.exe -a 3000 -A "Houdini 4 Pro x64" -W -p Top8_RoundRobin_e1e_Pgn.pgn -o Top8a.txt

   # PLAYER                      : RATING    POINTS  PLAYED    (%)
   1 Komodo 9.3 64-bit           : 3069.7       9.0      14   64.3%
   2 Stockfish 051215 64 BMI2    : 3046.0       8.5      14   60.7%
   3 Stockfish 070915 64 BMI2    : 3022.8       8.0      14   57.1%
   4 Houdini 4 Pro x64           : 3000.0       7.5      14   53.6%
   5 Andscacs 0.84               : 2954.7       6.5      14   46.4%
   6 Protector 1.9.0             : 2931.9       6.0      14   42.9%
   7 Gull 3 x64                  : 2908.7       5.5      14   39.3%
   8 Nirvanachess 2.2            : 2885.0       5.0      14   35.7%
Last edited by Dan Cooper on Mon Dec 14, 2015 12:24 am, edited 1 time in total.

Dan Cooper
Posts: 184
Joined: Sun Nov 01, 2015 2:15 am

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by Dan Cooper » Mon Dec 14, 2015 12:21 am

I'd like to run this again with new versions of Texel, Ginkgo, Fritz and Hannibal after this one concludes.

carldaman
Posts: 2094
Joined: Sat Jun 02, 2012 12:13 am

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by carldaman » Mon Dec 14, 2015 12:57 am

Hi Dan, I see 2 versions of Stockfish but no Fire. Could you also include Fritz/Rybka in the future? Just a friendly suggestion - this is your tournament, of course.

Dan Cooper wrote:These are approximately the top 8 actively developed engines with Houdini and Gull thrown in as gatekeepers/benchmarks. SF-YBW representative included as well.

Nice result for the latest Andscacs so far.

Intel i7 5960X @ 4.5Ghz
8GB hash
Syzygy 6-man EGTB (Protector with 5-man Nalimov)
Ponder off
Barracuda 2015 book
40 moves in 40 minutes

Default UCI settings except increased move overhead for some engines and set Houdini contempt to 0.

Standings after round 14 of 56

Code: Select all

 RANK   ENGINE                                    GAMES  POINTS   1  2  3  4  5  6  7  8  
-------------------------------------------------------------------------------------------
   1.   KOMODO 9.3 64-BIT                           14     9.0    ** == == == 1= 1= =1 =1 
   2.   STOCKFISH 051215 64 BMI2                    14     8.5    == ** == == == =1 1= 1= 
   3.   STOCKFISH 070915 64 BMI2                    14     8.0    == == ** == == =1 == 1= 
   4.   HOUDINI 4 PRO X64                           14     7.5    == == == ** == == == 1= 
   5.   ANDSCACS 0.84                               14     6.5    0= == == == ** 01 == == 
   6.   PROTECTOR 1.9.0                             14     6.0    0= =0 =0 == 10 ** =1 == 
   7.   GULL 3 X64                                  14     5.5    =0 0= == == == =0 ** == 
   8.   NIRVANACHESS 2.2                            14     5.0    =0 0= 0= 0= == == == ** 

Total games = 56

ordo-win64.exe -a 3000 -A "Houdini 4 Pro x64" -W -p Top8_RoundRobin_e1e_Pgn.pgn -o Top8a.txt

   # PLAYER                      : RATING    POINTS  PLAYED    (%)
   1 Komodo 9.3 64-bit           : 3069.7       9.0      14   64.3%
   2 Stockfish 051215 64 BMI2    : 3046.0       8.5      14   60.7%
   3 Stockfish 070915 64 BMI2    : 3022.8       8.0      14   57.1%
   4 Houdini 4 Pro x64           : 3000.0       7.5      14   53.6%
   5 Andscacs 0.84               : 2954.7       6.5      14   46.4%
   6 Protector 1.9.0             : 2931.9       6.0      14   42.9%
   7 Gull 3 x64                  : 2908.7       5.5      14   39.3%
   8 Nirvanachess 2.2            : 2885.0       5.0      14   35.7%

Dann Corbit
Posts: 12122
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by Dann Corbit » Mon Dec 14, 2015 3:22 am

I guess that the two SF versions are YBW and Lazy. I am curious to see the result.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

User avatar
cdani
Posts: 2198
Joined: Sat Jan 18, 2014 9:24 am
Location: Andorra
Contact:

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by cdani » Mon Dec 14, 2015 1:03 pm

Thanks!! Will see how this ends :-)

User avatar
Nathanael Russell
Posts: 803
Joined: Sat Jan 31, 2015 10:50 pm
Location: Philadelphia, USA
Contact:

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by Nathanael Russell » Mon Dec 14, 2015 2:23 pm

Dann Corbit wrote:I guess that the two SF versions are YBW and Lazy. I am curious to see the result.
Stockfish YBW - 291115
Stockfish Lazy - 021115
Core_Engine_Tester_CCRL
Image

Dann Corbit
Posts: 12122
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by Dann Corbit » Mon Dec 14, 2015 7:38 pm

YBW is newer than Lazy?
Can't be.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

User avatar
Nathanael Russell
Posts: 803
Joined: Sat Jan 31, 2015 10:50 pm
Location: Philadelphia, USA
Contact:

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by Nathanael Russell » Mon Dec 14, 2015 9:44 pm

Author: mbootsector
Date: Tue Oct 20 06:58:08 2015 +0200
Timestamp: 1445317088

Lazy SMP

Start all threads searching on root position and
use only the shared TT table as synching scheme.

It seems this scheme scales better than YBWC for
high number of threads.


Verified for nor regression at STC 3 threads
LLR: -2.95 (-2.94,2.94) [-3.00,1.00]
Total: 40232 W: 6908 L: 7130 D: 26194

Verified for nor regression at LTC 3 threads
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 28186 W: 3908 L: 3798 D: 20480

Verified for nor regression at STC 7 threads
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 3607 W: 674 L: 526 D: 2407

Verified for nor regression at LTC 7 threads
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 4235 W: 671 L: 528 D: 3036

Tested with fixed games at LTC with 20 threads
ELO: 44.75 +-7.6 (95%) LOS: 100.0%
Total: 2069 W: 407 L: 142 D: 1520

Tested with fixed games at XLTC (120secs) with 20 threads
ELO: 28.01 +-6.7 (95%) LOS: 100.0%
Total: 2275 W: 349 L: 166 D: 1760

Original patch of mbootsector, with additional work
from Ivan Ivec (log formula), Joerg Oster (id loop
simplification) and Marco Costalba (assorted formatting
and rework).

Bench: 8116244
I missed this update on http://abrok.eu/stockfish/?page=2. I wasn't sure if this patch was merged permanently with the master branch. Thanks.
Core_Engine_Tester_CCRL
Image

Dan Cooper
Posts: 184
Joined: Sun Nov 01, 2015 2:15 am

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by Dan Cooper » Wed Dec 16, 2015 5:55 pm

Standings after round 21 of 56.

Code: Select all

 RANK   ENGINE                                    GAMES  POINTS   1   2   3   4   5   6   7   8    
--------------------------------------------------------------------------------------------------
   1.   KOMODO 9.3 64-BIT                           21    13.5    --- === === ==1 =1= 1=1 =1= 1==
   2.   STOCKFISH 070915 64 BMI2                    21    13.0    === --- === === === ==1 1=1 =11
   3.   STOCKFISH 051215 64 BMI2                    21    13.0    === === --- === 1=1 === 1== =11
   4.   HOUDINI 4 PRO X64                           21    10.5    ==0 === === --- === === 1== ===
   5.   GULL 3 X64                                  21     9.0    =0= === 0=0 === --- === === =01
   6.   ANDSCACS 0.84                               21     9.0    0=0 ==0 === === === --- === 01=
   7.   NIRVANACHESS 2.2                            21     8.0    =0= 0=0 0== 0== === === --- ===
   8.   PROTECTOR 1.9.0                             21     8.0    0== =00 =00 === =10 10= === ---   

ordo-win64.exe -a 3000 -A "Houdini 4 Pro x64" -W -p Top8_RoundRobin_e1e_Pgn.pgn -o Top8a.txt 

   # PLAYER                      : RATING    POINTS  PLAYED    (%)
   1 Komodo 9.3 64-bit           : 3093.0      13.5      21   64.3%
   2 Stockfish 051215 64 BMI2    : 3077.1      13.0      21   61.9%
   3 Stockfish 070915 64 BMI2    : 3077.0      13.0      21   61.9%
   4 Houdini 4 Pro x64           : 3000.0      10.5      21   50.0%
   5 Andscacs 0.84               : 2960.2       9.0      21   42.9%
   6 Gull 3 x64                  : 2954.4       9.0      21   42.9%
   7 Protector 1.9.0             : 2923.2       8.0      21   38.1%
   8 Nirvanachess 2.2            : 2922.9       8.0      21   38.1%

jdart
Posts: 4115
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by jdart » Wed Dec 16, 2015 6:45 pm

What kind of book is the "Barracuda 2015 book?"

--Jon

Post Reply