8 engine, 8CPU, octo roundrobin 40/40 tournament

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

Dan Cooper
Posts: 184
Joined: Sun Nov 01, 2015 3:15 am

8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by Dan Cooper »

These are approximately the top 8 actively developed engines with Houdini and Gull thrown in as gatekeepers/benchmarks. SF-YBW representative included as well.

Nice result for the latest Andscacs so far.

Intel i7 5960X @ 4.5Ghz
8GB hash
Syzygy 6-man EGTB (Protector with 5-man Nalimov)
Ponder off
Barracuda 2015 book
40 moves in 40 minutes

Default UCI settings except increased move overhead for some engines and set Houdini contempt to 0.

Standings after round 14 of 56

Code: Select all

 RANK   ENGINE                                    GAMES  POINTS   1  2  3  4  5  6  7  8  
-------------------------------------------------------------------------------------------
   1.   KOMODO 9.3 64-BIT                           14     9.0    ** == == == 1= 1= =1 =1 
   2.   STOCKFISH 051215 64 BMI2                    14     8.5    == ** == == == =1 1= 1= 
   3.   STOCKFISH 070915 64 BMI2                    14     8.0    == == ** == == =1 == 1= 
   4.   HOUDINI 4 PRO X64                           14     7.5    == == == ** == == == 1= 
   5.   ANDSCACS 0.84                               14     6.5    0= == == == ** 01 == == 
   6.   PROTECTOR 1.9.0                             14     6.0    0= =0 =0 == 10 ** =1 == 
   7.   GULL 3 X64                                  14     5.5    =0 0= == == == =0 ** == 
   8.   NIRVANACHESS 2.2                            14     5.0    =0 0= 0= 0= == == == ** 

Total games = 56

ordo-win64.exe -a 3000 -A "Houdini 4 Pro x64" -W -p Top8_RoundRobin_e1e_Pgn.pgn -o Top8a.txt

   # PLAYER                      : RATING    POINTS  PLAYED    (%)
   1 Komodo 9.3 64-bit           : 3069.7       9.0      14   64.3%
   2 Stockfish 051215 64 BMI2    : 3046.0       8.5      14   60.7%
   3 Stockfish 070915 64 BMI2    : 3022.8       8.0      14   57.1%
   4 Houdini 4 Pro x64           : 3000.0       7.5      14   53.6%
   5 Andscacs 0.84               : 2954.7       6.5      14   46.4%
   6 Protector 1.9.0             : 2931.9       6.0      14   42.9%
   7 Gull 3 x64                  : 2908.7       5.5      14   39.3%
   8 Nirvanachess 2.2            : 2885.0       5.0      14   35.7%
Last edited by Dan Cooper on Mon Dec 14, 2015 1:24 am, edited 1 time in total.
Top
Dan Cooper
Posts: 184
Joined: Sun Nov 01, 2015 3:15 am

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by Dan Cooper »

I'd like to run this again with new versions of Texel, Ginkgo, Fritz and Hannibal after this one concludes.
Top
carldaman
Posts: 2283
Joined: Sat Jun 02, 2012 2:13 am

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by carldaman »

Hi Dan, I see 2 versions of Stockfish but no Fire. Could you also include Fritz/Rybka in the future? Just a friendly suggestion - this is your tournament, of course.

Dan Cooper wrote:These are approximately the top 8 actively developed engines with Houdini and Gull thrown in as gatekeepers/benchmarks. SF-YBW representative included as well.

Nice result for the latest Andscacs so far.

Intel i7 5960X @ 4.5Ghz
8GB hash
Syzygy 6-man EGTB (Protector with 5-man Nalimov)
Ponder off
Barracuda 2015 book
40 moves in 40 minutes

Default UCI settings except increased move overhead for some engines and set Houdini contempt to 0.

Standings after round 14 of 56

Code: Select all

 RANK   ENGINE                                    GAMES  POINTS   1  2  3  4  5  6  7  8  
-------------------------------------------------------------------------------------------
   1.   KOMODO 9.3 64-BIT                           14     9.0    ** == == == 1= 1= =1 =1 
   2.   STOCKFISH 051215 64 BMI2                    14     8.5    == ** == == == =1 1= 1= 
   3.   STOCKFISH 070915 64 BMI2                    14     8.0    == == ** == == =1 == 1= 
   4.   HOUDINI 4 PRO X64                           14     7.5    == == == ** == == == 1= 
   5.   ANDSCACS 0.84                               14     6.5    0= == == == ** 01 == == 
   6.   PROTECTOR 1.9.0                             14     6.0    0= =0 =0 == 10 ** =1 == 
   7.   GULL 3 X64                                  14     5.5    =0 0= == == == =0 ** == 
   8.   NIRVANACHESS 2.2                            14     5.0    =0 0= 0= 0= == == == ** 

Total games = 56

ordo-win64.exe -a 3000 -A "Houdini 4 Pro x64" -W -p Top8_RoundRobin_e1e_Pgn.pgn -o Top8a.txt

   # PLAYER                      : RATING    POINTS  PLAYED    (%)
   1 Komodo 9.3 64-bit           : 3069.7       9.0      14   64.3%
   2 Stockfish 051215 64 BMI2    : 3046.0       8.5      14   60.7%
   3 Stockfish 070915 64 BMI2    : 3022.8       8.0      14   57.1%
   4 Houdini 4 Pro x64           : 3000.0       7.5      14   53.6%
   5 Andscacs 0.84               : 2954.7       6.5      14   46.4%
   6 Protector 1.9.0             : 2931.9       6.0      14   42.9%
   7 Gull 3 x64                  : 2908.7       5.5      14   39.3%
   8 Nirvanachess 2.2            : 2885.0       5.0      14   35.7%
Top
Dann Corbit
Posts: 12540
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by Dann Corbit »

I guess that the two SF versions are YBW and Lazy. I am curious to see the result.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Top
User avatar
cdani
Posts: 2204
Joined: Sat Jan 18, 2014 10:24 am
Location: Andorra

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by cdani »

Thanks!! Will see how this ends :-)
Top
User avatar
Nathanael Russell
Posts: 803
Joined: Sat Jan 31, 2015 11:50 pm
Location: Philadelphia, USA

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by Nathanael Russell »

Dann Corbit wrote:I guess that the two SF versions are YBW and Lazy. I am curious to see the result.
Stockfish YBW - 291115
Stockfish Lazy - 021115
Core_Engine_Tester_CCRL
Image
Top
Dann Corbit
Posts: 12540
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by Dann Corbit »

YBW is newer than Lazy?
Can't be.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Top
User avatar
Nathanael Russell
Posts: 803
Joined: Sat Jan 31, 2015 11:50 pm
Location: Philadelphia, USA

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by Nathanael Russell »

Author: mbootsector
Date: Tue Oct 20 06:58:08 2015 +0200
Timestamp: 1445317088

Lazy SMP

Start all threads searching on root position and
use only the shared TT table as synching scheme.

It seems this scheme scales better than YBWC for
high number of threads.

Verified for nor regression at STC 3 threads
LLR: -2.95 (-2.94,2.94) [-3.00,1.00]
Total: 40232 W: 6908 L: 7130 D: 26194

Verified for nor regression at LTC 3 threads
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 28186 W: 3908 L: 3798 D: 20480

Verified for nor regression at STC 7 threads
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 3607 W: 674 L: 526 D: 2407

Verified for nor regression at LTC 7 threads
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 4235 W: 671 L: 528 D: 3036

Tested with fixed games at LTC with 20 threads
ELO: 44.75 +-7.6 (95%) LOS: 100.0%
Total: 2069 W: 407 L: 142 D: 1520

Tested with fixed games at XLTC (120secs) with 20 threads
ELO: 28.01 +-6.7 (95%) LOS: 100.0%
Total: 2275 W: 349 L: 166 D: 1760

Original patch of mbootsector, with additional work
from Ivan Ivec (log formula), Joerg Oster (id loop
simplification) and Marco Costalba (assorted formatting
and rework).

Bench: 8116244
I missed this update on http://abrok.eu/stockfish/?page=2. I wasn't sure if this patch was merged permanently with the master branch. Thanks.
Core_Engine_Tester_CCRL
Image
Top
Dan Cooper
Posts: 184
Joined: Sun Nov 01, 2015 3:15 am

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by Dan Cooper »

Standings after round 21 of 56.

Code: Select all

 RANK   ENGINE                                    GAMES  POINTS   1   2   3   4   5   6   7   8    
--------------------------------------------------------------------------------------------------
   1.   KOMODO 9.3 64-BIT                           21    13.5    --- === === ==1 =1= 1=1 =1= 1==
   2.   STOCKFISH 070915 64 BMI2                    21    13.0    === --- === === === ==1 1=1 =11
   3.   STOCKFISH 051215 64 BMI2                    21    13.0    === === --- === 1=1 === 1== =11
   4.   HOUDINI 4 PRO X64                           21    10.5    ==0 === === --- === === 1== ===
   5.   GULL 3 X64                                  21     9.0    =0= === 0=0 === --- === === =01
   6.   ANDSCACS 0.84                               21     9.0    0=0 ==0 === === === --- === 01=
   7.   NIRVANACHESS 2.2                            21     8.0    =0= 0=0 0== 0== === === --- ===
   8.   PROTECTOR 1.9.0                             21     8.0    0== =00 =00 === =10 10= === ---   

ordo-win64.exe -a 3000 -A "Houdini 4 Pro x64" -W -p Top8_RoundRobin_e1e_Pgn.pgn -o Top8a.txt 

   # PLAYER                      : RATING    POINTS  PLAYED    (%)
   1 Komodo 9.3 64-bit           : 3093.0      13.5      21   64.3%
   2 Stockfish 051215 64 BMI2    : 3077.1      13.0      21   61.9%
   3 Stockfish 070915 64 BMI2    : 3077.0      13.0      21   61.9%
   4 Houdini 4 Pro x64           : 3000.0      10.5      21   50.0%
   5 Andscacs 0.84               : 2960.2       9.0      21   42.9%
   6 Gull 3 x64                  : 2954.4       9.0      21   42.9%
   7 Protector 1.9.0             : 2923.2       8.0      21   38.1%
   8 Nirvanachess 2.2            : 2922.9       8.0      21   38.1%
Top
jdart
Posts: 4366
Joined: Fri Mar 10, 2006 5:23 am
Location: http://www.arasanchess.org

Re: 8 engine, 8CPU, octo roundrobin 40/40 tournament

Post by jdart »

What kind of book is the "Barracuda 2015 book?"

--Jon
Top

Return to “Computer Chess Club: Tournaments and Matches”