Komodo 8 vs Stockfish dev’s (sept14)

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
beram
Posts: 1187
Joined: Wed Jan 06, 2010 2:11 pm

Komodo 8 vs Stockfish dev’s (sept14)

Post by beram » Sat Sep 27, 2014 10:12 am

I have collected some recent test publications to bring some light on the strenght difference between Komodo 8 and the latest Stockfish dev versions (from sept2014)
Conclusion: the SFdev’s of sept14 score between 54 % and 57 % against Komodo 8, from blitz till LTC
See for test results:
LTC 150m30 s (6x) WSET (Sedat Canbaz and Tom Casanovas) - 55% for SF - 50 games
http://www.talkchess.com/forum/viewtopic.php?t=53582

TC 5m3s (1core) - Owl Chess - 55,5% for SF - 100 games
http://chessowl.blogspot.fr/2014/09/sto ... 8-x64.html

TC 2m2s and 4m0s (6x and 4x) - Tom Casanovas ongoing SFdev tests - 54,61 % for SF - 640 games avg
STOCKFISH 090914a MZ vs Komodo 8 (3266) = 51.87% 160 games;
STOCKFISH 140914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH 190914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH DEv 210914 vs Komodo 8 (3266) = 52.19% 160 games
http://www.talkchess.com/forum/viewtopi ... &start=360

TC 1min0s (6x), Bobby Johnsson – 57,1% for SF - 500games
1 Stockfish 2109 +50 +167/=237/-96 57.10% 285.5/500
2 Komodo 8 64-bit
http://www.talkchess.com/forum/viewtopic.php?t=53819

Hugo
Posts: 775
Joined: Tue Dec 01, 2009 10:10 am

Re: Komodo 8 vs Stockfish dev’s (sept14)

Post by Hugo » Sat Sep 27, 2014 11:00 am

Hello Sir :-)

I started 2 days ago a LTC match between SF from 23Sept and Komodo 8.
It will take some time, I get 8 games per day maximum.
conditions:
timer 40 mvs in 120 min + 60 min
12 cpu both engines, ponder OFF, 8Gb hashtables
5 moves book
shredder gui, 2 games at the same time (running on Xeon E5-2697 v2 server)
5 GB of syzygy bases
5 moves book, no repeated color openings

I hope I do not loose my mood or need PC for other stuff in the next 2 -3 weeks.

best regards

Clemens

beram
Posts: 1187
Joined: Wed Jan 06, 2010 2:11 pm

Re: Komodo 8 vs Stockfish dev’s (sept14)

Post by beram » Sat Sep 27, 2014 11:08 am

Thats nice Clemens,

What are the results sofar ?

kind regards Bram

Adam Hair
Posts: 3201
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: Komodo 8 vs Stockfish dev’s (sept14)

Post by Adam Hair » Sat Sep 27, 2014 11:30 am

beram wrote:I have collected some recent test publications to bring some light on the strenght difference between Komodo 8 and the latest Stockfish dev versions (from sept2014)
Conclusion: the SFdev’s of sept14 score between 54 % and 57 % against Komodo 8, from blitz till LTC
See for test results:
LTC 150m30 s (6x) WSET (Sedat Canbaz and Tom Casanovas) - 55% for SF - 50 games
http://www.talkchess.com/forum/viewtopic.php?t=53582

TC 5m3s (1core) - Owl Chess - 55,5% for SF - 100 games
http://chessowl.blogspot.fr/2014/09/sto ... 8-x64.html

TC 2m2s and 4m0s (6x and 4x) - Tom Casanovas ongoing SFdev tests - 54,61 % for SF - 640 games avg
STOCKFISH 090914a MZ vs Komodo 8 (3266) = 51.87% 160 games;
STOCKFISH 140914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH 190914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH DEv 210914 vs Komodo 8 (3266) = 52.19% 160 games
http://www.talkchess.com/forum/viewtopi ... &start=360

TC 1min0s (6x), Bobby Johnsson – 57,1% for SF - 500games
1 Stockfish 2109 +50 +167/=237/-96 57.10% 285.5/500
2 Komodo 8 64-bit
http://www.talkchess.com/forum/viewtopic.php?t=53819
I have tested Komodo 8 (Drawscore=0) and Stockfish 140919 at 40 moves/3 minutes (equivalent to CCRL 40/4) on a couple of different computers (Intel E8400 and Intel Xeon L5420), in Windows XP 64-bit and Linux Mint 15, using both the Linux and Windows compiles (including via Wine), and using various openings formats (Sedat's Perfect 2014t, Lyudmil Tsvetkov's 2 move pgn, and a subset of the 8 move pgn used to create the TCEC openings). The cumulative score was:

Code: Select all

   # PLAYER                     : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish 140919 64-bit    :   41.5    7.9   1985.0    3555   55.8%
   2 Komodo 8 64-bit            :    0.0   ----   1570.0    3555   44.2%

Hugo
Posts: 775
Joined: Tue Dec 01, 2009 10:10 am

Re: Komodo 8 vs Stockfish dev’s (sept14)

Post by Hugo » Sat Sep 27, 2014 12:27 pm

beram wrote:Thats nice Clemens,

What are the results sofar ?

kind regards Bram
tooo early ;-)
only 8 games finnished.


C.K.

Hugo
Posts: 775
Joined: Tue Dec 01, 2009 10:10 am

Re: Komodo 8 vs Stockfish dev’s (sept14)

Post by Hugo » Sat Sep 27, 2014 12:29 pm

Adam Hair wrote:
beram wrote:I have collected some recent test publications to bring some light on the strenght difference between Komodo 8 and the latest Stockfish dev versions (from sept2014)
Conclusion: the SFdev’s of sept14 score between 54 % and 57 % against Komodo 8, from blitz till LTC
See for test results:
LTC 150m30 s (6x) WSET (Sedat Canbaz and Tom Casanovas) - 55% for SF - 50 games
http://www.talkchess.com/forum/viewtopic.php?t=53582

TC 5m3s (1core) - Owl Chess - 55,5% for SF - 100 games
http://chessowl.blogspot.fr/2014/09/sto ... 8-x64.html

TC 2m2s and 4m0s (6x and 4x) - Tom Casanovas ongoing SFdev tests - 54,61 % for SF - 640 games avg
STOCKFISH 090914a MZ vs Komodo 8 (3266) = 51.87% 160 games;
STOCKFISH 140914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH 190914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH DEv 210914 vs Komodo 8 (3266) = 52.19% 160 games
http://www.talkchess.com/forum/viewtopi ... &start=360

TC 1min0s (6x), Bobby Johnsson – 57,1% for SF - 500games
1 Stockfish 2109 +50 +167/=237/-96 57.10% 285.5/500
2 Komodo 8 64-bit
http://www.talkchess.com/forum/viewtopic.php?t=53819
I have tested Komodo 8 (Drawscore=0) and Stockfish 140919 at 40 moves/3 minutes (equivalent to CCRL 40/4) on a couple of different computers (Intel E8400 and Intel Xeon L5420), in Windows XP 64-bit and Linux Mint 15, using both the Linux and Windows compiles (including via Wine), and using various openings formats (Sedat's Perfect 2014t, Lyudmil Tsvetkov's 2 move pgn, and a subset of the 8 move pgn used to create the TCEC openings). The cumulative score was:

Code: Select all

   # PLAYER                     : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish 140919 64-bit    :   41.5    7.9   1985.0    3555   55.8%
   2 Komodo 8 64-bit            :    0.0   ----   1570.0    3555   44.2%
Thanks Adam! quite intresting. Do you think drawscore Zero is no good setup?

C.K.

x3
Posts: 13
Joined: Mon Sep 08, 2014 4:54 pm

Re: Komodo 8 vs Stockfish dev’s (sept14)

Post by x3 » Sat Sep 27, 2014 12:31 pm

K8-SF 290714 3m+1s Book 8 moves repeating ponder off Syzygy 5
188,5-170,5 52,5% +17 elo xeon 16 core ht off

x3

User avatar
Leto
Posts: 2028
Joined: Thu May 04, 2006 1:40 am
Location: Dune

Re: Komodo 8 vs Stockfish dev’s (sept14)

Post by Leto » Sat Sep 27, 2014 8:30 pm

x3 wrote:K8-SF 290714 3m+1s Book 8 moves repeating ponder off Syzygy 5
188,5-170,5 52,5% +17 elo xeon 16 core ht off

x3
Thanks, although the July 29 version of Stockfish is a bit outdated by now.

Adam Hair
Posts: 3201
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: Komodo 8 vs Stockfish dev’s (sept14)

Post by Adam Hair » Sat Sep 27, 2014 8:51 pm

Hugo wrote:
Adam Hair wrote:
beram wrote:I have collected some recent test publications to bring some light on the strenght difference between Komodo 8 and the latest Stockfish dev versions (from sept2014)
Conclusion: the SFdev’s of sept14 score between 54 % and 57 % against Komodo 8, from blitz till LTC
See for test results:
LTC 150m30 s (6x) WSET (Sedat Canbaz and Tom Casanovas) - 55% for SF - 50 games
http://www.talkchess.com/forum/viewtopic.php?t=53582

TC 5m3s (1core) - Owl Chess - 55,5% for SF - 100 games
http://chessowl.blogspot.fr/2014/09/sto ... 8-x64.html

TC 2m2s and 4m0s (6x and 4x) - Tom Casanovas ongoing SFdev tests - 54,61 % for SF - 640 games avg
STOCKFISH 090914a MZ vs Komodo 8 (3266) = 51.87% 160 games;
STOCKFISH 140914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH 190914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH DEv 210914 vs Komodo 8 (3266) = 52.19% 160 games
http://www.talkchess.com/forum/viewtopi ... &start=360

TC 1min0s (6x), Bobby Johnsson – 57,1% for SF - 500games
1 Stockfish 2109 +50 +167/=237/-96 57.10% 285.5/500
2 Komodo 8 64-bit
http://www.talkchess.com/forum/viewtopic.php?t=53819
I have tested Komodo 8 (Drawscore=0) and Stockfish 140919 at 40 moves/3 minutes (equivalent to CCRL 40/4) on a couple of different computers (Intel E8400 and Intel Xeon L5420), in Windows XP 64-bit and Linux Mint 15, using both the Linux and Windows compiles (including via Wine), and using various openings formats (Sedat's Perfect 2014t, Lyudmil Tsvetkov's 2 move pgn, and a subset of the 8 move pgn used to create the TCEC openings). The cumulative score was:

Code: Select all

   # PLAYER                     : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish 140919 64-bit    :   41.5    7.9   1985.0    3555   55.8%
   2 Komodo 8 64-bit            :    0.0   ----   1570.0    3555   44.2%
Thanks Adam! quite intresting. Do you think drawscore Zero is no good setup?

C.K.
Hi Clemens,

I think that Drawscore=0 is the best setting to use against Stockfish.

shrapnel
Posts: 1248
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: Komodo 8 vs Stockfish dev’s (sept14)

Post by shrapnel » Sun Sep 28, 2014 5:22 am

Adam Hair wrote:I think that Drawscore=0 is the best setting to use against Stockfish.
+1.
No two ways about it ; even -1 doesn't cut it.
Also, the kind of tests that Beram is doing will always be favorable to Stockfish, no matter the drawscore.
I can confirm from my online games that Komodo 8 really comes into its own in really Long Time Control Games on really fast computers with loads of RAM. There is a dramatic rise in ELO, at a rate much higher than what one would see if Stockfish or Houdini 4 were given that benefit !
For this very reason, I predict that Komodo 8 will easily win the TCEC, especially since Stockfish won't have the Syzygy EGTB props, on which it heavily depends.
I'll post some games later to show what I mean.
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

Post Reply