Komodo 8 vs Stockfish dev’s (sept14)

[beram]

I have collected some recent test publications to bring some light on the strenght difference between Komodo 8 and the latest Stockfish dev versions (from sept2014)
Conclusion: the SFdev’s of sept14 score between 54 % and 57 % against Komodo 8, from blitz till LTC
See for test results:
LTC 150m30 s (6x) WSET (Sedat Canbaz and Tom Casanovas) - 55% for SF - 50 games
http://www.talkchess.com/forum/viewtopic.php?t=53582

TC 5m3s (1core) - Owl Chess - 55,5% for SF - 100 games
http://chessowl.blogspot.fr/2014/09/sto ... 8-x64.html

TC 2m2s and 4m0s (6x and 4x) - Tom Casanovas ongoing SFdev tests - 54,61 % for SF - 640 games avg
STOCKFISH 090914a MZ vs Komodo 8 (3266) = 51.87% 160 games;
STOCKFISH 140914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH 190914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH DEv 210914 vs Komodo 8 (3266) = 52.19% 160 games
http://www.talkchess.com/forum/viewtopi ... &start=360

TC 1min0s (6x), Bobby Johnsson – 57,1% for SF - 500games
1 Stockfish 2109 +50 +167/=237/-96 57.10% 285.5/500
2 Komodo 8 64-bit
http://www.talkchess.com/forum/viewtopic.php?t=53819

[Hugo]

Hello Sir

I started 2 days ago a LTC match between SF from 23Sept and Komodo 8.
It will take some time, I get 8 games per day maximum.
conditions:
timer 40 mvs in 120 min + 60 min
12 cpu both engines, ponder OFF, 8Gb hashtables
5 moves book
shredder gui, 2 games at the same time (running on Xeon E5-2697 v2 server)
5 GB of syzygy bases
5 moves book, no repeated color openings

I hope I do not loose my mood or need PC for other stuff in the next 2 -3 weeks.

best regards

Clemens

[beram]

Thats nice Clemens,

What are the results sofar ?

kind regards Bram

[Adam Hair]

I have collected some recent test publications to bring some light on the strenght difference between Komodo 8 and the latest Stockfish dev versions (from sept2014)
Conclusion: the SFdev’s of sept14 score between 54 % and 57 % against Komodo 8, from blitz till LTC
See for test results:
LTC 150m30 s (6x) WSET (Sedat Canbaz and Tom Casanovas) - 55% for SF - 50 games
http://www.talkchess.com/forum/viewtopic.php?t=53582

TC 5m3s (1core) - Owl Chess - 55,5% for SF - 100 games
http://chessowl.blogspot.fr/2014/09/sto ... 8-x64.html

TC 2m2s and 4m0s (6x and 4x) - Tom Casanovas ongoing SFdev tests - 54,61 % for SF - 640 games avg
STOCKFISH 090914a MZ vs Komodo 8 (3266) = 51.87% 160 games;
STOCKFISH 140914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH 190914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH DEv 210914 vs Komodo 8 (3266) = 52.19% 160 games
http://www.talkchess.com/forum/viewtopi ... &start=360

TC 1min0s (6x), Bobby Johnsson – 57,1% for SF - 500games
1 Stockfish 2109 +50 +167/=237/-96 57.10% 285.5/500
2 Komodo 8 64-bit
http://www.talkchess.com/forum/viewtopic.php?t=53819

I have tested Komodo 8 (Drawscore=0) and Stockfish 140919 at 40 moves/3 minutes (equivalent to CCRL 40/4) on a couple of different computers (Intel E8400 and Intel Xeon L5420), in Windows XP 64-bit and Linux Mint 15, using both the Linux and Windows compiles (including via Wine), and using various openings formats (Sedat's Perfect 2014t, Lyudmil Tsvetkov's 2 move pgn, and a subset of the 8 move pgn used to create the TCEC openings). The cumulative score was:

Code: Select all

   # PLAYER                     : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish 140919 64-bit    :   41.5    7.9   1985.0    3555   55.8%
   2 Komodo 8 64-bit            :    0.0   ----   1570.0    3555   44.2%

[Hugo]

Thats nice Clemens,

What are the results sofar ?

kind regards Bram

tooo early
only 8 games finnished.


C.K.

[Hugo]

I have collected some recent test publications to bring some light on the strenght difference between Komodo 8 and the latest Stockfish dev versions (from sept2014)
Conclusion: the SFdev’s of sept14 score between 54 % and 57 % against Komodo 8, from blitz till LTC
See for test results:
LTC 150m30 s (6x) WSET (Sedat Canbaz and Tom Casanovas) - 55% for SF - 50 games
http://www.talkchess.com/forum/viewtopic.php?t=53582

TC 5m3s (1core) - Owl Chess - 55,5% for SF - 100 games
http://chessowl.blogspot.fr/2014/09/sto ... 8-x64.html

TC 2m2s and 4m0s (6x and 4x) - Tom Casanovas ongoing SFdev tests - 54,61 % for SF - 640 games avg
STOCKFISH 090914a MZ vs Komodo 8 (3266) = 51.87% 160 games;
STOCKFISH 140914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH 190914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH DEv 210914 vs Komodo 8 (3266) = 52.19% 160 games
http://www.talkchess.com/forum/viewtopi ... &start=360

TC 1min0s (6x), Bobby Johnsson – 57,1% for SF - 500games
1 Stockfish 2109 +50 +167/=237/-96 57.10% 285.5/500
2 Komodo 8 64-bit
http://www.talkchess.com/forum/viewtopic.php?t=53819

I have tested Komodo 8 (Drawscore=0) and Stockfish 140919 at 40 moves/3 minutes (equivalent to CCRL 40/4) on a couple of different computers (Intel E8400 and Intel Xeon L5420), in Windows XP 64-bit and Linux Mint 15, using both the Linux and Windows compiles (including via Wine), and using various openings formats (Sedat's Perfect 2014t, Lyudmil Tsvetkov's 2 move pgn, and a subset of the 8 move pgn used to create the TCEC openings). The cumulative score was:

Code: Select all

   # PLAYER                     : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish 140919 64-bit    :   41.5    7.9   1985.0    3555   55.8%
   2 Komodo 8 64-bit            :    0.0   ----   1570.0    3555   44.2%

Thanks Adam! quite intresting. Do you think drawscore Zero is no good setup?

C.K.

[x3]

K8-SF 290714 3m+1s Book 8 moves repeating ponder off Syzygy 5
188,5-170,5 52,5% +17 elo xeon 16 core ht off

x3

[Leto]

K8-SF 290714 3m+1s Book 8 moves repeating ponder off Syzygy 5
188,5-170,5 52,5% +17 elo xeon 16 core ht off

x3

Thanks, although the July 29 version of Stockfish is a bit outdated by now.

[Adam Hair]

I have collected some recent test publications to bring some light on the strenght difference between Komodo 8 and the latest Stockfish dev versions (from sept2014)
Conclusion: the SFdev’s of sept14 score between 54 % and 57 % against Komodo 8, from blitz till LTC
See for test results:
LTC 150m30 s (6x) WSET (Sedat Canbaz and Tom Casanovas) - 55% for SF - 50 games
http://www.talkchess.com/forum/viewtopic.php?t=53582

TC 5m3s (1core) - Owl Chess - 55,5% for SF - 100 games
http://chessowl.blogspot.fr/2014/09/sto ... 8-x64.html

TC 2m2s and 4m0s (6x and 4x) - Tom Casanovas ongoing SFdev tests - 54,61 % for SF - 640 games avg
STOCKFISH 090914a MZ vs Komodo 8 (3266) = 51.87% 160 games;
STOCKFISH 140914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH 190914 MZ vs Komodo 8 (3266) = 57.19% 160 games;
STOCKFISH DEv 210914 vs Komodo 8 (3266) = 52.19% 160 games
http://www.talkchess.com/forum/viewtopi ... &start=360

TC 1min0s (6x), Bobby Johnsson – 57,1% for SF - 500games
1 Stockfish 2109 +50 +167/=237/-96 57.10% 285.5/500
2 Komodo 8 64-bit
http://www.talkchess.com/forum/viewtopic.php?t=53819

I have tested Komodo 8 (Drawscore=0) and Stockfish 140919 at 40 moves/3 minutes (equivalent to CCRL 40/4) on a couple of different computers (Intel E8400 and Intel Xeon L5420), in Windows XP 64-bit and Linux Mint 15, using both the Linux and Windows compiles (including via Wine), and using various openings formats (Sedat's Perfect 2014t, Lyudmil Tsvetkov's 2 move pgn, and a subset of the 8 move pgn used to create the TCEC openings). The cumulative score was:

Code: Select all

   # PLAYER                     : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish 140919 64-bit    :   41.5    7.9   1985.0    3555   55.8%
   2 Komodo 8 64-bit            :    0.0   ----   1570.0    3555   44.2%

Thanks Adam! quite intresting. Do you think drawscore Zero is no good setup?

C.K.

Hi Clemens,

I think that Drawscore=0 is the best setting to use against Stockfish.

[shrapnel]

I think that Drawscore=0 is the best setting to use against Stockfish.

+1.
No two ways about it ; even -1 doesn't cut it.
Also, the kind of tests that Beram is doing will always be favorable to Stockfish, no matter the drawscore.
I can confirm from my online games that Komodo 8 really comes into its own in really Long Time Control Games on really fast computers with loads of RAM. There is a dramatic rise in ELO, at a rate much higher than what one would see if Stockfish or Houdini 4 were given that benefit !
For this very reason, I predict that Komodo 8 will easily win the TCEC, especially since Stockfish won't have the Syzygy EGTB props, on which it heavily depends.
I'll post some games later to show what I mean.