Testing Komodo 5.1 MP - 480 GAMES

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Testing Komodo 5.1 MP - 480 GAMES

Post by Tomcass »

480 GAMES KOMODO 5.1 MP (Not own book)

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 4 min +0 sec/ game
No tablebases

201306_Komodo5.1MPx6_nob__4+0_120games 2013

Komodo 5.1 64-bit_x6_nob - Critter 1.6 64-bitx6_nob 20.5 - 19.5 +9/=23/-8 51.25%
Komodo 5.1 64-bit_x6_nob - Deep Rybka 4.1 SSE42 x64 (x6) 23.5 - 16.5 +13/=21/-6 58.75%
Komodo 5.1 64-bit_x6_nob - Houdini 3 Pro x64_6 11.5 - 28.5 +5/=13/-22 28.75%

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 2min +2 sec/ game
No tablebases

201306_Komodo5.1MPx6_nob__2+2_120games 2013

Komodo 5.1 64-bit_x6_nob - Critter 1.6 64-bitx6_nob 21.5 - 18.5 +12/=19/-9 53.75%
Komodo 5.1 64-bit_x6_nob - Deep Rybka 4.1 SSE42 x64 (x6) 19.5 - 20.5 +10/=19/-11 48.75%
Komodo 5.1 64-bit_x6_nob - Houdini 3 Pro x64_6 14.0 - 26.0 +4/=20/-16 35.00%

X6 240 games= http://www.mediafire.com/?q4ut3aazc0pgeyz

Overall average with 6 cores= 110.5-129.5 (46.04%)

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 4 min + 0 sec/ game
Ponder: Off
No tablebases

20130618_Komodo5.1MPnob_4+0_x4-1 2013

Komodo 5.1 64-bitx4nob - Critter 1.6a 64-bitnob_4 20.0 - 20.0 +8/=24/-8 50.00%
Komodo 5.1 64-bitx4nob - Deep Rybka 4 x64 (x4) 23.0 - 17.0 +9/=28/-3 57.50%
Komodo 5.1 64-bitx4nob - Houdini 3 Pro x64_4 20.0 - 20.0 +11/=18/-11 50.00%

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 2 min + 2 sec/ game
Ponder: Off
No tablebases

20130618_Komodo5.1MPnob_2+2_x4 2013

Komodo 5.1 64-bitx4nob - Critter 1.6a 64-bitnob_4 24.0 - 16.0 +12/=24/-4 60.00%
Komodo 5.1 64-bitx4nob - Deep Rybka 4 x64 (x4) 23.0 - 17.0 +12/=22/-6 57.50%
Komodo 5.1 64-bitx4nob - Houdini 3 Pro x64_4 17.5 - 22.5 +6/=23/-11 43.75%

X4 240 games = http://www.mediafire.com/?3tj1opeex0ty5rj

Overall average with 4 cores= 127.5 – 112.5 (53.12%)


Global Score: 238.0-242.0 = 49.58%
Against Critter 1.6: 53.75% Deep Rybka 4: 55.62% Houdini 3.0: 39.37%
Estimated ELO Performance: 3.091

Regards from Barcelona.

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Komodo 5.1 MP - 480 GAMES

Post by Tomcass »

TESTING KOMODO 5.1r1 MP = 480 games

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 4 min +0 sec/ game
No tablebases

201306_Komodo5.1r1MPx6_nob__4+0_120game 2013

Komodo 5.1r1 64-bitx6nob - Critter 1.6 64-bitx6_nob 23.5 - 16.5 +13/=21/-6 58.75%
Komodo 5.1r1 64-bitx6nob - Deep Rybka 4.1 SSE42 x64 (x6) 23.5 - 16.5 +14/=19/-7 58.75%
Komodo 5.1r1 64-bitx6nob - Houdini 3 Pro x64_6 18.5 - 21.5 +8/=21/-11 46.25%


I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 2min +2 sec/ game
No tablebases

201306_Komodo5.1r1MPx6_nob__2+2_120game 2013

Komodo 5.1r1 64-bitx6nob - Critter 1.6 64-bitx6_nob 18.5 - 21.5 +8/=21/-11 46.25%
Komodo 5.1r1 64-bitx6nob - Deep Rybka 4.1 SSE42 x64 (x6) 24.5 - 15.5 +19/=11/-10 61.25%
Komodo 5.1r1 64-bitx6nob - Houdini 3 Pro x64_6 17.0 - 23.0 +8/=18/-14 42.50%

X6 240 GAMES = http://www.mediafire.com/?e5qzl68uf5c6ob6
Overall average with 6 cores: (125.5-114.5 ) = 52.29%


i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 4 min + 0 sec/ game
Ponder: Off
No tablebases

Komodo 5.1r1 64-bitx4nob - Critter 1.6a 64-bitnob_4 18.5 - 21.5 +4/=29/-7 46.25%
Komodo 5.1r1 64-bitx4nob - Deep Rybka 4 x64 (x4) 28.0 - 12.0 +20/=16/-4 70.00%
Komodo 5.1r1 64-bitx4nob - Houdini 3 Pro x64_4 17.5 - 22.5 +6/=23/-11 43.75%

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 2 min + 2 sec/ game
Ponder: Off
201306_Komodo5.1r1MPx6_nob__2+2_120game 2013

Komodo 5.1r1 64-bitx4nob - Critter 1.6a 64-bitnob_4 24.5 - 15.5 +14/=21/-5 61.25%
Komodo 5.1r1 64-bitx4nob - Deep Rybka 4 x64 (x4) 21.5 - 18.5 +9/=25/-6 53.75%
Komodo 5.1r1 64-bitx4nob - Houdini 3 Pro x64_4 21.0 - 19.0 +11/=20/-9 52.50%


X4 240 GAMES = http://www.mediafire.com/?4ckg1obkn4ui75p
Overall average with 4 cores: (131.0-109.0) = 54.58%

Global Score: 256.5 – 223.5 = 53.44%

Against Critter 1.6: 53.12% Deep Rybka 4: 60.94% Houdini 3.0: 46.25% Estimated ELO Performance: 3118

Accortding to my tests, this result is around:

56 ELO points below Houdini 3.0 Pro
15 ELO points better than the latest Stockfish development version tested
27 ELO point better than Komodo 5.1MP (Before the latest upgrade to 5.1r1MP)

Regards from Barcelona.

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Komodo 5.1 MP - 480 GAMES

Post by Tomcass »

TESTING KOMODO 5.1r2 = 480 GAMES

7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 4 min +0 sec/ game
No tablebases
Relative Speed: 28.66
Knodes per second: 13.759

Time control 4+0

Komodo 5.1r2 64-bitx6nob - Critter 1.6a 64-bitnob_6 24.5 - 15.5 +13/=23/-4 61.25%
Komodo 5.1r2 64-bitx6nob - Deep Rybka 4 x64 (x6) 23.5-16.5 +13/=21/-6 58.75%
Komodo 5.1r2 64-bitx6nob - Houdini 3 Pro x64_6 20.0 - 20.0 +13/=14/-13 50.00%

Time control 2+2

Komodo 5.1r2 64-bitx6nob - Critter 1.6 64-bitx6_nob 23.0-17.0 +14/=18/-8 57.50%
Komodo 5.1r2 64-bitx6nob - Deep Rybka 4.1 SSE42 x64 25.0-15.0 +14/=22/-4 62.50%
Komodo 5.1r2 64-bitx6nob - Houdini 3 Pro x64_6 19.0-21.0 +10/=18/-12 47.50%

240 games x6 cores= http://www.mediafire.com/?afk4zux7x4e1o2f
Global Score with 6 cores= 135.0 – 105.0 = 56.25%


i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Ponder: Off
No tablebases
Relative Speed: 20.62
Knodes per second: 9.899

Time control 4+0

Komodo 5.1r2 64-bitnob - Critter 1.6a 64-bitnob_4 18.0 - 22.0 +4/=28/-8 45.00%
Komodo 5.1r2 64-bitnob - Deep Rybka 4 x64 (x4) 28.0 - 12.0 +19/=18/-3 70.00%
Komodo 5.1r2 64-bitnob - Houdini 3 Pro x64_4 16.0 - 24.0 +6/=20/-14 40.00%

Time control 2+2

Komodo 5.1r2 64-bitnob - Critter 1.6a 64-bitnob_4 24.0 - 16.0 +13/=22/-5 60.00%
Komodo 5.1r2 64-bitnob - Deep Rybka 4 x64 (x4) 23.0 - 17.0 +11/=24/-5 57.50%
Komodo 5.1r2 64-bitnob - Houdini 3 Pro x64_4 21.5 - 18.5 +9/=25/-6 53.75%

240 games x 4 cores= http://www.mediafire.com/?goron510kjhzitp

Global Score with 4 cores = 130.5 – 109.5 = 54.37%

Global Score: = 265.5 – 214.5 = 55.31%
Against Critter 1.6: 55.94% Deep Rybka 4: 62.19% Houdini 3.0: 47.81%
Estimated ELO Performance: 3131

--------------------------------------------------------------

Only as a reference, this was the score for Komodo 5.1r1 in the same 480 games test:

Global Score: 256.5 – 223.5 = 53.44%
Against Critter 1.6: 53.12% Deep Rybka 4: 60.94% Houdini 3.0: 46.25%
Estimated ELO Performance: 3118

According with my tests, the improvement is 13 Elo points for Komodo 5.1r2 over Komodo 5.1r1 version. Only 41 Elo points below Houdini 3.0 Pro (3172). Excellent!.

Regards from Barcelona.

Tom.
Top
Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Re: Testing Komodo 5.1 MP - 480 GAMES

Post by Sven »

Thanks for your nice test. It would be interesting for me to see the ratings + error bars for the four different TC/core configurations. If I did not make a mistake then the overall scores for Komodo were:

Code: Select all

1) 4+0 (6 cores)

Komodo 5.1 64-bit_x6_nob  55.5 - 64.5
Komodo 5.1r1 64-bitx6nob  65.5 - 54.5
Komodo 5.1r2 64-bitx6nob  68.0 - 52.0

2) 2+2 (6 cores)

Komodo 5.1 64-bit_x6_nob  54.0 - 66.0
Komodo 5.1r1 64-bitx6nob  60.0 - 60.0
Komodo 5.1r2 64-bitx6nob  67.0 - 53.0

3) 4+0 (4 cores)

Komodo 5.1 64-bit_x6_nob  63.0 - 57.0
Komodo 5.1r1 64-bitx6nob  64.0 - 56.0
Komodo 5.1r2 64-bitx6nob  62.0 - 58.0

4) 2+2 (4 cores)

Komodo 5.1 64-bit_x6_nob  64.5 - 55.5
Komodo 5.1r1 64-bitx6nob  67.0 - 53.0
Komodo 5.1r2 64-bitx6nob  68.5 - 51.5
and depending on the error bars this *might* be a very small hint that the improvement 5.1->r1->r2 is bigger with 6 cores than with 4.

Sven
Top

Return to “Computer Chess Club: Tournaments and Matches”