Testing Komodo 9.1 12CPU for CEGT 40/4

[Leto]

900 games done, just 100 more to go. Here are the results so far:

match 1:

1 Komodo 9.1 64-bit +35 +46/=183/-21 55.00% 137.5/250
2 Stockfish 6 64 POPCNT 12 -35 +21/=183/-46 45.00% 112.5/250
(K9 12CPU scored 52% after 250 games)


gauntlet 1:

Komodo 9.1 64-bit - Critter 1.6 64-bit 4 42.0 - 8.0 +35/=14/-1 84.00%
Komodo 9.1 64-bit - Stockfish 6 64 POPCNT 8 28.0 - 22.0 +10/=36/-4 56.00%
Komodo 9.1 64-bit - Deep Rybka 4.1 SSE42 x64 12 43.5 - 6.5 +38/=11/-1 87.00%
Komodo 9.1 64-bit - Stockfish 5 64 SSE4.2 12 32.0 - 18.0 +16/=32/-2 64.00%
Komodo 9.1 64-bit - Stockfish 5 64 SSE4.2 1 37.0 - 13.0 +25/=24/-1 74.00%
Komodo 9.1 64-bit - Houdini 4 x64 12a 39.0 - 11.0 +31/=16/-3 78.00%


gauntlet 2:

Komodo 9.1 64-bit - Stockfish 6 64 POPCNT 4 30.0 - 20.0 +13/=34/-3 60.00%
Komodo 9.1 64-bit - Houdini 3 x64 12 33.5 - 16.5 +23/=21/-6 67.00%
Komodo 9.1 64-bit - Gull 3 x64 12 38.5 - 11.5 +28/=21/-1 77.00%
Komodo 9.1 64-bit - Houdini 4 x64 4a 37.5 - 12.5 +27/=21/-2 75.00%
Komodo 9.1 64-bit - Stockfish 6 64 POPCNT 1 35.0 - 15.0 +21/=28/-1 70.00%
Komodo 9.1 64-bit - Gull 3 x64 4 37.0 - 13.0 +26/=22/-2 74.00%

match 2:

1 Komodo 9.1 64-bit +123 +19/=29/-2 67.00% 33.5/50
2 Stockfish 5 64 SSE4.2 8 -123 +2/=29/-19 33.00% 16.5/50
(K9 12CPU scored 69% after 50 games)


Rating list will be updated within the coming weeks. Here's the link to the CEGT 40/4 rating list:
http://www.husvankempen.de/nunn/40_4_Ra ... liste.html

[Leto]

Match 3 and 4:

Komodo 9.1 64-bit - Houdini 4 Pro x64 8 40.0 - 10.0 +31/=18/-1 80.00%
Komodo 9.1 64-bit - Houdini 4 Pro x64 4a 38.5 - 11.5 +27/=23/-0 77.00%

and if my calculations are correct that'd put K9.1 12CPU at a performance of 71% against average opposition of 3261, 6% higher than K9.

[Laskos]

Match 3 and 4:

Komodo 9.1 64-bit - Houdini 4 Pro x64 8 40.0 - 10.0 +31/=18/-1 80.00%
Komodo 9.1 64-bit - Houdini 4 Pro x64 4a 38.5 - 11.5 +27/=23/-0 77.00%

and if my calculations are correct that'd put K9.1 12CPU at a performance of 71% against average opposition of 3261, 6% higher than K9.

Thanks, very important and conclusive tests. Seems close to 40 ELO points improvement over 9.0. Komodo 9.1 is a clear favorite to win the next TCEC.

[lkaufman]

Match 3 and 4:

Komodo 9.1 64-bit - Houdini 4 Pro x64 8 40.0 - 10.0 +31/=18/-1 80.00%
Komodo 9.1 64-bit - Houdini 4 Pro x64 4a 38.5 - 11.5 +27/=23/-0 77.00%

and if my calculations are correct that'd put K9.1 12CPU at a performance of 71% against average opposition of 3261, 6% higher than K9.

It is simply amazing to me that Komodo is beating Houdini 4 by 58 wins to one loss, draws excluded. I wouldn't have thought that even ten years from now a program would exist that could do so. The elo gains from version to version hide the true improvement, due to the number of draws.