60 games Komodo 5 against Top4 at 120m+3s

[TimoK]

I've started a new series of testing quite similar to my former "Clash of the Titans" match at longer TCs:

Komodo vs.
- Houdini: http://www.team-oh.de/livegames2/
- Stockfish: http://www.team-oh.de/livegames3/
- Critter: http://www.team-oh.de/livegames4/
- Rybka: http://www.team-oh.de/livegames5/

All engines use 1 core (AMD Phenom II @ 3GHz), 512MB Hash, 120m+3s, Ponder=On, 30 Openings (Noomen Test Suite 2012) have to be played with both colors.

Let's see how the new Komodo 5 fights the others compared to my first test: http://www.team-oh.de/Computerschach/Clash.htm

Best regards
Timo

P.S.: I know that the test sample is of no statistical relevance (only 4 x 60 games), but it should give us a first glance of how Komodo 5 behaves at longer TCs against top opponents.

It is possible to download the games using "games.pgn" added to the above links. E.g. http://www.team-oh.de/livegames2/games.pgn downloads the games from the Komodo vs. Houdini match.

[gleperlier]

Very very nice tournaments !

Gab

[jdart]

Interesting endgame position:

[D] 5nkb/1r2p2p/3p2pP/2pP2P1/1pK2P2/1P5B/2PB4/4R3 w - - 0 62

Komodo played Be6+ with a +1.7 score. Houdini didn't see this, apparently.

(Komodo5-Houdini game 13).

--Jon

[zullil]

Interesting endgame position:

[D] 5nkb/1r2p2p/3p2pP/2pP2P1/1pK2P2/1P5B/2PB4/4R3 w - - 0 62

Komodo played Be6+ with a +1.7 score. Houdini didn't see this, apparently.

(Komodo5-Houdini game 13).

--Jon

Nice find for a single core engine. Here's Critter's search:

Code: Select all

Critter v1.6a 64-bit, by Richard Vida
hardware POPCNT enabled
setboard 5nkb/1r2p2p/3p2pP/2pP2P1/1pK2P2/1P5B/2PB4/4R3 w - - 0 62
info string Opening book: "book.cbk" - 57853 entries [native]
info string GTB Init OK (5 piece set)
info string Using 8 thread(s), 2048 MB hash LP=off

    A   B   C   D   E   F   G   H
  +---+---+---+---+---+---+---+---+
8 |   |   |   |   |   | n | k | b |
  +---+---+---+---+---+---+---+---+
7 |   | r |   |   | p |   |   | p |
  +---+---+---+---+---+---+---+---+
6 |   |   |   | p |   |   | p | P |
  +---+---+---+---+---+---+---+---+
5 |   |   | p | P |   |   | P |   |
  +---+---+---+---+---+---+---+---+
4 |   | p | K |   |   | P |   |   |
  +---+---+---+---+---+---+---+---+
3 |   | P |   |   |   |   |   | B |
  +---+---+---+---+---+---+---+---+
2 |   |   | P | B |   |   |   |   |
  +---+---+---+---+---+---+---+---+
1 |   |   |   |   | R |   |   |   |
  +---+---+---+---+---+---+---+---+
FEN: 5nkb/1r2p2p/3p2pP/2pP2P1/1pK2P2/1P5B/2PB4/4R3 w - - 0 62
Hashkey: 779f58d736c1211e mgpsq: +0.02
Pawnkey: cd891bf1e73f081e egpsq: +0.80
Mtrlkey: febab9b5

go infinite
 2/ 2  00:00      554       0  +0.97 Bc8 Ra7
 2/ 2  00:00      603       0  +1.03 Bg4 Bd4
 3/ 4  00:00      816  816000  +0.96 Bg4 Bd4 Be3 Bxe3 Rxe3
 4/ 5  00:00     1170   1170K  +0.89 Bg4 Bd4 Be3 Bc3
 5/ 6  00:00     2918   1459K  +0.78 Bg4 Bd4 Bc1 Kf7 f5 Ra7 fxg6+ Kxg6
 5/ 6  00:00     3849   1283K  +0.82 Re4 Bd4 Be3 Bxe3 Rxe3 Kf7
 6/ 8  00:00     5191   1297K  +0.73 Re4 Bd4 Bc1 Kf7 f5 Ra7
 6/ 9  00:00     8563   1427K  +0.83 Bc1 Bc3 Re2 Bd4 Bg4 Kf7
 7/ 9  00:00    10541   1505K  +0.62 Bc1 Bc3 Re2 Ra7 f5 Nd7 fxg6 Ne5+ Kb5 Nxg6 Be6+ Kf8
 7/ 9  00:00    12883   1431K  +0.77 Re4 Bd4 Bg4 Kf7 f5 Be5 fxg6+ Kxg6
 7/14  00:00    17404   1450K  +0.84 f5 gxf5 Bxf5 Be5 Rf1 Ra7 Be4 Ra2
 8/14  00:00    26191   1455K  +0.79 f5 Bd4 Kd3 gxf5 Bxf5 Be5 Ke4 Ra7 Bf4 Bxf4 Kxf4
 9/14+ 00:00    34760   1448K  +0.88 f5 Bd4 Kd3 gxf5 Bxf5 Be5 Ke4 Ra7 Bf4 Bc3
 9/14  00:00    36752   1413K  +0.94 f5 Bd4 Rf1 Ra7 Bxb4 Nd7 Kd3 Ne5+ Ke4 cxb4 Kxd4
10/15- 00:00    58011   1567K  +0.85 f5 Bd4 Rf1 Ra7 Bxb4 Nd7 Kd3 Ne5+ Ke4 cxb4 Kxd4
10/15  00:00    77370   1719K  +0.79 f5 gxf5 Bxf5 Ra7 Bc1 Bd4 Rh1 Nd7 Be6+ Kh8 Bxd7 Rxd7 Rf1 Kg8
11/15  00:00    95468   1801K  +0.78 f5 gxf5 Bxf5 Ra7 Bc1 Bd4 Rh1 Ra8 Rf1 Ra2 Rf3 Be5
12/16  00:00   165861   2211K  +0.76 f5 gxf5 Bxf5 Ra7 Bc1 Bd4 Kb5 Kf7 Rf1 Ke8 Kc4 Ra2 Be4
13/20  00:00   358370   3011K  +0.72 f5 gxf5 Bxf5 Ra7 Bc1 Be5 Kb5 Rb7+ Kc6 Ra7 Rf1 Ra2 Kb5 Bc3 Kc4 Be5 Bf4 Bxf4 Rxf4 Ra1 Bh3 Ra2
14/20  00:00   516611   3376K  +0.72 f5 gxf5 Bxf5 Ra7 Bc1 Be5 Kb5 Rb7+ Kc6 Ra7 Rf1 Ra2 Kb5 Bc3 Kc4 Be5 Bf4 Bxf4 Rxf4 Ra1 Bh3 Ra2
15/22  00:00   722075   3840K  +0.72 f5 gxf5 Bxf5 Ra7 Bc1 Be5 Kb5 Rb7+ Kc6 Ra7 Rf1 Ra2 Kb5 Bc3 Kc4 Be5 Bf4 Bxf4 Rxf4 Ra1 Bh3 Ra2
16/28  00:00    1799K   5519K  +0.72 f5 gxf5 Bxf5 Ra7 Kd3 Be5 Be3 Bc3 Rh1 Bh8 Bd2 Ra2 Re1 Be5 Kc4 Kf7 Rf1 Kg8 Bc1 Ra1 Bd3 Ra2 Be3 Bc3 Bf5 Be5 Rf2 Ng6 Kd3 Nf8 Rf1 Ra7 Kc4 Ra2
17/28  00:00    2777K   6065K  +0.72 f5 gxf5 Bxf5 Ra7 Kd3 Be5 Be3 Bc3 Rh1 Bh8 Bd2 Ra2 Rf1 Be5 Rf3 Kh8 Kc4 Kg8 Bc1 Bd4 Bd2 Be5
18/28  00:00    4971K   7031K  +0.72 f5 gxf5 Bxf5 Ra7 Kd3 Be5 Be3 Bc3 Rh1 Bh8 Bd2 Be5 Kc4 Ra2 Rf1 Bh8 Kd3 Be5 Rf3 Kh8 Kc4 Ng6 Bxg6 hxg6 Rf8+ Kh7 Rf7+ Kg8 Rxe7 Rxc2+ Kd3 Ra2
19/28  00:00    7534K   7587K  +0.72 f5 gxf5 Bxf5 Ra7 Kd3 Be5 Be3 Bc3 Rh1 Bh8 Bd2 Be5 Kc4 Ra2 Rf1 Bh8 Kd3 Be5 Bg4 Ng6 Be6+ Kh8 Bf5 Nf8 Kc4 Kg8 Bd3 Nd7 Bf5 Nf8
20/30  00:01   11330K   8127K  +0.72 f5 gxf5 Bxf5 Ra7 Kd3 Be5 Be3 Bc3 Rh1 Bh8 Bd2 Be5 Kc4 Ra2 Rf1 Bh8 Kd3 Be5 Rf3 Ra5 Kc4 Ra2 Bf4 Bd4 Bh2 Kh8 Bg3 Kg8 Rf1 Be3 Bf4 Bxf4 Rxf4 Ra1 Bh3 Ra2
21/35  00:02   22828K   8817K  +0.80 f5 gxf5 Bxf5 Ra7 Kd3 Be5 Rf1 Ra3 Bg4 Ra2 Bh5 Ng6 Bxg6 hxg6 Rf3 Bh8 Bc1 Ra1 Bd2 Ra2
22/39  00:05   50752K   9568K  +0.80 f5 gxf5 Bxf5 Bb2 Kd3 Be5 Rf1 e6 dxe6 Re7 g6 hxg6 Be4 Nxe6 Bxg6 Bd4 Be4 Kh8 Bg5 Re8 Bc1 Re7 Kc4 Be5 Rg1
23/39  00:06   65472K   9770K  +0.80 f5 gxf5 Bxf5 Bb2 Kd3 Ra7 Rf1 Ra6 Bg4 Be5 Bh5 Ra7 Bf7+ Kh8 Be6 Ng6 Bf5 Kg8 Bxg6 hxg6 Be3 Ra2 Re1 Ra8 Re2 Kh7 Rf2 Kg8 Bd2 Ra2 Rf1 Ra7
24/39  00:09  100636K  10223K  +0.80 f5 gxf5 Bxf5 Bb2 Kd3 Ra7 Rf1 Ra6 Bg4 Be5 Bh5 Ra7 Bf7+ Kh8 Be6 Ng6 Bf5 Kg8 Bxg6 hxg6 Be3 Ra2 Re1 Ra8 Re2 Kh7 Rf2 Kg8 Bd2 Ra2 Rf1 Ra7
25/45  00:15  160503K  10476K  +0.80 f5 gxf5 Bxf5 Bb2 Kd3 Ra7 Rf1 Ra6 Bg4 Be5 Bh5 Ra7 Bf7+ Kh8 Be6 Ng6 Rf7 Bd4 Ke2 Ra2 Bf5 Ra7 Bxg6 hxg6 Kd3 Kg8 Rf1 Be5 Be3 Ra2 Re1 Ra8 Re2 Kh7 Rf2 Kg8 Bd2 Ra2 Rf1 Ra7
26/45  00:25  279008K  10880K  +0.80 f5 gxf5 Bxf5 Bb2 Kd3 Ra7 Rf1 Ra6 Bg4 Be5 Bh5 Ra7 Bf7+ Kh8 Be6 Ng6 Bf5 Nh4 Be4 Kg8 Bf4 Bh8 Bg3 Ng6 Bxg6 hxg6 Be1 Ra2 Bd2 Bd4 Bc1 Be5 Be3 Bh8 Rf3 Be5 Bd2 Bh8 Bf4 Kh7 Be3 Kg8 Bd2 Be5 Rf1 Ra7 Be3 Ra2 Re1 Ra8 Re2 Kh7 Rf2 Kg8 Bd2 Ra2 Rf1
27/54  00:41  460379K  11124K  +0.80 f5 gxf5 Bxf5 Bb2 Kd3 Ra7 Rf1 Ra6 Bg4 Be5 Bh5 Ra7 Bf7+ Kh8 Be6 Ng6 Bg4 Ra2 Bh5 Kg8 Bxg6 hxg6 Be3 Bh8 Rf3 Be5 Bd2 Bh8 Bf4 Kh7 Be3 Kg8 Bd2 Be5 Rf1 Ra7 Be3 Ra2 Re1 Ra8 Re2 Kh7 Rf2 Kg8 Bd2 Ra2 Rf1
27/54+ 01:07  752536K  11211K  +0.89 Be6+ Nxe6 Rxe6 Kf7
27/55+ 01:28  986578K  11201K  +1.02 Be6+ Nxe6 Rxe6 Kf7
27/56+ 01:55    1302M  11242K  +1.21 Be6+ Nxe6 Rxe6 Kf7
27/64+ 03:06    2128M  11388K  +1.50 Be6+ Nxe6 Rxe6 Kf7
27/68+ 07:39    5341M  11632K  +1.92 Be6+ Nxe6 Rxe6 Kf7
27/75  15:34   10966M  11738K  +2.10 Be6+ Nxe6 dxe6 Rb8 f5 Rc8 fxg6 hxg6 Rf1 d5+ Kb5 Bc3 Bxc3 bxc3 Rf7 d4 Rg7+ Kh8 Kc4 Rc7 Rxg6 Rc6 Rg7 Rxe6 Rf7 Kg8 Rf5 Rd6 Kd3 Rc6 Re5 Kf7 h7 Kg7 Rxe7+ Kh8 Ke4 Rd6 Kf5 d3 g6 Rd8 cxd3 c2 Re1 Kg7 Rc1 Rf8+ Kg5

[MM]

How is that Komodo is losing all matches?

Are the settings right?

Best Regards

[Sedat Canbaz]

How is that Komodo is losing all matches?

Are the settings right?

Best Regards

I think Komodo's performance suffers due to critical openings (especially with Blacks)




Btw,for example Perfect 2012b book has no any similar openings,which are played in the current match

Best,
Sedat

[Houdini]

How is that Komodo is losing all matches?

Are the settings right?

Best Regards

Yes, it's very strange. Komodo is losing all 4 matches rather badly, must be a string of bad luck.

As always in Timo's tournaments there are some very interesting games.
In the current game 17 against Houdini, Komodo completely misses a classical Houdini king-side attack with bishop sacrifice on h3.
This is the position after white move 26.Qb2, Komodo evals this as +0.20 for White:
[D] 2b1r1k1/1p3pp1/p1p3r1/3n2qp/1PNPp3/P3P2P/1Q3PPK/2R1RB2 b - -
Komodo doesn't expect 26...Nf6! (Houdini evals this as +1.43 for Black) and 10 moves and some sacrifices later this is the attacking position:
[D] 4k3/1p4p1/p1p1P1r1/7p/1P2p3/P3Pn2/1Q3PP1/2RR1K1q w - -
Evaluation 2.7 in favor of Black.

Five moves later the game ends 0-1:
[D]4k3/1p4p1/p1p1P3/4n1r1/PP2p1q1/4P2p/1Q2KP2/3R3R w - -

The complete game:
(17) Komodo 5 64-bit - Houdini 2.0c Pro x64
1.d4 d5 2.c4 e6 3.Nc3 Nf6 4.Nf3 Be7 5.Bg5 h6 6.Bh4 0-0 7.e3 Ne4 8.Bxe7 Qxe7 9.Rc1 c6
10.Qc2 Nd7 11.Be2 Nxc3 12.Qxc3 dxc4 13.Qxc4 e5 14.0-0 Re8 15.Qc2 e4 16.Nd2 Nf6
17.a3 Bd7 18.h3 Nd5 19.Rfe1 Rad8 20.b4 a6 21.Nb3 Bc8 22.Bf1 Rd6
23.Kh2 Rg6 24.Nd2 h5 25.Nc4 Qg5 26.Qb2 Nf6 27.Ne5 Rxe5 28.dxe5 Bxh3 29.Kxh3 Qg4+
30.Kh2 Qh4+ 31.Kg1 Ng4 32.Bc4 Nh2 33.Bxf7+ Kxf7 34.e6+ Ke8 35.Red1 Nf3+
36.Kf1 Qh1+ 37.Ke2 Qxg2 38.Rh1 Rg5 39.Rcd1 Qg4 40.Rhf1 h4 41.a4 h3 42.Rh1 Ne5+ 0-1

Robert

[lkaufman]

My casual impression is that Komodo is moving much too fast. This time control is for all practical purposes just game in two hours, with the 3 seconds just preventing forfeits due to system overhead. With ponder on, I think Komodo should be averaging over five minutes per unpredicted move for the early moves out of book, but that doesn't appear to be the case, though I could be mistaken. Is there any way to tell (without adding up all the times manually for each game) how much time remained on each clock in each game after say 20 or 30 moves?
Of course we never tested any similar time control, but according to my understanding of the algorithm used it should treat this as essentially a sudden death game, but perhaps something is wrong. We'll have to investigate. It seems that Komodo performs well enough in games with normal increments (say 1/60 or 1/100 of the base time), but poorly in repeating time controls and perhaps also in sudden death. Looks like we have some work to do there.

[MM]

My casual impression is that Komodo is moving much too fast. This time control is for all practical purposes just game in two hours, with the 3 seconds just preventing forfeits due to system overhead. With ponder on, I think Komodo should be averaging over five minutes per unpredicted move for the early moves out of book, but that doesn't appear to be the case, though I could be mistaken. Is there any way to tell (without adding up all the times manually for each game) how much time remained on each clock in each game after say 20 or 30 moves?
Of course we never tested any similar time control, but according to my understanding of the algorithm used it should treat this as essentially a sudden death game, but perhaps something is wrong. We'll have to investigate. It seems that Komodo performs well enough in games with normal increments (say 1/60 or 1/100 of the base time), but poorly in repeating time controls and perhaps also in sudden death. Looks like we have some work to do there.

I don't know how Komodo behaves in long time control. Anyway i think that generally an engine should spend much time during the opening and the middlegame, increment or not. I'm not able to state a formula but i believe that this is the best time management.
Note that i think this is the best time management in short time controls too, especially with time increment.

In my testing Komodo has not problems in blitz games sudden death but the risk of running out of time (that sometimes happens, very rarely at 3+0, sometimes at 1+0, often at 10'' per game).

It is clear to me that, especially if there's an increment, an engine should use ALL its time to go deeper in the position, it is a nonsense finishing games having still on the clock 30/40 minutes, like i see often watching some long games (e.g. 120 m x games).

Another question is WHEN use more or less time. It is very interesting talking about the predicted and unpredicted moves.

I think there's no basically reason to move immediately if the move predicted is right. If i ponder for 5 minutes about a predicted move and that move is played, well i have other minutes to go deeper in the position to analyze better.

For example, i think that Rybka TM is the best around (it thinks a lot, often goes into time trouble, but goes very deeply into the position).

Best Regards

[Uri Blass]

My casual impression is that Komodo is moving much too fast. This time control is for all practical purposes just game in two hours, with the 3 seconds just preventing forfeits due to system overhead. With ponder on, I think Komodo should be averaging over five minutes per unpredicted move for the early moves out of book, but that doesn't appear to be the case, though I could be mistaken. Is there any way to tell (without adding up all the times manually for each game) how much time remained on each clock in each game after say 20 or 30 moves?
Of course we never tested any similar time control, but according to my understanding of the algorithm used it should treat this as essentially a sudden death game, but perhaps something is wrong. We'll have to investigate. It seems that Komodo performs well enough in games with normal increments (say 1/60 or 1/100 of the base time), but poorly in repeating time controls and perhaps also in sudden death. Looks like we have some work to do there.

Something seems to me wrong with the time management of komodo

game number 20 against houdini.

Komodo used 19 minutes and 37 seconds for 10...Bb7 and got depth 25 and later used only 26 seconds for 11...0-0 and got depth 21.

(20) Houdini 2.0c Pro x64 - Komodo 5 64-bit
Komodo5-Houdini (20.1), 22.07.2012
1.d4 Nf6 2.c4 e6 3.Nf3 b6 4.g3 Ba6 5.Qc2 Bb4+ 6.Bd2 Be7 7.Bg2 c6 8.0-0 d5 9.cxd5 0.35/21 2:44 cxd5 0.30/22 1:10 10.Bf4 0.35/23 1:40 Bb7 0.14/25 19:37 11.Ne5 (Qa4+) 0.25/25 0 11...0-0 (Nfd7) 0.07/21 26 12.Rc1 (Rd1) 0.28/23 4:35 12...Ne8 -0.03/21 2:02 13.Nc3 (Qa4) 0.22/22 2:36 13...f6 -0.01/22 1:05 14.Nd3 0.16/22 2:13 Nc6 0.07/24 2:11 15.Qa4 0.15/23 42 g5 (Rc8) 0.02/23 6:34 16.Bd2 0.18/21 3:06 Qd7 -0.01/22 1:16 17.Re1 (f4) 0.12/22 1:54 17...Rd8 -0.03/24 10:59 18.Red1 (Rac1) 0.10/24 4:42 18...Nd6 -0.21/23 1:28 19.e3 0.08/23 1:19 Rc8 -0.22/23 1:07 20.Rac1 0.08/22 4:04 Kh8 (Kg7) -0.21/24 31 21.Bh3 (Be1) 0.08/20 2:22 21...f5 -0.22/21 2:55 22.Bf1 (Bg2) 0.08/21 23 22...Bf6 (Nc4) -0.20/21 5:18 23.Be1 (Bg2) 0.16/20 5:13 23...Rfe8 (Kg8) -0.20/21 2:46 24.Bg2 0.15/19 2:37 Bg7 -0.17/20 2:39 25.b3 (Qc2) 0.11/20 38 25...Bf8 (Qe7) -0.12/21 5:23 26.Rc2 (Qa3) 0.03/20 2:31 26...a6 (Red8) -0.21/23 2:21 27.Qa3 0.14/23 3:59 Nc4 -0.18/25 30 28.Qc1 0.13/24 0 Na3 -0.19/24 1:06 29.Rcd2 (Re2) 0.06/24 1:01 29...Qc7 (Qg7) -0.15/23 1:56 30.Nb1 0.09/22 1:53 Nb5 (Qe7) -0.13/23 1:33 *