That's a nice one.
--Jon
60 games Komodo 5 against Top4 at 120m+3s
Moderators: hgm, Rebel, chrisw
-
jdart
- Posts: 4366
- Joined: Fri Mar 10, 2006 5:23 am
- Location: http://www.arasanchess.org
-
carldaman
- Posts: 2283
- Joined: Sat Jun 02, 2012 2:13 am
Re: 60 games Komodo 5 against Top4 at 120m+3s
This is why test suites are important. A line like 1.e4 e6 2.d4 d5 3.Nc3 Bb4 certainly belongs in a test suite, where engines will have to play both sides of the opening line, and the one that can play this type of position better will be rewarded for it.
However, as I mentioned in a different post, a test suite position that one side (here White) tends to handle much better will create ratings distortions if there's too much of a rating gap between the opponents, by favoring the weaker engine (by gifting it the White side of such a variation).
This seems to pose a dilemma. Either rated test suite matches should only be run between closely matched engines, or they should be rated differently.
Carl
However, as I mentioned in a different post, a test suite position that one side (here White) tends to handle much better will create ratings distortions if there's too much of a rating gap between the opponents, by favoring the weaker engine (by gifting it the White side of such a variation).
This seems to pose a dilemma. Either rated test suite matches should only be run between closely matched engines, or they should be rated differently.
Carl
-
carldaman
- Posts: 2283
- Joined: Sat Jun 02, 2012 2:13 am
Re: 60 games Komodo 5 against Top4 at 120m+3s
Hi Sedat,
If I may make a friendly suggestion -- at some point you may want to put together a book that has more variety, but at the expense of being less perfect, where some lines such as 1.e4 e6 2.d4 d5 3.Nc3 Bb4 and others that we know from general opening theory to be playable (for ex. the Philidor, Old Indian) can be included. Of course, you would still exclude lines that are outright busted/unplayable. You can call it a Variety book if you like.
That way testers can test with your Perfect book, and then test with the expanded Variety book as well, for better comparison.
Thank you for your very good work!
Carl
If I may make a friendly suggestion -- at some point you may want to put together a book that has more variety, but at the expense of being less perfect, where some lines such as 1.e4 e6 2.d4 d5 3.Nc3 Bb4 and others that we know from general opening theory to be playable (for ex. the Philidor, Old Indian) can be included. Of course, you would still exclude lines that are outright busted/unplayable. You can call it a Variety book if you like.
That way testers can test with your Perfect book, and then test with the expanded Variety book as well, for better comparison.
Thank you for your very good work!
Carl
-
carldaman
- Posts: 2283
- Joined: Sat Jun 02, 2012 2:13 am
Re: 60 games Komodo 5 against Top4 at 120m+3s
Houdini pulled a Gelfand in this one 
Maybe it is human after all
CL
Maybe it is human after all
CL
-
Uri Blass
- Posts: 10282
- Joined: Thu Mar 09, 2006 12:37 am
- Location: Tel-Aviv Israel
Re: 60 games Komodo 5 against Top4 at 120m+3s
Or maybe it is the opposite and there is now a rating distortion that cause top engines to perform better relative to weak engines.carldaman wrote:This is why test suites are important. A line like 1.e4 e6 2.d4 d5 3.Nc3 Bb4 certainly belongs in a test suite, where engines will have to play both sides of the opening line, and the one that can play this type of position better will be rewarded for it.
However, as I mentioned in a different post, a test suite position that one side (here White) tends to handle much better will create ratings distortions if there's too much of a rating gap between the opponents, by favoring the weaker engine (by gifting it the White side of such a variation).
This seems to pose a dilemma. Either rated test suite matches should only be run between closely matched engines, or they should be rated differently.
Carl
I think that knowledge to play the Bb4 line well is part of the knowledge in chess.
I am also surprised to read that top engines perform so poorly in this line
for black because my experience is that they cannot find the right moves for white.
The common move for white is 4.e5 but
I tested stockfish and it seems to suggest exd5 that is not the common move for white and I also tested houdini and it suggests 4.a3.
I wonder if these moves give stockfish and houdini 70% result with white against top engines.
[D]rnbqk1nr/ppp2ppp/4p3/3p4/1b1PP3/2N5/PPP2PPP/R1BQKBNR w KQkq - 2 4
Stockfish-222-64-ja-intel:
1/1 00:00 148 526 +0.68 Bf1-d3 Bb4xc3+ b2xc3
2/3 00:00 570 2,028 +0.60 Qd1-g4 Bb4xc3+ b2xc3
3/4 00:00 1,286 4,576 +0.76 Qd1-g4 d5xe4 Qg4xe4
4/5 00:00 2,448 6,818 +0.76 Qd1-g4 d5xe4 Qg4xg7 Qd8-f6
5/7 00:00 5,651 15,740 +0.60 Qd1-g4 Bb4xc3+ b2xc3 g7-g6 e4xd5 e6xd5
6/8 00:00 10,437 29,072 +0.56 Qd1-g4 g7-g6 e4xd5 Bb4xc3+ b2xc3 e6xd5 Qg4-g3
7/10 00:00 22,968 61,248 +0.28 Qd1-g4 g7-g6 e4xd5 Ng8-f6 Qg4-f3 Bb4xc3+ Qf3xc3 O-O d5xe6 Bc8xe6
8/10 00:00 26,518 70,714 +0.28 Qd1-g4 g7-g6 e4xd5 Ng8-f6 Qg4-f3 Bb4xc3+ Qf3xc3 O-O d5xe6 Bc8xe6
9/13 00:00 65,600 168,205 +0.12 Qd1-g4 Ke8-f8 e4-e5 Ng8-h6 Bc1xh6 Bb4xc3+ b2xc3 g7xh6 O-O-O Nb8-c6
10/15 00:00 118,109 290,908 +0.24 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 Bb4xc3+ b2xc3 O-O O-O Nf6-e4 Bd3xe4 d5xe4
11/17 00:00 240,923 531,838 +0.20 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O Nb8-c6 Bc1-g5 Bb4xc3 b2xc3 h7-h6 Bg5-f4 Nf6-e4
12/22 00:00 349,442 698,884 +0.16 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O Nb8-c6 Bc1-g5 Bb4xc3 b2xc3 h7-h6 Bg5-f4 g7-g5 Bf4-g3 Nf6-e4 Bd3xe4 d5xe4
13/22 00:00 492,838 902,633 +0.16 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O Nb8-c6 Bc1-g5 Bb4xc3 b2xc3 h7-h6 Bg5-f4 g7-g5 Bf4-g3 Nf6-e4 Bd3xe4 d5xe4
14/22 00:00 733,384 1,117,963 +0.16 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O Nb8-c6 Bc1-g5 Bb4xc3 b2xc3 h7-h6 Bg5-f4 g7-g5 Bf4-g3 Nf6-e4 Bd3xe4 d5xe4
15/22 00:00 1,127,606 1,445,648 +0.16 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O Nb8-c6 Bc1-g5 Bb4xc3 b2xc3 h7-h6 Bg5-f4 g7-g5 Bf4-g3 Nf6-e4 Bd3xe4 d5xe4
16/22 00:00 1,646,390 1,758,963 +0.24 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O Nb8-c6 Bc1-g5 Bb4xc3 b2xc3 h7-h6 Bg5-f4 g7-g5 Bf4-g3 Nf6-e4 Bd3xe4 d5xe4 Nf3-d2
17/22 00:01 2,543,889 2,063,170 +0.24 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O Nb8-c6 Bc1-g5 Bb4xc3 b2xc3 h7-h6 Bg5-f4 g7-g5 Bf4-g3 Nf6-e4 Bd3xe4 d5xe4 Nf3-d2
18/22 00:01 3,874,676 2,342,609 +0.24 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O Nb8-c6 Bc1-g5 Bb4xc3 b2xc3 h7-h6 Bg5-f4 g7-g5 Bf4-g3 Nf6-e4 Bd3xe4 d5xe4 Nf3-d2
19/25- 00:02 6,220,835 2,622,611 +0.16 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O c7-c5 d4xc5 Bb4xc5 Bc1-f4 Nb8-c6 Nc3-a4 Bc5-d6 Bf4xd6 Qd8xd6
19/25- 00:03 9,319,571 2,844,801 +0.08 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O c7-c5 d4xc5 Bb4xc5 Bc1-f4 Nb8-c6 Nc3-a4 Bc5-d6 Bf4xd6 Qd8xd6
19/29 00:07 22,988,557 3,102,369 0.00 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O c7-c5 d4xc5 Nb8-d7 Nc3-b5 Nd7xc5 Bc1-g5 Bb4-a5 Nb5-d4 Ba5-c7 Nd4-b5 Bc7-a5
20/29+ 00:08 24,846,212 3,110,442 +0.08 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O c7-c5 d4xc5 Nb8-d7 Nc3-b5 Nd7xc5 a2-a3 Nc5xd3 Qd1xd3 Bb4-e7 Bc1-e3
20/29+ 00:08 25,384,668 3,116,977 +0.16 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O c7-c5 d4xc5 Nb8-d7 Nc3-b5 Nd7xc5 a2-a3 Nc5xd3 Qd1xd3 Bb4-e7 Bc1-e3
20/29 00:08 26,931,940 3,133,078 +0.16 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O c7-c5 d4xc5 Nb8-d7 Nc3-b5 Nd7xc5 a2-a3 Nc5xd3 Qd1xd3 Bb4-c5 Bc1-g5 Bc8-e6 Nf3-e5 Rf8-e8 Bg5-e3 Bc5xe3 Qd3xe3 Nf6-g4 Ne5xg4 Be6xg4
21/33+ 00:10 32,216,884 3,143,110 +0.24 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O c7-c5 d4xc5 Nb8-d7 Bc1-e3 Nd7xc5 a2-a3 Bb4xc3 Be3xc5 Bc3xb2 Ra1-b1
21/33+ 00:10 34,211,964 3,159,874 +0.32 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O Bb4xc3 b2xc3 Nf6-e4 Bc1-a3 Rf8-e8 c3-c4 Ne4-c3 Qd1-d2
21/33 00:12 38,748,010 3,172,166 +0.28 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O c7-c5 d4xc5 Bb4xc5 Bc1-f4 Nb8-c6 a2-a3 Bc8-e6 Nf3-g5 Bc5-d6 Qd1-d2 Qd8-c7 Bf4xd6 Qc7xd6 Nc3-b5 Qd6-b8 Ng5xe6 f7xe6 Rf1-e1
22/33- 00:15 48,101,745 3,086,412 +0.20 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O Nb8-c6 Rf1-e1 Bc8-g4 Bc1-g5 Bb4-e7 Bd3-e2 Bg4-f5 Nf3-e5 Nc6-b4 Ne5-d3 Nb4-c6 Nd3-e5
22/33 00:17 55,963,253 3,119,467 +0.16 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-b5+ Nb8-c6 O-O O-O Bb5xc6 b7xc6 Nf3-e5 c6-c5 Bc1-g5 c5xd4 Bg5xf6 Qd8xf6 Qd1xd4 Bb4xc3 Qd4xc3 Qf6-d6 Ra1-d1 Bc8-f5
23/33+ 00:21 66,765,163 3,153,763 +0.24 e4xd5 e6xd5 Ng1-f3 Ng8-f6 Bf1-d3 O-O O-O Nb8-c6 Rf1-e1 Bc8-g4 Bc1-g5 Bb4-e7 Bd3-e2 Bg4-f5 a2-a3 Rf8-e8 Be2-b5 Nf6-e4 Bg5xe7 Re8xe7 Nc3xe4 d5xe4 Bb5xc6 b7xc6 Nf3-e5
23/33+ 00:24 78,872,436 3,163,882 +0.32 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 O-O O-O Nb8-c6 Bc1-g5 Bb4-e7 Rf1-e1 Nc6-b4 Ne2-g3 Nb4xd3 Qd1xd3 Bc8-e6 Ng3-f5 Be7-b4 Re1-e5 c7-c6 Bg5-d2
23/33 00:26 84,904,481 3,169,733 +0.28 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 O-O O-O Nb8-c6 Bc1-g5 Bb4-e7 Rf1-e1 Nc6-b4 Ne2-f4 c7-c6 Qd1-e2 Nb4xd3 Nf4xd3 Be7-d6 Qe2-f3 Bc8-e6 Bg5xf6 Qd8xf6 Qf3xf6 g7xf6
24/33- 00:35 113,288,926 3,169,186 +0.20 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-f3 O-O O-O c7-c5 d4xc5 Bb4xc5 Bc1-f4 Nb8-c6 a2-a3 Rf8-e8 Rf1-e1 Re8xe1+ Qd1xe1 Bc8-g4 b2-b4 Bc5-b6 Nf3-e5 Bb6-d4 Ne5xg4 Nf6xg4 Bf4-g3
24/33+ 00:39 125,034,979 3,181,227 +0.36 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Nb8-c6 O-O h7-h6 a2-a3 Bb4xc3 b2xc3 O-O c3-c4 d5xc4 Bd3xc4 Bc8-e6 Bc4xe6 f7xe6 Ra1-b1
24/33 00:41 130,642,721 3,183,690 +0.28 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Nb8-c6 O-O Bc8-g4 f2-f3 Bg4-e6 a2-a3 Bb4xc3 b2xc3 O-O Ne2-f4 Rf8-e8 Rf1-e1 Qd8-d6 Nf4xe6 Re8xe6 Re1xe6 f7xe6 Bc1-e3 b7-b6 Qd1-e2 a7-a5
25/33+ 00:46 148,031,741 3,197,713 +0.36 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Nb8-c6 O-O Bc8-g4 f2-f3 Bg4-e6 a2-a3 Bb4xc3 b2xc3 O-O Ne2-f4 Rf8-e8 Rf1-e1 Qd8-d6 Nf4xe6 Re8xe6 Re1xe6 f7xe6 Bc1-e3 b7-b6 c3-c4 Ra8-e8 c2-c3
25/34 00:54 174,904,887 3,213,981 +0.40 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Nb8-c6 O-O Bc8-g4 f2-f3 Bg4-e6 a2-a3 Bb4xc3 b2xc3 O-O Bc1-f4 a7-a6 Ra1-b1 Ra8-b8 Qd1-e1 Nf6-h5 Bf4-d2 Qd8-e7 Ne2-f4 Nh5xf4 Bd2xf4 g7-g6 Bf4-h6
26/35- 00:59 191,275,400 3,217,795 +0.32 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Nb8-c6 O-O Bc8-g4 f2-f3 Bg4-e6 a2-a3 Bb4xc3 b2xc3 O-O Bc1-f4 a7-a6 Ra1-b1 Ra8-b8 Qd1-e1 Nf6-h5 Bf4-d2 Rf8-e8 g2-g4 Nh5-f6 g4-g5 Nf6-d7
26/35 01:08 221,244,990 3,230,325 +0.28 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Nb8-c6 O-O Bc8-g4 f2-f3 Bg4-e6 a2-a3 Bb4xc3 b2xc3 O-O Bc1-f4 a7-a6 Ra1-b1 Ra8-b8 Rf1-e1 h7-h6 Ne2-g3 Rf8-e8 Bd3-f5 Qd8-d7 Bf5xe6 Re8xe6 Re1xe6 f7xe6
27/35+ 01:25 277,483,331 3,230,645 +0.36 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Nb8-c6 O-O Bc8-g4
27/35 01:41 325,026,883 3,217,355 +0.28 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Nb8-c6 O-O O-O Bc1-g5 Bb4-e7 Ne2-f4 Nc6-b4 Rf1-e1 h7-h6 Bg5xf6 Be7xf6 a2-a3 Nb4xd3 Qd1xd3 c7-c6 Nf4-h5 Bc8-g4 Nh5xf6+ Qd8xf6 Re1-e5 Ra8-e8 Ra1-e1 Re8xe5 d4xe5 Qf6-g5 Qd3-d4 b7-b6
28/37+ 02:05 401,293,393 3,207,986 +0.36 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Nb8-c6 O-O O-O a2-a3 Bb4-d6 Nc3-b5 Nf6-g4 Nb5xd6 Qd8xd6 Bc1-f4 Qd6-d7
28/38 02:48 544,119,248 3,220,974 +0.28 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Bb4-e7 O-O O-O Bc1-f4 a7-a6 Bf4-e3 Nb8-d7 Rf1-e1 Nd7-b6 Ne2-g3 Bc8-e6 a2-a3 Be7-d6 Qd1-d2 c7-c6 Be3-f4 Rf8-e8 Ra1-c1 Bd6-c7 Ng3-f5 Nb6-c4 Bd3xc4 Be6xf5
29/39+ 03:27 669,588,900 3,229,891 +0.36 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Bb4-e7 O-O O-O Bc1-e3 Nb8-c6 a2-a3 Rf8-e8 Rf1-e1 Nf6-g4 Be3-f4 Be7-d6 f2-f3 Ng4-f6 Qd1-d2 Bd6xf4 Ne2xf4 Re8xe1+ Ra1xe1 Nc6xd4 Bd3xh7+
29/41- 05:28 1,074,908,577 3,274,873 +0.20 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Bb4-e7 O-O O-O Bc1-e3 Nb8-a6 Ne2-g3 Rf8-e8 Rf1-e1 Na6-b4 Bd3-e2 Be7-d6 a2-a3 Nb4-c6 Be2-f3 Bc8-e6 Nc3-b5 Bd6xg3 h2xg3 a7-a6 Nb5-c3 Be6-f5
29/43 06:10 1,210,416,271 3,270,069 +0.28 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Bb4-e7 O-O O-O Bc1-e3 Nb8-a6 Bd3xa6 b7xa6 Qd1-d3 Rf8-e8 Be3-g5 c7-c6 Rf1-e1 Ra8-b8 b2-b3 Nf6-h5 Bg5xe7 Qd8xe7 a2-a3 Bc8-b7 Qd3-d2 Qe7-h4 Ne2-g3 Nh5-f4 Ng3-f5
30/43+ 07:22 1,449,662,779 3,275,526 +0.36 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Bb4-e7 O-O O-O Bc1-e3 Nb8-a6 Bd3xa6 b7xa6 Qd1-d3 Rf8-e8 Be3-g5 c7-c6 Rf1-e1 Ra8-b8 b2-b3 Nf6-h5 Bg5xe7 Qd8xe7 a2-a3 Bc8-b7 Qd3-d2 Qe7-h4 Ne2-g3 Nh5-f4 Re1xe8+ Rb8xe8 Ra1-e1
30/43 08:11 1,610,709,378 3,274,784 +0.32 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Bb4-e7 O-O O-O Bc1-e3 Rf8-e8 Rf1-e1 Nb8-c6 a2-a3 Nf6-g4 Be3-f4 g7-g5 Bf4-g3 a7-a6 Qd1-d2 Be7-f6 h2-h3 Ng4-h6 h3-h4 Nh6-f5 Bd3xf5 Bc8xf5 h4xg5 Bf6xg5 Ne2-f4 Re8xe1+ Ra1xe1 Bf5-e6
31/43 11:17 2,221,971,494 3,279,178 +0.32 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Ng1-e2 Bb4-e7 O-O O-O Bc1-e3 c7-c6 Ne2-g3 Be7-d6 Qd1-d2 Bc8-e6 Be3-f4 Bd6xf4 Qd2xf4 Nb8-d7 a2-a3 Qd8-b6 Rf1-e1 Rf8-e8 b2-b4 g7-g6 f2-f3 a7-a6 Ra1-b1 Nf6-e4 Ng3xe4
[D]rnbqk1nr/ppp2ppp/4p3/3p4/1b1PP3/2N5/PPP2PPP/R1BQKBNR w KQkq - 2 4
Houdini_15a_x64:
4/16 00:00 3,700 97,000 +0.54 Qd1-g4 d5xe4 Qg4xg7 Qd8-f6 Bc1-h6 Nb8-c6 Ng1-e2 Qf6xg7 Bh6xg7
5/17 00:00 5,502 141,000 +0.54 Qd1-g4 d5xe4 Qg4xg7 Qd8-f6 Bc1-h6 Nb8-c6 Ng1-e2 Qf6xg7 Bh6xg7
6/18 00:00 9,177 218,000 +0.54 Qd1-g4 d5xe4 Qg4xg7 Qd8-f6 Bc1-h6 Nb8-c6 Ng1-e2 Qf6xg7 Bh6xg7
7/18 00:00 22,653 190,000 +0.25 Qd1-g4 Ng8-f6 Qg4xg7 Rh8-g8 Qg7-h6 d5xe4 a2-a3 Bb4-d6
8/21 00:00 51,503 381,000 +0.25 e4xd5 e6xd5 Bf1-d3 Ng8-f6 Qd1-e2+ Qd8-e7 Qe2xe7+ Bb4xe7 Ng1-f3 Nb8-c6 O-O
9/27 00:00 86,382 579,000 +0.31 e4xd5 e6xd5 Bf1-d3 Nb8-c6 Ng1-f3 Qd8-e7+ Bc1-e3 Ng8-f6 O-O O-O Qd1-e2 Bc8-e6 Nf3-e5 Nc6xe5 d4xe5 Bb4xc3 b2xc3
10/27- 00:00 120,203 737,000 +0.24 e4xd5 e6xd5
10/27 00:00 187,333 996,000 +0.27 e4xd5 e6xd5 Bf1-d3 Nb8-c6 Ng1-f3 Qd8-e7+ Bc1-e3 Ng8-f6 O-O O-O Rf1-e1 Bb4-d6
11/27 00:00 261,696 1,234,000 +0.25 e4xd5 e6xd5 Bf1-d3 Nb8-c6 Ng1-f3 Qd8-e7+ Bc1-e3 Ng8-f6 O-O O-O Rf1-e1 Bb4-d6 Qd1-d2 Bc8-e6 Nf3-g5 h7-h6
12/27- 00:00 399,304 1,578,000 +0.18 e4xd5 e6xd5
12/27 00:00 533,188 1,813,000 +0.31 e4-e5 Ng8-e7 Bf1-d3 O-O Qd1-g4 Ne7-f5 Ng1-f3 h7-h6 O-O Nb8-c6 a2-a3 Bb4-e7 Bc1-d2
13/27- 00:00 671,686 1,993,000 +0.24 e4-e5 Ng8-e7
13/27 00:00 1,237,555 2,541,000 +0.20 e4-e5 Ng8-e7 Ng1-f3 O-O Bf1-e2 Nb8-c6 O-O h7-h6 a2-a3 Bb4xc3 b2xc3 b7-b6 Be2-b5 Bc8-b7
14/28 00:00 1,740,171 2,829,000 +0.16 e4-e5 Ng8-e7 Ng1-f3 O-O Bf1-e2 Nb8-c6 O-O h7-h6 a2-a3 Bb4xc3 b2xc3 b7-b6 Be2-b5 Bc8-b7 Qd1-d3 Ne7-f5
14/42+ 00:01 4,665,616 3,385,000 +0.27 a2-a3
14/42 00:01 5,876,633 3,471,000 +0.20 a2-a3 Bb4xc3+ b2xc3 d5xe4 Qd1-g4 Ng8-f6 Qg4xg7 Rh8-g8 Qg7-h6 Nb8-c6 Bc1-g5 Rg8-g6 Qh6-h4 e6-e5 d4xe5 Nc6xe5 h2-h3 h7-h6 Bg5xf6 Qd8xf6 Qh4xe4 Qf6-e7 Ra1-d1 f7-f5 Qe4-e3 Rg6-e6
15/42+ 00:01 6,473,721 3,504,000 +0.27 a2-a3
15/42 00:02 7,741,431 3,520,000 +0.27 a2-a3 Bb4xc3+ b2xc3 d5xe4 Qd1-g4 Ng8-f6 Qg4xg7 Rh8-g8 Qg7-h6 Rg8-g6 Qh6-e3 Nb8-c6 Ng1-e2 Nc6-e7 Ne2-g3 Ne7-d5 Qe3-d2 Ke8-f8 c3-c4 Nd5-e7 Qd2-e3 b7-b6 Bf1-e2 Ne7-c6 Ng3xe4 Nf6xe4 Qe3xe4 Qd8xd4
16/42 00:02 10,051,058 3,631,000 +0.27 a2-a3 Bb4xc3+ b2xc3 d5xe4 Qd1-g4 Ng8-f6 Qg4xg7 Rh8-g8 Qg7-h6 Bc8-d7 Ng1-e2 Bd7-c6 Ne2-f4 Nb8-d7 a3-a4 Nd7-b6 a4-a5 Nb6-d5 Nf4xd5 e6xd5 a5-a6 b7-b6 Bc1-a3 Bc6-d7 h2-h3
17/42- 00:03 14,344,730 3,765,000 +0.21 a2-a3 Bb4xc3+
17/42 00:07 29,601,517 4,021,000 +0.19 a2-a3 Bb4xc3+ b2xc3 d5xe4 Qd1-g4 Ng8-f6 Qg4xg7 Rh8-g8 Qg7-h6 Bc8-d7 Bc1-g5 Rg8-g6 Qh6-h4 h7-h6 Bg5xh6 Rg6-g4 Qh4-h3 Nb8-c6 Ng1-e2 e6-e5 Qh3-e3 Nf6-d5 Qe3-d2 e5xd4 Ne2xd4 Qd8-f6 h2-h3 Rg4-g6 Bf1-c4 Nd5xc3 Nd4xc6 Bd7xc6
18/42 00:09 37,648,989 4,067,000 +0.23 a2-a3 Bb4xc3+ b2xc3 d5xe4 Qd1-g4 Ng8-f6 Qg4xg7 Rh8-g8 Qg7-h6 Bc8-d7 Bc1-g5 Rg8-g6 Qh6-h4 h7-h6 Bg5xh6 Rg6-g4 Qh4-h3 Nb8-c6 Ng1-e2 Rg4-g8 Bh6-d2 Qd8-e7 c3-c4 e6-e5 Qh3-e3 e5xd4 Ne2xd4 O-O-O Nd4xc6 Bd7xc6 Qe3xa7
19/42 00:11 46,430,095 4,113,000 +0.23 a2-a3 Bb4xc3+ b2xc3 d5xe4 Qd1-g4 Ng8-f6 Qg4xg7 Rh8-g8 Qg7-h6 Bc8-d7 Bc1-g5 Rg8-g6 Qh6-h4 h7-h6 Bg5xh6 Rg6-g4 Qh4-h3 Nb8-c6 Ng1-e2 Rg4-g8 Bh6-d2 Qd8-e7 c3-c4 e6-e5 Qh3-e3 e5xd4 Ne2xd4 O-O-O Nd4xc6 Bd7xc6 Qe3xa7
20/49- 00:15 65,727,734 4,164,000 +0.16 a2-a3 Bb4xc3+
20/49 00:41 179,233,113 4,351,000 +0.25 a2-a3 Bb4xc3+ b2xc3 d5xe4 Qd1-g4 Ng8-f6 Qg4xg7 Rh8-g8 Qg7-h6 Bc8-d7 Bc1-g5 Rg8-g6 Qh6-h4 h7-h6 Bg5xh6 Rg6-g4 Qh4-h3 Rg4-g8 Ra1-b1 e6-e5 Qh3-e3 Nf6-g4 Qe3-d2 Bd7-c6 d4xe5 Nb8-d7 e5-e6 f7xe6 Rb1-d1 Ng4xh6 Qd2xh6 Qd8-f6 Qh6xf6 Nd7xf6 h2-h4 e4-e3
21/49 00:44 191,220,609 4,353,000 +0.23 a2-a3 Bb4xc3+ b2xc3 d5xe4 Qd1-g4 Ng8-f6 Qg4xg7 Rh8-g8 Qg7-h6 Bc8-d7 Bc1-g5 Rg8-g6 Qh6-h4 h7-h6 Bg5xh6 Rg6-g4 Qh4-h3 Nb8-c6 Ng1-e2 Rg4-g8 Bh6-d2 Qd8-e7 c3-c4 e6-e5 Qh3-e3 e5xd4 Ne2xd4 O-O-O Nd4xc6 Bd7xc6 Qe3xa7
22/51 01:12 315,437,744 4,385,000 +0.27 a2-a3 Bb4xc3+ b2xc3 d5xe4 Qd1-g4 Ng8-f6 Qg4xg7 Rh8-g8 Qg7-h6 Bc8-d7 Bc1-g5 Rg8-g6 Qh6-h4 h7-h6 Bg5xh6 Rg6-g4 Qh4-h3 Rg4-g8 Ra1-b1 e6-e5 Qh3-e3 Nf6-g4 Qe3-d2 Bd7-c6 d4xe5 Nb8-d7 Bh6-f4 Ng4xe5 Bf4-e3 Ne5-g4 Be3-d4 Nd7-e5 Ng1-h3 Qd8-h4
23/51 01:52 495,847,026 4,405,000 +0.27 a2-a3 Bb4xc3+ b2xc3 d5xe4 Qd1-g4 Ng8-f6 Qg4xg7 Rh8-g8 Qg7-h6 Bc8-d7 Bc1-g5 Rg8-g6 Qh6-h4 h7-h6 Bg5xh6 Rg6-g4 Qh4-h3 Rg4-g8 Ra1-b1 e6-e5 Qh3-e3 Nf6-g4 Qe3-d2 Bd7-c6 d4xe5 Nb8-d7 Bh6-f4 Ng4xe5 Bf4-e3 Ne5-g4 Be3-d4 Nd7-e5 Ng1-h3 Qd8-h4
-
Sedat Canbaz
- Posts: 3018
- Joined: Thu Mar 09, 2006 11:58 am
- Location: Antalya/Turkey
Re: 60 games Komodo 5 against Top4 at 120m+3s
I've just done some analysis with the current French line,where i used a few top engines
And its interesting to note that almost all the current tested engines played 3...Nf6
Exception,only Komodo 4 and Strelka 5.5 preferred to play 3...Bb4
Conditions:
Intel Core i7-970 @4.0GHz
Windows XP Professional x64
Fritz 12 GUI
1024 MB Hashtable size
60 Minutes per game
[D]rnbqkbnr/ppp2ppp/4p3/3p4/3PP3/2N5/PPP2PPP/R1BQKBNR b KQkq - 0 3
Note:The current all MP engines are tested with 1 core and 6 cores
For Full Analysis:
http://www.sedatcanbaz.com/chess/files/french.txt
Best,
Sedat
And its interesting to note that almost all the current tested engines played 3...Nf6
Exception,only Komodo 4 and Strelka 5.5 preferred to play 3...Bb4
Conditions:
Intel Core i7-970 @4.0GHz
Windows XP Professional x64
Fritz 12 GUI
1024 MB Hashtable size
60 Minutes per game
[D]rnbqkbnr/ppp2ppp/4p3/3p4/3PP3/2N5/PPP2PPP/R1BQKBNR b KQkq - 0 3
Note:The current all MP engines are tested with 1 core and 6 cores
For Full Analysis:
http://www.sedatcanbaz.com/chess/files/french.txt
Best,
Sedat
-
Sedat Canbaz
- Posts: 3018
- Joined: Thu Mar 09, 2006 11:58 am
- Location: Antalya/Turkey
Re: 60 games Komodo 5 against Top4 at 120m+3s
Dear Carl,carldaman wrote:Hi Sedat,
If I may make a friendly suggestion -- at some point you may want to put together a book that has more variety, but at the expense of being less perfect, where some lines such as 1.e4 e6 2.d4 d5 3.Nc3 Bb4 and others that we know from general opening theory to be playable (for ex. the Philidor, Old Indian) can be included. Of course, you would still exclude lines that are outright busted/unplayable. You can call it a Variety book if you like.
That way testers can test with your Perfect book, and then test with the expanded Variety book as well, for better comparison.
Thank you for your very good work!
Carl
Not at all...
Thanks too for your useful comments
Yes...your idea sounds not bad,actually i was considering about releasing such varied neutral short book
But later i've changed my mind and do you know why,see below please
If we will start testing the engines with such various openings,then the game results will be look like more interesting,more exiting...
But then i am afraid that we will need to run minimum 3.000 - 5.000 games per player
For example,1000 games (per player) are enough data to show the real strength,if the engines are using Perfect 2012b version
Plus in case of releasing a such varied book,many engines Elo standings will be effected
Probably such varied book will be a good idea,if we start completely new testing or rating list (with new conditions)
Of course,i agree with you that those engines,which can play this type of disadvantage positions should be rewarded for it
But unfortunately,there are some opening positions, where mostly of the engines are falling in real trouble
And i think its a mistake and injustice,if we will allow the engines to be tested in such critical positions
Note also that almost all Top Human Players or Top Book Makers dont prefer various openings,each of them has own favorite lines
I think the answer is no so hard to know about why they prefer not varied openings...
Best,
Sedat
-
Nelson Hernandez
- Posts: 101
- Joined: Sun Nov 14, 2010 9:36 pm
- Location: U.S.
Re: 60 games Komodo 5 against Top4 at 120m+3s
Did you ever read "1984"?
I can imagine you leading the "two minutes hate".
I can imagine you leading the "two minutes hate".
-
Nelson Hernandez
- Posts: 101
- Joined: Sun Nov 14, 2010 9:36 pm
- Location: U.S.
Re: 60 games Komodo 5 against Top4 at 120m+3s
The position you cite has been seen many times, and while 10.cxd5 is the most common response 10.b3 has actually done well because most black players have played 10...b5? instead of the the much better 10...Bb7. What appears best in the subsequent position is 11.Ba3, which has resulted in five white wins against one loss and three draws. The key is exchanging off those bishops so the queen can dominate that diagonal.
-
TimoK
- Posts: 98
- Joined: Sun Jan 03, 2010 12:28 pm
- Location: Hamburg
Re: 60 games Komodo 5 against Top4 at 120m+3s
Test finished!
Here are the results (Komodo 4 results in brackets):
Komodo 5 vs. Houdini 2.0c: 26.5-33.5 (K4: 28.0-32.0)
Komodo 5 vs. Stockfish 2.2.2: 30.5-29.5 (K4: 31.5-28.5)
Komodo 5 vs. Critter 1.6: 29.5-30.5 (K4: 29.5-30.5)
Komodo 5 vs. Rybka 4.1: 30.0-30.0 (K4: 33.0-27.0)
Possible conclusions:
a) 160 games are by far too less to measure the improvement of an engine, especially when you test it against opponents of nearly equal strength.
b) Komodo 5 shows no improvement over Komodo 4 for the used test conditions, i.e. long TCs with low increment, AMD CPU, strong opponents, Noomen Opening Suite 2012.
I think the probable conclusion should be (a), but I can't hide that I'm a little bit disappointed that K5 didn't manage to win a single match clearly to show some measureable improvement. Of course that doesn't mean that I question the 30 Elo improvement that Don and Larry claim - not at all. But it's still possible that this improvement doesn't show under these special circumstances.
Best regards
Timo
Here are the results (Komodo 4 results in brackets):
Komodo 5 vs. Houdini 2.0c: 26.5-33.5 (K4: 28.0-32.0)
Komodo 5 vs. Stockfish 2.2.2: 30.5-29.5 (K4: 31.5-28.5)
Komodo 5 vs. Critter 1.6: 29.5-30.5 (K4: 29.5-30.5)
Komodo 5 vs. Rybka 4.1: 30.0-30.0 (K4: 33.0-27.0)
Possible conclusions:
a) 160 games are by far too less to measure the improvement of an engine, especially when you test it against opponents of nearly equal strength.
b) Komodo 5 shows no improvement over Komodo 4 for the used test conditions, i.e. long TCs with low increment, AMD CPU, strong opponents, Noomen Opening Suite 2012.
I think the probable conclusion should be (a), but I can't hide that I'm a little bit disappointed that K5 didn't manage to win a single match clearly to show some measureable improvement. Of course that doesn't mean that I question the 30 Elo improvement that Don and Larry claim - not at all. But it's still possible that this improvement doesn't show under these special circumstances.
Best regards
Timo