Crafty 23.0 - can it beat The King 3.50 ?

[Werner]

Hi,
I have started the match Crafty 23.0 x64 2CPU - The King 3.50 x64 2CPU adapted for CEGT 40/20 conditions on my quad (as The King uses 25% CPU when idle).
Conditions:
eng -eng match with ponder off
25 different opening positions (top25.pgn) - 50 games
hash: Crafty 440 MB, The King 132 MB
GUI Arena
The King 3.50 default
8000 kns against 1500 kns

The first game was a win for The King:

[Event "Crafty23-2"]
[Site "SPEEDSTER"]
[Date "2009.04.01"]
[Round "1"]
[White "Crafty 23.0 x64 2CPU"]
[Black "TheKing350_64 2CPU"]
[Result "0-1"]
[BlackElo "2200"]
[Time "08:35:26"]
[WhiteElo "2200"]
[TimeControl "40/600:40/600:40/600"]
[Termination "normal"]
[PlyCount "126"]
[WhiteType "program"]
[BlackType "program"]

1. c4 e5 2. Nc3 Nf6 3. Nf3 Nc6 4. g3 Bb4 5. Bg2 O-O 6. O-O e4 7. Ng5 Bxc3
8. dxc3 Re8 9. c5 {-0.15/16 15} Qe7 {+0.21/225 16} 10. Be3 {-0.25/17 14} b6
{+0.33/201 40} 11. b4 {-0.26/16 14} Ba6 {+0.40/225 17} 12. a4 {-0.31/16 14}
Bc4 {+0.50/201 19} 13. Qd2 {-0.31/16 14} h6 {+0.37/225 21} 14. Nh3
{-0.42/16 15} Ne5 {+0.43/225 30} 15. Rab1 {-0.26/15 15} bxc5 {+0.48/225 18}
16. Bxc5 {-0.38/16 15} d6 {+0.71/201 18} 17. Be3 {-0.47/15 15} c5
{+0.71/225 25} 18. Rfd1 {-0.37/15 17} Rad8 {+0.73/225 12} 19. a5 {-0.66/16
22} Neg4 {+0.92/225 12} 20. b5 {-0.86/16 17} Nxe3 {+1.23/225 10} 21. fxe3
{-1.37/17 15} d5 {+1.20/225 11} 22. Rb2 {-1.50/18 15} Qd7 {+1.32/225 14}
23. b6 {-1.63/18 16} Rb8 {+1.26/249 16} 24. bxa7 {-1.61/18 16} Ra8
{+1.97/178 22} 25. Nf2 {-1.86/17 16} Rxa7 {+2.07/178 17} 26. Bh3 {-1.96/18
16} Qc6 {+2.08/178 33} 27. Rdb1 {-1.96/18 17} Rxa5 {+2.13/178 20} 28. Ng4
{-1.96/17 17} Nd7 {+2.26/201 9} 29. Nf2 {-1.96/16 17} Ra7 {+2.27/201 17}
30. Qd1 {-2.30/16 18} Rea8 {+2.36/225 10} 31. Bf5 {-2.35/16 18} Qc7
{+2.35/225 8} 32. Bxd7 {-2.39/17 18} Qxd7 {+2.45/225 6} 33. Qc2 {-2.61/18
27} Qf5 {+2.56/225 10} 34. Kg2 {-2.79/18 19} Qe5 {+2.59/225 13} 35. Qd2
{-2.62/17 21} Ra2 {+2.56/225 10} 36. Rxa2 {-2.87/20 35} Rxa2 {+2.55/154 10}
37. Rb2 {-2.86/20 18} Rxb2 {+2.58/178 14} 38. Qxb2 {-2.93/20 6} Qh5
{+2.70/155 18} 39. Qb8 {-2.76/20 22} Kh7 {+1.86/209 0} 40. Qf4 {-2.81/20
22} Bxe2 {+2.95/131 19} 41. g4 {-3.23/20 23} Qg5 {+2.95/131 21} 42. Kg3
{-3.16/20 11} f6 {+2.96/155 12} 43. Qf5 {-2.57/17 11} Qxf5 {+2.93/108 13}
44. gxf5 {-2.94/20 4} g5 {+3.03/113 12} 45. fxg6 {-2.69/22 12} Kxg6
{+2.95/90 9} 46. Kf4 {-2.68/23 12} Bf3 {+3.00/90 14} 47. Nh3 {-2.62/23 12}
Bg2 {+2.89/90 15} 48. Nf2 {-2.54/23 12} h5 {+2.83/90 12} 49. Nd1 {-2.54/25
12} Bf1 {+2.86/90 14} 50. Nb2 {-2.55/24 13} Bb5 {+2.87/90 11} 51. h4
{-2.65/24 21} Kf7 {+2.86/90 12} 52. Nd1 {-2.85/24 19} Ke6 {+2.83/90 15} 53.
Nf2 {-3.12/24 12} Bf1 {+2.77/90 11} 54. Nh1 {-3.43/25 18} Be2 {+2.66/67 13}
55. Ng3 {-4.03/25 12} Bg4 {+2.76/90 12} 56. Nf1 {-4.09/25 20} Kd6 {+3.03/90
16} 57. Nd2 {-4.38/24 12} Kc6 {+3.11/90 18} 58. Kg3 {-5.52/23 16} Kb5
{+3.01/113 14} 59. Kg2 {-7.12/24 15} Be2 {+4.63/90 18} 60. Kf2 {-10.51/24
17} Bd3 {+7.09/67 15} 61. Nb3 {-13.63/24 33} Kc4 {+7.49/67 17} 62. Na5
{-15.51/24 53} Kxc3 {+9.37/67 8} 63. Nb7 {-15.51/23 71} c4 {+10.36/90 11}
64. ... {Weiss gibt auf} 0-1

[Werner]

After 10 games each engine has 3 wins and 4 draws.

Here is a nice endgame win of The King (without tbs...):

[Event "Crafty23-2"]
[Site "SPEEDSTER"]
[Date "2009.04.01"]
[Round "10"]
[White "TheKing350_64 2CPU"]
[Black "Crafty 23.0 x64 2CPU"]
[Result "1-0"]
[BlackElo "2200"]
[Time "14:19:23"]
[WhiteElo "2200"]
[TimeControl "40/600:40/600:40/600"]
[Termination "normal"]
[PlyCount "265"]
[WhiteType "program"]
[BlackType "program"]

1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 Nc6 6. Bg5 e6 7. Qd2 a6 8.
O-O-O h6 9. Nxc6 {+0.62/201 22} bxc6 {+0.90/16 14} 10. Bf4 {+0.62/201 18}
d5 {+0.71/14 14} 11. Qe3 {+0.56/225 20} Bb4 {+0.73/15 14} 12. exd5
{+0.59/249 13} cxd5 {+0.35/15 14} 13. a3 {+0.48/249 12} Ba5 {+0.37/16 14}
14. Be2 {+0.31/225 19} O-O {+0.35/15 15} 15. Qg3 {+0.32/225 20} Bxc3
{+0.24/18 14} 16. Qxc3 {+0.28/225 8} Bd7 {+0.26/17 15} 17. Qd4 {+0.26/249
17} Bb5 {0.00/16 15} 18. Bf3 {+0.39/201 15} Rc8 {0.00/15 15} 19. b3
{+0.50/201 27} a5 {+0.04/16 15} 20. a4 {+0.51/225 15} Bd7 {+0.20/15 15} 21.
Kb2 {+0.67/249 13} Rc6 {+0.33/15 16} 22. Rhe1 {+0.66/249 17} Rb6 {+0.26/16
16} 23. c3 {+0.74/249 18} Ne8 {+0.24/16 16} 24. Ka2 {+0.73/249 17} Nd6
{+0.14/16 16} 25. Qc5 {+0.59/17 11} Rc6 {+0.21/16 16} 26. Qa7 {+0.65/249
10} Re8 {-0.01/15 16} 27. Rd2 {+0.56/249 22} Bc8 {+0.04/15 22} 28. c4
{+0.64/249 20} Nxc4 {-0.01/17 17} 29. bxc4 {+0.73/225 8} Rxc4 {-0.01/16 17}
30. Be5 {+0.50/225 16} Rxa4 {-0.01/16 18} 31. Kb1 {+0.77/225 9} Qg5
{-0.01/16 18} 32. Re3 {+0.88/249 18} Re7 {+0.28/14 55} 33. Qa8 {+1.04/225
16} Qf5 {+0.41/15 20} 34. Rc2 {+0.96/225 8} Rc4 {+0.78/16 16} 35. Be4
{+0.75/225 17} Rxe4 {+0.45/16 17} 36. Qxc8 {+0.68/225 13} Kh7 {+0.30/17 5}
37. Rxe4 {+0.60/202 15} dxe4 {+0.58/18 24} 38. Bb2 {+0.53/202 22} Qg4
{+0.53/18 22} 39. Ka2 {+0.46/226 18} a4 {+0.56/18 23} 40. g3 {+0.99/202 21}
a3 {+0.68/18 21} 41. Bxa3 {+1.16/178 21} Ra7 42. Rd2 {+1.12/178 20} Qg5
{+0.45/16 13} 43. Qc3 {+1.09/178 20} Qb5 {+0.39/15 12} 44. Qd4 {+1.00/202
14} Ra4 {+0.31/17 12} 45. Qe3 {+1.00/178 16} f5 {+0.30/17 12} 46. Qc3
{+0.99/202 15} Ra7 {+0.29/15 12} 47. h3 {+0.94/202 26} Qa4 {+0.32/16 12}
48. h4 {+0.89/226 13} Qb5 {+0.30/17 12} 49. h5 {+0.87/202 14} Ra8 {+0.22/16
12} 50. Rc2 {+0.87/202 17} Qd5 {+0.16/16 13} 51. Qb3 {+0.77/202 7} Qd1
{+0.24/18 12} 52. Rb2 {+0.66/178 17} Qd6 {+0.23/18 12} 53. Re2 {+0.48/178
17} e5 {+0.07/18 13} 54. Rb2 {+0.36/178 17} Qd4 {+0.07/18 13} 55. Kb1
{+0.56/202 13} Ra6 {-0.01/18 13} 56. f3 {+0.07/226 11} Qa7 {-0.80/17 13}
57. Ka2 {-0.23/178 12} exf3 {-0.86/18 13} 58. Qxf3 {-0.24/202 8} Qf7
{-0.65/19 13} 59. Kb1 {-0.26/226 6} e4 {-1.02/18 13} 60. Qc3 {-0.33/202 17}
Qxh5 61. Rd2 {-0.34/250 12} Qh3 {-0.77/16 17} 62. Kc2 {-0.41/250 13} Rg6
{-0.86/17 14} 63. Bd6 {-0.25/202 11} Qf1 {-1.05/17 14} 64. Qd4 {-0.31/226
16} Rf6 {-0.82/16 22} 65. Qe5 {-0.33/250 10} Qc4 {-0.95/15 14} 66. Kd1
{-0.21/226 5} Qb3 {-0.90/16 14} 67. Kc1 {-0.26/202 11} Rf7 68. Be7
{-0.14/226 19} Qc4 {-0.49/15 21} 69. Rc2 {-0.17/226 9} Qf1 {-0.57/17 14}
70. Kb2 {-0.12/202 7} Qa6 {-0.56/18 15} 71. Kc1 {-0.11/226 11} Qa7
{-0.56/18 15} 72. Bc5 {-0.05/202 15} Qa5 {-0.56/16 15} 73. Qd5 {-0.09/226
12} Re7 {-0.36/16 32} 74. Qxf5 {-0.02/226 11} g6 {-0.83/17 21} 75. Qd5
{0.00/179 21} e3 {-0.01/17 20} 76. Kb1 {0.00/202 16} Ra7 {-0.01/17 14} 77.
Rb2 {+0.36/202 23} Qe1 {-0.01/17 17} 78. Kc2 {+0.27/209 0} Qe2 {-0.01/16
16} 79. Kc3 {+0.27/202 15} Qe1 80. Kb3 {+0.27/226 12} Rc7 {-0.01/16 21} 81.
Qd6 {+0.23/250 18} Rf7 {-0.01/16 12} 82. Qe6 {+0.19/226 21} Rb7 {-0.01/15
11} 83. Ka2 {+0.15/179 11} Rxb2 {-0.01/16 13} 84. Kxb2 {-0.09/209 0} Qd2
{-0.01/16 12} 85. Kb3 {+0.09/203 9} Qd3 {-0.01/16 12} 86. Ka4 {+0.09/227 7}
e2 {-0.01/16 12} 87. Qf7 {+0.22/227 15} Kh8 88. Qf8 {+0.21/203 19} Kh7 89.
Qe7 {+0.20/179 21} Kg8 {-0.01/16 13} 90. Bb4 {+0.17/203 21} Qd1 {-0.01/15
13} 91. Ka3 {+0.17/203 8} Qa1 {-0.01/17 13} 92. Kb3 {+0.39/209 0} Qb1
{-0.01/17 13} 93. Kc4 {+0.16/179 12} Qf1 {-0.01/18 14} 94. Qe6 {+0.21/203
15} Kh7 {-0.01/18 13} 95. Kd4 {+0.24/203 20} Qd1 {-0.01/16 14} 96. Kc5
{+0.17/227 9} Qg1 {-0.01/15 13} 97. Kc6 {+0.24/227 9} Qh1 {-0.01/15 14} 98.
Kc7 {+0.24/227 7} Qf3 {-0.01/15 13} 99. Qe7 {+0.33/227 14} Kg8 100. Qe5
{+0.36/227 17} Kf7 {-0.01/15 14} 101. Kd6 {+0.57/227 17} g5 {-0.01/14 22}
102. Qd5 {+0.97/179 15} Qxd5 {+0.24/20 18} 103. Kxd5 {0.00/90 0} h5
{+0.53/29 14} 104. Ke4 {+1.24/254 14} Ke6 {+M34/29 14} 105. Kf3 {+1.12/185
17} Ke5 {+M30/29 14} 106. Ba5 {+2.91/231 14} Kf5 {+M28/31 15} 107. Bc3
{+4.42/208 16} g4 {+M27/31 15} 108. Kf2 {+3.04/208 8} e1=Q {+M27/51 15}
109. Bxe1 {+4.39/185 7} Ke4 {+M26/60 2} 110. Ke2 {+4.62/116 21} Kd4
{+M25/60 0} 111. Bf2 {+4.52/208 16} Ke4 {+M24/60 0} 112. Bb6 {+4.77/162 23}
Ke5 {+M23/60 1} 113. Ke3 {+5.00/162 24} Kf5 {+M22/60 0} 114. Bc7 {+5.24/208
25} Kg5 {+M21/60 0} 115. Ke4 {+5.29/162 23} Kg6 {+M20/60 0} 116. Kf4
{+5.35/185 17} Kf6 {+M19/60 0} 117. Be5 {+5.37/185 22} Kg6 118. Bd4
{+8.75/208 14} Kh6 {+M17/60 0} 119. Kf5 {+9.37/185 21} h4 120. gxh4
{+11.85/254 9} g3 121. Kg4 {+11.89/21 20} Kg6 122. h5 {+11.98/67 15} Kh6
123. Be3 {+M16/90 15} Kg7 124. Kf5 {+M12/113 14} g2 125. h6 {+M11/136 4}
Kh7 126. Kg5 {+M10/136 1} Kg8 127. Kg6 {+M9/182 0} Kf8 128. h7 {+M8/228 0}
g1=Q 129. Bxg1 {+M7/228 0} Ke7 130. h8=Q {+M6/19 0} Kd6 131. Qc8 {+M4/90 0}
Ke5 132. Qc4 {+M3/137 0} Kd6 133. Kf6 {+M2/161 0} ... {Schwarz gibt auf}
1-0

[swami]

I'd guess The King wins 26-24 in a 50 games match. Crafty has certainly improved but I'm still sceptic about the claim that it's very highly improved.

[Werner]

after 20 games now:

Code: Select all

1   TheKing350_64 2CPU    2200   1½0½0½01½1½10½0111½½  11.0/20
2   Crafty 23.0 x64 2CPU  2200  0½1½1½10½0½01½1000½½    9.0/20

First time the King leads with 2 points.

[Werner]

Next results:

Code: Select all

1   TheKing350_64 2CPU    2200   1½0½0½01½1½10½0111½½½01½1½10½½  16.5/30
2   Crafty 23.0 x64 2CPU  2200  0½1½1½10½0½01½1000½½½10½0½01½½   13.5/30

and after 40 games again The King 2 points better:

Code: Select all

1   TheKing350_64 2CPU    2200  +14/-12/=14 52.50%   21.0/40
2   Crafty 23.0 x64 2CPU  2200  +12/-14/=14 47.50%   19.0/40

[Werner]

With the last game Crafty lost this match with the closest result:

Code: Select all

1   TheKing350_64 2CPU    2200  +17/-16/=17 51.00%   25.5/50
2   Crafty 23.0 x64 2CPU  2200  +16/-17/=17 49.00%   24.5/50

On Sunday we will see how close the ratings are..l

[bob]

I'd guess The King wins 26-24 in a 50 games match. Crafty has certainly improved but I'm still sceptic about the claim that it's very highly improved.

What, You don't believe millions of games, and prefer to use 20 game or 50 game matches?

[swami]

I'd guess The King wins 26-24 in a 50 games match. Crafty has certainly improved but I'm still sceptic about the claim that it's very highly improved.

What, You don't believe millions of games, and prefer to use 20 game or 50 game matches?

Yeah I do believe in the value of millions of games... but who has got all that time?! especially with thousands of engines in the field nowadays and testers are left to divide time equally to test each and every version of an engine, thus limiting the number of games for each engine? I think 1000~2000 games for Crafty in CEGT after few months is normally Ok. Ratings get near accurate.