OliverBr wrote: ↑Fri Jan 08, 2021 10:39 am
Graham Banks wrote: ↑Thu Jan 07, 2021 11:30 pm
Small sample size, plus the unpredictability that tournaments can throw up.
OliThink 5.9.2 64-bit is performing around 2750 Elo in CCRL after a small number of games, but the gauntlet I'm starting today will sort out how accurate that rating is.
Thank you so much, Graham! You are such a huge support for the chess programmer community!
You are right about the small sample size, but there is a pattern that I observe since quite some time in other tourneys, too.
Actually I am running a Gauntlet, too, on my 32-core Linux server with 32 concurrent games. It's the same time control as yours, 25 minutes for 40 games and ponder off.
Differences: I am using 3moves book with thousands of opening, no tablebases at all, machines use their default hash size.
So there are not that many games, the error is still quite large. It's running for 12 hours and this is an intermediate result :
Code: Select all
# PLAYER : RATING ERROR POINTS PLAYED (%) W D L D(%) CFS(%)
1 OliThink 5.9.2 : 0 ---- 145.0 198 73.2 119 52 27 26.3 100
2 Delocto 0.6 : -121 75 22.5 66 34.1 12 21 33 31.8 93
3 K2 v.0.96dev : -202 77 16.5 66 25.0 8 17 41 25.8 75
4 Drosophila 1.6 : -241 86 14.0 66 21.2 7 14 45 21.2 ---
White advantage = 80.18 +/- 24.87
Draw rate (equal opponents) = 33.23 % +/- 4.29
Of course we have to consider the fact that OliThink has been developed on that very Linux server, so it's optimizied for that.
Other than that, there is notable difference in terms of performance which I cannot explain.
With 43 games so far in CCRL (long time) the error bar is extremely large as already stated.
Here is a game when OliThink never realizes it will lose and long insists on being near draw.
I let it crunch here for a very very long time in WB (no popcount version) and I already thought it is somehow stuck
until it popped out depth 35! Look at the search explosion. Also it still had never considered the GreKo answer
to Qa1. It seems the move is sacrificed by some reduction?
(On top of this, this was done after some back and forward analysing for some moves before)
Code: Select all
dep score nodes time (not shown: tbhits knps seldep)
35 0.00 2.52G 26:32.88 g3g4 c3c2 a7b8 d5g2 h2g2 c2c1q b8f8 f6g6 f8g8 g6f6 g8f8
34 0.00 308.6M 2:57.45 a7a1 f6g6 a1a4 c3c2 g2c2 d4c2 a4c2 d5d4 c2a2 g6f6 h2h1 d4c3 h1h2 c3d4
33 0.00 179.8M 1:39.42 a7a1 f6g6 a1a4 c3c2 a4e8 g6f6 e8f8 f6g6 f8g8 g6f6 g8f8
32 0.00 121.3M 1:06.53 a7a1 f6g6 a1a4 c3c2 g2c2 d4f3 h2g2 f3e1 g2f1 e1c2 a4e8 g6f6 e8f8 f6g6 f8e8
31 0.00 63.8M 0:34.91 a7a1 f6g6 a1a4 d4f3 h2h1 c3c2 a4c2 f3e1 c2a2 d5e4 h1g1 e1g2 a2g2 e4e1 g1h2 e1d1 g2f2 g6f6 f2e3 d1c2 h2g1 c2b1 g1f2 b1a2 f2g1 a2b1
30 0.00 54.2M 0:29.55 a7a1 f6g6 a1a4 d4f3 h2h1 c3c2 a4c2 f3e1 c2a2 d5e4 h1g1 e1g2 a2g2 e4e1 g1h2 h6h5 g2g1 e1e2 h2h1 e2f3 h1h2 f3e2
29 0.00 39.3M 0:21.32 a7a1 f6g6 a1a4 d4f3 h2h1 c3c2 a4c2 f3e1 c2a2 d5e4 h1g1 e1g2 a2g2 e4e1 g1h2 h6h5 g2g1 e1e2 h2h1 e2f3 h1h2 f3e2
28 0.00 30.1M 0:16.27 a7a1 f6g6 a1a4 d4f3 h2h1 c3c2 a4c2 f3e1 c2a2 d5e4 h1g1 e1g2 a2g2 e4e1 g1h2 h6h5 g2g1 e1e2 h2h1 e2f3 h1h2 f3e2
27 0.00 22.9M 0:12.32 a7a1 f6g6 a1a4 d4f3 h2h1 c3c2 a4c2 f3e1 c2a2 d5e4 h1g1 e1g2 a2g2 e4e1 g1h2 h6h5 g2g1 e1e2 g1g2 e2d1 g2b2 d1e1 b2d4 e1e2 h2g1 e2e1 g1g2 e1e2 g2h1 e2e1 h1h2
26 0.00 15.1M 0:08.17 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
25 0.00 10.6M 0:05.77 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
24 0.00 8.07M 0:04.41 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
23 0.00 5.37M 0:02.96 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
22 0.00 3.79M 0:02.11 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
21 0.00 2.28M 0:01.29 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
20 0.00 1.36M 0:00.81 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
19 0.00 921028 0:00.56 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
18 0.00 559661 0:00.33 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
17 0.00 392830 0:00.22 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
16 0.00 269502 0:00.16 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
15 0.00 175579 0:00.11 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
14 0.00 121019 0:00.08 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
13 0.00 48440 0:00.03 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
12 0.00 38798 0:00.03 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
11 0.00 29579 0:00.01 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
10 0.00 21868 0:00.01 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
9 0.00 18916 0:00.01 a7a1 f6g6 a1a4 d4f3 h2h1 f3e1 a4e8 g6g7 e8e7 g7g6 e7e8
8 0.00 7229 0:00.01 g3g4 c3c2 a7c7 d5e4 c7d8 f6f7 d8c7 f7f6
7 0.00 6127 0:00.00 g3g4 c3c2 a7c7 d5e4 c7d8 f6f7 d8d7 f7f6 d7d8
6 +0.50 2875 0:00.00 g3g4 c3c2 a7c7 e6e5 g2f2 d4f3 f2f3
5 +0.31 1154 0:00.00 g3g4 f5g4 g2g4 f6f5 g4g2
4 +0.38 382 0:00.00 g3g4 c3c2 a7c7 e6e5
3 +0.93 142 0:00.00 g3g4 c3c2 a7c7
2 +0.70 22 0:00.00 h2g1 c3c2
1 +0.70 9 0:00.00 h2g1
[pgn]
[Event "CCRL 40/15"]
[Site "CCRL"]
[Date "2021.01.05"]
[Round "752.1.163"]
[White "OliThink 5.9.2 64-bit"]
[Black "GreKo 2020.03 64-bit"]
[Result "0-1"]
[ECO "A10"]
[Opening "English opening"]
[PlyCount "108"]
[BlackElo "2716"]
[Number "24"]
1. c4 b6
2. b3 c5
3. Bb2 Bb7
4. f4 f5
5. Nf3 Nf6
6. g3 g6
7. Bg2 Bg7
8. O-O O-O
9. Nc3 {+0.05/24 29} Nc6 {-0.08/14 25}
10. e3 {+0.03/23 49} e6 {-0.17/14 25}
11. Qe2 {+0.19/23 45} Qe7 {-0.05/14 35}
12. d4 {+0.04/23 30} d5 {-0.29/14 28}
13. dxc5 {+0.00/24 58} bxc5 {-0.08/14 37}
14. Rad1 {-0.01/24 36} dxc4 {-0.11/15 28}
15. bxc4 {+0.10/24 28} Rfd8 {-0.11/14 50}
16. h3 {+0.03/24 32} Rab8 {+0.20/15 27}
17. Ba1 {+0.00/25 86} Na5 {+0.36/14 28}
18. Rxd8+ {+0.00/24 44} Rxd8 {+0.40/17 49}
19. Rd1 {-0.10/26 31} Nd7 {+0.50/18 41}
20. e4 {+0.06/28 30} Ba6 {+0.26/17 53}
21. Nb5 {+0.00/28 33} Bxa1 {+0.38/17 46}
22. Rxa1 {+0.00/29 30} Nb6 {+0.18/18 45}
23. exf5 {+0.00/29 32} Naxc4 {+0.18/17 43}
24. Na3 {+0.00/27 80} gxf5 {+0.46/18 38}
25. Rc1 {+0.06/33 42} Nxa3 {+0.38/19 55}
26. Qxa6 {+0.00/32 38} c4 {+0.00/20 51}
27. Kh2 {+0.16/33 44} h6 {+0.00/18 56}
28. Qa5 {+0.29/28 31} Rd5 {-0.28/20 48}
29. Qc3 {+0.33/29 40} Nb5 {+0.00/18 37}
30. Qe1 {+0.00/26 37} Rc5 {+0.00/19 58}
31. Ne5 {+0.00/31 81} c3 {+0.00/18 30}
32. Qd1 {+0.00/30 51} Kh7 {+0.00/18 34}
33. a3 {+0.00/31 33} Nxa3 {+0.75/18 52}
34. Qb3 {+0.00/36 43} Nb5 {+0.83/18 66}
35. Nd3 {+0.00/30 62} Nd5 {+0.83/17 66}
36. Nxc5 {+0.00/31 32} Qxc5 {+1.07/16 38}
37. Qa2 {+0.00/32 78} Kg6 {+0.96/17 54}
38. Qa6 {+0.00/29 44} Kf7 {+0.99/16 80}
39. Bxd5 {+0.00/29 36} Qxd5 {+1.00/16 80}
40. Rc2 {+0.00/35 52} Ke7 {+0.99/17 20}
41. Rg2 {+0.00/29 25} Nd4 {+1.42/15 21}
42. Qxa7+ {+0.00/27 36} Kf6 {+1.44/14 39}
43. Qa1 {+0.00/31 29} Qc5 {+2.85/16 21}
44. Qa8 {+0.00/32 25} c2 {+3.43/17 36}
45. Rg1 {-3.55/30 71} Ne2 {+3.66/18 26}
46. Qd8+ {-3.73/31 35} Kf7 {+3.79/19 28}
47. Qd7+ {-3.95/34 24} Qe7 {+3.88/20 40}
48. Qc6 {-4.40/37 27} Qa7 {+3.90/20 40}
49. Rf1 {-4.70/37 33} c1=Q {+4.13/21 27}
50. Rxc1 {-5.95/36 30} Qf2+ {+4.26/20 26}
51. Qg2 {-5.78/35 23} Qxg2+ {+5.46/24 30}
52. Kxg2 {-6.75/36 46} Nxc1 {+5.55/25 29}
53. Kf3 {-11.20/33 36} Kf6 {+5.56/22 42}
54. Kf2 {-11.41/33 26} Nb3 {+6.03/22 27}
0-1[/pgn]
[d]8/Q7/4pk1p/3q1p2/3n1P2/2p3PP/6RK/8 w - - 1 43
