Hard-Talkchess-2020 set, final release
Moderators: hgm, Dann Corbit, Harvey Williamson
Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
-
- Posts: 233
- Joined: Sat Mar 18, 2006 3:01 am
- Location: Cold
Re: Hard-Talkchess-2020 set, final release
Engine: SugaR AI ICCF 1.20 (8192 MB)
by Stockfish Team, Marco Zerbinati, Khalid
1/01 0:00 +1.25 1.Bc2 (23.748) 5937
2/02 0:00 +1.25 1.Bc2 bxc3 (36.365) 7273
3/03 0:00 +1.25 1.Bc2 bxc3 2.bxc3 (94.748) 15791
...
19/33 0:15 +0.57 1.Nh4 bxc3 2.bxc3 Nxh4 3.Qh3 Nf5
4.Qxh5 Rb2 5.Bg4 Bd7 6.Kh1 Rxf2
7.Rg1 Bxe5 8.Bxe5 Qe7 9.Bxf5 exf5
10.g6 fxg6 11.Qxg6 Rf7 12.Qg3 Re2 (373.026.292) 23759
...
best move: Nf3-h4 time: 0:17.906 min n/s: 23.652.532 nodes: 423.758.766
Threads=24
Engine Options:
multiPV Search=4
Re: Hard-Talkchess-2020 set, final release
Lc0 don't consider other moves:
Analysis by Lc0 v0.26.3:
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/23 00:00:00 77kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/24 00:00:00 77kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/24 00:00:00 77kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/24 00:00:00 78kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Qf7 10.h4 Ba5 11.h5 Rxh5 12.Nxh5 gxh5
+/= (0.63) Depth: 15/26 00:00:02 81kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5 gxh5
+/= (0.63) Depth: 15/27 00:00:02 81kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5 gxh5
+/= (0.64) Depth: 15/27 00:00:07 92kN
Analysis by Lc0 v0.26.3:
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/23 00:00:00 77kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/24 00:00:00 77kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/24 00:00:00 77kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/24 00:00:00 78kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Qf7 10.h4 Ba5 11.h5 Rxh5 12.Nxh5 gxh5
+/= (0.63) Depth: 15/26 00:00:02 81kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5 gxh5
+/= (0.63) Depth: 15/27 00:00:02 81kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5 gxh5
+/= (0.64) Depth: 15/27 00:00:07 92kN
Jouni
Re: Hard-Talkchess-2020 set, final release
I settled for minimal analysis depth = 10 and Abort if the correct move is held = 3 with activate abort analysis set to ON.Vinvin wrote: ↑Mon Dec 28, 2020 3:51 pmAt left, it's not very important, it's just the lines displayed after n-depth. I set to "10" but you can leave at "2".
I activate "abort", minimal depth= "24", if correct move is held "3" additional plies.
Results, Lc0 is at rank 26.
Code: Select all
+----+----------------------+------------+---------------+-------------+------------------------------+----------------+-----------+
| | name | numsolve | meantimesec | pospermin | cpu | gpu | threads |
|----+----------------------+------------+---------------+-------------+------------------------------+----------------+-----------|
| 0 | Bluefish-FD-XI-DC | 191 | 25 | 5 | AMD Ryzen Threadripper 3970X | na | 2 |
| 1 | Bluefish-FD-XI-TD | 181 | 153 | 30 | i7-4930K | na | 1 |
| 2 | Eman_5.40_fad_12 | 167 | 193 | 30 | i7-4930K | na | 1 |
| 3 | SugaR.AI.ICCF.1.00 | 166 | 113 | 30 | i7-4930K | na | 1 |
| 4 | ShashChess15.1 | 166 | 170 | 30 | i7-4930K | na | 1 |
| 5 | Bluefish-12-W | 162 | 187 | 30 | i7-4930K | na | 1 |
| 6 | Crystal-2020-06-29 | 159 | 199 | 30 | i7-4930K | na | 1 |
| 7 | Bluefish-XI | 159 | 222 | 30 | i7-4930K | na | 1 |
| 8 | CorChess v6 | 156 | 327 | 30 | i7-4930K | na | 1 |
| 9 | Houdidi_6.03_tac_1 | 155 | 196 | 30 | i7-4930K | na | 1 |
| 10 | Stockfish_2020-07-19 | 155 | 239 | 30 | i7-4930K | na | 1 |
| 11 | ShashChess 12.1 | 154 | 248 | 30 | i7-4930K | na | 1 |
| 12 | Stockfish_20121407 | 153 | 229 | 30 | i7-4930K | na | 1 |
| 13 | Stockfish_20062710 | 153 | 331 | 30 | i7-4930K | na | 1 |
| 14 | Eman 6.61 | 153 | 258 | 30 | i7-4930K | na | 1 |
| 15 | SugaR.AI.1.00 | 149 | 280 | 30 | i7-4930K | na | 1 |
| 16 | Stockfish 11 | 141 | 366 | 30 | i7-4930K | na | 1 |
| 17 | Honey-XI | 139 | 290 | 30 | i7-4930K | na | 1 |
| 18 | Stockfish 10 | 137 | 450 | 30 | i7-4930K | na | 1 |
| 19 | Black-Diamond-XI | 136 | 267 | 30 | i7-4930K | na | 1 |
| 20 | Black-Diamond-XR7 | 136 | 231 | 30 | i7-4930K | na | 1 |
| 21 | Crystal 2020-05-02 | 136 | 187 | 30 | i7-4930K | na | 1 |
| 22 | Crystal 1.1 | 135 | 270 | 30 | i7-4930K | na | 1 |
| 23 | Black-Diamond-XI-r2 | 135 | 158 | 30 | i7-4930K | na | 1 |
| 24 | McCain-v2 | 130 | 389 | 30 | i7-4930K | na | 1 |
| 25 | Stockfish 9 | 130 | 473 | 30 | i7-4930K | na | 1 |
| 26 | Houdidi_6.03_tac_def | 125 | 349 | 30 | i7-4930K | na | 1 |
| 27 | Lc0 v0.26.3 s1 | 123 | 10 | 2 | i7-2600K | GTX 1650 super | 2 |
| 28 | Sting-SF-16 | 122 | 234 | 30 | i7-4930K | na | 1 |
| 29 | Sting-SF-20 | 119 | 240 | 30 | i7-4930K | na | 1 |
| 30 | Ethereal 12.25 | 98 | 396 | 30 | i7-4930K | na | 1 |
| 31 | Houdini 1.5 x64 | 97 | 283 | 30 | i7-4930K | na | 1 |
| 32 | Critter 1.6a 64bit | 96 | 240 | 30 | i7-4930K | na | 1 |
| 33 | Stockfish 8 | 94 | 507 | 30 | i7-4930K | na | 1 |
| 34 | Xiphos 0.6 | 81 | 330 | 30 | i7-4930K | na | 1 |
| 35 | Ethereal 11.75 | 77 | 613 | 30 | i7-4930K | na | 1 |
| 36 | Stockfish 7 | 76 | 487 | 30 | i7-4930K | na | 1 |
+----+----------------------+------------+---------------+-------------+------------------------------+----------------+-----------+
Header Legend:
Code: Select all
Stockfish_2020-07-19: Stockfish.sse42.halfkp_256x2-32-32.Profile-nnue.2020-07-19
Bluefish-FD-XI-DC: Bluefish-FD-XI-T=2;D=N;DC 3970X 5 minutes
Bluefish-FD-XI-TD: Bluefish-FD-XI-(Tact=2,defens=no)
Houdidi_6.03_tac_1: Houdidit603-tact=1
Houdidi_6.03_tac_def: Houdidit603(default)
Eman_5.40_fad_12: Eman 5.40, FAD=12
Lc0 v0.26.3 s1: Lc0 v0.26.3 cudnn-fp16, 66680, GTX 1650 super, i7-2600K, 2 minutes/pos
Re: Hard-Talkchess-2020 set, final release
NN engines should score 100% soon! Either with learning data or by NN learning, if someone shows position in learning? No?
Jouni
Re: Hard-Talkchess-2020 set, final release
Yes I've just discovered this myself. Incredible.Jouni wrote: ↑Tue Dec 29, 2020 8:28 amLc0 don't consider other moves:
Analysis by Lc0 v0.26.3:
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/23 00:00:00 77kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/24 00:00:00 77kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/24 00:00:00 77kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/24 00:00:00 78kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Qf7 10.h4 Ba5 11.h5 Rxh5 12.Nxh5 gxh5
+/= (0.63) Depth: 15/26 00:00:02 81kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5 gxh5
+/= (0.63) Depth: 15/27 00:00:02 81kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5 gxh5
+/= (0.64) Depth: 15/27 00:00:07 92kN
Is position 23 (this position) regarded as the hardest position in the testsuite?
Re: Hard-Talkchess-2020 set, final release
For others engines than LC0, yes !Werewolf wrote: ↑Tue Dec 29, 2020 7:13 pmYes I've just discovered this myself. Incredible.Jouni wrote: ↑Tue Dec 29, 2020 8:28 amLc0 don't consider other moves:
Analysis by Lc0 v0.26.3:
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/23 00:00:00 77kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/24 00:00:00 77kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/24 00:00:00 77kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5
+/= (0.62) Depth: 15/24 00:00:00 78kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Qf7 10.h4 Ba5 11.h5 Rxh5 12.Nxh5 gxh5
+/= (0.63) Depth: 15/26 00:00:02 81kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5 gxh5
+/= (0.63) Depth: 15/27 00:00:02 81kN
1.Nh4 Kh7 2.Bxh5 Rh8 3.Qg3 a5 4.Ng2 Kg8 5.Bxg6 fxg6 6.Nf4 a4 7.a3 bxc3 8.bxc3 Rb3 9.Re3 Ba5 10.h4 Qf7 11.h5 Rxh5 12.Nxh5 gxh5
+/= (0.64) Depth: 15/27 00:00:07 92kN
Is position 23 (this position) regarded as the hardest position in the testsuite?
Re: Hard-Talkchess-2020 set, final release
Behavior with 1 thread and several threads can be very different.perejaslav wrote: ↑Tue Dec 29, 2020 7:09 am
Engine: SugaR AI ICCF 1.20 (8192 MB)
by Stockfish Team, Marco Zerbinati, Khalid
...
19/33 0:15 +0.57 1.Nh4 bxc3 2.bxc3 Nxh4 3.Qh3 Nf5
4.Qxh5 Rb2 5.Bg4 Bd7 6.Kh1 Rxf2
7.Rg1 Bxe5 8.Bxe5 Qe7 9.Bxf5 exf5
10.g6 fxg6 11.Qxg6 Rf7 12.Qg3 Re2 (373.026.292) 23759
...
best move: Nf3-h4 time: 0:17.906 min n/s: 23.652.532 nodes: 423.758.766
Threads=24
Engine Options:
multiPV Search=4
If you test with several threads, please run it 4 or 5 times and make the average time.
I know my tests become not very realistic because almost everybody uses multi-thread.
I plan to buy a new computer with an AMD 5950x in late January. Likewise, I will be able to run the set faster and with multi-threading.
Re: Hard-Talkchess-2020 set, final release
I was expecting to see this position in Hard-TalkChess. A lot harder than lots of positions in this set.
Taken from:
Re: Hard-Talkchess-2020 set, final release
Here is a list of the 25 hardest positions for Houdini, Stockfish and their family ...
Code: Select all
23) Nf3-h4
33) Nf7-d6
38) Ne4-c3
7) Rh4xh1
74) Kb7-c8
57) Nc2-d4
67) f3-f4
204) Bc1-g5
184) .. Bf5-g4
68) Qc7-d8
77) Nc4-a5
91) Be3xc5
93) Nc6xe7
131) Na8-c7
70) a7-a8B
153) Ne2-d4
69) Re8xe5
150) Bd4-f6
26) Ne2-f4
99) .. Qd3xg3
144) Bg5-f6
78) Qd6xe5
130) f2-f4
89) .. Re2xd2
18) Be1-d2