http://talkchess.com/forum/viewtopic.php?t=66530
Uri showed some Stockfish performance in opening/early midgame, but just for one position.
In the past, I tried to build a positional test-suite for openings, with no tactical shots. I used: human and engine databases CB MegaBase 2017, CB Live Book, online Chess Tempo database and Noomen.ctg opening book, and several engines analyzing.
My last version was Openings200beat07.epd is listed here:
http://talkchess.com/forum/viewtopic.ph ... t&start=49
Although Dann Corbit shows that dozens of positions are not solved by corroborated engines even at long TC, I am pretty satisfied by this positional opening suite containing 200 positions. I estimate that out of the 200 positions, about 15 have wrong solutions, and about 15 are too hard for any current engine at any time control.
I tested in Polyglot with these settings:
Code: Select all
-min-depth 3 -max-depth 99 -min-time 0.1 -max-time 5 -depth-delta 2
I used 1000 positions suite, these 200 openings repeated 5 times, to dampen statistical noise, and by jackknifing, I get that 1 standard deviation of the difference in results is about 9-10 points out of 1000.
The results:
Code: Select all
Komodo 11.2.2
score=666/1000 [averages on correct positions: depth=13.2 time=0.71 nodes=3853146]
Houdini 6.03
score=656/1000 [averages on correct positions: depth=14.3 time=0.87 nodes=6708492]
Stockfish 9
score=641/1000 [averages on correct positions: depth=14.0 time=0.74 nodes=4793389]
Andscacs 0.93
score=598/1000 [averages on correct positions: depth=12.2 time=0.69 nodes=3202933]
Shredder 13
score=573/1000 [averages on correct positions: depth=14.3 time=0.79 nodes=4678511]