First, here is the move matching procedure I am using and the rationale behind it:
Code: Select all
I start matching at the 15 move - an attempt to avoid book.
I use Stockfish for pre-test. If Stockfish plays the same move on
depth 1-12 (without varying) and so does Houdini and the player - I
ignore this move. It was probably an obvious recapture or other
obvious move.
Otherwise, I consider the move a match if Houdini would have played
the move AT ANY POINT starting at depth 7.
I tell Houdini the position and set it to play as if there were 30
seconds + 0.6 second increment on the clock. I must play at a fast
pace to be able to feasily process thousands of games this way.
The reasoning for step 3 is as follows:
If I start matching at a low depth, even the computer can make a
relatively shallow tactical blunder. So the human may not be using
a computer, and yet blunder and still get a match if I accept
shallow depth matches.
Matching whatever is shown over a few iterations is better than
matching the top N moves because it is typical for a program to be
"torn" between two or three moves. If it appears as the first move
in any PV it means Houdini might have played it given the right
depth - and of course we cannot know what level the human used.
I've currently process about 1,300 games. Here is some statistics:
There are 34 games that match 100 percent. However none of these games sampled more than 6 moves and 32 of them sampled 4 mores or less. If you consider all games, regardless of their sample count you would have to match 77.78 percent or more to be within the 95th percentile.
If you require at least 10 samples no player yet has matched Houdini's move 100% of the time. Here is the data of the top 10 matches given a sample of at least 10:
Code: Select all
92.31 12 13 Gavrilov,Al1 23rd Czech Open A
88.24 15 17 Edouard,R 45th Biel Masters Open
84.21 16 19 Vachier Lagrave,M 45th Biel Masters Open
83.33 10 12 Bakre,T 20th Montcada Open
81.82 9 11 Svidler,P 65th ch-RUS
80.00 8 10 Shomoev,A Corporation Centre Cup A
79.31 23 29 Yu Yangyi 51st World Juniors
78.57 11 14 Ganguly,S 21st Kavala Open A
76.92 10 13 Fridman,D 2nd Riga Tech Open
75.00 9 12 Berg,E Manhem GM 2012
Code: Select all
79.31 23 29 Yu Yangyi 51st World Juniors
75.00 18 24 Kononenko,Dmitry1 23rd Czech Open A
75.00 15 20 Wang Hao 45th Biel GM
75.00 15 20 Jansa,Vl 23rd Czech Open A
73.91 17 23 Wen Yang 45th Biel Masters Open
71.79 28 39 Kuzubov,Y 81st ch-UKR 2012
71.43 25 35 Vovk,Y 81st ch-UKR 2012
70.83 17 24 Vallejo Pons,F CECLUB 1st Div GpB 2012
70.00 14 20 Jakovenko,D 65th ch-RUS
69.70 23 33 Ipatov,Alexander 51st World JuniorsUsing this methodology it is clearly possible to get high match rates that are significantly above the median of about 47% in a single game. If you require at least 20 samples the median is 45.5 percent.
I want to next determine if certain players are stylistically inclined to match Houdini much more than others.
