Discussion of anything and everything relating to chess playing software and machines.
Moderators: hgm , Rebel , chrisw
chrisw
Posts: 4313Joined: Tue Apr 03, 2012 4:28 pm
Post by chrisw Mon Sep 03, 2018 2:18 pm
sim predictions for CCCC first round (Rapid Rumble), after 93 rounds, using elos taken from CCCC list.
(some features not implemented yet, and still using TCEC tiebreak rules, will fix that soonishly)
Code: Select all
Engine Tournament Initial First Second Third Fourth Fifth Sixth Seventh Eighth
Stockfish 3439 3439 0.448 0.305 0.148 0.056 0.026 0.010 0.004 0.001 0.001 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
Komodo 3404 3404 0.365 0.317 0.180 0.074 0.038 0.015 0.006 0.003 0.001 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
Houdini 3400 3400 0.133 0.227 0.290 0.183 0.077 0.043 0.024 0.011 0.008 0.003 0.001 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
Fire 3326 3326 0.020 0.054 0.118 0.177 0.183 0.148 0.112 0.071 0.057 0.032 0.016 0.007 0.003 0.001 0.001 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
Shredder 3287 3287 0.017 0.036 0.083 0.132 0.168 0.154 0.131 0.139 0.048 0.039 0.025 0.015 0.006 0.003 0.002 0.001 0.001 0.000 0.000 0.000 0.000 0.000 0.000 0.000
Lc0 3300 3300 0.006 0.025 0.070 0.150 0.141 0.151 0.142 0.096 0.094 0.059 0.032 0.016 0.008 0.004 0.002 0.001 0.001 0.000 0.000 0.000 0.000 0.000 0.000 0.000
Ethereal 3283 3283 0.006 0.022 0.063 0.096 0.163 0.166 0.145 0.109 0.096 0.061 0.035 0.019 0.010 0.005 0.002 0.001 0.001 0.000 0.000 0.000 0.000 0.000 0.000 0.000
Andscacs 3244 3244 0.002 0.003 0.010 0.031 0.035 0.058 0.087 0.139 0.124 0.139 0.122 0.089 0.061 0.038 0.024 0.016 0.008 0.005 0.003 0.001 0.002 0.001 0.000 0.000
Fizbo 3259 3259 0.001 0.005 0.018 0.047 0.070 0.099 0.125 0.131 0.153 0.125 0.090 0.057 0.034 0.020 0.011 0.006 0.004 0.002 0.001 0.001 0.001 0.000 0.000 0.000
Booot 3276 3276 0.001 0.004 0.016 0.041 0.071 0.100 0.122 0.129 0.151 0.127 0.093 0.058 0.035 0.022 0.012 0.007 0.004 0.002 0.001 0.001 0.001 0.000 0.000 0.000
Laser 3226 3226 0.000 0.001 0.003 0.008 0.018 0.034 0.053 0.072 0.112 0.141 0.149 0.124 0.092 0.066 0.045 0.029 0.022 0.013 0.008 0.004 0.004 0.002 0.001 0.000
Nirvana 3186 3186 0.000 0.000 0.001 0.002 0.003 0.006 0.013 0.033 0.035 0.061 0.091 0.122 0.133 0.117 0.101 0.073 0.065 0.048 0.035 0.021 0.021 0.012 0.005 0.000
Gull 3184 3184 0.000 0.000 0.000 0.001 0.002 0.006 0.013 0.020 0.041 0.065 0.095 0.120 0.126 0.118 0.100 0.077 0.066 0.049 0.037 0.028 0.018 0.011 0.005 0.000
Fritz 3200 3200 0.000 0.000 0.000 0.001 0.003 0.007 0.013 0.023 0.042 0.068 0.097 0.101 0.139 0.124 0.102 0.077 0.065 0.047 0.033 0.021 0.019 0.012 0.005 0.000
Xiphos 3179 3179 0.000 0.000 0.000 0.000 0.001 0.002 0.004 0.007 0.016 0.031 0.051 0.077 0.094 0.107 0.110 0.083 0.108 0.088 0.068 0.051 0.046 0.033 0.019 0.002
Nemorino 3099 3099 0.000 0.000 0.000 0.000 0.000 0.001 0.001 0.010 0.005 0.012 0.024 0.048 0.056 0.078 0.094 0.099 0.108 0.106 0.097 0.077 0.080 0.061 0.037 0.005
Texel 3144 3144 0.000 0.000 0.000 0.000 0.000 0.000 0.001 0.002 0.004 0.009 0.019 0.031 0.048 0.066 0.082 0.089 0.105 0.108 0.107 0.121 0.081 0.070 0.049 0.008
Ivanhoe 3115 3115 0.000 0.000 0.000 0.000 0.000 0.000 0.001 0.002 0.004 0.010 0.019 0.039 0.044 0.063 0.078 0.088 0.102 0.106 0.106 0.089 0.098 0.083 0.059 0.009
Senpai 3112 3112 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.001 0.003 0.007 0.015 0.026 0.038 0.055 0.071 0.082 0.098 0.105 0.109 0.096 0.109 0.096 0.074 0.015
Vajolet 3101 3101 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.001 0.002 0.005 0.012 0.022 0.032 0.047 0.063 0.142 0.065 0.086 0.097 0.097 0.114 0.108 0.087 0.018
Pedone 3090 3090 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.001 0.003 0.007 0.015 0.023 0.035 0.051 0.062 0.080 0.097 0.113 0.156 0.114 0.118 0.101 0.022
Wasp 3041 3041 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.001 0.002 0.007 0.008 0.015 0.024 0.034 0.050 0.069 0.093 0.132 0.135 0.172 0.199 0.058
Arasan 3123 3123 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.001 0.003 0.006 0.008 0.015 0.024 0.033 0.048 0.065 0.087 0.097 0.139 0.175 0.224 0.074
Crafty 3013 3013 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.001 0.002 0.004 0.008 0.018 0.045 0.133 0.787
frankp
Posts: 228Joined: Sun Mar 12, 2006 3:11 pm
Post by frankp Mon Sep 03, 2018 3:11 pm
that should help fill out the chess.com 'guess the final positions' competition
.
Out of interest, how did you score the two lc0 adjudicated games (for example, there were other crashes too), in terms of the chess performance to aid the prediction: +1 against ivanhoe and 0.5 gull?
I guess predicting crashes is harder.
Some surprising finishing places compared to the current table.
After 94 games (I am watching game 95 now).
Still many days to go yet.
chrisw
Posts: 4313Joined: Tue Apr 03, 2012 4:28 pm
Post by chrisw Mon Sep 03, 2018 3:56 pm
frankp wrote: ↑ Mon Sep 03, 2018 3:11 pm
that should help fill out the chess.com 'guess the final positions' competition
.
Out of interest, how did you score the two lc0 adjudicated games (for example, there were other crashes too), in terms of the chess performance to aid the prediction: +1 against ivanhoe and 0.5 gull?
I guess predicting crashes is harder.
Some surprising finishing places compared to the current table.
After 94 games (I am watching game 95 now).
Still many days to go yet.
I got the results from directly parsing the CCCC schedule, so if that doesn't account for crashes and so on, neither will mine (at the moment).
The sim code is a hacked version of the TCEC sim code where I had to completely change the input data parser (the TCEC code was written to read the TCEC cross table, then hacked to read the new format beta cross table, then completely changed to read CCCC because their cross table doesn't let me get at the formatting very easily.
So, it may be a bit broken at the moment. Will check for weirdnesses ....
chrisw
Posts: 4313Joined: Tue Apr 03, 2012 4:28 pm
Post by chrisw Tue Sep 04, 2018 1:02 pm
sim predictions for CCCC first round (Rapid Rumble), after 123 rounds, using elos taken from CCCC list.
Code: Select all
Engine Tournament Init 1st 2nd 3rd 4th 5th 6th 7th ....
Komodo 3455 3404 0.37 0.26 0.19 0.12 0.04 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Houdini 3449 3400 0.28 0.28 0.22 0.14 0.05 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Stockfish 3446 3439 0.23 0.27 0.25 0.17 0.06 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Shredder 3405 3287 0.10 0.15 0.21 0.28 0.16 0.07 0.02 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Fire 3373 3326 0.01 0.03 0.07 0.17 0.34 0.21 0.09 0.04 0.02 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Lc0 3348 3300 0.00 0.01 0.03 0.09 0.22 0.32 0.17 0.08 0.04 0.02 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Booot 3318 3276 0.00 0.00 0.01 0.02 0.07 0.15 0.24 0.20 0.13 0.09 0.05 0.03 0.01 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Andscacs 3291 3244 0.00 0.00 0.00 0.01 0.03 0.09 0.18 0.21 0.17 0.12 0.08 0.05 0.03 0.01 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Xiphos 3263 3179 0.00 0.00 0.00 0.00 0.01 0.03 0.07 0.13 0.16 0.17 0.15 0.11 0.07 0.05 0.03 0.01 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Ethereal 3277 3283 0.00 0.00 0.00 0.00 0.01 0.03 0.08 0.13 0.16 0.17 0.15 0.11 0.07 0.04 0.02 0.01 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Fritz 3270 3200 0.00 0.00 0.00 0.00 0.01 0.04 0.09 0.14 0.18 0.17 0.14 0.10 0.06 0.04 0.02 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Fizbo 3168 3259 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.01 0.02 0.04 0.06 0.10 0.15 0.21 0.22 0.13 0.04 0.01 0.00 0.00 0.00
Texel 3191 3144 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.02 0.03 0.06 0.09 0.12 0.14 0.16 0.15 0.12 0.07 0.03 0.01 0.00 0.00 0.00 0.00
Vajolet 3206 3101 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.02 0.04 0.07 0.10 0.13 0.15 0.16 0.13 0.09 0.05 0.02 0.00 0.00 0.00 0.00 0.00
Gull 3235 3184 0.00 0.00 0.00 0.00 0.00 0.00 0.02 0.04 0.06 0.10 0.13 0.16 0.15 0.12 0.09 0.06 0.03 0.02 0.00 0.00 0.00 0.00 0.00 0.00
Laser 3203 3226 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.01 0.03 0.05 0.09 0.12 0.15 0.16 0.16 0.12 0.06 0.02 0.01 0.00 0.00 0.00 0.00
Ivanhoe 3211 3115 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.02 0.03 0.05 0.08 0.12 0.15 0.16 0.14 0.11 0.07 0.04 0.01 0.00 0.00 0.00 0.00 0.00
Pedone 3149 3090 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.01 0.03 0.05 0.08 0.13 0.21 0.25 0.14 0.05 0.02 0.01 0.00 0.00
Arasan 3111 3123 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.02 0.05 0.10 0.20 0.34 0.16 0.07 0.03 0.01 0.00
Nemorino 3059 3099 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.03 0.09 0.22 0.24 0.23 0.17 0.01
Senpai 3063 3112 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.04 0.10 0.22 0.24 0.22 0.16 0.01
Wasp 3040 3041 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.02 0.07 0.19 0.24 0.25 0.21 0.01
Nirvana 3048 3186 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.04 0.10 0.17 0.25 0.39 0.03
Crafty 2915 3013 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.05 0.93
JJJ
Posts: 1346Joined: Sat Apr 19, 2014 1:47 pm
Post by JJJ Tue Sep 04, 2018 1:53 pm
lc0 has still chance to finish first. Also Komodo and Houdini are on part to be first, I would say 50/50. Stockfish might finished first as well. Fire and Shredder won't hold for long. So I disagree with your simulation.
chrisw
Posts: 4313Joined: Tue Apr 03, 2012 4:28 pm
Post by chrisw Tue Sep 04, 2018 3:32 pm
JJJ wrote: ↑ Tue Sep 04, 2018 1:53 pm
lc0 has still chance to finish first. Also Komodo and Houdini are on part to be first, I would say 50/50. Stockfish might finished first as well. Fire and Shredder won't hold for long. So I disagree with your simulation.
To me it's 25% for Stockfish Houdini Komodo and Lc0 and perhaps a little less for Lc0.
Could be. I can't do much more than splatter (formulaically randomise) the results of games between X and Y according to their elo ratings. Thus the intial elos (given from the CCCC listings) and the tournament elo adjustments are critical. For example, Shredder is right at the top in performance, but the sim marks it down because the start elo is quite low wrt top three. Maybe I should increase the influence of the tournament elo, but I need to make sure there aren't any errors in the algorithm or my coding first.
JJJ
Posts: 1346Joined: Sat Apr 19, 2014 1:47 pm
Post by JJJ Tue Sep 04, 2018 4:03 pm
Yes I think it would be a good idea.
frankp
Posts: 228Joined: Sun Mar 12, 2006 3:11 pm
Post by frankp Tue Sep 04, 2018 4:36 pm
Shredder has yet to play any of the top four, discounting itself of course.
Milos
Posts: 4190Joined: Wed Nov 25, 2009 1:47 am
Post by Milos Wed Sep 05, 2018 1:51 am
chrisw wrote: ↑ Tue Sep 04, 2018 1:02 pm
sim predictions for CCCC first round (Rapid Rumble), after 123 rounds, using elos taken from CCCC list.
Code: Select all
Engine Tournament Init 1st 2nd 3rd 4th 5th 6th 7th ....
Komodo 3455 3404 0.37 0.26 0.19 0.12 0.04 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Houdini 3449 3400 0.28 0.28 0.22 0.14 0.05 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Stockfish 3446 3439 0.23 0.27 0.25 0.17 0.06 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Shredder 3405 3287 0.10 0.15 0.21 0.28 0.16 0.07 0.02 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Fire 3373 3326 0.01 0.03 0.07 0.17 0.34 0.21 0.09 0.04 0.02 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Lc0 3348 3300 0.00 0.01 0.03 0.09 0.22 0.32 0.17 0.08 0.04 0.02 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Booot 3318 3276 0.00 0.00 0.01 0.02 0.07 0.15 0.24 0.20 0.13 0.09 0.05 0.03 0.01 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Andscacs 3291 3244 0.00 0.00 0.00 0.01 0.03 0.09 0.18 0.21 0.17 0.12 0.08 0.05 0.03 0.01 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Xiphos 3263 3179 0.00 0.00 0.00 0.00 0.01 0.03 0.07 0.13 0.16 0.17 0.15 0.11 0.07 0.05 0.03 0.01 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Ethereal 3277 3283 0.00 0.00 0.00 0.00 0.01 0.03 0.08 0.13 0.16 0.17 0.15 0.11 0.07 0.04 0.02 0.01 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Fritz 3270 3200 0.00 0.00 0.00 0.00 0.01 0.04 0.09 0.14 0.18 0.17 0.14 0.10 0.06 0.04 0.02 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Fizbo 3168 3259 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.01 0.02 0.04 0.06 0.10 0.15 0.21 0.22 0.13 0.04 0.01 0.00 0.00 0.00
Texel 3191 3144 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.02 0.03 0.06 0.09 0.12 0.14 0.16 0.15 0.12 0.07 0.03 0.01 0.00 0.00 0.00 0.00
Vajolet 3206 3101 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.02 0.04 0.07 0.10 0.13 0.15 0.16 0.13 0.09 0.05 0.02 0.00 0.00 0.00 0.00 0.00
Gull 3235 3184 0.00 0.00 0.00 0.00 0.00 0.00 0.02 0.04 0.06 0.10 0.13 0.16 0.15 0.12 0.09 0.06 0.03 0.02 0.00 0.00 0.00 0.00 0.00 0.00
Laser 3203 3226 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.01 0.03 0.05 0.09 0.12 0.15 0.16 0.16 0.12 0.06 0.02 0.01 0.00 0.00 0.00 0.00
Ivanhoe 3211 3115 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.02 0.03 0.05 0.08 0.12 0.15 0.16 0.14 0.11 0.07 0.04 0.01 0.00 0.00 0.00 0.00 0.00
Pedone 3149 3090 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.01 0.03 0.05 0.08 0.13 0.21 0.25 0.14 0.05 0.02 0.01 0.00 0.00
Arasan 3111 3123 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.02 0.05 0.10 0.20 0.34 0.16 0.07 0.03 0.01 0.00
Nemorino 3059 3099 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.03 0.09 0.22 0.24 0.23 0.17 0.01
Senpai 3063 3112 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.04 0.10 0.22 0.24 0.22 0.16 0.01
Wasp 3040 3041 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.02 0.07 0.19 0.24 0.25 0.21 0.01
Nirvana 3048 3186 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.04 0.10 0.17 0.25 0.39 0.03
Crafty 2915 3013 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.05 0.93
Based on these results, it seems you are not using any advantage regarding opening side. Shredder only won games with white against 6th and 7th engines and didn't play any of top 5.
And again using tournament rating is just ridiculous. Primer information from CCRL should have at least 90% of weight and games max 10% and that's after the whole tournament.
SF still has at least 50% of chance of finishing first, and H and K around 22%. All other engines combined should be around 6%.
JJJ
Posts: 1346Joined: Sat Apr 19, 2014 1:47 pm
Post by JJJ Wed Sep 05, 2018 2:01 pm
Lc0 still on part with the top 3. I would give lc0 8% to reach top 1. Other engine like fire and shredder doesn't have 2% to me.
.