(1): Excluding the case of a SMP engine which I know would add non-deterministic results, isn't the search function deterministic? Henceforth, if I tested two versions of my engine against itself, wouldn't it always play the same game over and over? So, I'd really only have to play 2 games in testing -- engine one plays white and one with black.
(2): I've seen several mentions of "I tested 5K+ games and here's the results" or something similar. How/What are you testing? Given the assumption that non-SMP engines are deterministic, why so many games? The engines must be playing somewhat randomly then -- right? How would I go about testing my engine against itself?
(3): Currently, my engine selects a random move from the set of move with the highest score. In other words, from the scores that are returned by negascout on the root node move list, I find the max value and return a random move that has that score. Is this how moves to play are selected, if I want some randomness in moving? I've seen the use of a PV move, but is there more than one?
(4): I'm trying to calculate some form of statistics from a set of games. What stats are normally used? Currently I'm returning: white wins, black wins, ties, average ply. I've seen LOS (likelihood of superiority) here, but they don't clearly explain enough for me to convert it to code. The formula looks like:
Code: Select all
LOS = phi(x: score_difference/N(1-draw_ratio)) = (erf(x/sqrt(2)) + 1) / 2