I like to run test games at a fixed number of nodes per move, because it improves reproducibility, and because it enables me to run other CPU-intensive tasks on the computer without influencing the results of the running tests.
Unfortunately, it seems that almost no engine authors bother to support the UCI "go nodes ..." command, which means that I can currently only test against older versions of my own program.
UCI engine authors, could you please implement "go nodes ..." in your next version? It's only a minute of work, after all. Every time you increment your move counter, check if it exceeds the maximum limit, and stop the search if it does.
I like to run test games at a fixed number of nodes per move, because it improves reproducibility, and because it enables me to run other CPU-intensive tasks on the computer without influencing the results of the running tests.
Unfortunately, it seems that almost no engine authors bother to support the UCI "go nodes ..." command, which means that I can currently only test against older versions of my own program.
UCI engine authors, could you please implement "go nodes ..." in your next version? It's only a minute of work, after all. Every time you increment your move counter, check if it exceeds the maximum limit, and stop the search if it does.
Tord
Gaviota supports fixed nodes/move but it is not UCI, it only supports winboard/xboard. Besides, Gaviota is not a strong engine. On the other hand, being a strong engine is not necessarily a requirement for testing as long as it is possible to play games with handicaps.
I think that movei can play with fixed number of nodes. What are the engines capable to do that. Uri?
Another advantage to play with fixed number of nodes is that we can used the debug versions to play once in while or we can choose to play really "fast" games without compromising the reliability of the test. There is a lot of things that can screw a game/2 seconds but a game played with 10,000 nodes/move is more reliable (or whatever number you wish).
When we are at it, then, I would also want to encourage authors of WinBoard engines to implement a command 'sn N', which limits the search to N nodes, similar to the way 'st' limits the serach time and 'sd' limits the depth.
You might want to send a 'feature sn=1' to announce to WinBoard that you understand it, to prevent that WinBoard will warn the user that the engine might not support the command if the user tries to set the nodes through the WinBoard menu command I will supply for this purpose. It is safe to do this in protocol 2, as protocol 2 specifies that WinBoard will simply reject unimplemented features without further consequences.
The user can always force sending of an 'sd N' to the engine by making it part of the initString.
hgm wrote:When we are at it, then, I would also want to encourage authors of WinBoard engines to implement a command 'sn N', which limits the search to N nodes, similar to the way 'st' limits the serach time and 'sd' limits the depth.
You might want to send a 'feature sn=1' to announce to WinBoard that you understand it, to prevent that WinBoard will warn the user that the engine might not support the command if the user tries to set the nodes through the WinBoard menu command I will supply for this purpose. It is safe to do this in protocol 2, as protocol 2 specifies that WinBoard will simply reject unimplemented features without further consequences.
The user can always force sending of an 'sd N' to the engine by making it part of the initString.
Yord
Being as not everyone counts nodes in exactly the same way why don't you just play them all to a fixed ply depth. It seems to be basically the same thing and everyone counts plys in the same way.
Bill
Bill Rogers wrote:Yord
Being as not everyone counts nodes in exactly the same way why don't you just play them all to a fixed ply depth. It seems to be basically the same thing and everyone counts plys in the same way.
Bill
But plies aren't equal either. One program does lots of extensions, and another has few extensions, and you get one program that gets 5x as much time as the other.
Robert Pope wrote:But plies aren't equal either. One program does lots of extensions, and another has few extensions, and you get one program that gets 5x as much time as the other.
Not only that, but reaching a certain depth gives a big distribution over both time and strength for the different phases of the game. E.g. reaching depth 10 in the opening and endgame will take different times and result in different levels of play.
I like to run test games at a fixed number of nodes per move, because it improves reproducibility, and because it enables me to run other CPU-intensive tasks on the computer without influencing the results of the running tests.
Unfortunately, it seems that almost no engine authors bother to support the UCI "go nodes ..." command, which means that I can currently only test against older versions of my own program.
UCI engine authors, could you please implement "go nodes ..." in your next version? It's only a minute of work, after all. Every time you increment your move counter, check if it exceeds the maximum limit, and stop the search if it does.
Tord
Make a note that this can give highly biased results. A simple change to your eval changes the shape of the tree ever-so-slightly, but more than enough to change the game outcome even if the change is no better or worse than without...
I've played with this idea extensively and discarded it as being unusable...
Bill Rogers wrote:Yord
Being as not everyone counts nodes in exactly the same way why don't you just play them all to a fixed ply depth. It seems to be basically the same thing and everyone counts plys in the same way.
Bill
I does not matter how you count them. It is just approximately proportional to time. As an example, rather than playing a game 1 second a move, you play N nodes a move (when N equals approx, NPS of the the engine). Since NPS increase on endgames, the game will slightly speed up at the end, which is not bad.
I like to run test games at a fixed number of nodes per move, because it improves reproducibility, and because it enables me to run other CPU-intensive tasks on the computer without influencing the results of the running tests.
Unfortunately, it seems that almost no engine authors bother to support the UCI "go nodes ..." command, which means that I can currently only test against older versions of my own program.
UCI engine authors, could you please implement "go nodes ..." in your next version? It's only a minute of work, after all. Every time you increment your move counter, check if it exceeds the maximum limit, and stop the search if it does.
Tord
Gaviota supports fixed nodes/move but it is not UCI, it only supports winboard/xboard. Besides, Gaviota is not a strong engine. On the other hand, being a strong engine is not necessarily a requirement for testing as long as it is possible to play games with handicaps.
I think that movei can play with fixed number of nodes. What are the engines capable to do that. Uri?
Another advantage to play with fixed number of nodes is that we can used the debug versions to play once in while or we can choose to play really "fast" games without compromising the reliability of the test. There is a lot of things that can screw a game/2 seconds but a game played with 10,000 nodes/move is more reliable (or whatever number you wish).
Miguel
Movei can do it as uci engine
I used it mainly to test against older versions.
I know that rybka also supports it.
I do not know much about the other engines that support it because I did not test much.
