Code: Select all
A couple of issues.
(1) in general, I've found that fast games are perfectly acceptable, so long as you are careful to not go so fast that a program starts to lose on time. After makihng several changes, you can always do an occasional verification using longer time controls, just to be safe...
Yes this was the issue I was running into while trying to find some good Stockfish settings. I was using 10 seconds per game with .1 increment. Which is exactly why I want to use fixed time per move.
Code: Select all
(2) a fixed time per move is OK, although you cut out a part of the engine's character, since you do not allow it to use more or less time for some moves, which most programs will do.
How much does this affect playing strength?
Code: Select all
(3) I do not like just playing games between A and A'. My testing has shown that this often gives results that are either inflated, or even wrong. It's better to play against several programs, preferably that are about the same strength as the engine you are testing, or a bit stronger. Don't test against a bunch of patsies, nor should you play against opponents you can only win one out of every hundred games.
Agreed. I was planning on running the 2 individual Stockfish settings against these engines:
- Critter 0.80
Naum 4.2
Komodo 1.2
Rybka 4
Houdini 1.03a
The testing methods I use are roughly the same as yours. Of course I do not have a cluster to test with...maybe "LittleBlitzer" can help me with that.
I want good accurate results and I want them quickly which is why I want to use a fixed time per move.
