Ovyron wrote:I've been thinking about getting serious about chess engine testing, but I still have to decide on the time control, since I don't have many CPU time for this, I have to make a compromise, and these are the two time controls that I'd want to use:
1' +3" Games: This is the fastest that I would go, and I would get a decent amount of games much faster.
1' +14" Games: The idea is to have a good increment time so the game's quality remains constant and it depends on the length of the game. For example, a 40 moves game would have the quality of a 10' + 0" game, but a 80 moves game would have the quality of a 20' + 0" game, and so on. A longer game would get a better quality.
On 1' +3" long games, the quality gets severely bad because the engines have to play too fast at the end and the result is basically random. But 1' +14" games take too long to finish and it would take me a long time to get a good amount of games.
What do you suggest?
When you run fast games, I think it adds a bit more randomness, and requires that you check the results more carefully (you will see lots of losses for time, for instance).
When the chess is slow, the games are more beautiful. It would be nice if, during a thousand hours of chess contests, besides and answer "A is stronger than B." we would also have a beautiful byproduct of nice to follow games.
Now, the big problem here is that slow games take a long time, and nobody has the patience for it. So the way that this sort of thing can work is to join a consortium like SSDF, CEGT, or CCRL and add your games to the pile.
Now, if you are against doing joint projects, then probably some sort of blitz will be necessary unless you don't mind contests that take a very long time to complete, and give results for chess engines that are obsolete by the time that the contest is done.
I think for side projects if you are going to do something by yourself, a themeatic tournament might be intersting. An oddball opening that looks interesting could yield something interesting.
But for the bottom line, I would say:
"Do exactly what _you_ want to do."