testing strategies

[flok]

Hello,

I would like to test multiple version of my chess-program. Some are different in algorithm, some are speed-improvements that should not (that's why I test them) affect the game-play.

What is best: play them all in one big match with a few reference programs? Or should I run multiple matches where every time only 1 version is tested? (gauntlet)

Second question: if I always use the same time-control on the same computer, can I the pgn-database and just keep adding new games to it to have a large list of versions with their scores? Or does that only work if all versions played against all versions?

[flok]

(Note: for the testing of multiple versions of 1 program, each version would have its own identifier (e.g. Dog_git-tag).)