Some Questions on Engine Testing

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

Echidna
Posts: 1
Joined: Sat Jun 11, 2022 5:07 am
Full name: Elliott Jones

Some Questions on Engine Testing

Post by Echidna »

Hello. I'm new to chess programming and had some questions on how to test if changes to my engine improve it or not.
1. What software is appropriate for testing engines? I have been using Arena GUI, but I have seen people recommending cutechess-cli.
2. What opening book is appropriate for testing? I have been using one of the built-in opening books that are installed with Arena but I'm unsure if it's good enough.
3. Should I play against the previous version of my engine or other engines?
4. How many games need to be played before I can be reasonably confident with the result?
5. How many wins does there have to be for me to be reasonably confident that the change is an improvement?
Thanks