Aaron Becker wrote: I have spent a little time with Cute Chess's source, and it's extremely well-engineered.
I second this, currently it is the best source I have ever read regarding computer chess stuff.
If you wonder what it means to write good code then give it a look.
...well, after Ilari spent good words on Daydreamer I gave a look also to this engine and I have to say it is well written too...but cuteches is still the best
My personal opinion is that if an engine is well written and has clear and simple yet effective style it is far easier to 'find' good improving ideas and is far easier to implement them. So I would not separate style from content because I think they are intimately bound. Another my opinion is that a clearly written engine is an hint that the author knows what he is doing and has a clear vision. A poorely written, messy engine reflects the messiness of its developer.
I'll second your praise for these tools, except for cutechess, but only because I don't use it and know nothing about it. I engineered my own testing scheme to be just what I need it to be.
But I make heavy use of bayeselo and pgn-extract, these are definitely tools that belong in your toolbox.
Aaron Becker wrote:I'm really happy with the progress I've been making with Daydreamer lately, and I credit my success to the fast testing scheme I've adopted.
Every change I make is recorded in version control, and all testing results can be associated with a git commit-id that allows me to see exactly what the code looked like when I ran the tests.
I test my changes using several thousand games against multiple at 1s+0.1s time control. Cutechess-cli makes this very easy, and its concurrency feature simplifies things a lot.
Bayeselo makes it very easy to quantify the effect of my changes, complete with likelihood-of-superiority numbers.
PGN-Extract lets me look at the kinds of games that Daydreamer tends to win or lose and lets me home in on the areas that need work. For instance, it's easy to isolate just the games that feature pawn endgames, or just the games that end in 20 moves or fewer.
These tools are all free, and all open source, and all incredibly useful to me. I wanted to create this thread for two reasons. First, to thank the authors of these tools for their efforts--thanks to David Barnes, Rémi Coulom, Ilari Pihlajisto, Arto Jonsson, and all the authors of git. I appreciate all your hard work. Second, to promote pervasive high-speed testing as a road to engine improvement. It seems perverse to test at such fast time controls, but as I've migrated to faster and faster games and more and more of them, my results have been consistently good.
It's certainly possible to create an engine that's weak at fast time controls and strong at longer time controls, but I've never seen an engine that's extremely strong at fast time controls that's not strong in other scenarios as well. When you combine the large number of games from high speed testing with a reliable way to sort through those games and extract useful information about engine strengths and weaknesses, you have an incredibly powerful tool to improve your engine, and it's completely practical to run on modest desktop hardware.
