Today the CCRL-list was updated with the test for Rustic Alpha 2:
Rustic Alpha 2
Thanks for testing guys
Even so, I think the engine's rating is disappointing: the addition of the TT + TTMove sorting yielded +200 Elo in self-play, and +165 in my gauntlets. That would put the engine at around 1840, but it only achieved 1781 in the CCRL-list. There can be a few possibilities:
- I forgot to setup hash tables for all the other engines in my gauntlet, and ran both Rustic Alpha 1 and 2 against those engines. It could be that both massively overperformed. I'm rerunning two new gauntlets, with both Alpha 1 and Alpha 2.
- I only test Rustic against other engines: the other engines don't play one another. (I don't have the computing power to do this; if I upgrade to a 16+ core computer at some time, I may have the option to do a few round-robin tournaments.) It could be that my results are skewed because of this.
- CCRL used different time controls (2m+1s for them, and 1m+0.6s for me), different opening books, and apart from one, different opponents.
- It could also be that I mistakenly uploaded the binary with the TT move sorting bug in it, which _would_ have caused the engine to underperform 60 Elo. I'll ask Gabor to redownload it to be sure. Even so, it doesn't matter. Even if I did, the next test with Alpha 3 will just have a bigger Elo jump.