I plan to run 3 AH_LTC divisions or classes, each one containing 3 engines and a candidate, and report ELO differences between engines within each class.lkaufman wrote:Assuming you plan to continue to run matches at this level among top engines (as they are released), why not have a rating list at this level based on your games? It isn't much work I think. Actually anyone could make this list based on your posts, but it makes more sense for you to do it. I think the only major issue is whether to use BayesElo or Ordo for ratings; I personally would use Ordo as there are no decisions to make regarding parameters and the ratings for a match always correspond with the ratings you get with the standard Elo/FIDE formula.
This would be the only standard time control list that is even close to being up to date. I think it would fill an important void. True, it might be limited to new versions of SF, Houdini, and Komodo unless another unrelated engine is even close to competitive, but those are the three that most people are interested in I suppose. As long as you play each new version against the two best unrelated versions, each rating would be based on at least 600 games.
Every still-to-determinate time I will run a match between number 1 of each class and the group above's candidate.
Due to CPU time limits I will only test engines in actual development and only the ones I'm interested in.
Stay tuned!