CCRL question re bayeselo

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

Modern Times
Posts: 3752
Joined: Thu Jun 07, 2012 11:02 pm

Re: CCRL question re bayeselo

Post by Modern Times »

lkaufman wrote: Tue Jan 09, 2024 6:46 pm
If the books are heavily unbalanced, with White always the superior side, then the White advantage (the elo parameter in bayeselo for this) would be enormous. Would setting it at that enormous value make BayesElo work properly, or would it still be incorrect?
I updated the Chess324 list with "mm 1 1" and the effect there was to compress the ratings slightly vs before, and the error margins reduced.

https://computerchess.org.uk/ccrl/Chess324/
Michel
Posts: 2292
Joined: Mon Sep 29, 2008 1:50 am

Re: CCRL question re bayeselo

Post by Michel »

lkaufman wrote: Tue Jan 09, 2024 6:46 pm
Michel wrote: Tue Jan 09, 2024 12:07 pm This research https://unclejerry9466728.wordpress.com/2018/12/20/172/ seems to suggest that requiring that drawElo depends linearly on the "common mode" (the average elo rating of the opponents) should give reasonably good results. This would replace drawElo by two parameters (slope and intercept).

I guess I would need to download the CCRL database to double check.

Another more complicated issue with BayesElo is that it does not properly account for heavily unbalanced books in case openings are replayed with reversed colors.
If the books are heavily unbalanced, with White always the superior side, then the White advantage (the elo parameter in bayeselo for this) would be enormous. Would setting it at that enormous value make BayesElo work properly, or would it still be incorrect?
If White (or Black) is always the superior side then using the whiteAdvantage parameter is correct.

In general it is theoretically incorrect but I do not really know what the effect of this is. The only case I know is the case of two engines. In that case the Elo values are correct but the error bars are too large.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
lkaufman
Posts: 6259
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: CCRL question re bayeselo

Post by lkaufman »

Modern Times wrote: Tue Jan 09, 2024 8:27 pm
lkaufman wrote: Tue Jan 09, 2024 6:46 pm
If the books are heavily unbalanced, with White always the superior side, then the White advantage (the elo parameter in bayeselo for this) would be enormous. Would setting it at that enormous value make BayesElo work properly, or would it still be incorrect?
I updated the Chess324 list with "mm 1 1" and the effect there was to compress the ratings slightly vs before, and the error margins reduced.

https://computerchess.org.uk/ccrl/Chess324/
For this dataset, the White advantage parameter being determined from the data by mm 1 1 is the important one I believe; draw percentage is fairly normal in that data, so I think this makes sense. But if you did the same thing for the 40/15 list, where balanced/normal opening books are mostly used, I would expect a noticeable expansion of ratings, since the draw percentage would be the main issue. If you only ran the top engines data, say over 3000 for example, it should be even more dramatic.
Komodo rules!
Frank Quisinsky
Posts: 7056
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: CCRL question re bayeselo

Post by Frank Quisinsky »

http://kirr.homeunix.org/chess/engines/Jim%20Ablett/

I believe very unkown the work Ferdinand Mosca do on Bayesian.
Can be found on the site by Jim Ablett.

Best
Frank