ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

CCRL live lists with 100 Elo reduction
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Tournaments and Matches Flat
View previous topic :: View next topic  
Author Message
Charles Roberson



Joined: 13 Mar 2006
Posts: 1697
Location: North Carolina, USA
PostPost subject: Re: CCRL live lists with 100 Elo reduction    Posted: Wed May 16, 2012 6:13 pm Reply to topic Reply with quote
michiguel wrote:
Adam Hair wrote:
To anyone who reads this:

What would be your reaction if we purposely disconnected the CCRL from any comparison to human ratings?

What if we make the rating for the top engine equal 0 Elo, so that the ratings are such that the rating of each engine directly indicates how many Elo it is behind the leading program?


This is philosophically sound, but problematic from an experimental point of view. Points of reference need to be as stable as possible. That is the reason why in science the ones that are chosen are the ones that do not obviously change and their measurement is the most precise.
For that reason, the best way to have a reference for a list like this, IMHO, is to get the ~16 engines with more games (i.e. lowest error) in a wide span of the spectrum, average them, and set that average to a fix convenient number. That will guarantee the maximum stability.

for instance, set the average of the elo of these engines to a given number.

Rybka 3 64-bit
Zappa Mexico II 64-bit
Fritz 11
Grapefruit 1.0 32 bit
Bright 0.4a
Spike 1.2 turin (8491 games!)
Ktulu 8
Chess Tiger 2007
Movei 00.8.438 (10 10 10)
Gandalf 6
Aristarch 4.5
Amyan 1.72
SOS 5.1
Ufim 8.02
AnMon 5.60
Ares 1.004

Miguel


This far better than having one engine as the base value, but still doesn't solve the FIDE alignment problem. If you are going to make your ratings look human then they should align with human ratings.

Of course, Miguel's idea can be used even with having the top rating as 0.

Can you really align with humans?

So pick a subset of the listed engines and have enough FIDE players with "nearly static" playing strength and accurate ratings
play enough games. (problem: people learn while playing a match). Then create an average as Miguel suggests or run a round robin tournament with all the engines listed by Miguel and then create an average.

Now, we are down to two fundamental problems:
1) How to deal with new faster hardware?
2) Is a 2400 player of 2012 the same as a 2400 player of 1998? Does "rating drift" exist? If so, recalibration is a periodic need.
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Subject Author Date/Time
CCRL live lists with 100 Elo reduction Graham Banks Tue May 15, 2012 5:07 am
      Re: CCRL live lists with 100 Elo reduction Charles Roberson Wed May 16, 2012 1:30 am
            Re: CCRL live lists with 100 Elo reduction Graham Banks Wed May 16, 2012 1:35 am
      Re: CCRL live lists with 100 Elo reduction Adam Hair Wed May 16, 2012 1:39 pm
            Re: CCRL live lists with 100 Elo reduction Martin Sedlak Wed May 16, 2012 2:10 pm
                  Re: CCRL live lists with 100 Elo reduction Adam Hair Wed May 16, 2012 6:35 pm
            Re: CCRL live lists with 100 Elo reduction Dan Honeycutt Wed May 16, 2012 2:37 pm
                  Re: CCRL live lists with 100 Elo reduction Adam Hair Wed May 16, 2012 6:38 pm
            Re: CCRL live lists with 100 Elo reduction Mincho Georgiev Wed May 16, 2012 2:42 pm
                  Re: CCRL live lists with 100 Elo reduction Adam Hair Wed May 16, 2012 7:56 pm
                        Re: CCRL live lists with 100 Elo reduction Mincho Georgiev Thu May 17, 2012 6:00 am
                              Re: CCRL live lists with 100 Elo reduction Adam Hair Thu May 17, 2012 11:18 am
            Re: CCRL live lists with 100 Elo reduction Charles Roberson Wed May 16, 2012 2:44 pm
                  Re: CCRL live lists with 100 Elo reduction Mincho Georgiev Wed May 16, 2012 2:49 pm
                  Re: CCRL live lists with 100 Elo reduction Adam Hair Wed May 16, 2012 7:53 pm
            Re: CCRL live lists with 100 Elo reduction Miguel A. Ballicora Wed May 16, 2012 3:54 pm
                  Re: CCRL live lists with 100 Elo reduction Charles Roberson Wed May 16, 2012 6:13 pm
                  Re: CCRL live lists with 100 Elo reduction Adam Hair Wed May 16, 2012 8:53 pm
            Re: CCRL live lists with 100 Elo reduction Ingo Bauer Wed May 16, 2012 5:29 pm
                  Re: CCRL live lists with 100 Elo reduction Charles Roberson Wed May 16, 2012 6:02 pm
                        Re: CCRL live lists with 100 Elo reduction Ingo Bauer Wed May 16, 2012 6:06 pm
                              Re: CCRL live lists with 100 Elo reduction Mark Loftus Thu May 17, 2012 4:08 pm
                                    Re: CCRL live lists with 100 Elo reduction Ingo Bauer Thu May 17, 2012 4:58 pm
      Re: CCRL live lists with 100 Elo reduction Dimitri Yasenko Wed May 16, 2012 2:23 pm
      Re: CCRL live lists with 100 Elo reduction Jorge Pichard Mon May 28, 2012 10:42 am
            Re: CCRL live lists with 100 Elo reduction Sven Schüle Mon May 28, 2012 12:57 pm
                  Re: CCRL live lists with 100 Elo reduction Sedat Canbaz Mon May 28, 2012 1:15 pm
                        Re: CCRL live lists with 100 Elo reduction Sedat Canbaz Tue May 29, 2012 12:28 am
      Re: CCRL live lists with 100 Elo reduction Piotr Cichy Sun Jun 24, 2012 12:46 pm
            Re: CCRL live lists with 100 Elo reduction Jorge Pichard Mon Jun 25, 2012 12:18 pm
                  Re: CCRL live lists with 100 Elo reduction Adam Hair Mon Jun 25, 2012 12:50 pm
                        Re: CCRL live lists with 100 Elo reduction Jorge Pichard Mon Jun 25, 2012 4:59 pm
                              Re: CCRL live lists with 100 Elo reduction Adam Hair Mon Jun 25, 2012 5:31 pm
                                    Re: CCRL live lists with 100 Elo reduction Piotr Cichy Mon Jun 25, 2012 6:02 pm
                                          Re: CCRL live lists with 100 Elo reduction Adam Hair Mon Jun 25, 2012 6:41 pm
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Tournaments and Matches

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads