Computer ELO ratings compared to human ELO ratings ?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Henk
Posts: 7221
Joined: Mon May 27, 2013 10:31 am

Re: Computer ELO ratings compared to human ELO ratings ?

Post by Henk »

Sven Schüle wrote:
Henk wrote:
Henk wrote:http://www.ascotti.org/programming/chess/elo.htm

Yes you're right if the difference in ELO rating is greater than 700 ELO than winning chances are practically zero.
It can never happen that the best human player wins from the best chess program 6 times in a row or more the chances will be less than 0.000001
We simply don't know the approximate ELO rating of the best chess program that it would have in a human ELO rating list, we can only guess. All ratings within a rating list are relative to all other players in that rating pool itself but are unrelated to players outside the pool. This is one of several reasons why today human ratings and computer chess ratings are not directly comparable.

Apart from that, I certainly agree that a 6-0 result of the best human player against the best engine is very unlikely. But the same applies to any other two strong chess players, especially if we consider today's draw rates, so why is that so interesting for you?
I do not know.

It is a quick performance test to see if ELO rating has improved or not.
Play six games against a player which has a 200 ELO higher rating. If you win all of them you can be certain that ELO has improved.

Play six games against a player which has 200 ELO lower rating. If you loose all of them you can be certain that your playing strength got worse.

This can also be used for manic-depression moods.

If you think you are the best player ever lived, just proof yourself by winning six games in a row from the best chess program.

If you think you are the worst chess player ever lived ...


http://www.chessgames.com/perl/chess.pl?tid=79224

Maybe this 6-0 comes from Fischer-Taimanov candidates match 1971.
Henk
Posts: 7221
Joined: Mon May 27, 2013 10:31 am

Re: Computer ELO ratings compared to human ELO ratings ?

Post by Henk »

Henk wrote:
Sven Schüle wrote:
Henk wrote:
Henk wrote:http://www.ascotti.org/programming/chess/elo.htm

Yes you're right if the difference in ELO rating is greater than 700 ELO than winning chances are practically zero.
It can never happen that the best human player wins from the best chess program 6 times in a row or more the chances will be less than 0.000001
We simply don't know the approximate ELO rating of the best chess program that it would have in a human ELO rating list, we can only guess. All ratings within a rating list are relative to all other players in that rating pool itself but are unrelated to players outside the pool. This is one of several reasons why today human ratings and computer chess ratings are not directly comparable.

Apart from that, I certainly agree that a 6-0 result of the best human player against the best engine is very unlikely. But the same applies to any other two strong chess players, especially if we consider today's draw rates, so why is that so interesting for you?
I do not know.

It is a quick performance test to see if ELO rating has improved or not.
Play six games against a player which has a 200 ELO higher rating. If you win all of them you can be certain that ELO has improved.

Play six games against a player which has 200 ELO lower rating. If you loose all of them you can be certain that your playing strength got worse.

This can also be used for manic-depression moods.

If you think you are the best player ever lived, just proof yourself by winning six games in a row from the best chess program.

If you think you are the worst chess player ever lived ...


http://www.chessgames.com/perl/chess.pl?tid=79224

Maybe this 6-0 comes from Fischer-Taimanov candidates match 1971.
http://www.chessgames.com/perl/chess.pl?tid=79179
Or Fisher-Larsen 6-0 candidates match 1971.
Henk
Posts: 7221
Joined: Mon May 27, 2013 10:31 am

Re: Computer ELO ratings compared to human ELO ratings ?

Post by Henk »

overlord wrote:It would be interesting to see some of top chess engines participatng in top human events.Anyway,default engines ELO ratings are rather overestimated.
http://computerchess.org.uk/ccrl/4040/
http://www.husvankempen.de/nunn//40120n ... liste.html


Yes I see two total different rating lists. Maybe the smaller ratings are more reliable?
Lyudmil Tsvetkov
Posts: 6052
Joined: Tue Jun 12, 2012 12:41 pm

Re: Computer ELO ratings compared to human ELO ratings ?

Post by Lyudmil Tsvetkov »

overlord wrote:It would be interesting to see some of top chess engines participatng in top human events.Anyway,default engines ELO ratings are rather overestimated.
Stockfish in Wijk aan Zee seems like an interesting idea :D
lkaufman
Posts: 5966
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA

Re: Computer ELO ratings compared to human ELO ratings ?

Post by lkaufman »

We have quite a lot of evidence about the ratings top engines would get against top human players.
Let's with Rybka 3 on one core as a standard, because more evidence exists for it than for any other engine. CCRL rates it at 3057 (40/40), CEGT at 2900 (40/20), IPON at 2908. Rybka 3 on 1 core is clearly much stronger than all the engines that played matches against Kramnik and Kasparov a decade ago (with overall slightly plus score for the engines), so a rating of 2900 seems reasonable just based on that. Furthermore Rybka 3 and earier versions (between Rybks 2.3 and Rybka 3) played many matches against prominent Grandmasters under a wide variety of handicap conditions, with the overall impression that on 4 cores it was playing around 3000 level, which means about 2900 on one core.
So my conclusion is that the ratings of CEGT and IPON are quite accurate relative to human FIDE ratings, while CCRL should reduce its ratings by at least one hundred elo if the intent is to match the level of human ratings.
One other consideration is that rating differences in engine lists are somewhat exaggerated relative to differences that would be seen against humans, for various technical reasons, so it follows that if Rybka 3 on one core is properly rated at 2900 or so by CEGT and IPON, the ratings of Houdini 3 and Komodo MP on 4 or 6 cores will be too high compared to what they would be against humans, by roughly 20% of the excess over 2900. So a rating by CEGT of 3200 would probably translate to something like 3140 if it played in enough human events.
Naturally all ratings assume that the opponents don't have access to the book used by the engines, and that MP (or something else) provides enough randomness that the opponents won't be able to predict the computers' moves to any significant degree prior to the game. If I can use the computer's book during a game (as Kramnik was allowed to do against Deep Fritz in the final match), even I can get a draw occasionally against Komodo MP if I have White, and I'm only in the 2300s now. But that's no way to rate an opponent.
overlord
Posts: 198
Joined: Sun Jun 03, 2012 6:46 pm
Location: Trinec, Czech Republic

Re: Computer ELO ratings compared to human ELO ratings ?

Post by overlord »

Larry,do you belive that those matches against Fritz,Junior or Hiarcs were normal?I would bet a lot of money that Kasparov or Kramnik got some finances not to slaughter engine.You dont have to be GM to smash Junor version that played against Kasparov.Junior is very,very weak in strategy...
lkaufman
Posts: 5966
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA

Re: Computer ELO ratings compared to human ELO ratings ?

Post by lkaufman »

overlord wrote:Larry,do you belive that those matches against Fritz,Junior or Hiarcs were normal?I would bet a lot of money that Kasparov or Kramnik got some finances not to slaughter engine.You dont have to be GM to smash Junor version that played against Kasparov.Junior is very,very weak in strategy...
Yes, I believe they were normal. I rarely believe in conspiracy theories without a lot of evidence. A company or even a government risks a lot by doing things like this; one disgruntled employee can ruin everything (see Snowden case for example). In the case of these matches, the results were quite consistent, all the matches were evenly fought until the Fritz win over Kramnik. Surely if Kramnik were paid to lose the match, it would not be done by having him overlook mate in 1! Probably a good anti-computer strategy might have worked against Junior or another engine of the time, but probably the champs either were unfamiliar with the details of the weaknesses of the engines or felt it would be unsuitable/unsportsmanlike to play in too artificial a manner. Finally, the book has a lot to do with whether you can get positions from which you can beat an engine. Maybe you can beat an engine with a general book or one that you know well, but to beat one with a book prepared for a match with a human is another matter entirely.
overlord
Posts: 198
Joined: Sun Jun 03, 2012 6:46 pm
Location: Trinec, Czech Republic

Re: Computer ELO ratings compared to human ELO ratings ?

Post by overlord »

To explain why I write about the match between Kasparov and Junior.In one game Kasparov fully demonstrated anticomputer strategy and won in funny manner.Anyway,in other games he didnt play this way...it is suspicious for me.If he wanted to win the match he would use it all the time...
overlord
Posts: 198
Joined: Sun Jun 03, 2012 6:46 pm
Location: Trinec, Czech Republic

Re: Computer ELO ratings compared to human ELO ratings ?

Post by overlord »

Another argument is that about Fritz 8-10 it was well known that they had bad pawn handling (overestimated doubled pawns disadvantage).Such kind of weakness should be sufficient for top GM.
carldaman
Posts: 2283
Joined: Sat Jun 02, 2012 2:13 am

Re: Computer ELO ratings compared to human ELO ratings ?

Post by carldaman »

Henk wrote:
overlord wrote:It would be interesting to see some of top chess engines participatng in top human events.Anyway,default engines ELO ratings are rather overestimated.
http://computerchess.org.uk/ccrl/4040/
http://www.husvankempen.de/nunn//40120n ... liste.html


Yes I see two total different rating lists. Maybe the smaller ratings are more reliable?
I doubt that very much. I tend to think the higher ratings are more reliable.

10 years or so ago, Chess Tiger did play in some human GM/IM events and simply destroyed the opposition. That was 10+ years ago. Then a few years later, world champion Kramnik lost his match against Deep Fritz 4-2, and afterwards Hydra destroyed very solid super-GM Michael Adams in a 6 game match where Adams only managed one draw. This last match was in 2005, and that was 8 years ago.

Nowadays top engines can give pawn odds to the best human players. Why would you think these engines should be lower rated ?!

The same thing can be said about low end engines, rated 1800-2100. If you're human player in that rating range try playing some games against such engines and see how easy it is. :wink: It is not easy at all -- it actually feels like you're playing a master.

Regards,
CL