Computer ELO ratings compared to human ELO ratings ?

[lkaufman]

It would be interesting to see some of top chess engines participatng in top human events.Anyway,default engines ELO ratings are rather overestimated.

http://computerchess.org.uk/ccrl/4040/
http://www.husvankempen.de/nunn//40120n ... liste.html


Yes I see two total different rating lists. Maybe the smaller ratings are more reliable?

I doubt that very much. I tend to think the higher ratings are more reliable.

10 years or so ago, Chess Tiger did play in some human GM/IM events and simply destroyed the opposition. That was 10+ years ago. Then a few years later, world champion Kramnik lost his match against Deep Fritz 4-2, and afterwards Hydra destroyed very solid super-GM Michael Adams in a 6 game match where Adams only managed one draw. This last match was in 2005, and that was 8 years ago.

Nowadays top engines can give pawn odds to the best human players. Why would you think these engines should be lower rated ?!

The same thing can be said about low end engines, rated 1800-2100. If you're human player in that rating range try playing some games against such engines and see how easy it is. It is not easy at all -- it actually feels like you're playing a master.

Regards,
CL

This just confirms my claim that the scale must be compressed by 20% or so. As it is, the IPON and CEGT lists might be correct (in human terms) at 2900 level, too high at 3200 level, but too low at 2000 level. Also, it has not yet been shown that any engine can give pawn odds to "the best human players", only to very good ones (Grandmasters)!

[lkaufman]

Another argument is that about Fritz 8-10 it was well known that they had bad pawn handling (overestimated doubled pawns disadvantage).Such kind of weakness should be sufficient for top GM.

Knowing such things is clearly helpful at times, but if the opening book produces positions where they are irrelevant, what can you do?

[carldaman]

It would be interesting to see some of top chess engines participatng in top human events.Anyway,default engines ELO ratings are rather overestimated.

http://computerchess.org.uk/ccrl/4040/
http://www.husvankempen.de/nunn//40120n ... liste.html


Yes I see two total different rating lists. Maybe the smaller ratings are more reliable?

I doubt that very much. I tend to think the higher ratings are more reliable.

10 years or so ago, Chess Tiger did play in some human GM/IM events and simply destroyed the opposition. That was 10+ years ago. Then a few years later, world champion Kramnik lost his match against Deep Fritz 4-2, and afterwards Hydra destroyed very solid super-GM Michael Adams in a 6 game match where Adams only managed one draw. This last match was in 2005, and that was 8 years ago.

Nowadays top engines can give pawn odds to the best human players. Why would you think these engines should be lower rated ?!

The same thing can be said about low end engines, rated 1800-2100. If you're human player in that rating range try playing some games against such engines and see how easy it is. It is not easy at all -- it actually feels like you're playing a master.

Regards,
CL

This just confirms my claim that the scale must be compressed by 20% or so. As it is, the IPON and CEGT lists might be correct (in human terms) at 2900 level, too high at 3200 level, but too low at 2000 level. Also, it has not yet been shown that any engine can give pawn odds to "the best human players", only to very good ones (Grandmasters)!

Good point - I overgeneralized with that phrase. We may infer this about the average Grandmaster, but not necessarily elite GMs.

[overlord]

Playing against Fritz it is no problem for me to make many draw even on PC.So I believe that someone like Kasparov must beat it easily.It necessary to mention that the first really modern engine that understand positional game was Rybka.So Fritz or Junior couldnt be competitors for top GM.If someone knows personally e.g. Anand,Carlsen or Kramnik...to ask them...

[Lyudmil Tsvetkov]

Playing against Fritz it is no problem for me to make many draw even on PC.So I believe that someone like Kasparov must beat it easily.It necessary to mention that the first really modern engine that understand positional game was Rybka.So Fritz or Junior couldnt be competitors for top GM.If someone knows personally e.g. Anand,Carlsen or Kramnik...to ask them...

I know an interview of Kariakin, in which he states: 'Playing Carlsen is easy, when I play against Houdini, the feeling of being doomed never leaves me'.
But of course, having knowledge of your opponent greatly helps. We, who play chess computers, are extremely favoured by the knowledge we have of the respective engine, while the engine does not know who it is playing against.

It might also be true that people playing regularly chess engines, while not competing in human tournaments, improve noticeably.

On the other hand, I am not very clear on what a computer is: why should top GMs compete against 8-16-32 core machines instead of a single core machine? Maybe in such a case the GMs should be entitled to more thinking time.

[carldaman]

Playing against Fritz it is no problem for me to make many draw even on PC.So I believe that someone like Kasparov must beat it easily.It necessary to mention that the first really modern engine that understand positional game was Rybka.So Fritz or Junior couldnt be competitors for top GM.If someone knows personally e.g. Anand,Carlsen or Kramnik...to ask them...

Yet, the same engines drew and, respectively, defeated Kasparov and Kramnik in matches a decade ago. Knowing some history puts things in better perspective.

[lkaufman]

Playing against Fritz it is no problem for me to make many draw even on PC.So I believe that someone like Kasparov must beat it easily.It necessary to mention that the first really modern engine that understand positional game was Rybka.So Fritz or Junior couldnt be competitors for top GM.If someone knows personally e.g. Anand,Carlsen or Kramnik...to ask them...

Yet, the same engines drew and, respectively, defeated Kasparov and Kramnik in matches a decade ago. Knowing some history puts things in better perspective.

Well, he is implying that they took bribes to draw or lose those matches. I don't believe it myself. For one thing, the additional sales generated by the results would probably not have justified a bribe of more than perhaps $20,000 or so, which would be far too little to even offer to Kasparov or Kramnik to risk their reputation in this way. Furthermore there would be the risk that they would go public with the bribe offer, and the risk that someone in the company might reveal it. It just doesn't make sense, especially if you have to assume that both Kasparov and Kramnik were guilty of selling out their integrity for a pittance, as well as aassuming that multiple companies were in on the fraud.

[Don]

Another argument is that about Fritz 8-10 it was well known that they had bad pawn handling (overestimated doubled pawns disadvantage).Such kind of weakness should be sufficient for top GM.

Knowing such things is clearly helpful at times, but if the opening book produces positions where they are irrelevant, what can you do?

Computers are less adaptable than humans and so I believe that in reality you can subtract a few ELO from your 2900 estimate. Humans can learn over time how to improve their results against such a rigid opponent but computers cannot normally.

Most of the results we know about were played in these short matches, so I could easily imagine that computers had an advantage. Perhaps 2850 would be a better estimate?

[lkaufman]

Another argument is that about Fritz 8-10 it was well known that they had bad pawn handling (overestimated doubled pawns disadvantage).Such kind of weakness should be sufficient for top GM.

Knowing such things is clearly helpful at times, but if the opening book produces positions where they are irrelevant, what can you do?

Computers are less adaptable than humans and so I believe that in reality you can subtract a few ELO from your 2900 estimate. Humans can learn over time how to improve their results against such a rigid opponent but computers cannot normally.

Most of the results we know about were played in these short matches, so I could easily imagine that computers had an advantage. Perhaps 2850 would be a better estimate?

Well, if you mean the rating the engine would get after a 50 game match with the opponent, or the rating it would get if the opponent has a copy of the exact engine to prepare with, then I agree that the rating would be lower. But usually ratings are based on tournaments where each player plays no more than 2 games with any given opponent, and of course without a copy of the opponent to train against, and so on that basis a rating based on 6 or 8 game matches might even be too low.

[Don]

Another argument is that about Fritz 8-10 it was well known that they had bad pawn handling (overestimated doubled pawns disadvantage).Such kind of weakness should be sufficient for top GM.

Knowing such things is clearly helpful at times, but if the opening book produces positions where they are irrelevant, what can you do?

Computers are less adaptable than humans and so I believe that in reality you can subtract a few ELO from your 2900 estimate. Humans can learn over time how to improve their results against such a rigid opponent but computers cannot normally.

Most of the results we know about were played in these short matches, so I could easily imagine that computers had an advantage. Perhaps 2850 would be a better estimate?

Well, if you mean the rating the engine would get after a 50 game match with the opponent, or the rating it would get if the opponent has a copy of the exact engine to prepare with, then I agree that the rating would be lower. But usually ratings are based on tournaments where each player plays no more than 2 games with any given opponent, and of course without a copy of the opponent to train against, and so on that basis a rating based on 6 or 8 game matches might even be too low.

I guess what I am suggesting is that the computer gets a free ride here - at least in principle because there is no mechanism to take into consideration the humans extra ability to adapt. The thought experiment here is what would happen if a top GM was paid for every win and half for every draw and was allowed to play one game per day for as long as he wanted to. The computer team was allowed to adjust the opening book only and see all the games. I think the human's results would improve over time, but the computers result obviously wouldn't. In a similar human/human scenario there is no reason to believe one player would improve and not the other.

I'm thinking back to the old days when you would buy a machine and it would beat you convincingly and feel really strong, but once you got used to playing it you would develop a certain degree of contempt for it, discovering that it was far from perfect as it seemed at first.

I agree that in typical conditions there probably is not time for the human to adapt appreciably but I don't think that factor should be ignored just because we have arranged things such that the computer is not penalized for it.