Elo differences between: Human vs Computer

[Frank Quisinsky]

Hi Sedat,

you comes with a lot of proofs and good arguments.

I think we should see the situation (if we compare human / computer chess) with the time control in computer chess at first position.

I believe in blitz games, compare to the human (1 Core, older Dual Core Hardware)

3.000 ELO for Shredder 12
2.800 ELO for Shredder 12 with middle time controls
2.700 ELO for Shredder 12 with tournament time controls.

GM Georg Meier played in the time I created the interview with GM Hickl many games vs. Kasporow on chess server and told me that indeed GMs discuss about the playing strength in computer chess. It's not the opinion I wrote before from GM Meier alone. Other GMs have the same opinion but the most compared with Rybka 3 at this time. Rybka 3 with around 2.900 ELO is in the brains of GMs if I understand all correct in middle time controls.

GMs must be right that Computer chess programs have the most problems in openings. Here I am sure too, that GMs around 100 - 200 ELO stronger against the number 1 on Quad Core on actual Hardware.

I think with middle time controls is around 3.100 ELO for Houdini on i7 with 1 Core very realistic. Also possible that GMs underestimate the playing stregth from Computer chess programs.

Topic is complicated!
And if I thinking about it I am very fast mate in my observation.

Interesting what you find out with the databases and analyses.

Best
Frank

[Sedat Canbaz]

Hello Frank,

Thanks again...your comments are always appreciated!

Actually the current issue is not complicated, its very easy,
just is needed a little bit Mathematica and to see/check more carefully again my published results

You know that instead of comments I prefer games, I prefer to relay on Elo calculation programs...
I don't care too much what will say X GM about the latest chess engines are approx. 2900 Elo or 3100 Elo

Actually this is very normal reaction that:
-X Top Grandmaster will never accept that X Top chess program is 500 Elo is stronger than him...

But please look at the statistic table ..., you will notice that GM are played mainly between 1991-1997


Do you know why??, because in that period of time (15-20 years ago) the chess programs were around
2600 Elo , that Elo engine number was approx. same as average on the same level as many GM
People were interested a lot to see: who is stronger Human or Computer !!

And do you know why after 2006 there is no any official man-machine world championships ?
-Simply because almost all GM understood perfectly that Rybka is UNBEATABLE, later appeared handicapped matches and so on...

Note also that Deep Fritz 10 won against the world champion Kramnik by 4:2, for example Deep Rybka 4 is at least 400 elo stronger than Deep Fritz 10

And nowadays,(in 2013) Stockfish, Houdini, Komodo are at least 500 Elo stronger than Deep Fritz 10

Dont forget also that between 1995 and 2014, the processor speed Elo difference is least 400-500 Elo
*Note for Fritz Bench results: Hardware of 1995 year = 60 kns, where nowadays latest fast hardwares = 30.000 kns
And from experience I can say, I mean (based mainly on my data, testings):
*SCCT Hardware Test: there was approx. 300 Elo difference between Celeron 1.7GHz 1c to QX9650 @3.66 GHz 4c

You know too that I don't like much comments, I like more working (as you),
Maybe you noticed that I have done many hardware processor testings, engine testings, rating calculations and so on...
Wirh respect to all, you know that my work does not look to others...
In other words, I have a little bit experience in Computerchess, you can try to trust to my data too
I know too that we a little bit more than 40 years old, and after 40 age people is not changing much, but we should try sometimes

Btw, If GMs will say only a few games by Fritz ... then I suggest them to look in my current rating list,
there more than 2000 games played by different chess engines vs GMs, this is really clear proof even 20 years ago...

And if still they (GM) will not believe, then I suggest them to check also:
SSDF,CCRL,CEGT,SCCT,FCP,TCEC and to look at many other engine ratings too



Best,
Sedat

[Rebel]

Even more than 20 years ago, not bad performance by Fritz 2

You can safely drop this one, note the year. PC was relative new, the mouse just invented. All of the GM's were put behind Fritz2 and had to input the moves themselves. All of them had troubles handling a mouse and the software.

[Sedat Canbaz]

Even more than 20 years ago, not bad performance by Fritz 2

You can safely drop this one, note the year. PC was relative new, the mouse just invented. All of the GM's were put behind Fritz2 and had to input the moves themselves. All of them had troubles handling a mouse and the software.

Thanks Ed...I did not understand what you mean exactly... ?!

So what should be the right Crosstable ?

Best,
Sedat

[Guenther]

Even more than 20 years ago, not bad performance by Fritz 2

You can safely drop this one, note the year. PC was relative new, the mouse just invented. All of the GM's were put behind Fritz2 and had to input the moves themselves. All of them had troubles handling a mouse and the software.

Thanks Ed...I did not understand what you mean exactly... ?!

So what should be the right Crosstable ?

Best,
Sedat

Forget about all blitz/rapid games and all games when the conditions applied made it useless for the Human to win/draw at all.
(Sometimes they might even have not won because of a relationship to the sponsor or whatever and it seemed more lucrative not to win ...)
If you count only games under real conditions where there was something to win or lose and the tc was at least 1h/40moves you probably end up with some dozens of games. Here you can start.
Of course programs are better than Humans since a decade perhaps, but not as much as some here think in real games.
The problem is, real games won't happen at all. They would involve training, money and a lot more... as it happens between Human Pros.

And explicetely for Eds comment: you cannot count games especially at a fast tc when the Humans hardly did know how to move the cursor with a mouse and never before played via a 2D monitor view.

Guenther

[Sedat Canbaz]

Forget about all blitz/rapid games and all games when the conditions applied made it useless for the Human to win/draw at all.
(Sometimes they might even have not won because of a relationship to the sponsor or whatever and it seemed more lucrative not to win ...)
If you count only games under real conditions where there was something to win or lose and the tc was at least 1h/40moves you probably end up with some dozens of games. Here you can start.
Of course programs are better than Humans since a decade perhaps, but not as much as some here think in real games.
The problem is, real games won't happen at all. They would involve training, money and a lot more... as it happens between Human Pros.

And explicetely for Eds comment: you cannot count games especially at a fast tc when the Humans hardly did know how to move the cursor with a mouse and never before played via a 2D monitor view.

Guenther

Hello Guenther,

Thanks for your interest too

And now let's starting...

According to CCRL 40/40 (Slow Time Control)

Houdini 4 64-bit 4CPU 3245
Deep Fritz 10.1 4CPU 2894

Note that Deep Fritz 10.1 is newer version and stronger than Deep Fritz 10
*I did not notice Deep Fritz 10, so I picked v10.1 (for a comparing)
So I expect under CCRL Slow time conditions, Deep Fritz 10 4CPU to be approx. 2850 Elo
That means there is approx. 400 Elo difference between H4 and DF10, even at slow time control !!

----------------------------------------

CCRL 40/4 (Blitz)
Houdini 4 64-bit 4CPU 3343
Deep Fritz 10.1 4CPU 2921
Deep Fritz 10 4CPU 2842
Fritz 10 1CPU 2797

Under CCRL Blitz conditions, we see that there is at least 500 Elo difference between H4 and DF10 !!

---------------------------------------

And as far as I remember,
World championship 2006 (Deep Fritz vs Kramnik, where Fritz won by 4:2) was at slow time control, or maybe am I not right ?

So once more I am asking to all ?
-What is the real Elo of Deep Fritz 10 or Fritz 10 for hardwares of 2006?

And from my experience I can say that the speed processors difference between 2006 and 2014:
-Mostly of the chess engines are expecting min 250 Elo points diffrence !!

One thing more,
CCRL ratings perfectly suits to Human ratings, because I expect under these conditions, Engines and Humans to have similar Elo points
Note: I mean for chess engines performance,which will have similar Elo points, e.g under AMD 4600 2.40 GHz,


Btw,
•Personally I’m not a big fan testing the engines with fast time controls
•Blitz games are less important and the results cannot be taken as very seriously
•But however, blitz is the most used and played time control in chess

And In all around the world,
80-90 % of the played chess games are belonging to blitz/rapid time controls!

And last,
No patience to see 'Man vs Machine' World Championship 2014

Best Regards,
Sedat

[Sedat Canbaz]

I decided to publish my posting (from Rybkachess forum) here too,
And I hope, all my data will be useful for those chess friends,
who still believe that the Top chess engines are not min 500 Elo points stronger than Top GMs

Actually there are only 98 games (human vs computer) from 1977 up to 1990
Rest human vs computer games (approx. 1700 games) played/belong to between 1990-1999 years
So in that period of time (between 1990-1999) the engines were much weaker than 2000-2007
Plus (between 1990-1999)the engines are played under much slower hardwares than 2000-2007

Actually I wanted to publish separately each engine participant,
but due to not many games per engine player,and for more accurate measuring I split them in one...

Btw, here is the below Top 20 rating list, which belongs to the current used database:
Note: the database includes games from 1890 up to 2011 years and only Grandmasters average 2700 Elo

So...do you think that the current Magnus Carlsen's Elo is 2784 ??
I am quite sure, if the database was based only of 2012/2013 games,
then M.Carlsen's rating will be approx. 2800-2850 Elo points...

In other words, this is the same thing with the chess engines too
I mean day by day the engines are becoming stronger and stronger...the processor speed is playing another BIG role too
For example, nowadays Stockfish,Houdini,Komodo,Rybka,Gull are much stronger than Fritz 5.32,Junior 5...
Plus the latest fast processors are much faster than Pentium MMX 233 260 MHz...

And the current Top chess engines probably will hit very easy 3400-3450 Elo on a BIG hardware + superior opening book

Where the strongest Grandmasters (Kasparov,Carlsen,Anand, Kramnik...) chess playing strength for nowadays is up to 2861 Elo

Note:Bayesian calculation is used and Garry Kasparov's rating is fixed to 2850 Elo

Code: Select all

Rank Name                            Elo    +    -  games score oppo. draws
     1.Machine 2000-2007             2903   19   19   390   74%  2774   34%
     2.Kasparov, Garry               2850   18   18   628   61%  2776   56%
     3.Anand, Viswanathan            2836   10   10  1996   62%  2754   53%
     4.Kramnik, Vladimir             2829   11   11  1602   60%  2762   56%
     5.Ivanchuk, Vassily             2802   10   10  2254   59%  2744   53%
     6.Aronian, Levon                2797   15   15   887   57%  2748   54%
     7.Topalov, Veselin              2795   12   12  1377   55%  2758   47%
     8.Karpov, Anatoly               2792   10   10  2186   60%  2727   53%
     9.Morozevich, Alexander         2788   15   15   987   56%  2741   37%
    10.Grischuk, Alexander           2785   15   15   909   57%  2736   51%
    11.Carlsen, Magnus               2784   16   16   797   56%  2743   49%
    12.Svidler, Peter                2783   12   12  1337   57%  2738   55%
    13.Gelfand, Boris                2779   11   11  1828   55%  2744   56%
    14.Shirov, Alexei                2779   17   17   729   48%  2795   50%
    15.Leko, Peter                   2775   12   12  1286   52%  2760   63%
    16.Ponomariov, Ruslan            2771   16   16   743   54%  2740   53%
    17.Adams, Michael                2769   11   11  1561   56%  2730   53%
    18.Mamedyarov, Shakhriyar        2768   19   19   558   56%  2726   51%
    19.Radjabov, Teimour             2766   16   16   735   53%  2747   59%
    20.Fischer, Robert James         2763   23   23   408   65%  2656   41% 
    *  Machine 1977-1999             2588   11   11  1802   51%  2590   23%

Best,
Sedat

[Sedat Canbaz]

Just to make it more clear,
I am not in the side of the Computers, I am a Human too

And I really wish the Top GMs to be on same level as Computers, then will be much more interesting to see (as 10-20 years ago) Man vs Machine games....
But the latest processor technology and chess engine's progress do not allow that... this is a fact, we should see the reality!

Actually we should be thankful to chess engine programmers and happy in the same time....
Because many chess friends (including GMs) can learn from the latest Top chess programs, where we can improve our playing strength!

Greetings,
Sedat

[mehmet karaman]

Dear Sedat thanks for your work.

In 2012 i made an experimental work and i examined near 500 games of chess programs vs. human matches at slower time controls. (at fast time controls chess programs are more stronger according to slower time controls)

I found this formula: Elo grades of chess programs according to Human= Cegt (40/20) Elo + 232 (For Amd x2 4200)

I didn't add the the elo grade improvements of opening books and elo grade improvements of tablebases

http://immortalchess.net/forum/showthread.php?t=17420

[Sedat Canbaz]

Dear Sedat thanks for your work.

In 2012 i made an experimental work and i examined near 500 games of chess programs vs. human matches at slower time controls. (at fast time controls chess programs are more stronger according to slower time controls)

I found this formula: Elo grades of chess programs according to Human= Cegt (40/20) Elo + 232 (For Amd x2 4200)

I didn't add the the elo grade improvements of opening books and elo grade improvements of tablebases

http://immortalchess.net/forum/showthread.php?t=17420

Dear Mehmet,

Not at all...its my pleasure and I am glad to see you, a long time passed ...

Also Welcome to Talkchess, here is a very good place and I hope you will enjoy here

To be honest,
You really surprised me again, not only with Houdini Kayra settings:
http://www.sedatcanbaz.com/chess/?page_id=127

And now I see even you have a such professional knowledge regarding Man vs Machine matches too

So your formula is just great, well-done !!!

And now let's look at CEGT 40/20
Fritz 10 is rated at 2629 Elo + 232 Elo (your formula) = 2852 Elo

As I expected... I never used any formula, but I have a little bit experience in chess

And my chess experience is working like a computer, like formula

So what is your opinion about to fix Fritz 10 at 2850 Elo ?
Do you find these values true for SCCT or maybe slightly to increase Fritz 10 to 2900 Elo
Because I use faster hardwares than Amd X2 4200 (SCCT's hardwares: i7 980X+QX9650)


Best Wishes,
Sedat