Dedicated Chess Machine Elo vs Human Elo, a least squares analysis

[JayRod]

Inspired by something said by GM Kaufman here: http://www.talkchess.com/forum3/viewtop ... 2&start=30 I went ahead and found some data and below is the correlation between dedicated chess machine Elo (SSDF) and human Elos, the human pool was apparently from 1987-1991 in Europe. Data below, some of the links are now dead, but I had saved a local copy of the data.

The conclusion is that there's a strong correlation between engine Elo and human Elo, which is comforting for people like me who don't play in human tournaments but track their Elo by playing vs an engine.

I suspect software is similar to this dedicated hardware, I don't see why not, at least at the below super GM level.

JR


Human pool from 1987-1991 in Europe.

For similar hardware I combined the games played and averaged the Elos. The numbers from left to right are: SSDF rating, FIDE vs humans , #Number of Games, and the final entry are comments.

SSDF rating FIDE vs humans # Games Comments
1621 1617 23 Novag Super Forte A 6502 5 MHz, Fidelity Excellence 6502 3 MHz, 24. Novag Super Constellation 6502 4MHz
1716 1861 28 17. Novag Forte B 6502 5 MHz
1780 1999 28 8. CXG Sphinx Galaxy 6502 4 MHz
1815 1866 28 15. Mephisto Mega IV 6502 5 MHz
1816 1722 18 21. Saitek Maestro D 6502 10 MHz
1817.5 1894 18 13. Saitek Maestro A 6502 6 MHz, 14. Novag Super Expert B 6502 6 MHz
1839 2057 21 6. Mephisto Academy 6502 5 MHz
1860 1996 15 9. Novag Super Forte C 6502 6 MHz
1872.3 1967.0 23 10. Mephisto Roma 68020 14 MHz, 11. Novag Diablo 68000 16 MHz, 12. Psion Atari 68000 8 MHz
1893 2067 25 5. Fidelity Mach III 68000 16 MHz
1923 1866 15 15. Mephisto Dallas 68000 12 MHz
1973 2030 26 7. Mephisto Almeria 68020 12 Mhz
1977 2177 19 3. Fidelity Mach IV 68020 12 MHz
2145 2217 22 1. Mephisto Lyon 68020 12 MHz & 2. Mephisto Portorose 68020 12 MHz, combined


Correlation: 0.80126309 very high for this sample

Least fit equations and graphic: Red is the Fide rating vs human players, Blue is the SSDF rating:

https://pasteboard.co/Iu4XoPE.png

raw data from: (dead link) http://home.interact.se/~w100107/level.htm

[pedrox]

If you are interested in knowing the relationship between dedicated machines and humans, in addition to the SDFF list you should know the list of Elo Aktiv.

https://www.schach-computer.info/wiki/i ... -Elo-Liste

As you can check this list, it tests (currently) many more machines than the SDFF list and does it at all levels of Strength. The Aktive list is related to the USCF Elo (although I don't know where that relationship comes from), and the USCF can be passed to FIDE.

I suggest that you read the following website:

http://www.spacious-mind.com/html/ratin ... ments.html

And finally if you know the Elo FIDE, if you want with Kai's formula you could go to Elo CCRL (To Kai's formula I would make a modification for Elo FIDE less than 1400)

Code: Select all

ELO USCF <--> ELO ACTIVE
Elo USCF = Elo Info-Active + 68

ELO FIDE <--> ELO USCF
Elo USCF = 20 + (1.02×Elo FIDE) if Elo FIDE > 1886
Elo USCF = -570 + (Elo FIDE/0.75) if Elo FIDE ≤ 1886
Elo FIDE = (Elo USCF-20) / 1.02 if Elo USCF > 1945
Elo FIDE = 0.75 * (570 + Elo USCF) if Elo USCF ≤ 1945

ELO FIDE <--> ELO CCRL
Elo FIDE = (0.7 x Elo CCRL) + 840

Code: Select all

CCRL		FIDE	USCF	AKTIV

2600		2660	2733	2665
2500		2590	2662	2594
2400		2520	2590	2522
2300		2450	2519	2451
2200		2380	2448	2380
2100		2310	2376	2308
2000		2240	2305	2237
1900		2170	2233	2165
1800		2100	2162	2094
1700		2030	2091	2023
1600		1960	2019	1951
1500		1890	1948	1880
1400		1820	1857	1789
1300		1750	1763	1695
1200		1680	1670	1602
1100		1610	1577	1509
1000		1540	1483	1415
900		1470	1390	1322
800		1400	1297	1229
700		1330	1203	1135
600		1260	1110	1042
500		1190	1017	949
400		1120	923	855
300		1050	830	762
200		980	737	669
100		910	643	575
0		840	550	482

The experience I have of visiting some dedicated machine forums is that the members of this forum (I also) seem to agree more with the Elo indicated by the Aktiv list than by the SDFF. In general, the Elo indicated for dedicated machines in the SDFF list is about 100 or 200 points lower than the Aktiv list.

Perhaps an explanation of that lower value has something to do with the fact that the SDFF list at one time decided to lower the value of 100 points for each of its participants.

There is a moment that you speak indistinctly of engine and dedicated machine, perhaps because you base your study on the SDFF. In my case it sounds strange to mix engines and dedicated machines because I have more in mind CCRL and Aktiv list.

[pedrox]

On the following website there is a list with dedicated machines taken from the SDFF in the year 93, I think before the drop of 100 points. On the web they indicate that they prefer this list to the current ones and that Elo are similar to humans.

http://www.meca-web.es/articulos/elo.htm

[JayRod]

\

And finally if you know the Elo FIDE, if you want with Kai's formula you could go to Elo CCRL (To Kai's formula I would make a modification for Elo FIDE less than 1400)

Code: Select all

CCRL		FIDE	USCF	AKTIV


2000		2240	2305	2237
1900		2170	2233	2165

The experience I have of visiting some dedicated machine forums is that the members of this forum (I also) seem to agree more with the Elo indicated by the Aktiv list than by the SDFF. In general, the Elo indicated for dedicated machines in the SDFF list is about 100 or 200 points lower than the Aktiv list.

Perhaps an explanation of that lower value has something to do with the fact that the SDFF list at one time decided to lower the value of 100 points for each of its participants.

There is a moment that you speak indistinctly of engine and dedicated machine, perhaps because you base your study on the SDFF. In my case it sounds strange to mix engines and dedicated machines because I have more in mind CCRL and Aktiv list.

Oh wow! Thank you! This is incredible on many levels, including I don't believe it...let me explain.

I play CCRL chess engines that I estimate are between 1900-2000 CCRL Elo based on hardware (compared to the CCRL AMD Athlon baseline) and time limits, including Kaufman's rule of about 75 points doubling per doubling of thinking time (and engines I believe are not set to think on opponents time, by default, for the most part).

Without a clock I am at roughly 1950 Elo on average, sometimes over 2000, and if I play fast under 1900. My sample size is usually N=30 games. I then take lessons from certain masters and high expert human players, mostly from Indian and SE Asia including the Philippines. Against them, and most of these guys are rated around 2200-2300 Fide Elo by their national organizations (not Fide but Fide affiliated), I win (in points, draws being 0.5 pts) about 15-25% of the time. So their Elo is consistent with my understanding of my Elo of around 1950 Elo Fide.

However, using the above table, if I can hold my own (be equal) with CCRL engines set between 1900 to 2000 Elo CCRL, it means I'm actually rated 2170-2240 Fide Elo, which is about 200 points higher than my estimated rating. If so, it suggests various possibilities: (1) the table is wrong, (2) I'm actually stronger than I thought (yes! I would love for this to be true but I'm afraid it's not), (3) I'm incorrectly calculating the CCRL Elo for the time limits I play (I usually play about 12-15 seconds per move for the engine, but unlimited time to reply for me), or, (4) the human masters I play against are sandbagging their rating or their rating is higher than they say (a common complaint actually, since a lot of tournaments for money require you to have a low Elo, and these guys all like to play for money, also I've read that SE Asian players Elos are higher than normal European Fide Elos, but I always assumed the difference is around 50 points, not 200 points).

Finally, in Lichess and Chess.com my rating is usually in blitz around the 1900s. I'm a terrible blitz player but there is a linear correlation between blitz strength and classical strength. So I doubt my classical chess is 200 points higher than my blitz chess, but I would be thrilled if it was true.

What's the truth?

JayRod

[pedrox]

You say:

(3) I'm incorrectly calculating the CCRL Elo for the time limits I play (I usually play about 12-15 seconds per move for the engine, but unlimited time to reply for me)

In my opinion if you want to calculate a CCRL rating, then you should play with the same time as the engine.

If you are already playing with players with Elo FIDE and you have some results, that calculation can be approximate and good, as you say your Elo FIDE could be about 1950 points.

But your calculation of Elo CCRL may be higher than it should be if you spend much more time than the engine responding.

I suggest that you face engines of approximately 1600 Elo CCRL points and see if you can win them by complying with time control. If the engine plays in 15 seconds and you respond in 2 minutes, you are using 8 times more time, this could mean by your rule of 75 points; 75 + 75 + 75 = 225 points more for engine.

[todd]

Another possibility is that you are experienced at playing against lower rated engines and dealing with their specific strengths and weaknesses. They don't play much like humans at the same level, and some people are much better than others at playing against them. Practice helps a lot.

I'm also experienced at playing against engines (both weak and strong) and the results I get are generally a lot better than those of my friends who are rated similarly (USCF master).

I'd give the results from human games a lot more weight when trying to determine what your rating would be within a human-only rating pool like FIDE's.

[JayRod]

I suggest that you face engines of approximately 1600 Elo CCRL points and see if you can win them by complying with time control. If the engine plays in 15 seconds and you respond in 2 minutes, you are using 8 times more time, this could mean by your rule of 75 points; 75 + 75 + 75 = 225 points more for engine.

Thanks for the reply. The above suggestion however would violate my rules for engagement with PC engines. First, I found nonlinear effects if you do too many 'doublings'--after a while the rule doesn't work. I like to take no more than about two doublings. As an extreme example, suppose you 'halve' (similar to doubling) the engine's time from 180 seconds to 1 second, say it's Stockfish. It will not play very strong I would imagine, since 1 second is barely enough time to load the chess tree variations into memory (speaking loosely, though I've read that many programmers will not incorporate a Elo raising idea into their program unless it can show an improvement in Elo in about five seconds of time, showing they are optimizing their engines for blitz). In any event, I have found nonlinear effects if you do too many doublings (or halvings) so what I do is use the CCRL 40/4 Elo list (which is about six seconds a move on average) for Elo, I double time to 12 seconds a move, so you add 75 points to the CCRL 40/4 list, then I sometimes add three seconds just to account for program overhead and make it a bit more challenging, then I, myself, move in "classical' time controls (about 2.3 minutes per move, or faster, in practice, though I don't use a clock), so I should be measuring my classic Elo vs the PC's Elo, an apples to apples comparison (approximately). As I say, my informal estimates vs humans shows that my assumptions are more or less correct, though to be fair my human opponents are not using a clock either, and I'm not sure how motivated they are to play me, so whether they really are playing at their true Elo level, and my sample size vs human masters is small (my online Elo is suspect since I think at many online sites there's chess cheating, some cheaters in fact turn on or off the engine at crucial times). What I found with master chess teachers, or some of them, is that it seems on occasion they let you win a game to pique your interest in the game (not unlike chess hustlers if you play a number of games vs them, they let you win so you don't walk away). But some masters in fact fight hard. I know because with some of them at the IM level I've never won a game from them (though I've drawn some games).

Thanks to todd too for the reply, and I agree that some people can learn to play better against engines than other people. For an obvious strategy, avoid tactics, play a closed opening, and make positional sacrifices, as many engines don't respond well to positional sacrifices.

JR

[Uri Blass]

If you are interested in knowing the relationship between dedicated machines and humans, in addition to the SDFF list you should know the list of Elo Aktiv.

https://www.schach-computer.info/wiki/i ... -Elo-Liste

As you can check this list, it tests (currently) many more machines than the SDFF list and does it at all levels of Strength. The Aktive list is related to the USCF Elo (although I don't know where that relationship comes from), and the USCF can be passed to FIDE.

I suggest that you read the following website:

http://www.spacious-mind.com/html/ratin ... ments.html

And finally if you know the Elo FIDE, if you want with Kai's formula you could go to Elo CCRL (To Kai's formula I would make a modification for Elo FIDE less than 1400)

Code: Select all

ELO USCF <--> ELO ACTIVE
Elo USCF = Elo Info-Active + 68

ELO FIDE <--> ELO USCF
Elo USCF = 20 + (1.02×Elo FIDE) if Elo FIDE > 1886
Elo USCF = -570 + (Elo FIDE/0.75) if Elo FIDE ≤ 1886
Elo FIDE = (Elo USCF-20) / 1.02 if Elo USCF > 1945
Elo FIDE = 0.75 * (570 + Elo USCF) if Elo USCF ≤ 1945

ELO FIDE <--> ELO CCRL
Elo FIDE = (0.7 x Elo CCRL) + 840

Code: Select all

CCRL		FIDE	USCF	AKTIV

2600		2660	2733	2665
2500		2590	2662	2594
2400		2520	2590	2522
2300		2450	2519	2451
2200		2380	2448	2380
2100		2310	2376	2308
2000		2240	2305	2237
1900		2170	2233	2165
1800		2100	2162	2094
1700		2030	2091	2023
1600		1960	2019	1951
1500		1890	1948	1880
1400		1820	1857	1789
1300		1750	1763	1695
1200		1680	1670	1602
1100		1610	1577	1509
1000		1540	1483	1415
900		1470	1390	1322
800		1400	1297	1229
700		1330	1203	1135
600		1260	1110	1042
500		1190	1017	949
400		1120	923	855
300		1050	830	762
200		980	737	669
100		910	643	575
0		840	550	482

The experience I have of visiting some dedicated machine forums is that the members of this forum (I also) seem to agree more with the Elo indicated by the Aktiv list than by the SDFF. In general, the Elo indicated for dedicated machines in the SDFF list is about 100 or 200 points lower than the Aktiv list.

Perhaps an explanation of that lower value has something to do with the fact that the SDFF list at one time decided to lower the value of 100 points for each of its participants.

There is a moment that you speak indistinctly of engine and dedicated machine, perhaps because you base your study on the SDFF. In my case it sounds strange to mix engines and dedicated machines because I have more in mind CCRL and Aktiv list.

The facts that I read in the link that you give are obviously wrong

http://www.spacious-mind.com/html/ratin ... ments.html

"This old formula was probably never meant to go below converting a 1400 ELO FIDE player to USCF. Which becomes obvious when you realize that FIDE ratings only start at 1400 ELO for Novices. Nothing below 1400 ELO exists for FIDE."

Sorry but the lowest fide rating is 1000 elo and novices do not get fide elo because they lose all their games even against players with slightly more than 1000 elo.

"USCF rating for a Beginner starts at 750 ELO"

This is clearly wrong and the lowest USCF rating is 100

See the following link

http://www.glicko.net/ratings/rating.system.pdf

bottom of page 2:

"If the resulting rating from Step 3 for the unrated player is less than 100, then change
the rating to 100."


practically there are many players with rating 100.
You can find in every state all the players with rating 100(I guess most of them did not continue to play chess after they got initial rating

http://www.uschess.org/datapage/player- ... &mode=Find

I am not sure if today new players in USA get rating of 100 but I am sure that a player who lost all his games got a rating of 101 in 2019

see
http://www.uschess.org/msa/MbrDtlTnmtHst.php?16976526

[pedrox]

Uri,

The article tries to establish a relationship between the Aktiv list of dedicated chess machines and the FIDE list. The fact that the Elo FIDE starts at 1000 points or the Elo USCF at 100 is not something critical for what the article tries to explain. The article is not recent, maybe 4 years old and as you know it is possible that at this time the minimum EIo FIDE has been going down from 1400 to 1200 to 1000 and who knows if in the future there will be Elo FIDE minors.

The article tries to explain that if the official USCF conversion formulas between the Elo FIDE and Elo USCF are used, then the Elo of the dedicated machines of the Aktiv list seems to have a good relationship when dealing with Elo (FIDE) above 2000. And if the official formula for lower Elo is used then the relationship between the Aktiv list and the FIDE list does not seem to fit reality, for that reason a different formula is proposed in the article. But all this has to be seen from the point of view of dedicated chess machines.

You are right that there may be USCF players with Elo of 100 and these appear to be class J (100-199). But even so I believe that it is possible to start in some cases with USCF of 750 or with different ones according to other criteria. It is indicated in the document in section 2.1 point 5. But I am unaware of the USCF system.

[pedrox]

I suggest that you face engines of approximately 1600 Elo CCRL points and see if you can win them by complying with time control. If the engine plays in 15 seconds and you respond in 2 minutes, you are using 8 times more time, this could mean by your rule of 75 points; 75 + 75 + 75 = 225 points more for engine.

Thanks for the reply. The above suggestion however would violate my rules for engagement with PC engines. First, I found nonlinear effects if you do too many 'doublings'--after a while the rule doesn't work. I like to take no more than about two doublings. As an extreme example, suppose you 'halve' (similar to doubling) the engine's time from 180 seconds to 1 second, say it's Stockfish. It will not play very strong I would imagine, since 1 second is barely enough time to load the chess tree variations into memory (speaking loosely, though I've read that many programmers will not incorporate a Elo raising idea into their program unless it can show an improvement in Elo in about five seconds of time, showing they are optimizing their engines for blitz). In any event, I have found nonlinear effects if you do too many doublings (or halvings) so what I do is use the CCRL 40/4 Elo list (which is about six seconds a move on average) for Elo, I double time to 12 seconds a move, so you add 75 points to the CCRL 40/4 list, then I sometimes add three seconds just to account for program overhead and make it a bit more challenging, then I, myself, move in "classical' time controls (about 2.3 minutes per move, or faster, in practice, though I don't use a clock), so I should be measuring my classic Elo vs the PC's Elo, an apples to apples comparison (approximately). As I say, my informal estimates vs humans shows that my assumptions are more or less correct, though to be fair my human opponents are not using a clock either, and I'm not sure how motivated they are to play me, so whether they really are playing at their true Elo level, and my sample size vs human masters is small (my online Elo is suspect since I think at many online sites there's chess cheating, some cheaters in fact turn on or off the engine at crucial times). What I found with master chess teachers, or some of them, is that it seems on occasion they let you win a game to pique your interest in the game (not unlike chess hustlers if you play a number of games vs them, they let you win so you don't walk away). But some masters in fact fight hard. I know because with some of them at the IM level I've never won a game from them (though I've drawn some games).

Thanks to todd too for the reply, and I agree that some people can learn to play better against engines than other people. For an obvious strategy, avoid tactics, play a closed opening, and make positional sacrifices, as many engines don't respond well to positional sacrifices.

JR

FIDE allows the introduction of chess programs (and engines) or dedicated chess machines in tournaments played by humans. There is a special regulation for this. Whether a program plays a tournament with humans depends mainly if the referee of the tournament allows it. Although the referees usually take into account the decision of the players since many can simply boycott the tournament for not wanting to play against computers even if the computers cannot qualify for cash prizes or their games not count for the Elo.

Among the rules to be followed; The game is played on a board and with a clock. The computer operator must make the moves on the board and press the clock. The program operator can adjust the program clock with the board clock if necessary.

Of course, the playing time of the program is the same as that of the player. The only difference is that in that same time the operator has to make the moves of the computer on the board and viceversa. I would say that you could consider about 5 seconds for each case.

If the game is played with 90 minutes + 30 seconds increment, you could program the program with 90 minutes + 20 seconds increment and the clocks will be quite synchronized.

If I introduce to the tournament an engine with an Elo CCRL 40/4 of 1900, it is easy, the engine will continue playing with 90 minutes + about 20 increment. He will not play with 15 seconds and the human with 2 minutes.

I am sure that said engine in a tournament with Elo FIDE could perfectly reach performances of 2170 FIDE or higher on a notebook. If in these conditions you win the engine, then I personally would think that your Elo FIDE would be greater than 1950 and maybe if those 2170.

(Maybe I can understand that you say that CCRL uses older computers as reference, but then I don't understand why you consider CCRL 40/4 to justify the 15 s of the engine and not CCRL 40/40 when you want to set an Elo in a classic time control).