CCRL scaling versus human player

[voffka]

I asked a friend of mind who is rated not so high, but still not so bad : 1900 FIDE to play against Igel 1.2 and Igel 1.3 with 1 thread, 1 mb of hash and not tablebases and no book.

The result is that he had absolutely no chances, but most games were quick time controls.

[PK]

I'm no master, but I'd gladly take on an engine in 1800-2000 range.

[Raphexon]

I asked a friend of mind who is rated not so high, but still not so bad : 1900 FIDE to play against Igel 1.2 and Igel 1.3 with 1 thread, 1 mb of hash and not tablebases and no book.

The result is that he had absolutely no chances, but most games were quick time controls.

You'd need something really primitive if you want it to be weaker than an strong human at quick time controls.

I'd wager even the original turing engine on modern hardware would match him on a strong CPU.

[Raphexon]

I asked a friend of mind who is rated not so high, but still not so bad : 1900 FIDE to play against Igel 1.2 and Igel 1.3 with 1 thread, 1 mb of hash and not tablebases and no book.

The result is that he had absolutely no chances, but most games were quick time controls.

It may be worth to test Shallow-Blue since it's on github and its releases have a range from really weak to still kinda weak.

https://github.com/GunshipPenguin/shallow-blue/releases

http://ccrl.chessdom.com/ccrl/404/cgi/c ... +opponents

Old Pirarucu releases would work too.

https://github.com/ratosh/pirarucu/releases

http://ccrl.chessdom.com/ccrl/404/cgi/c ... +opponents

[lkaufman]

I guess this is already a well documented subject but let me ask it my way.

I wonder if some kind (very) good chess player would accept to play some games against middle range engines (2200-2700 CCRL elo rating) at CCRL 40/4 TC in order to rescale CCRL rating to FIDE rating.

One easy way to do this: chess.com has established blitz ratings for all the Komodo levels, and I think their blitz ratings are not too far away from normal human ratings. So you could just run 40/4 games of the engines you are interested in against appropriate Komodo levels to determine human blitz ratings for them. I would avoid the top few levels, as their chess.com ratings are deflated by cheating (yes, players use computers to cheat to beat computers!!), but the first 15 or so levels are not affected by this, cheaters challenge only the top few levels. Note also that these Komodo levels (up thru 19) don't use noticeable time, so their rating is absolute, not based on time control. But for a 40/4 list, no problem.

[xr_a_y]

I guess this is already a well documented subject but let me ask it my way.

I wonder if some kind (very) good chess player would accept to play some games against middle range engines (2200-2700 CCRL elo rating) at CCRL 40/4 TC in order to rescale CCRL rating to FIDE rating.

One easy way to do this: chess.com has established blitz ratings for all the Komodo levels, and I think their blitz ratings are not too far away from normal human ratings. So you could just run 40/4 games of the engines you are interested in against appropriate Komodo levels to determine human blitz ratings for them. I would avoid the top few levels, as their chess.com ratings are deflated by cheating (yes, players use computers to cheat to beat computers!!), but the first 15 or so levels are not affected by this, cheaters challenge only the top few levels. Note also that these Komodo levels (up thru 19) don't use noticeable time, so their rating is absolute, not based on time control. But for a 40/4 list, no problem.

That would be helpul indeed !
Is there an UCI option in the free Komodo 10 in order to specify this "level" ?
Also do you have a link to a page with the komodo level versus elo rating table, I couldn't find it by myself yesterday.

[lkaufman]

I guess this is already a well documented subject but let me ask it my way.

I wonder if some kind (very) good chess player would accept to play some games against middle range engines (2200-2700 CCRL elo rating) at CCRL 40/4 TC in order to rescale CCRL rating to FIDE rating.

One easy way to do this: chess.com has established blitz ratings for all the Komodo levels, and I think their blitz ratings are not too far away from normal human ratings. So you could just run 40/4 games of the engines you are interested in against appropriate Komodo levels to determine human blitz ratings for them. I would avoid the top few levels, as their chess.com ratings are deflated by cheating (yes, players use computers to cheat to beat computers!!), but the first 15 or so levels are not affected by this, cheaters challenge only the top few levels. Note also that these Komodo levels (up thru 19) don't use noticeable time, so their rating is absolute, not based on time control. But for a 40/4 list, no problem.

That would be helpul indeed !
Is there an UCI option in the free Komodo 10 in order to specify this "level" ?
Also do you have a link to a page with the komodo level versus elo rating table, I couldn't find it by myself yesterday.

I think that the levels were introduced in the Komodo 11.xx series. Komodo 11 isn't free but is very cheap so if you buy that and send us an email requesting the version with the levels, we'll do that for you. The rough estimate for the chess.com ratings is 700 + level * 100. To get the latest current ratings, go to chess.com, then try to play a game against a computer, and you should see all the levels with their blitz ratings displayed. But the top few levels are better than the displayed (or calculated) ratings. Level 19 scored overwhelmingly in 5' + 2" games against GMs except for 2 of the world's top 3 blitz players (Nakamura and MVL) and Level 18 made a healthy plus score against same, so perhaps by FIDE blitz standards they would be about 2850 and 2750 respectively. Level 20 is full Komodo strength but on just one thread, no ponder, no TBS, and limited opening book. Nakamura scored one out of about 20 games with it, so it would be about 3400 FIDE Blitz.

[Chessqueen]

I feel it is conservative to estimate 2800 CCRL at only 2800 FIDE. That being said there are a lot of factors that complicate matters.

Humans memorize openings, this means they will play far stronger than their ratings in the opening, until they are out of book. I don't know if the context you are thinking of allows you to handle opening books, but to smooth things out I would consider adding an opening book. The opening book could be based on human games of the specified rating.

Engine strength is hardware dependent. You may want to consider trying to come up with ways to normalize engine strength across different hardware. I would bet on Carlsen against Minic on a Raspberry PI, but I would bet on Minic against Carlsen on TCEC hardware.

The CCRL list is indeed conservative by FIDE standards. Let's assume the engine gets a good book which tries to get out of theory early when it doesn't cost too much. Assume also the hardware specified as standard by CCRL, which is way below current PC speeds. Note that number of threads is not an issue as they rate 1 and 4 threads separately. Let's use CCRL 40/40 as it is obviously more relevant for tournament chess than 40/4. In 1998 Junior 5 earned a 2700 FIDE result in 9 games vs. top players, and while it is too old to even appear on CCRL, by extrapolation it would surely not be higher than 2600 on this list. But the hardware in 1998 was way below even the modest hardware specified by CCRL. The same conclusion would be reached by looking at the various matches from around 2003 or the Kramnik vs Fritz match of 2006; even a 2700 engine on CCRL would get a FIDE rating above 2800 with the above assumptions. Note that Deep Fritz 10, which beat Kramnik in that match, is at 2830 on the list, but it gave several handicaps to Kramnik such as letting him see the engine's book during the game (!), giving him a copy of the engine to practice against for months, limiting TBs to 5 man, etc. On the CCRL specified hardware with no special conditions like those, I believe that Fritz 10 on 4 cpus would easily defeat Carlsen in a match today.
One other point: CCRL uses Bayeselo which contracts the ratings considerably from normal elo, so although I totally agree with Kai about scaling engine results down to 70%, this is really correct just for CEGT. For CCRL much of the contraction is already done by bayeselo, so maybe 85% or so might be the right figure for CCRL 40/40. Of course blitz results are more spread out, so maybe 70% is actually about right for CCRL 40/4.

You wrote limiting TBs to 5 man, etc. But how how many GMs play perfectly all the positions with 5 man ?
Here is the top endgame player of all time Carlsen blundering a 5 pieces endgame ==> https://www.youtube.com/watch?v=5b-1u5XjaFU

[Sven]

Here is the top endgame player of all time Carlsen blundering a 5 pieces endgame ==> https://www.youtube.com/watch?v=5b-1u5XjaFU

This is OT of course but in the video I only see KRBPPP vs KRBPPP positions, that is 12 pieces. What did I miss?

[Chessqueen]

Here is the top endgame player of all time Carlsen blundering a 5 pieces endgame ==> https://www.youtube.com/watch?v=5b-1u5XjaFU

This is OT of course but in the video I only see KRBPPP vs KRBPPP positions, that is 12 pieces. What did I miss?

You missed the moment when Carlsen blundered which was at the end of the video with 5 pieces