What engine breaks even with GMs in blitz?

[lkaufman]

In blitz (let's say 3' + 2" or as close to this as possible), the top engines today are far beyond human level. But how far down the list do we have to go to find engines (and specified hardware) that score evenly against GMs, preferably ones with known identities and ratings? I'm sure there is plenty of data to answer this question as countless games have been played online over the years, but does anyone actually have some data, such as "Engine xyz on one thread scored 50% against GMs averaging 2600 FIDE" for example? The question I'd like to answer is: How much would we have to add to CCRL blitz ratings to estimate the FIDE blitz rating of a human GM who would score 50% against it at 3' + 2"?

[Laskos]

In blitz (let's say 3' + 2" or as close to this as possible), the top engines today are far beyond human level. But how far down the list do we have to go to find engines (and specified hardware) that score evenly against GMs, preferably ones with known identities and ratings? I'm sure there is plenty of data to answer this question as countless games have been played online over the years, but does anyone actually have some data, such as "Engine xyz on one thread scored 50% against GMs averaging 2600 FIDE" for example? The question I'd like to answer is: How much would we have to add to CCRL blitz ratings to estimate the FIDE blitz rating of a human GM who would score 50% against it at 3' + 2"?

I have no such examples or data, but I would say: add much, maybe 500+ points to CCRL 40/4 ratings.

[jdart]

That sounds about right. I think even 2300 level engines (CEGT) would be hard competition, especially with a decent opening book.

--Jon

[lkaufman]

That sounds about right. I think even 2300 level engines (CEGT) would be hard competition, especially with a decent opening book.

--Jon

I'm remembering some relevant data now. I see that on the CCRL Fritz 5.32 (from 1998) is rated 2540. Around 1990 I operated for Rexchess in a blitz tournament with several GMs, and I recall we were fairly even with them (beating most everyone else), so let's call it a 2550 FIDE performance. Rex was probably about a hundred elo off from the best engine of the day, but engines progressed considerably from 1990 to 1998, so perhaps Rexchess would get a rating of 2200 to 2300 on CCRL blitz now (just a guess) (CEGT would be 150 to 200 elo lower). But that is running on hardware from just a few years ago; in 1990 we ran on a 25 MHz machine of the generation before Pentium came out (I forget the processor number, but I think it "4" was the digit defining the generation). I would imagine that this would knock hundreds of elo off the blitz rating, not sure how many hundred. This seems more or less in line with the 500 estimate for CCRL and also suggests that if you meant for 2300 CEGT (2450 or a bit more CCRL blitz) to be competitive with Carlsen in blitz you might be right. Carlsen is 2954 FIDE blitz (Nakamura and MVL next at 2934 and 2933) so is 2300 CEGT/2450CCRL blitz the right level for these top three guys? Feel free to revise your estimates based on the info about Rexchess. Too bad it's too old to be on the rating lists.

[mwyoung]

That sounds about right. I think even 2300 level engines (CEGT) would be hard competition, especially with a decent opening book.

--Jon

I'm remembering some relevant data now. I see that on the CCRL Fritz 5.32 (from 1998) is rated 2540. Around 1990 I operated for Rexchess in a blitz tournament with several GMs, and I recall we were fairly even with them (beating most everyone else), so let's call it a 2550 FIDE performance. Rex was probably about a hundred elo off from the best engine of the day, but engines progressed considerably from 1990 to 1998, so perhaps Rexchess would get a rating of 2200 to 2300 on CCRL blitz now (just a guess) (CEGT would be 150 to 200 elo lower). But that is running on hardware from just a few years ago; in 1990 we ran on a 25 MHz machine of the generation before Pentium came out (I forget the processor number, but I think it "4" was the digit defining the generation). I would imagine that this would knock hundreds of elo off the blitz rating, not sure how many hundred. This seems more or less in line with the 500 estimate for CCRL and also suggests that if you meant for 2300 CEGT (2450 or a bit more CCRL blitz) to be competitive with Carlsen in blitz you might be right. Carlsen is 2954 FIDE blitz (Nakamura and MVL next at 2934 and 2933) so is 2300 CEGT/2450CCRL blitz the right level for these top three guys? Feel free to revise your estimates based on the info about Rexchess. Too bad it's too old to be on the rating lists.

[pgn][Event "2nd Harvard Cup"] [Site "Boston, MA USA"] [Date "1991.05.03"] [EventDate "?"] [Round "?"] [Result "1/2-1/2"] [White "Rex (Computer)"] [Black "Boris Gulko"] [ECO "B07"] [WhiteElo "?"] [BlackElo "?"] [PlyCount "107"] 1.d4 d6 2.e4 c6 3.Nc3 Nf6 4.Nf3 Bg4 5.Be2 e6 6.h3 Bh5 7.Bf4 Be7 8.Nd2 Bg6 9.e5 Nd5 10.Nxd5 cxd5 11.Nf3 Nc6 12.O-O O-O 13.c3 a6 14.exd6 Bxd6 15.Bxd6 Qxd6 16.Nh4 Be4 17.f3 Bg6 18.Nxg6 hxg6 19.f4 Rac8 20.Qd3 Ne7 21.Rae1 Qb6 22.Qd2 Nf5 23.Bd3 Rc6 24.Bxf5 gxf5 25.Re2 Qd8 26.Kh2 b5 27.Rfe1 g6 28.Re3 a5 29.a3 Kg7 30.Qe2 Qb8 31.Qf2 Rh8 32.Kh1 Rc4 33.Qg3 Qd6 34.R3e2 b4 35.axb4 axb4 36.Ra1 Rhc8 37.Re3 bxc3 38.bxc3 Qc7 39.Ra3 Rb8 40.Re2 Rb1+ 41.Kh2 Rc1 42.Re3 Rc6 43.Rb3 Ra6 44.Qf2 Raa1 45.g3 Qc4 46.Rb4 Qa2 47.Rb2 Rh1+ 48.Kg2 Rag1+ 49.Kf3 Qa1 50.Ra2 Qd1+ 51.Qe2 Qxe2+ 52.Rexe2 Rxh3 53.Rg2 Rc1 54.Ra3 1/2-1/2 [/pgn]

[Albert Silver]

In blitz (let's say 3' + 2" or as close to this as possible), the top engines today are far beyond human level. But how far down the list do we have to go to find engines (and specified hardware) that score evenly against GMs, preferably ones with known identities and ratings? I'm sure there is plenty of data to answer this question as countless games have been played online over the years, but does anyone actually have some data, such as "Engine xyz on one thread scored 50% against GMs averaging 2600 FIDE" for example? The question I'd like to answer is: How much would we have to add to CCRL blitz ratings to estimate the FIDE blitz rating of a human GM who would score 50% against it at 3' + 2"?

Hardware is also a big factor. Back in 2001 I gave private computer classes to GM Mecking, whose ICC blitz rating was the no.1. It turned out that he got this rating in great part by beating up on Chess Tiger 13 and Fritz 6 accounts in blitz. I was surprised by this, but saw him play 4-5 games in which he regularly bamboozled them in endgames. However, could he still do this, even with that same skill level, if they ran on a modern machine, even with one core? Maybe, but I am not nearly so confident.

[jdart]

you meant for 2300 CEGT (2450 or a bit more CCRL blitz) to be competitive with Carlsen in blitz you might be right. Carlsen is 2954 FIDE blitz (Nakamura and MVL next at 2934 and 2933) so is 2300 CEGT/2450CCRL blitz the right level for these top three guys? Feel free to revise your estimates based on the info about Rexchess. Too bad it's too old to be on the rating lists.

This is just an imprecise guess and assuming nothing too far off current hardware. So this would be something like Tao 5.6, Yace, Arasan version 11 or so, Crafty v. 19 or so, etc., on a modern machine.

Blitz is tough because the computers don't miss anything easy to moderate and tactical. These older engines weren't so good in endgames though and I think none of those mentioned had tablebases, except maybe Crafty. I remember going back and testing Gnuchess 4.77 once and its endgame play was awful. So a GM might gain a few points there.

--Jon

[lkaufman]

That sounds about right. I think even 2300 level engines (CEGT) would be hard competition, especially with a decent opening book.

--Jon

I'm remembering some relevant data now. I see that on the CCRL Fritz 5.32 (from 1998) is rated 2540. Around 1990 I operated for Rexchess in a blitz tournament with several GMs, and I recall we were fairly even with them (beating most everyone else), so let's call it a 2550 FIDE performance. Rex was probably about a hundred elo off from the best engine of the day, but engines progressed considerably from 1990 to 1998, so perhaps Rexchess would get a rating of 2200 to 2300 on CCRL blitz now (just a guess) (CEGT would be 150 to 200 elo lower). But that is running on hardware from just a few years ago; in 1990 we ran on a 25 MHz machine of the generation before Pentium came out (I forget the processor number, but I think it "4" was the digit defining the generation). I would imagine that this would knock hundreds of elo off the blitz rating, not sure how many hundred. This seems more or less in line with the 500 estimate for CCRL and also suggests that if you meant for 2300 CEGT (2450 or a bit more CCRL blitz) to be competitive with Carlsen in blitz you might be right. Carlsen is 2954 FIDE blitz (Nakamura and MVL next at 2934 and 2933) so is 2300 CEGT/2450CCRL blitz the right level for these top three guys? Feel free to revise your estimates based on the info about Rexchess. Too bad it's too old to be on the rating lists.

[pgn][Event "2nd Harvard Cup"] [Site "Boston, MA USA"] [Date "1991.05.03"] [EventDate "?"] [Round "?"] [Result "1/2-1/2"] [White "Rex (Computer)"] [Black "Boris Gulko"] [ECO "B07"] [WhiteElo "?"] [BlackElo "?"] [PlyCount "107"] 1.d4 d6 2.e4 c6 3.Nc3 Nf6 4.Nf3 Bg4 5.Be2 e6 6.h3 Bh5 7.Bf4 Be7 8.Nd2 Bg6 9.e5 Nd5 10.Nxd5 cxd5 11.Nf3 Nc6 12.O-O O-O 13.c3 a6 14.exd6 Bxd6 15.Bxd6 Qxd6 16.Nh4 Be4 17.f3 Bg6 18.Nxg6 hxg6 19.f4 Rac8 20.Qd3 Ne7 21.Rae1 Qb6 22.Qd2 Nf5 23.Bd3 Rc6 24.Bxf5 gxf5 25.Re2 Qd8 26.Kh2 b5 27.Rfe1 g6 28.Re3 a5 29.a3 Kg7 30.Qe2 Qb8 31.Qf2 Rh8 32.Kh1 Rc4 33.Qg3 Qd6 34.R3e2 b4 35.axb4 axb4 36.Ra1 Rhc8 37.Re3 bxc3 38.bxc3 Qc7 39.Ra3 Rb8 40.Re2 Rb1+ 41.Kh2 Rc1 42.Re3 Rc6 43.Rb3 Ra6 44.Qf2 Raa1 45.g3 Qc4 46.Rb4 Qa2 47.Rb2 Rh1+ 48.Kg2 Rag1+ 49.Kf3 Qa1 50.Ra2 Qd1+ 51.Qe2 Qxe2+ 52.Rexe2 Rxh3 53.Rg2 Rc1 54.Ra3 1/2-1/2 [/pgn]

Yes, that was a rapid, not blitz game, something like 30' + inc or so. In the last one or two Harvard Cups Rex (or successor Socrates) was at least even with strong U.S. GMs of the early 90s at rapid, running on a Pentium, so probably already near World Champ. level in blitz. I'm checking out "Giraffe" as a candidate for being an engine of strong GM level in blitz now, partly because it was a predecessor to AlphaZero and therefore to Lc0.

[lkaufman]

In blitz (let's say 3' + 2" or as close to this as possible), the top engines today are far beyond human level. But how far down the list do we have to go to find engines (and specified hardware) that score evenly against GMs, preferably ones with known identities and ratings? I'm sure there is plenty of data to answer this question as countless games have been played online over the years, but does anyone actually have some data, such as "Engine xyz on one thread scored 50% against GMs averaging 2600 FIDE" for example? The question I'd like to answer is: How much would we have to add to CCRL blitz ratings to estimate the FIDE blitz rating of a human GM who would score 50% against it at 3' + 2"?

Hardware is also a big factor. Back in 2001 I gave private computer classes to GM Mecking, whose ICC blitz rating was the no.1. It turned out that he got this rating in great part by beating up on Chess Tiger 13 and Fritz 6 accounts in blitz. I was surprised by this, but saw him play 4-5 games in which he regularly bamboozled them in endgames. However, could he still do this, even with that same skill level, if they ran on a modern machine, even with one core? Maybe, but I am not nearly so confident.

Yes, I'm sure hardware makes a big difference. What do you think the hardware was like that he played against? Also, what time limit? An easy way to raise your blitz rating online is to play something like 12' +4"against engines which is technically "blitz" but of course not really so. Against modern engines it's not easy to even reach an endgame without being obviously lost.

[Vinvin]

Aubervilliers Open Rapid in 1993, Chess Genius made a 2530 perf : Lost against Spiridonov (2420), but beat GM Bachar Kouatly (2505) and IM Lupu (2435). -> http://www.blitzchess.fr/fr/biblio/lapu ... ce-n3.html

Rebel vs Yusupov, June 1997, 5 minutes, 7 games, Rebel won 6-1.
http://www.rebel.nl/june2.htm

Rebel vs Anand, July 1998, 5m+5s, 4 games, Rebel won 3-1.
http://www.rebel.nl/july21.htm