Grandmaster-Level Chess Without Search. Large-scale attention-based architectures and datasets of unprecedented scale.

[Vinvin]

Paper : https://arxiv.org/pdf/2402.04494.pdf

"Lichess blitz Elo of 2895 against humans", my estimation is around top 50 FIDE Blitz players.

"We also show that our model outperforms AlphaZero’s policy and value networks (without MCTS) and GPT-3.5-turbo-instruct."

[smatovic]

Nice.

Our agent’s aggressive style is highly successful
against human opponents and achieves a grandmaster-
level Lichess Elo of 2895. However, we ran another
instance of the bot and allowed other engines to play
it. Its estimated Elo was far lower, i.e., 2299. Its ag-
gressive playing style does not work as well against
engines that are adept at tactical calculations, par-
ticularly when there is a tactical refutation to a sub-
optimal move. Most losses against bots can be ex-
plained by just one tactical blunder in the game that
the opponent refutes.

Moreover, GPT-3.5-turbo-
instruct also requires whole games encoded via PGN
to reduce hallucinations according to Carlini (2023),
who also finds that GPT-4 struggles to play full games
without making illegal moves, so we do not compare
against GPT-4.

--
Srdja

[towforce]

Paper : https://arxiv.org/pdf/2402.04494.pdf

"Lichess blitz Elo of 2895 against humans", my estimation is around top 50 FIDE Blitz players.

"We also show that our model outperforms AlphaZero’s policy and value networks (without MCTS) and GPT-3.5-turbo-instruct."

Wow!

[towforce]

P19 of the paper - it only gets 2300 against computers due to tactical errors.

[Uri Blass]

P19 of the paper - it only gets 2300 against computers due to tactical errors.

I wonder if you do not compare apples with orange.

If the rating of the computer opponents is not based on games against humans but on games against computers then it may be misleading.

[Werewolf]

2900 LiChess would be, what? 2500 FIDE Elo?

[Whiskers]

P19 of the paper - it only gets 2300 against computers due to tactical errors.

I wonder if you do not compare apples with orange.

If the rating of the computer opponents is not based on games against humans but on games against computers then it may be misleading.

https://lichess.org/@/Polaris2Bot (Polaris is about 3000 elo on the CCRL scale) is 2400 on lichess playing against other engines, a huge gap between human rating and engine rating is entirely normal.

[Vinvin]

Video from today :

[towforce]

Video from today

Thank you: I've just watched it, and I can tell you that this video is of ABSOLUTELY FUNDAMENTAL IMPORTANCE:

* Researchers at Google DeepMind previously created an AI system that can play chess at a grandmaster level, significantly outperforming humans.

* In their new paper, the researchers developed a novel AI technique that can also play chess at a grandmaster level, but without using the same approaches as their previous system.

* The new system was trained on 15 billion chess board positions and the moves played by the powerful Stockfish chess engine, rather than learning by playing against itself.

* Despite its relatively small size (270 million parameters, 3000x smaller than GPT-4), the new chess AI can make 20 moves per second on a $200 GPU, or 2 moves per second on a CPU.

* The key innovation is that this chess AI learns to approximate the underlying chess algorithm, rather than just memorizing tactical patterns. This could enable the extraction of understandable chess algorithms from the neural network.

* This work is seen as a step towards developing AI systems that can not just provide answers, but learn to generate useful algorithms that can be applied to various problems, beyond just the game of chess.

In the past, NN's were encoding a large number of surface features. Now they're going after the deep underlying patterns.

This is the most important thing that's ever happened in computer chess - by a wide margin: whatever the second most important thing is, it doesn't even come close to this.

[Magnum]

Nice.

Our agent’s aggressive style is highly successful
against human opponents and achieves a grandmaster-
level Lichess Elo of 2895. However, we ran another
instance of the bot and allowed other engines to play
it. Its estimated Elo was far lower, i.e., 2299. Its ag-
gressive playing style does not work as well against
engines that are adept at tactical calculations, par-
ticularly when there is a tactical refutation to a sub-
optimal move. Most losses against bots can be ex-
plained by just one tactical blunder in the game that
the opponent refutes.

Moreover, GPT-3.5-turbo-
instruct also requires whole games encoded via PGN
to reduce hallucinations according to Carlini (2023),
who also finds that GPT-4 struggles to play full games
without making illegal moves, so we do not compare
against GPT-4.

--
Srdja

Like I always said Lichess Elo is crap.
I must know it 3500
Calculate at least -600 to -1000 to get real elo.