AlphaZero Chess is not that strong ...

[Vinvin]

The results of the match AZ vs SF8 is translated in +100 Elo for AZ.
The weakened part of Stockfish for this match is already known :
1) Only 1 GB hashtable (64GB to 128GB would have been more way more suitable)
2) 1 minute by move fixed time control
3) No opening book for Stockfish (AZ saved his opening knowledge from its previous games)
4) Only version 8 (the current development version is already about 40 Elo above)

With this 4 points upgraded to a regular level, the current Stockfish version would been already 100 Elo stronger.

One more thing, when you look at the graph : https://www.dropbox.com/s/kaq37zf0z66d6 ... g.png?dl=0
you see there's no progress from 300,000 to 700,000 steps. This AZ is already at his best !

[hgm]

1GB hash seems pretty big. Have you actually measured if there is any advantage whatsoever of increasing the hash size?

And how much would AlphaZero gain by using a book, and not having to run at fixed time per move, in your estimate?

[Eelco de Groot]

Yes, but just for the sake of argument, the lost games might have still been lost with better conditions, not counting an openingbook or tablebases for SF. Because of eval holes and Stockfish having no idea about winning probabilities (I consider that a form of Monte Carlo generated contempt). An opening book is not really fair because the opening book of Alpha Zero I think works more like pattern recognition? No opening moves are learnt, as far as I understand? (I may be wrong I did not read it all) Pattern recognition is accepted domain of the eval. It could be used in all the traditional rating lists and tournaments.

Stockfish getting a bit stronger with more hash and timemanagement, might still have lost 28 games Or close to it. Alpha Zero could also have some time management built in. There are no hash tables in Alpha Zero, this reminds a bit of the Deep Blue hardware that also had no no hashtables for the leaves, only for the part that was executed in software.

[gladius]

1GB hash seems pretty big. Have you actually measured if there is any advantage whatsoever of increasing the hash size?

And how much would AlphaZero gain by using a book, and not having to run at fixed time per move, in your estimate?

1GB hash is far too low. I have immense respect for the deepmind team, but this was a pretty serious error.

At 70 million positions/second, that 1GB of hash is being overwritten once every second (10 bytes per entry = 700MB overwritten/second). Even worse, it's being contended by all 64 threads on the same 1GB of RAM. To give some context, we usually play our 60 sec games with 64Mb of hash, with one thread!

In these games, at 1 minute/move, the hash table is all but useless. It should probably have been 32 or 64GB, and the machine certainly had that much RAM.

[CheckersGuy]

Yeah 100 elo above stockfish isn't like what they have accomplished with Go. However, I wonder how strong alphaZero could get given more training and optimizations specific to chess. Since AlphaZero was, basically, for any board game I think there might be a lot of improvements.

[mario carbonell]

One possible explanation it that chess engines are near the maximum strength possible, that may be around 3600 or 3700 elo.







We can see that in the game of go, AlphaZero was still learning, but at chess seemed to have arrived to a plateau. Maybe the maximum strength?

That also speaks about the very good work done in Stockfish, that is very near to A0.

The problem with chess is that the draw result exists, and even one or two pawns difference in material is not enough to win.

[hgm]

1GB hash is far too low. I have immense respect for the deepmind team, but this was a pretty serious error.

At 70 million positions/second, that 1GB of hash is being overwritten once every second (10 bytes per entry = 700MB overwritten/second). Even worse, it's being contended by all 64 threads on the same 1GB of RAM. To give some context, we usually play our 60 sec games with 64Mb of hash, with one thread!

But normally you would only start to notice any effect if you overload the hash table more than a factor 10, and even then the speed only goes down as the 12th root of the hash size. That would mean 5-8 Elo/memory doubling. The factor 10 is reached in 14 sec, so you miss only 2 doublings, or 10-16 Elo. And at fixed time per move, the last 15sec of each move might often not be used at all.

Have you actually measured how much Elo you lose by overloading the hash table a factor 40 at fixed time per move?

[Guenther]

One possible explanation it that chess engines are near the maximum strength possible, that may be around 3600 or 3700 elo.







We can see that in the game of go, AlphaZero was still learning, but at chess seemed to have arrived to a plateau. Maybe the maximum strength?

That also speaks about the very good work done in Stockfish, that is very near to A0.

The problem with chess is that the draw result exists, and even one or two pawns difference in material is not enough to win.

What if it just needs more steps than in go? Even in go it looks like it had already hit an early wall between 350K and 500k+ something, but in hindsight that assumption would have been wrong.

We would only know if it still tries to learn and we get more data.

[Vinvin]

One possible explanation it that chess engines are near the maximum strength possible, that may be around 3600 or 3700 elo.

I think they reached "the maximum strength possible" of this configuration of the neural network. But it can be improved !!

[Lyudmil Tsvetkov]

The results of the match AZ vs SF8 is translated in +100 Elo for AZ.
The weakened part of Stockfish for this match is already known :
1) Only 1 GB hashtable (64GB to 128GB would have been more way more suitable)
2) 1 minute by move fixed time control
3) No opening book for Stockfish (AZ saved his opening knowledge from its previous games)
4) Only version 8 (the current development version is already about 40 Elo above)

With this 4 points upgraded to a regular level, the current Stockfish version would been already 100 Elo stronger.

One more thing, when you look at the graph : https://www.dropbox.com/s/kaq37zf0z66d6 ... g.png?dl=0
you see there's no progress from 300,000 to 700,000 steps. This AZ is already at his best !

True Vincent.
Here you are perfectly right.

Chess is much more complex than Go and requires more knowledge-based algorithms they will never quite get.