Thank you, Zipproth. I was wondering what Cerebellum would say about these games.
I would also like to say, I am ready to buy Cerebellum, even if all features are not available yet. And please PM me if you need a beta tester!
-Carl
Cerebellum analysis of the AlphaZero - Stockfish Games
Moderators: hgm, Rebel, chrisw
-
- Posts: 186
- Joined: Fri Oct 10, 2014 10:05 pm
- Location: Berkeley, CA
-
- Posts: 6993
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Cerebellum analysis of the AlphaZero - Stockfish Games
That has amazed me too. AZ never follows the draw SF suggests when there are more than one move that lead to a forced draw, 3-fold repetitions mainly. It seems to know exactly which move to avoid. What other conclusion is there then that position is learned?Thomas Zipproth wrote: I calculated the AlphaZero games and added them to Cerebellum, which took some time because Stockfish naturally needed very long calculation times and many added nodes to finally get the (most likely) correct results.
http://www.zipproth.de/Brainfish/Cerebe ... aZero.html
I thought about a way to present a game calculation in a most comprehensive but simple way, and finally ended up with this images.
Of course all side lines are missing, only the best move is additionally displayed, if it is different from the move which was played in the game.
Some conclusions or assumptions gained from the results:
Stockfish's evaluation ist correct most times, but it does not distinguish between a position which is dead draw and other positions which are theoretically also draw, but very complicated with many occassions to make a wrong move. In such positions stockfish sometimes makes errors leading to a loss, even if most times the evaluation of close to zero is correct.
-
- Posts: 6052
- Joined: Tue Jun 12, 2012 12:41 pm
Re: Cerebellum analysis of the AlphaZero - Stockfish Games
I don't understand what you are talking about, what facts?Ras wrote:Btw., these facts somewhat clash with your claim that 80% of the games were lost right in the opening.Lyudmil Tsvetkov wrote:But they are checking facts, while you are just speculating.
Not qualifying: games 1, 2, 3, 7, 8, 10.
Qualifying: game 4.
Debatable: games 5, 6, 9 (move 18/19 is not really opening anymore, IMO).
Or, that is the alternative conclusion, the opening book wouldn't have changed much because it doesn't see the early losses that you can see.
Who claims that at least 4 games have not been lost until move 10 and 4 more close to losing?
If it is Cerebellum, than this is just SF, precisely the engine you should not be using for verification.
-
- Posts: 2488
- Joined: Tue Aug 30, 2016 8:19 pm
- Full name: Rasmus Althoff
Re: Cerebellum analysis of the AlphaZero - Stockfish Games
Take a look at the graphs from the original posting to see at which point they turned.Lyudmil Tsvetkov wrote:I don't understand what you are talking about, what facts?
Well of course you sometimes claim that a certain game is lost after move 2 of an established opening - but if it's "Tsvetkov loss" where no opposing engine is actually able to win that by force, it's not really relevant in an engine match. Or, if the issue is that Alpha0 now IS able to win that, then it is the better engine.Who claims that at least 4 games have not been lost until move 10 and 4 more close to losing?
Disagree. First, Alpha0 only had self-play, too, so it would be a an obvious measure to check what Stockfish can do on it's own if given enough time. The data indicate that the majority of the losses of these 10 games would have happened anyway. Actually, only game 4 is such an early loss that a book could have prevented that.If it is Cerebellum, than this is just SF, precisely the engine you should not be using for verification.
Second, Cerebellum was exactly the book a lot of people would have wanted to include because it's the best that Stockfish can have.
Third, what other book would you have suggested if not Cerebellum?
-
- Posts: 6052
- Joined: Tue Jun 12, 2012 12:41 pm
Re: Cerebellum analysis of the AlphaZero - Stockfish Games
Some people spot the loss 1 move before mate...Ras wrote:Take a look at the graphs from the original posting to see at which point they turned.Lyudmil Tsvetkov wrote:I don't understand what you are talking about, what facts?
Well of course you sometimes claim that a certain game is lost after move 2 of an established opening - but if it's "Tsvetkov loss" where no opposing engine is actually able to win that by force, it's not really relevant in an engine match. Or, if the issue is that Alpha0 now IS able to win that, then it is the better engine.Who claims that at least 4 games have not been lost until move 10 and 4 more close to losing?
Disagree. First, Alpha0 only had self-play, too, so it would be a an obvious measure to check what Stockfish can do on it's own if given enough time. The data indicate that the majority of the losses of these 10 games would have happened anyway. Actually, only game 4 is such an early loss that a book could have prevented that.If it is Cerebellum, than this is just SF, precisely the engine you should not be using for verification.
Second, Cerebellum was exactly the book a lot of people would have wanted to include because it's the best that Stockfish can have.
Third, what other book would you have suggested if not Cerebellum?
Others know the loss is there on move 2.
For example, 1.g4 loses, but how many on this forum would subscribe to this assertion?
I guess I will be the only one.
Cerebellum still plays 1. e4, and, on 1. e4, 1...e6.
Not much to discuss here.
That is what sent SF most convincingly down the drain: choosing the French, where it lost 39(!) games with black out of 50.
Also never fianchettoeing its bishop kingside, either with white or black, on that only it lost most of the QIDs.
-
- Posts: 1080
- Joined: Fri Sep 16, 2016 6:55 pm
- Location: USA/Minnesota
- Full name: Leo Anger
Re: Cerebellum analysis of the AlphaZero - Stockfish Games
Many people want to buy it.clumma wrote:Thank you, Zipproth. I was wondering what Cerebellum would say about these games.
I would also like to say, I am ready to buy Cerebellum, even if all features are not available yet. And please PM me if you need a beta tester!
-Carl
Advanced Micro Devices fan.
-
- Posts: 13447
- Joined: Wed Mar 08, 2006 9:02 pm
- Location: Dallas, Texas
- Full name: Matthew Hull
Re: Cerebellum analysis of the AlphaZero - Stockfish Games
This question seems a false dichotomy to me. The assumption is that a new program cannot see beyond what the current standard can know. But I have always had high expectations of NNs, that they could see be trained to find patterns that human programmers might entirely miss. And since algos are humanly derived...Rebel wrote:That has amazed me too. AZ never follows the draw SF suggests when there are more than one move that lead to a forced draw, 3-fold repetitions mainly. It seems to know exactly which move to avoid. What other conclusion is there then that position is learned?Thomas Zipproth wrote: I calculated the AlphaZero games and added them to Cerebellum, which took some time because Stockfish naturally needed very long calculation times and many added nodes to finally get the (most likely) correct results.
http://www.zipproth.de/Brainfish/Cerebe ... aZero.html
I thought about a way to present a game calculation in a most comprehensive but simple way, and finally ended up with this images.
Of course all side lines are missing, only the best move is additionally displayed, if it is different from the move which was played in the game.
Some conclusions or assumptions gained from the results:
Stockfish's evaluation ist correct most times, but it does not distinguish between a position which is dead draw and other positions which are theoretically also draw, but very complicated with many occassions to make a wrong move. In such positions stockfish sometimes makes errors leading to a loss, even if most times the evaluation of close to zero is correct.
NNs learn what we cannot see. That is the promise of NNs. That's why I'm less skeptical about this.
Matthew Hull
-
- Posts: 2488
- Joined: Tue Aug 30, 2016 8:19 pm
- Full name: Rasmus Althoff
Re: Cerebellum analysis of the AlphaZero - Stockfish Games
That AZ sees something better than a draw. That was easy.Rebel wrote:AZ never follows the draw SF suggests when there are more than one move that lead to a forced draw, 3-fold repetitions mainly. It seems to know exactly which move to avoid. What other conclusion is there then that position is learned?
-
- Posts: 6993
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Cerebellum analysis of the AlphaZero - Stockfish Games
Yep, 2 (or more) moves that lead to a draw, the SF move played has a W/L/D value of 50% hence they play the other one with +50.1% established during the training. Easy too as well.Ras wrote:That AZ sees something better than a draw. That was easy.Rebel wrote:AZ never follows the draw SF suggests when there are more than one move that lead to a forced draw, 3-fold repetitions mainly. It seems to know exactly which move to avoid. What other conclusion is there then that position is learned?
-
- Posts: 273
- Joined: Wed Aug 24, 2016 9:49 pm
Re: Cerebellum analysis of the AlphaZero - Stockfish Games
A neural network is not an opening book. Nough said