Alphazero news

[Laskos]

Is it because people who wrote those 'fully decent' papers are not Google-branded?

But in my view a 'decent paper' is still a decent paper even if it is published on a morning newspaper.

You have no idea how is to publish in "Nature" and "Science" (Deep Mind publishing in both on this same theme). HGM has a "Nature" paper. I don't have, although I do have (few) papers in the best journals of physics (Phys. Rev. Letters, for example). I don't know about morning newspapers, not exactly my field of expertise. Many, many very solid researchers never in their life get to have a "Nature" or a "Science" paper.

[matthewlai]

@Matthew

IBM had a "whole series of GM's" training Deep Blue, for the Kasparov matchup.
AlphaGo had one Go player as a consultant, I think.

Has DeepMind used the services of any past or present GMs or IMs while creating A0 ?
If yes, could we know who ?

Yes we did have Fan Hui as a consultant for AlphaGo.

There wasn't one for AlphaZero during development of this Science version, but we did have GM Matthew Sadler look at the games in the paper and to pick the ones he thought was most interesting. We also collaborated with him later on the World Chess Championship commentary [1][2], and their upcoming book [3]. We obviously learned a few things about AlphaZero in the process.

[1]: https://chess24.com/en/read/news/alphaz ... -games-1-8
[2]: https://chess24.com/en/read/news/alphaz ... games-9-12
[3]: https://www.newinchess.com/game-changer

[matthewlai]

Hi Matthew,

Nice to see you on the forum again!
I was a big fan of your probability based search. In fact, shortly after you published your thesis I read it and my only confusion was why you did it in chess and not Go, so I wrote my B.Sc. thesis doing exactly that. Naturally, I was wondering whether you got to try your probability based search further in AlphaZero or any other project?

With best regards,
Jonathan Rosenthal

Thanks! I am glad you found the idea interesting. It's something I wish I had time to explore further, because I really felt it makes more sense than depth-limited alpha-beta.

It wasn't used in AZ because AZ is based on MCTS, and my goal of the probability-based search was to basically "make alpha-beta more MCTS". MCTS already naturally shapes the tree the way a probability-based search would. It also makes incorporating prior much simpler. The only reason I didn't actually use MCTS was because, like everyone at the time, I didn't think it would be possible to use MCTS on a game that has so much sharp tactics. In fact, we weren't sure about that either until we actually tried AlphaZero on chess.

[matthewlai]

Matthew, thank you very much for participating in this thread! Everyone here should be grateful, that we are now getting the answers that were missing so much after releasing of the preliminary paper last December. Let's keep this discussion a technical one. With regards to this, I have a question, that came up while I was greedily digging through the papers. One number seems to be very odd to me: In the match against the SF with opening book (Brainfish), when AZ play black, it gets ~2/4 % wins against SF8/SF9, but ~18% wins when playing against the supposedly stronger Brainfish ?! Beside a docu bug, I can imagine two possible reasons: Either the number of games in this match has been very low or AZ discovered some poor opening book lines. The latter would be very interesting, especially how many and which lines are affected. While getting the game notation from these matches would be perfect, knowing how many games have been played in these tests would also be helpful.
Thanks again for your contribution, Matthew!

It's hard to say and I don't want to speculate much beyond what we have data to support, but my guess (and I can very well be wrong) is that there's much less diversity when SF uses the BF opening book. We already didn't have a lot of diversity from start position, but start position at least has several variations that are roughly equal and both AZ and SF have enough non-determinism (through multi-threaded scheduling mostly) that we still got reasonably diverse games. With the BF games we took out most of SF's non-determinism, and it's possible that SF just ends up playing a line that's not very good often, or something like that. In fact, we found that as we explained in the paper, if we force AZ to play some sub-optimal moves (in its opinion) to enforce diversity, we win even more games! I realise there's a lot of hand-waving here, but there are just too many possibilities.

I don't remember the number of games played, but it was more than high enough that the result is statistically significant.

We decided to release games from the start position and TCEC positions as the main result of the chess part of the paper because start position is more scientifically pure (they were actually playing the game of chess, not a game that's just like chess except you are forced to start from these positions), and from TCEC openings we show that we can play well even in openings that it wouldn't normally play.

[duncan]

You have no idea how is to publish in "Nature" and "Science" (Deep Mind publishing in both on this same theme). HGM has a "Nature" paper. I don't have, although I do have (few) papers in the best journals of physics (Phys. Rev. Letters, for example).

Do you have a link ?

[Laskos]

Matthew, thank you very much for participating in this thread! Everyone here should be grateful, that we are now getting the answers that were missing so much after releasing of the preliminary paper last December. Let's keep this discussion a technical one. With regards to this, I have a question, that came up while I was greedily digging through the papers. One number seems to be very odd to me: In the match against the SF with opening book (Brainfish), when AZ play black, it gets ~2/4 % wins against SF8/SF9, but ~18% wins when playing against the supposedly stronger Brainfish ?! Beside a docu bug, I can imagine two possible reasons: Either the number of games in this match has been very low or AZ discovered some poor opening book lines. The latter would be very interesting, especially how many and which lines are affected. While getting the game notation from these matches would be perfect, knowing how many games have been played in these tests would also be helpful.
Thanks again for your contribution, Matthew!

It's hard to say and I don't want to speculate much beyond what we have data to support, but my guess (and I can very well be wrong) is that there's much less diversity when SF uses the BF opening book. We already didn't have a lot of diversity from start position, but start position at least has several variations that are roughly equal and both AZ and SF have enough non-determinism (through multi-threaded scheduling mostly) that we still got reasonably diverse games. With the BF games we took out most of SF's non-determinism, and it's possible that SF just ends up playing a line that's not very good often, or something like that. In fact, we found that as we explained in the paper, if we force AZ to play some sub-optimal moves (in its opinion) to enforce diversity, we win even more games! I realise there's a lot of hand-waving here, but there are just too many possibilities.

I don't remember the number of games played, but it was more than high enough that the result is statistically significant.

We decided to release games from the start position and TCEC positions as the main result of the chess part of the paper because start position is more scientifically pure (they were actually playing the game of chess, not a game that's just like chess except you are forced to start from these positions), and from TCEC openings we show that we can play well even in openings that it wouldn't normally play.

I don't remember what Cerebellum book lines are chosen by what UI, but using for SF8 a regular polyglot opening book like the small, but good BookX.bin and with Lc0 without any book in Cutechess-Cli, I did get very varied openings. And a decrease of Lc0 strength of at least 50 Elo points compared to just playing from Initial Board position, but at short time controls. I think in Cutechess-Cli one has a random seed for a .bin book, but I don't remember well now.

[pohl4711]

It's hard to say and I don't want to speculate much beyond what we have data to support, but my guess (and I can very well be wrong) is that there's much less diversity when SF uses the BF opening book.

That is correct. When playing with the default settings (one of the default options is BestBookMove=true), BrainFish plays always the best move out of the Cerebellum-Library, which means: BrainFish always plays the same move in the same position.

Regards - Stefan (SPCC (main test-website of Brainfish)) https://www.sp-cc.de/

[Laskos]

It's hard to say and I don't want to speculate much beyond what we have data to support, but my guess (and I can very well be wrong) is that there's much less diversity when SF uses the BF opening book.

That is correct. When playing with the default settings (one of the default options is BestBookMove=true), BrainFish plays always the best move out of the Cerebellum-Library, which means: BrainFish always plays the same move in the same position.

Regards - Stefan (SPCC (main test-website of Brainfish)) https://www.sp-cc.de/

Yes, but didn't they take SF8 + book itself, and not the full BrainFish with its UCI option? That would be a good match: A0 versus full BrainFish of early 2018 with varied openings UCI option, but they seem to not have done that.

[matthewlai]

Matthew, thank you very much for participating in this thread! Everyone here should be grateful, that we are now getting the answers that were missing so much after releasing of the preliminary paper last December. Let's keep this discussion a technical one. With regards to this, I have a question, that came up while I was greedily digging through the papers. One number seems to be very odd to me: In the match against the SF with opening book (Brainfish), when AZ play black, it gets ~2/4 % wins against SF8/SF9, but ~18% wins when playing against the supposedly stronger Brainfish ?! Beside a docu bug, I can imagine two possible reasons: Either the number of games in this match has been very low or AZ discovered some poor opening book lines. The latter would be very interesting, especially how many and which lines are affected. While getting the game notation from these matches would be perfect, knowing how many games have been played in these tests would also be helpful.
Thanks again for your contribution, Matthew!

It's hard to say and I don't want to speculate much beyond what we have data to support, but my guess (and I can very well be wrong) is that there's much less diversity when SF uses the BF opening book. We already didn't have a lot of diversity from start position, but start position at least has several variations that are roughly equal and both AZ and SF have enough non-determinism (through multi-threaded scheduling mostly) that we still got reasonably diverse games. With the BF games we took out most of SF's non-determinism, and it's possible that SF just ends up playing a line that's not very good often, or something like that. In fact, we found that as we explained in the paper, if we force AZ to play some sub-optimal moves (in its opinion) to enforce diversity, we win even more games! I realise there's a lot of hand-waving here, but there are just too many possibilities.

I don't remember the number of games played, but it was more than high enough that the result is statistically significant.

We decided to release games from the start position and TCEC positions as the main result of the chess part of the paper because start position is more scientifically pure (they were actually playing the game of chess, not a game that's just like chess except you are forced to start from these positions), and from TCEC openings we show that we can play well even in openings that it wouldn't normally play.

I don't remember what Cerebellum book lines are chosen by what UI, but using for SF8 a regular polyglot opening book like the small, but good BookX.bin and with Lc0 without any book in Cutechess-Cli, I did get very varied openings. And a decrease of Lc0 strength of at least 50 Elo points compared to just playing from Initial Board position, but at short time controls. I think in Cutechess-Cli one has a random seed for a .bin book, but I don't remember well now.

It's true, there are many books that we could have chosen from. The problem is there wasn't one that we thought everyone will be happy with, and we do have a lot of "critics" (in quotes because they aren't the useful kind of critics) who will probably go into any book we choose, find a line that they think is bad using whatever their preferred analysis method is on the day, and say we deliberately chose that book because it's bad.

In the end we settled on Brainfish book just because it's actually generated by Stockfish, so it's about as "pure" as it gets.

I do believe that opening book will help quite a bit at short time control, since "intuitive play" is what AZ/Lc0 are good at. Though at the time controls we played with, SF does make some very reasonable opening moves (at least when it would still be in most books), and I'm not sure if it would have made a lot of difference.

[matthewlai]

It's hard to say and I don't want to speculate much beyond what we have data to support, but my guess (and I can very well be wrong) is that there's much less diversity when SF uses the BF opening book.

That is correct. When playing with the default settings (one of the default options is BestBookMove=true), BrainFish plays always the best move out of the Cerebellum-Library, which means: BrainFish always plays the same move in the same position.

Regards - Stefan (SPCC (main test-website of Brainfish)) https://www.sp-cc.de/

Thanks! That is good to know.