Google's AlphaGo team has been working on chess

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Post Reply
CheckersGuy
Posts: 272
Joined: Wed Aug 24, 2016 7:49 pm

Re: Google's AlphaGo team has been working on chess

Post by CheckersGuy » Sat Dec 09, 2017 4:05 pm

I think working on a similar engine to AlphaZero would be really intresting. One will obviously not get the same performance as AlphaZero but it would still be intresting to see, how well the algorithm scales (with additional hardware and time) compared to current state of the art engines.

brianr
Posts: 248
Joined: Thu Mar 09, 2006 2:01 pm

Re: Google's AlphaGo team has been working on chess

Post by brianr » Sat Dec 09, 2017 4:25 pm

See here:

https://github.com/Zeta36/chess-alpha-zero

I have gotten it to work, but it is painfully slow with just one gpu.

LeelaZero is trying a more distributed approach like Fishtest, albeit for Go.

https://github.com/gcp/leela-zero

Henk
Posts: 5083
Joined: Mon May 27, 2013 8:31 am

Re: Google's AlphaGo team has been working on chess

Post by Henk » Sat Dec 09, 2017 5:01 pm

That's fast show us some games.

CheckersGuy
Posts: 272
Joined: Wed Aug 24, 2016 7:49 pm

Re: Google's AlphaGo team has been working on chess

Post by CheckersGuy » Sat Dec 09, 2017 7:15 pm

brianr wrote:See here:

https://github.com/Zeta36/chess-alpha-zero

I have gotten it to work, but it is painfully slow with just one gpu.

LeelaZero is trying a more distributed approach like Fishtest, albeit for Go.

https://github.com/gcp/leela-zero
That's really nice. The new Titan V from Nvidia would be a really good gpu for training. (10 times the performance of a 1080TI) but it costs 3000$ :lol:

syzygy
Posts: 4221
Joined: Tue Feb 28, 2012 10:56 pm

Re: Google's AlphaGo team has been working on chess

Post by syzygy » Sat Dec 09, 2017 9:25 pm

Have there been any attempts to use MCTS just for tuning of the eval weights of a "regular" chess engine?

Tuning the eval to predict the outcome of an alpha-beta search is a bit hopeless because of all the tactics that can't be encoded in a typical evaluation function. MCTS might average out the tactics sufficiently that its results can directly be used for tuning. The engine would then use alpha-beta for playing games.

CheckersGuy
Posts: 272
Joined: Wed Aug 24, 2016 7:49 pm

Re: Google's AlphaGo team has been working on chess

Post by CheckersGuy » Sat Dec 09, 2017 9:32 pm

syzygy wrote:Have there been any attempts to use MCTS just for tuning of the eval weights of a "regular" chess engine?

Tuning the eval to predict the outcome of an alpha-beta search is a bit hopeless because of all the tactics that can't be encoded in a typical evaluation function. MCTS might average out the tactics sufficiently that its results can directly be used for tuning. The engine would then use alpha-beta for playing games.
I dont think that would work particularly well. No matter what optimization algorithm one uses we only have a linear funciton of evaluation parameters which I doubt can encode tactics (very well).

Neural networks aren't like that and can even learn non-linear functions well.
As for tuning the "regular" evaluation parameters and dont quite understand what you mean by "using mcts to train eval parameters"

syzygy
Posts: 4221
Joined: Tue Feb 28, 2012 10:56 pm

Re: Google's AlphaGo team has been working on chess

Post by syzygy » Sat Dec 09, 2017 9:52 pm

CheckersGuy wrote:As for tuning the "regular" evaluation parameters and dont quite understand what you mean by "using mcts to train eval parameters"
Run random simulations to get some sort of "averaged" estimation of the evaluation, the idea being that the tactics of the position are washed out sufficiently by the nature of mcts. Then tune the eval weights to better predict that washed out tactic-free estimation.

I don't really expect it to work, but I wouldn't expect AlphaZero Chess to work, either :)

brianr
Posts: 248
Joined: Thu Mar 09, 2006 2:01 pm

Re: Google's AlphaGo team has been working on chess

Post by brianr » Sat Dec 09, 2017 10:28 pm

Henk wrote:That's fast show us some games.
??

If this was meant for me, it would take thousands of cpu/gpu (both) hours to even just begin to train any reasonable NN. It has been estimated that AlphaGo Zero would take something like about 1,700 years on typical consumer hardware. For chess, I only got it to start running, and am still trying to understand it. There is a Connect4 game he did also that is a simpler starting point. The author is considering a distributed approach, I think.

Milos
Posts: 2984
Joined: Wed Nov 25, 2009 12:47 am

Re: Google's AlphaGo team has been working on chess

Post by Milos » Sat Dec 09, 2017 11:38 pm

syzygy wrote:
CheckersGuy wrote:As for tuning the "regular" evaluation parameters and dont quite understand what you mean by "using mcts to train eval parameters"
Run random simulations to get some sort of "averaged" estimation of the evaluation, the idea being that the tactics of the position are washed out sufficiently by the nature of mcts. Then tune the eval weights to better predict that washed out tactic-free estimation.

I don't really expect it to work, but I wouldn't expect AlphaZero Chess to work, either :)
Alpha0 NN has only tactics of depth 8 encoded in weights and it works.
MCTS should be used for playing not training.
Here is what could actually work.
Instead of alpha-beta use identical MCTS as in Alpha0.
When you reach leaf node instead of only preforming eval perform optimized mpv (of moves that wouldn't be cut by LMR) depth 8 search + QS and return score. Transform the best move score into v and ratio of move scores into probability for each move to be played all using some linear function.
If you could for example perform this mpv depth 8 search+QS in 10ms, using 64 cores machine you'd have 6400 MCT searches performed each second. That is still quite shy from 80k performed in Alpha0, but with more powerful machine you'd get there.
And despite all the hype about Alpha0's NN eval, I really doubt that it is better than depth 8 mpv search + QS of SF.

syzygy
Posts: 4221
Joined: Tue Feb 28, 2012 10:56 pm

Re: Google's AlphaGo team has been working on chess

Post by syzygy » Sun Dec 10, 2017 12:24 am

Milos wrote:
syzygy wrote:
CheckersGuy wrote:As for tuning the "regular" evaluation parameters and dont quite understand what you mean by "using mcts to train eval parameters"
Run random simulations to get some sort of "averaged" estimation of the evaluation, the idea being that the tactics of the position are washed out sufficiently by the nature of mcts. Then tune the eval weights to better predict that washed out tactic-free estimation.

I don't really expect it to work, but I wouldn't expect AlphaZero Chess to work, either :)
Alpha0 NN has only tactics of depth 8 encoded in weights and it works.
MCTS should be used for playing not training.
Here is what could actually work.
Instead of alpha-beta use identical MCTS as in Alpha0.
When you reach leaf node instead of only preforming eval perform optimized mpv (of moves that wouldn't be cut by LMR) depth 8 search + QS and return score. Transform the best move score into v and ratio of move scores into probability for each move to be played all using some linear function.
If you could for example perform this mpv depth 8 search+QS in 10ms, using 64 cores machine you'd have 6400 MCT searches performed each second. That is still quite shy from 80k performed in Alpha0, but with more powerful machine you'd get there.
And despite all the hype about Alpha0's NN eval, I really doubt that it is better than depth 8 mpv search + QS of SF.
I think this is more or less what Stéphane is now working on.

If I now understand the AlphaZero MCTS correctly, it is actually a sort of book-builder search. Build a tree by expanding leaf nodes and back up the results to the root, using some rule to select the next leaf node to be expanded.

If this works well with SF's alpha-beta search and evaluation, that would be very funny.

Post Reply