Alphazero news

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
duncan
Posts: 8156
Joined: Mon Jul 07, 2008 8:50 pm

Re: Alphazero news

Post by duncan » Thu Dec 06, 2018 11:42 pm

matthewlai wrote:
Thu Dec 06, 2018 11:00 pm

Unfortunately we weren't able to get our time machine up and running before we finished writing the paper in order to test against engines that hadn't been released yet. But don't worry, we are working hard on that!
I am not sure why you have to wait for your time machine. Just send the time machine you are going to create back in time to now and you can use it :)

jp
Posts: 259
Joined: Mon Apr 23, 2018 5:54 am

Re: Alphazero news

Post by jp » Thu Dec 06, 2018 11:44 pm

Daniel Shawul wrote:
Thu Dec 06, 2018 11:35 pm
matthewlai wrote:
Thu Dec 06, 2018 11:00 pm
While I sympathize with that statement, releasing A0 source code and networks for anyone to test sounds better.
Yes. That's the best solution in every way. Easy to do, gets everyone else to do all the hard testing work for you, etc...

noobpwnftw
Posts: 275
Joined: Sun Nov 08, 2015 10:10 pm

Re: Alphazero news

Post by noobpwnftw » Fri Dec 07, 2018 12:09 am

I have a few questions:
TCEC SuFi used 120' + 15" TC, Division P used 90' + 10" TC, since DM went for mimicking TCEC conditions this time like using a 44-core machine and with the same openings, why mess with time controls(again)?

Also, a year ago the NPS of A0 was 80K, now it is only around 60K, that's about 30% a nerf, what happened, people overclocked the TPUs a year ago?

jp
Posts: 259
Joined: Mon Apr 23, 2018 5:54 am

Re: Alphazero news

Post by jp » Fri Dec 07, 2018 12:36 am

noobpwnftw wrote:
Fri Dec 07, 2018 12:09 am
Also, a year ago the NPS of A0 was 80K, now it is only around 60K, that's about 30% a nerf, what happened, people overclocked the TPUs a year ago?
What was the nps they got with SF8 this time with the 44-core machine?

noobpwnftw
Posts: 275
Joined: Sun Nov 08, 2015 10:10 pm

Re: Alphazero news

Post by noobpwnftw » Fri Dec 07, 2018 12:42 am

A year ago, A0: 4TPUs @ 80k SF: 64 cores @ 70,000k
Now, A0: 4TPUs @ 63k (13k), SF: 44 cores @ 58,100k (24,000k)

"Evaluation speed is the average over entire games at full time controls from
the initial board position (the main evaluation in Figure 2), standard deviations are shown in
parentheses."

matthewlai
Posts: 778
Joined: Sun Aug 03, 2014 2:48 am
Location: London, UK
Contact:

Re: Alphazero news

Post by matthewlai » Fri Dec 07, 2018 1:04 am

sovaz1997 wrote:
Thu Dec 06, 2018 11:07 pm
matthewlai wrote:
Thu Dec 06, 2018 11:00 pm
Vinvin wrote:
Thu Dec 06, 2018 10:26 pm
Stockfish 8 ???
They really fear real Stockfish ...
Unfortunately we weren't able to get our time machine up and running before we finished writing the paper in order to test against engines that hadn't been released yet. But don't worry, we are working hard on that!
Do you have results against modern versions? Against Stockfish 9, for example?
We did play against a development version of Stockfish that was very close to SF 9 (we finished testing just before SF 9 was released), and the result was very similar to results against SF 8. It's in Fig 2 here - http://science.sciencemag.org/content/362/6419/1140

Elo differences from new versions are always greater between two versions of the same (or very similar) programs, and they don't usually translate to equal improvement against other opponents.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.

matthewlai
Posts: 778
Joined: Sun Aug 03, 2014 2:48 am
Location: London, UK
Contact:

Re: Alphazero news

Post by matthewlai » Fri Dec 07, 2018 1:06 am

mwyoung wrote:
Thu Dec 06, 2018 11:29 pm
They played a later version of stockfish. A January version but they did not give the score other then A0 won.
The score is given (in the form of a bar graph) in Fig 2. here: http://science.sciencemag.org/content/362/6419/1140
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.

matthewlai
Posts: 778
Joined: Sun Aug 03, 2014 2:48 am
Location: London, UK
Contact:

Re: Alphazero news

Post by matthewlai » Fri Dec 07, 2018 1:15 am

Daniel Shawul wrote:
Thu Dec 06, 2018 11:35 pm
matthewlai wrote:
Thu Dec 06, 2018 11:00 pm
Vinvin wrote:
Thu Dec 06, 2018 10:26 pm
Stockfish 8 ???
They really fear real Stockfish ...
Unfortunately we weren't able to get our time machine up and running before we finished writing the paper in order to test against engines that hadn't been released yet. But don't worry, we are working hard on that!
While I sympathize with that statement, releasing A0 source code and networks for anyone to test sounds better.
Many will not be satisfied with in-house testing with supposedly fair conditions.
That would be good, but it would also be a lot of work for us (AZ is tightly-coupled with DM and Google's systems) for not really much value to the scientific community. We feel that it's our ideas and algorithms that are important, not our implementation. That's why we have published all the algorithms we developed in detail, with almost-runnable pseudo-code, so that they can be replicated easily. And who knows, maybe someone will come up with a better implementation than we did! I am personally really excited about what Leela has already achieved, and hopefully it won't be long before it will be at the level of AlphaZero!
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.

matthewlai
Posts: 778
Joined: Sun Aug 03, 2014 2:48 am
Location: London, UK
Contact:

Re: Alphazero news

Post by matthewlai » Fri Dec 07, 2018 1:18 am

duncan wrote:
Thu Dec 06, 2018 11:38 pm
matthewlai wrote:
Thu Dec 06, 2018 11:00 pm
Vinvin wrote:
Thu Dec 06, 2018 10:26 pm
Stockfish 8 ???
They really fear real Stockfish ...
Unfortunately we weren't able to get our time machine up and running before we finished writing the paper in order to test against engines that hadn't been released yet. But don't worry, we are working hard on that!
Do you have an opinion on whether alpha zero would beat the latest version of stockfish ?
I don't think it would be really useful for me to speculate on that.

Suffice to say, AlphaZero today is not the same as AlphaZero in the paper either.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.

matthewlai
Posts: 778
Joined: Sun Aug 03, 2014 2:48 am
Location: London, UK
Contact:

Re: Alphazero news

Post by matthewlai » Fri Dec 07, 2018 1:19 am

duncan wrote:
Thu Dec 06, 2018 11:42 pm
matthewlai wrote:
Thu Dec 06, 2018 11:00 pm

Unfortunately we weren't able to get our time machine up and running before we finished writing the paper in order to test against engines that hadn't been released yet. But don't worry, we are working hard on that!
I am not sure why you have to wait for your time machine. Just send the time machine you are going to create back in time to now and you can use it :)
Doesn't that violate some laws of physics or something? :D
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.

Post Reply