Alpha Zero vs Stockfish 8 tournament conditions.

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.

Alpha Zero vs Stockfish 8 tournament conditions.

Poll ended at Fri Dec 08, 2017 4:15 am

The time per move and hardware etc was fair.
27
52%
Google set it up to give Alpha Zero an edge.
25
48%
 
Total votes: 52

Lyudmil Tsvetkov
Posts: 6037
Joined: Tue Jun 12, 2012 10:41 am

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Lyudmil Tsvetkov » Fri Dec 08, 2017 4:50 pm

clumma wrote:
Graham Banks wrote:The only thing I question is that AlphaZero had an opening book based on learning over millions of games, whereas it looks as though Stockfish may not have used an opening book at all.
Stockfish was fine out of the opening (in the 10 games at 1 min/move). AlphaZero also played 1200 games against SF from known opening positions and won with a score of 733 (see page 6 of the paper).

-Carl
SF was DEAD LOST in 40% straight out of the opening, and very bad in another 40%.
The myth Alpha is very strong in the endgame and won its games there, is just a myth.

Nay Lin Tun
Posts: 509
Joined: Mon Jan 16, 2012 5:34 am

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Nay Lin Tun » Fri Dec 08, 2017 5:21 pm

This cloud service will soon kill amazon EC2, at least for chess funs. :P
https://cloud.google.com/tpu/

Ras
Posts: 1135
Joined: Tue Aug 30, 2016 6:19 pm
Contact:

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Ras » Fri Dec 08, 2017 7:33 pm

Lyudmil Tsvetkov wrote:Most of your statements make very much sense, except the first one. So why do they weigh boxers and weight-lifters before each and every competition?
But they did - the energy consumption was about the same. You can't really object NN software using NN optimised hardware. Stockfish, being classic branch-oriented, sequential software uses an x86 that is optimised for this kind of SW, too, so both have the hardware that is appropriate for their way of computing.

The only exception would be, of course, if one had a much more advanced production process. You wouldn't compare the energy consumption plus performance of, say, a chip produced with 130nm process to a chip produced with 22nm. But I don't think this is the case because Intel's production facilities are well-known for being state of the art.

nabildanial
Posts: 104
Joined: Thu Jun 05, 2014 3:29 am
Location: Malaysia

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by nabildanial » Sat Dec 09, 2017 10:55 am

Graham Banks wrote:The only thing I question is that AlphaZero had an opening book based on learning over millions of games, whereas it looks as though Stockfish may not have used an opening book at all.
Learning over millions of (self-)games is the only way for AlphaZero to raking up its ELO. You don't say about the billions of games Stockfish uses to improve itself on Fishtest.

Damir
Posts: 2069
Joined: Mon Feb 11, 2008 2:53 pm

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Damir » Sat Dec 09, 2017 11:19 am

Not unless the Stockfish team implement learning capability in Stockfish. :)

Lyudmil Tsvetkov
Posts: 6037
Joined: Tue Jun 12, 2012 10:41 am

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Lyudmil Tsvetkov » Sat Dec 09, 2017 2:36 pm

nabildanial wrote:
Graham Banks wrote:The only thing I question is that AlphaZero had an opening book based on learning over millions of games, whereas it looks as though Stockfish may not have used an opening book at all.
Learning over millions of (self-)games is the only way for AlphaZero to raking up its ELO. You don't say about the billions of games Stockfish uses to improve itself on Fishtest.
This was over the course of many years, and besides, SF tunes with a (very crappy) book, so it does not tune for the very first moves, while tuning a lot of irrelevant stuff for the rest.

SF has to get rid of that book, if it has to progress.
They think this is not their main problem currently, but it is.

Lyudmil Tsvetkov
Posts: 6037
Joined: Tue Jun 12, 2012 10:41 am

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Lyudmil Tsvetkov » Sat Dec 09, 2017 2:39 pm

Milos wrote:
Lyudmil Tsvetkov wrote:
stavros wrote:to make it more practical: i want to know the price of 64core st hardware and the price of alpha zero hardware pls anyone? to make a more fair comparison
+ 10, most probably larger than 20/1, just as in the case with hardware speed.
And it used significantly more memory.
And an opening book, which actually decided the whole match.
I made a calculation here.

It would cost from 250k$ to half a million bucks.
Tell that Larry.

Lyudmil Tsvetkov
Posts: 6037
Joined: Tue Jun 12, 2012 10:41 am

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Lyudmil Tsvetkov » Sat Dec 09, 2017 2:44 pm

Ras wrote:
Lyudmil Tsvetkov wrote:Most of your statements make very much sense, except the first one. So why do they weigh boxers and weight-lifters before each and every competition?
But they did - the energy consumption was about the same. You can't really object NN software using NN optimised hardware. Stockfish, being classic branch-oriented, sequential software uses an x86 that is optimised for this kind of SW, too, so both have the hardware that is appropriate for their way of computing.

The only exception would be, of course, if one had a much more advanced production process. You wouldn't compare the energy consumption plus performance of, say, a chip produced with 130nm process to a chip produced with 22nm. But I don't think this is the case because Intel's production facilities are well-known for being state of the art.
By about the same you mean 5 times higher for Alpha?
As you know pretty well, TPUs are precisely designed to run at couple of times lower tension, but that only improves memory and computations.

Ras
Posts: 1135
Joined: Tue Aug 30, 2016 6:19 pm
Contact:

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Ras » Sat Dec 09, 2017 5:31 pm

Lyudmil Tsvetkov wrote:By about the same you mean 5 times higher for Alpha?
No, about equal means about equal. Both in the 300W ballpark or so. You are completely mistaken if you think that Google threw half a computing centre at the match. In the self-training, yes, that was a different story, but so was Stockfish's development (e.g. fishtest).

clumma
Posts: 177
Joined: Fri Oct 10, 2014 8:05 pm
Location: Berkeley, CA

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by clumma » Sat Dec 09, 2017 5:58 pm

Ras wrote:You are completely mistaken if you think that Google threw half a computing centre at the match. In the self-training, yes, that was a different story, but so was Stockfish's development (e.g. fishtest).
In particular, they played 44 million games in 9 hours. Fishtest does that in about 3 weeks.

-Carl

Post Reply