AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Lyudmil Tsvetkov
Posts: 6036
Joined: Tue Jun 12, 2012 10:41 am

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by Lyudmil Tsvetkov » Wed Dec 06, 2017 8:02 pm

jhellis3 wrote:
Why would I care for such an engine?
The better question is why the engine (or anyone) would care whether you care.
I am an anonymous person, and I don't claim I am able to beat Stockfish.
Oops, actually I do. In that case, the best thing I can think of is a direct clash between me and Alpha. :D

jhellis3
Posts: 347
Joined: Fri Aug 16, 2013 10:36 pm

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by jhellis3 » Wed Dec 06, 2017 8:04 pm

I think you are confused. You seem to believe you have something to offer.

Lyudmil Tsvetkov
Posts: 6036
Joined: Tue Jun 12, 2012 10:41 am

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by Lyudmil Tsvetkov » Wed Dec 06, 2017 8:07 pm

Dan Cooper wrote:
Lyudmil Tsvetkov wrote:
Alpha hardware equivalent was somewhere 1024 standard cores.
How 1024 cores compare with 64 cores?
How scientific is that.
Has Jonny on 2000 cores ever beaten Komodo 28-0?
I guess it has scored some wins out of very few games, maybe it is even a tie or Komodo has scored slightly more, so fully comparable.

But as said, SMP on so many cores is highly inefficient, while TPUs seem not to have that problem.
And also, most importantly, the built-in, or simulated, opening book.
That really makes the difference, if you have carefully browsed the games.

Lyudmil Tsvetkov
Posts: 6036
Joined: Tue Jun 12, 2012 10:41 am

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by Lyudmil Tsvetkov » Wed Dec 06, 2017 8:11 pm

kranium wrote:
Lyudmil Tsvetkov wrote:
clumma wrote:
Lyudmil Tsvetkov wrote:Alpha had considerable hardware advantage
That comparison is not straightforward, but this claim does not seem to be true. SF had 64 threads. I'm not up on the latest scaling behavior of the engine but that has got to be near saturation.

-Carl
From what I gleaned from hardware comparisons, the advantage is 16/1.
Why would one want to run a similar very unfair match?
Only one thing comes to mind: that the company will want to advertise its colossal breakthrough with TPUs and artificial intelligence and then sell its products.

But then, the achievement is not there.
The fact that Google has created a chess playing entity that crushes SF is notable (and fascinating).

TPUs are not for sale, and (at the moment) are applied only to Googles deep learning and research projects,
except when Google donates them to research for free.

https://techcrunch.com/2017/05/17/the-t ... cientists/
What would be the score between SF on 64 cores and SF on 1024 cores out of 100 games?
You think the bigger-hardware SF would score less than 64 points?
I guess at least 80.

So what is so new?
They applied some big hardware, that is all.
The real strength of Alpha is 2850, so around spot 97 or so among engines.
97 is not such a bad achievement, after all.

User avatar
cdani
Posts: 2059
Joined: Sat Jan 18, 2014 9:24 am
Location: Andorra
Contact:

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by cdani » Wed Dec 06, 2017 8:19 pm

Nothing strange to happen as I suggested some times.

Chess engines are good only in the subset of positions that are likely to navigate consequence of their algorithms. They are ignoring an incredible big part of the possible tree. So the feared death draw with the current engines is just a local minimum.

They exists much elaborated chess concepts that no engine is able to understand, some players intuit but human lack of precision always loses to current engines. But an engine with elaborated concepts and Stockfish like precision should be doable, and of course will destroy current Stockfish.

I hope google team will work a bit harder on this project and they organize some more serious match once they have trained its engine much more.

jhellis3
Posts: 347
Joined: Fri Aug 16, 2013 10:36 pm

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by jhellis3 » Wed Dec 06, 2017 8:23 pm

So what is so new?
Sigh.... I will try to put it terms you can understand.

AlphaZero is not like SF.

AlphaZero evaluated at 80 thousand nodes/sec while SF was at 80 million.

But the eval AlphaZero is using is a self constructed network. So the end result is more like Magnus Carlsen evaluating 80 thousand nodes per second with 0 mistakes. But actually it is worse than that because its "understanding" of the game is even better than Magnus's, it is beyond the human realm, and it is ever improving.

There are no gaps in its understanding for you to repeatedly exploit, and should a very large miracle occur and you were to find one. It would learn, on its own. It does not play chess as humans understand it, it plays chess as it understands it...

abulmo2
Posts: 131
Joined: Fri Dec 16, 2016 10:04 am
Contact:

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by abulmo2 » Wed Dec 06, 2017 8:24 pm

Milos wrote:It actually is, instead of 4TPUs required to run Alpha0 so far, on x64 hardware one would need around 2000 Haswell cores to achieve the same speed of NN
Details about Google's TPU are available here.
https://drive.google.com/file/d/0Bx4haf ... xtcEk/view

For NN inference, 1 TPU is around 15-30 times faster than an Haswell (with multiple cores, so your figure of 2000x faster for 4 TPUs vs 1 core is right), but other comparable properties favor the CPU:
TPU vs CPU
Die size: TPU is < half size of an Haswell
Frequency: 700 Mhz vs 2300
Power: 40W vs 145W
Memory Bandwitdth: 51 vs 34

The TPU is much more efficient than a CPU (for NN inference), but it does not eat more power, occupies more size, etc.
So using 4 TPU vs 64 Haswell cores, is fair I think.
Richard Delorme

abulmo2
Posts: 131
Joined: Fri Dec 16, 2016 10:04 am
Contact:

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by abulmo2 » Wed Dec 06, 2017 8:28 pm

Lyudmil Tsvetkov wrote:With what is this different from a self-tuning software, as widely used in autotuning engines, applied on a very large scale/involving tremendous hardware?
Most of current chess eval can be modelised as a single neuron (or perceptron). AlphaGo uses many neurons. That's a big difference.
Richard Delorme

Milos
Posts: 3174
Joined: Wed Nov 25, 2009 12:47 am

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by Milos » Wed Dec 06, 2017 8:29 pm

abulmo2 wrote:
Milos wrote:It actually is, instead of 4TPUs required to run Alpha0 so far, on x64 hardware one would need around 2000 Haswell cores to achieve the same speed of NN
Details about Google's TPU are available here.
https://drive.google.com/file/d/0Bx4haf ... xtcEk/view

For NN inference, 1 TPU is around 15-30 times faster than an Haswell (with multiple cores, so your figure of 2000x faster for 4 TPUs vs 1 core is right), but other comparable properties favor the CPU:
TPU vs CPU
Die size: TPU is < half size of an Haswell
Frequency: 700 Mhz vs 2300
Power: 40W vs 145W
Memory Bandwitdth: 51 vs 34

The TPU is much more efficient than a CPU (for NN inference), but it does not eat more power, occupies more size, etc.
So using 4 TPU vs 64 Haswell cores, is fair I think.
As fair as using SF running on a single CPU Haswell and 100 Ultrascale+ FPGA chips used only for evaluation. Sorry, but there is no f from fair in this comparison. If we miraculously got Alpha0 source code for x64, we would need around 2000 Haswell cores for the similar performance, or at least 100 top of the range 1080Ti GPUs.

Lyudmil Tsvetkov
Posts: 6036
Joined: Tue Jun 12, 2012 10:41 am

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by Lyudmil Tsvetkov » Wed Dec 06, 2017 8:32 pm

abulmo2 wrote:
Milos wrote:It actually is, instead of 4TPUs required to run Alpha0 so far, on x64 hardware one would need around 2000 Haswell cores to achieve the same speed of NN
Details about Google's TPU are available here.
https://drive.google.com/file/d/0Bx4haf ... xtcEk/view

For NN inference, 1 TPU is around 15-30 times faster than an Haswell (with multiple cores, so your figure of 2000x faster for 4 TPUs vs 1 core is right), but other comparable properties favor the CPU:
TPU vs CPU
Die size: TPU is < half size of an Haswell
Frequency: 700 Mhz vs 2300
Power: 40W vs 145W
Memory Bandwitdth: 51 vs 34

The TPU is much more efficient than a CPU (for NN inference), but it does not eat more power, occupies more size, etc.
So using 4 TPU vs 64 Haswell cores, is fair I think.
Apart from frequency, how would the other parameters affect speed?
Is not frequency already factored in the '20 times faster assessment'?

2000/1, with added SMP inefficiencies, will make it 3000/1, 3000/64,
in what way could that be fair?

Post Reply