AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

[Lyudmil Tsvetkov]

Good point re the book - Alphazero effevtibely has one. But it is still not a small achievement wiining against SF, even with unequal conditions. But many of us would like to see a more equal test.

And I have been wondering why in most games Alpha takes very early advantage.
Stockfish opening play seems normal for its level, but that thing plays like a beast.

[Lyudmil Tsvetkov]

Alpha had considerable hardware advantage

... but that has got to be near saturation.

-Carl

Excellent point.

I mean at that hardware, there won't be any fluctuation on the best candidate move by SF8 anyway.

Alpha hardware equivalent was somewhere 1024 standard cores.
How 1024 cores compare with 64 cores?
How scientific is that.

I don't know what saturation you are talking about, from what I read, without fully understanding it, the TPUs are a very different architecture and quite differently affected by general computer chess concepts.

How can you make up such claims when there are not enough data on the PDF file?
TPU instruction set and benchmark was not properly published.
Was only claimed in PDF report that SF8 was on 64 threads (but no clock speed). Was discussed years ago on this forum that clockspeed, including those of the buses,
would trump number of cores.

Would choose a 1-core 4.0 Ghz over 8-cores running at 2.0 GHz with buggy SMP implementation of engine.

I did not read carefully the whole pdf, because from the start I saw it is unreadable.
How can they use 'The modern chess instructor' by Steinitz to improve Alpha?
I read however a number of sites on the Internet, from where it became clear that:
- TPUs are the equivalent of at least 256 normal cores
- TPUs are highly efficient for calculations, and mostly lack the diminishing returns problem of SMP implementations; so that, actually, while SF has been using 64 cores and losing probably half of that hardware to SMP inefficiencies with larger number of cores, Alpha lost almost nothing from its tremendous hardware

So, that, in reality, the hardware difference is not 16/1, as I thought initially, but more like 30/1.
Add to this the early opening advantage Alpha gets due to the simulated book, and conducting the test has been fully meaningless.

Alpha would play not stronger than 1850 on a single core.
Why would I care for such an engine?

[jhellis3]

Why would I care for such an engine?

The better question is why the engine (or anyone) would care whether you care.

[Lyudmil Tsvetkov]

As has been mentioned previously, one can not really make direct core count comparisons in this case.

The most "fair" metric I can think of using is system power consumption, and I would guess that SF was at a bit of a disadvantage in this regard. Regardless, the writing is clearly on the wall.....

Your last sentence is enigmatic, to say the least.
What is on the wall?

What kind of metric is system power consumption, when TPUs are geared towards extremely low power consumption?

[Lyudmil Tsvetkov]

Why would I care for such an engine?

The better question is why the engine (or anyone) would care whether you care.

I am an anonymous person, and I don't claim I am able to beat Stockfish.
Oops, actually I do. In that case, the best thing I can think of is a direct clash between me and Alpha.

[jhellis3]

I think you are confused. You seem to believe you have something to offer.

[Lyudmil Tsvetkov]

Alpha hardware equivalent was somewhere 1024 standard cores.
How 1024 cores compare with 64 cores?
How scientific is that.

Has Jonny on 2000 cores ever beaten Komodo 28-0?

I guess it has scored some wins out of very few games, maybe it is even a tie or Komodo has scored slightly more, so fully comparable.

But as said, SMP on so many cores is highly inefficient, while TPUs seem not to have that problem.
And also, most importantly, the built-in, or simulated, opening book.
That really makes the difference, if you have carefully browsed the games.

[Lyudmil Tsvetkov]

Alpha had considerable hardware advantage

That comparison is not straightforward, but this claim does not seem to be true. SF had 64 threads. I'm not up on the latest scaling behavior of the engine but that has got to be near saturation.

-Carl

From what I gleaned from hardware comparisons, the advantage is 16/1.
Why would one want to run a similar very unfair match?
Only one thing comes to mind: that the company will want to advertise its colossal breakthrough with TPUs and artificial intelligence and then sell its products.

But then, the achievement is not there.

The fact that Google has created a chess playing entity that crushes SF is notable (and fascinating).

TPUs are not for sale, and (at the moment) are applied only to Googles deep learning and research projects,
except when Google donates them to research for free.

https://techcrunch.com/2017/05/17/the-t ... cientists/

What would be the score between SF on 64 cores and SF on 1024 cores out of 100 games?
You think the bigger-hardware SF would score less than 64 points?
I guess at least 80.

So what is so new?
They applied some big hardware, that is all.
The real strength of Alpha is 2850, so around spot 97 or so among engines.
97 is not such a bad achievement, after all.

[cdani]

Nothing strange to happen as I suggested some times.

Chess engines are good only in the subset of positions that are likely to navigate consequence of their algorithms. They are ignoring an incredible big part of the possible tree. So the feared death draw with the current engines is just a local minimum.

They exists much elaborated chess concepts that no engine is able to understand, some players intuit but human lack of precision always loses to current engines. But an engine with elaborated concepts and Stockfish like precision should be doable, and of course will destroy current Stockfish.

I hope google team will work a bit harder on this project and they organize some more serious match once they have trained its engine much more.

[jhellis3]

So what is so new?

Sigh.... I will try to put it terms you can understand.

AlphaZero is not like SF.

AlphaZero evaluated at 80 thousand nodes/sec while SF was at 80 million.

But the eval AlphaZero is using is a self constructed network. So the end result is more like Magnus Carlsen evaluating 80 thousand nodes per second with 0 mistakes. But actually it is worse than that because its "understanding" of the game is even better than Magnus's, it is beyond the human realm, and it is ever improving.

There are no gaps in its understanding for you to repeatedly exploit, and should a very large miracle occur and you were to find one. It would learn, on its own. It does not play chess as humans understand it, it plays chess as it understands it...