TalkChess.com

Adam Hair wrote:

EvgeniyZh wrote: You claimed TPU is 180 TOPS, while it is 180 TFLOPS per pod of four TPU, thus your evaluation 4 times higher than it should be.

From what I can find, Google seems to be calling a motherboard containing 4 ASICs a Cloud TPU, and a TPU pod is 64 Cloud TPUs.

Milos wrote:

Adam Hair wrote:

EvgeniyZh wrote: You claimed TPU is 180 TOPS, while it is 180 TFLOPS per pod of four TPU, thus your evaluation 4 times higher than it should be.

From what I can find, Google seems to be calling a motherboard containing 4 ASICs a Cloud TPU, and a TPU pod is 64 Cloud TPUs.

The guy is just a cocky kiddo with a keyboard faster than his brain. It's a lost cause really to argue anything he says.

Just a quote from https://ai.google/tools/cloud-tpus/:

Each device delivers up to 180 teraflops of floating-point performance, and these new TPUs are designed to be connected into even larger systems. A 64-TPU pod can apply up to 11.5 petaflops of computation to a single ML training task.

lkaufman wrote:

Lyudmil Tsvetkov wrote:

lkaufman wrote:

EvgeniyZh wrote:

Milos wrote:

clumma wrote:

Milos wrote:4 hours my ass (pardon my french).

Far fewer transistors and joules were used training AlphaZero than have been used training Stockfish. You can soon rent those TPUs on Google's cloud, or apply for free access now, so stop complaining. Furthermore it's an experimental project in early days and performance is obviously not optimal, so all the 'but-but-but 30 Elo because they used SF 8 instead of SF 8.00194' sounds really dumb.

Days of alpha-beta engines have come to an abrupt end.

-Carl

Sorry, that is pretty childish rent.
Google is obviously comparing apples and oranges and again doing marketing stunt and ppl are falling for it.
Days of Alpha0 on normal hardware are years away. But keep on dreaming, no one can take that from you.

P.S. Just as a small comparison. leelazero open source project trying to replicate alpha0 in Go, took 1 month to get the same games as AG0 got in 3 hours, that with constant 1000 volunteers.
For chess it would take even more.

Training AlphaZero would take tons of time. Just like creating SF from 0. However, running it took 4 TPU, which is comparable to whats available to (rich) consumers - you can get 6-8 NVIDIA V100 which would get you similar performance.

To me this is the most informative post in the whole thread, assuming it is accurate (I know nothing about TPUs). The only reasonable comparison I can think of between the AlphaZero hardware and the Stockfish hardware is cost of equivalent machines. It doesn't matter to me how much hardware was used to reach the current level of strength for both engines, just whether the playing conditions were fair. You seem to be implying that comparable hardware to the 4 TPUs would cost no more (maybe much less?) than the sixty-four core machine used by SF. Is this correct? I'm asking to learn, not making a claim myself either way.

The other conditions were of course not "fair", but reasonable given that AlphaZero only trained for a few hours. I suppose if Stockfish used a good book, was allowed to use its time management as if the time limit were pure increment, and used the latest dev. version, the match would have been much closer, but probably (judging by the infinite win to loss ratio and the actual games) SF would have still lost. The games were amazing.

Bottom line, assuming the comparable cost claim is accurate: If Google wants to optimize the software for a few weeks and sell it, rent it, or give it away, we have a revolution in computer chess. But my guess is that they won't do this, in which case the revolution may be delayed a couple years or so.

Larry, what kind of revolution, this is 30/1 hardware advantage.
Alpha is currently at 2850 level.

Based on the estimated $60k price for equivalent hardware vs. maybe $20k for the 64 core SF machine (my guess) it would be 3 to 1, not 30. The actual hardware used by Alpha would be useless for SF, so you can't compare the hardware any other way than by price, I think. It sounds like the cost of the type of hardware needed for Alpha is expected to plummet while the cost of normal CPUs just trends slightly lower. We already see the same thing in GO. Leela (top GO engine for pc, and free like SF) plays a stone or so stronger with even a cheap GPU than without one. So while Alpha might only play at 2850 level on your laptop, it might be super-strong in a year or so on something many people could afford. But if Google doesn't release it, that won't happen.

EvgeniyZh wrote:

Lyudmil Tsvetkov wrote:

Uri Blass wrote:

Lyudmil Tsvetkov wrote:

Uri Blass wrote:

Lyudmil Tsvetkov wrote:

kranium wrote:

Lyudmil Tsvetkov wrote:

clumma wrote:

Lyudmil Tsvetkov wrote:Alpha had considerable hardware advantage

That comparison is not straightforward, but this claim does not seem to be true. SF had 64 threads. I'm not up on the latest scaling behavior of the engine but that has got to be near saturation.

-Carl

From what I gleaned from hardware comparisons, the advantage is 16/1.
Why would one want to run a similar very unfair match?
Only one thing comes to mind: that the company will want to advertise its colossal breakthrough with TPUs and artificial intelligence and then sell its products.

But then, the achievement is not there.

The fact that Google has created a chess playing entity that crushes SF is notable (and fascinating).

TPUs are not for sale, and (at the moment) are applied only to Googles deep learning and research projects,
except when Google donates them to research for free.

https://techcrunch.com/2017/05/17/the-t ... cientists/

What would be the score between SF on 64 cores and SF on 1024 cores out of 100 games?
You think the bigger-hardware SF would score less than 64 points?
I guess at least 80.

So what is so new?
They applied some big hardware, that is all.
The real strength of Alpha is 2850, so around spot 97 or so among engines.
97 is not such a bad achievement, after all.

I doubt if SF on 1024 cores is going to score even 50%
Maybe after some point more cores are counter productive for stockfish.

I also doubt if it is possible to get at least 80 points against stockfish with 64 cores at 1 minute per move.

Why not?
How much would SF 16-cores vs SF single core score, that is easily reproducible.
The experts claim the TPUs lack any SMP inefficiencies.

If you give the engines 10 minutes per move then I doubt so they play almost perfect chess then I guess that you will get less than 80% and 1 minute per move for 64 cores is probably stronger than 10 minutes per move for the 1 core.

Actually, the hardware advantage + simulated opening book(very important, as SF lost most games already in the opening), was close to 50/1, so how will SF 50 cores fare against SF single core?

Certainly somewhere 95% or so.

Are you aware that the dependency of TTD vs. core number is sublinear? 64 cores to 1 is not like 4096 to 64. Moreover even if it were, the dependency of strength of play vs TTD is also sublinear, and, certainly, is practically bounded. It is actually demonstrated in paper, page 7. Did you read it?

EvgeniyZh wrote:

Adam Hair wrote:

EvgeniyZh wrote: You claimed TPU is 180 TOPS, while it is 180 TFLOPS per pod of four TPU, thus your evaluation 4 times higher than it should be.

From what I can find, Google seems to be calling a motherboard containing 4 ASICs a Cloud TPU, and a TPU pod is 64 Cloud TPUs.

Yeah, well, per device with 4 TPUs.

BTW, new gen NVIDIA GPUs are claimed to have 100+ FP16 TFLOPS, which would made AlphaZero on consumer PC reality. 2xTitan V will match 4 TPUs, however, that needs some test of course.

tmokonen wrote:

Lyudmil Tsvetkov wrote:Eelco, how can you buy into the SCAM too?
The hardware advantage was 50/1.
It plays 1850-elo chess on a single core.

Above diagram is already way way won for black; Stockfish blundered already in the opening with Nce5, this is already lost.

Alpha beating me? Gosh, I will shred it to pieces.
It understands absolutely nothing of closed positions, no such were encountered in the sample.

It is all about the hardware, 2 or 3 beautiful games, with the d4-e5-f6 chain outperforming a whole black minor piece, one great attack on the bare SF king and one more, all the rest is just exceding computations.
Nothing special about its eval.

Just another crappy, misinformed, false bravado post from a guy who quit his job to pursue the quixotic dream of finding the perfect old-school end point evaluation for an alpha beta searcher. He can't accept the fact that his years of painstaking effort have been rendered moot by a project that was just a "meh, let's spend a few hours and see what happens" lark by the team that already conquered Go, a much more complex game than chess.

kranium wrote:

Lyudmil Tsvetkov wrote:

chessmobile wrote:Seems to play a mean game of chess. The endgames is where it excels. Many games looked equal to the naked eye but Alpha went on to win. If this thing follows the Go project then expect in a few months a monster that will beat it's current version quite easily.

Again, 80% of games were already decided in the early opening.
Due to the opening book Alpha unfairly used.

What opening book?

Jhoravi wrote:Do you all think that AlphaZero from scratch only knows the rules of chess and nothing more? How about each piece values like 1 for pawn and 3 for knight etc? I would be so amazed if it learned all the pieces values from self play.

BeyondCritics wrote:

cdani wrote:
Will be very interesting to know which was the typical deep achieved by AlphaZero. I bet that much less than Stockfish..

You lost that bet
AlphaZero uses MCTS https://en.wikipedia.org/wiki/Monte_Carlo_tree_search. From this source https://www.arxiv-vanity.com/papers/1712.01815v1/:

Instead of an alpha-beta search with domain-specific enhancements, AlphaZero uses a general-purpose Monte-Carlo tree search (MCTS) algorithm. Each search consists of a series of simulated games of self-play that traverse a tree from root to leaf.

...

At the end of the game, the terminal position is scored according to the rules of the game to compute the game outcome -1 for a loss, 0 for a draw, and +1 for a win.

lkaufman wrote:

Lyudmil Tsvetkov wrote:In the 10-games sample, which is freely accessible, I see the following:

Game 1 and 2 feature this position:
[d]r1bqk2r/ppp2ppp/2p2n2/2b1p3/4P3/3P1N2/PPP2PPP/RNBQK2R w KQkq - 0 6

SF has traded bishop for knight and on move 6, already has worse position.

Game 3:

[d]rn1qkb1r/p2p1ppp/bp2pn2/2pP4/2P5/5NP1/PPQ1PP1P/RNB1KB1R b KQkq - 0 6

On move 6, SF already is much worse, if not lost at all.

Game 4:

[d]rnbqkb1r/pppn1ppp/4p3/3pP3/3P1P2/2N5/PPP3PP/R1BQKBNR b KQkq f3 0 5

On move 5, SF is already considerably worse.

Games 5 and 6:

[d]rn1q1rk1/pbppbppp/1p2pn2/3P4/2P5/5NP1/PP2PPBP/RNBQ1RK1 b - - 0 7

On move 7, SF is much much worse.

Game 9:

[d]rnbqkb1r/pppn1ppp/4p3/3pP3/3P1P2/2N5/PPP3PP/R1BQKBNR b KQkq f3 0 5

On move 5, SF is considerably worse.

Game 10: repetition of games 5 and 6

So, actually, only games 7 and 8 featured more balanced opening, all the rest was decided very early into the opening, with Alpha having trained the opening on human games. SF, on the other hand, does not rely on human opening knowledge, which is much superior.

So that my assessment 80% of the games were decided by the in-built opening knowledge is fully correct.

With that, my assessment is that the lack of openin gbook was the biggest disadvantage to SF.


It was basically an openings book match.

All of those positions are ones normally played by Grandmasters and considered to offer White no more than his normal opening edge. Maybe the French defense is a little better for White than say the Berlin. I'm pretty sure that Alpha zero would have held the draw playing the other side of them. The gambits played in the Queen's Indian are tricky, but not objectively much better for White.