Alpha Zero vs Stockfish 8 tournament conditions.

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Alpha Zero vs Stockfish 8 tournament conditions.

Poll ended at Fri Dec 08, 2017 4:15 am

The time per move and hardware etc was fair.
27
52%
Google set it up to give Alpha Zero an edge.
25
48%
 
Total votes: 52

Milos
Posts: 2903
Joined: Wed Nov 25, 2009 12:47 am

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Milos » Thu Dec 07, 2017 8:23 pm

Branko Radovanovic wrote:Why do you think SF should have played with an opening book and an EGTB? AlphaZero had none. As I gather, it used actual computation to play all its moves, rather than relying on instant lookup.
Alpha0 opening book is actually already stored in its network weights. It would most probably played weaker if you forced it to play from a real opening book.
SF surely lost 20-30 Elo at least due to fixed move time, but given the fact that AlphaZero itself played under the same conditions, there is no reason to believe the outcome would have been any different even with dynamic time management given to both sides. (Actually, if there is any mention of AlphaZero's time management in the paper, I've missed it.)
That's a bad excuse. If Google didn't have time or know how to optimally implement time management, there is no reason to handicap other engine.
Moreover, not going into too much details, classical engine time management is more complex because of the nature of alpha-beta search, when failed high/low engine simply needs much more time to do the research of a position. This is not the case with MCTS.
AlphaZero's hardware is better described as novel than specialized. There is no doubt it will become commonplace, perhaps in a matter of years, while FPGAs will remain exotic, however powerful and energy-efficient they might be.
That is not actually true. BTU that google is exclusively using is much more exotic than anything that exists. FPGAs are quite common and used.
What lacks is actually engineers who could use them efficiently.
Everyone today can learn to make some deep learning in Python in few hour following a YouTube course. Learning how to efficiently trranslate C++ to FPGA takes years. That's the main difference.
If I had to name just one reason why AlphaZero playing chess the way it plays is a monumental achievement, it is this: in the chapter title "Anatomy of a Computer Chess Program" the authors list a number of computer chess techniques (alpha-beta search, material imbalance tables, PSQT, mobility, pawn structure eval, king safety, QS, pruning, extensions, history, SEE, heuristics, TT, etc.) and conclude AlphaZero uses none of them. Not one. It appears that the only thing SF and AlphaZero have in common is the move generator. That's mind-boggling, and remains mind-boggling even if Google's results do appear to be oversold a bit.
This is all fine, but MCTS is also quite an old concept in chess programming and used since forever in Go and other games (it had also fair share of usage in chess quite some time ago). And Alpha0 without it would be nowhere near to actual performance. So yes the way they organized the features and trained the network is quite novel and certainly a breakthrough, but none of that would have been possible without monstrous hardware and "old" techniques such as MCTS an reinforcement learning (at least 25 years old concept in chess).

Ras
Posts: 922
Joined: Tue Aug 30, 2016 6:19 pm
Contact:

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Ras » Thu Dec 07, 2017 10:08 pm

The match was fair, and AlphaGo did NOT have massive hardware advantage, and it does NOT make sense to ask for it to run on x86. Just like it would not make sense to ask for Stockfish to run on TPUs.

Oh, the TPUs are more expensive? Of course they are because they are a brand new development, but the whole point of this match, from Google's point of view, was to prove that this hardware is suited to neural network applications. It was not specialised hardware like with Hydra or Deep Blue. It was general purpose NN hardware. The same hardware also plays Go and Shogi, something that Hydra or Deep Blue could not do - and neither could a Stockfish version boosted by FPGAs.

Oh, the hardware isn't for sale? Wrong point. Google doesn't sell them now because they can use their current monopoly better with cloud offers. But that creates a demand, and since Google has proven how capable these things are, others will catch up.

Just like others caught up when Intel started to put the 4004 into production, the first microprocessor actually available on the market. In fact, I think that Intel should finally wake up here after their failed adventures from Itanic to challenging ARM. Or maybe even AMD since they already have GPU competence. Or Nvidia.

Second, these TPUs with neural networks scale better than traditional cores which are subject to the law of diminishing returns, which is why Johnny with 2000 cores is still weaker than Stockfish. That's why these TPUs are not a one-time artifact like Hydra or Deep Blue - they are here to stay and change computing.

The energy consumption was comparable, that's the central point here, because that's what will determine the cost of the whole approach in the medium run.

Milos
Posts: 2903
Joined: Wed Nov 25, 2009 12:47 am

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Milos » Thu Dec 07, 2017 10:23 pm

Ras wrote:The energy consumption was comparable, that's the central point here, because that's what will determine the cost of the whole approach in the medium run.
That's the only meaningful comparison that can be made, for others there is zero argument, so unrealistic and pointless, basically just fan cheering, so no point really commenting.
But the argument that energy consumption will determine cost in the medium run is so ridiculous that I can't avoid laughing at it despite best effort.
You seems to think that computing hardware prices are formed like house heating systems, the more heat it produces the more expensive it is, I mean that's just brilliant, brilliant :lol: :lol: :lol:.

Ras
Posts: 922
Joined: Tue Aug 30, 2016 6:19 pm
Contact:

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Ras » Thu Dec 07, 2017 11:03 pm

Milos wrote:But the argument that energy consumption will determine cost in the medium run is so ridiculous
If you have two solutions with similar output, but one of them needs massively more energy, then the latter one usually isn't competetive. Energy consumption is a strong indicator here.

Obviously, I'm assuming that there will be competition that will drive down the hardware cost of TPU-like architectures in the future, but that's what I also wrote in my previous posting. Of course, in that scenario, you are welcome to use a 1000 CPU cores instead of one TPU for a NN application. Guess which solution will be inferior - lo and behold, that happens to be the one with more energy consumption.

User avatar
MikeB
Posts: 2505
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by MikeB » Fri Dec 08, 2017 3:04 am

Graham Banks wrote:The only thing I question is that AlphaZero had an opening book based on learning over millions of games, whereas it looks as though Stockfish may not have used an opening book at all.
+1 I believe you are 100% correct!

CheckersGuy
Posts: 272
Joined: Wed Aug 24, 2016 7:49 pm

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by CheckersGuy » Fri Dec 08, 2017 3:25 am

MikeB wrote:
Graham Banks wrote:The only thing I question is that AlphaZero had an opening book based on learning over millions of games, whereas it looks as though Stockfish may not have used an opening book at all.
+1 I believe you are 100% correct!
I still think that calling the weights of the NN an "opening boook" is kinda misleading. It is not an opening book as you would think of one

Milos
Posts: 2903
Joined: Wed Nov 25, 2009 12:47 am

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Milos » Fri Dec 08, 2017 3:37 am

CheckersGuy wrote:
MikeB wrote:
Graham Banks wrote:The only thing I question is that AlphaZero had an opening book based on learning over millions of games, whereas it looks as though Stockfish may not have used an opening book at all.
+1 I believe you are 100% correct!
I still think that calling the weights of the NN an "opening boook" is kinda misleading. It is not an opening book as you would think of one
Why, when NN evaluation function is exactly what an opening book gives you - for a given position a probability of selecting each move and wining probability of that position? And the closer position to the root, the better tuned weights are for it.

clumma
Posts: 144
Joined: Fri Oct 10, 2014 8:05 pm
Location: Berkeley, CA

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by clumma » Fri Dec 08, 2017 3:44 am

Graham Banks wrote:The only thing I question is that AlphaZero had an opening book based on learning over millions of games, whereas it looks as though Stockfish may not have used an opening book at all.
Stockfish was fine out of the opening (in the 10 games at 1 min/move). AlphaZero also played 1200 games against SF from known opening positions and won with a score of 733 (see page 6 of the paper).

-Carl

Nay Lin Tun
Posts: 246
Joined: Mon Jan 16, 2012 5:34 am

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Nay Lin Tun » Fri Dec 08, 2017 5:41 am

It is simple, and without doubt, machine can learn better than human. First step, machines can calculate better than human ( people already accept it). This is the next step of technological revolution ( machines can learn better than human).It would be a bitter experience for many programmers who spent signifiant years of their life to teach a machine(program a machine) rather than let the machine learn by his own.

Zenmastur
Posts: 272
Joined: Sat May 31, 2014 6:28 am

Re: Alpha Zero vs Stockfish 8 tournament conditions.

Post by Zenmastur » Fri Dec 08, 2017 6:07 am

It's been a long time since I have seen such BS posted on this board. Did IQ's suddenly drop while I was away?

It's clear to me that few posts on this thread are objective. Many posts are filled with insults, misinformation, and in some cases, out right lies. What the hell is wrong with you people?

I suggest you get a grip on reality!

Regards,

Forrest
Only 2 defining forces have ever offered to die for you.....Jesus Christ and the American Soldier. One died for your soul, the other for your freedom.

Post Reply