So Alpha Zero was a hoax?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
mjlef
Posts: 1402
Joined: Thu Mar 30, 2006 12:08 pm
Contact:

Re: So Alpha Zero was a hoax?

Post by mjlef » Sat Mar 17, 2018 3:43 am

Jouni wrote:I have feeling, that DeepMind is now ashamed of premature report and we get nothing new!
If you are going to call Alpha Zero a hoax, you really should have some evidence. "feelings" are not evidence.

Milos
Posts: 3348
Joined: Wed Nov 25, 2009 12:47 am

Re: So Alpha Zero was a hoax?

Post by Milos » Sat Mar 17, 2018 4:17 am

jhellis3 wrote:Yeah, 1 Titan V should get you at least 75% of the performance of 1 TPU gen2.
Another load of bollocks. A0 uses 3x3 kernel sizes. Tensor cores are performing 4x4 matrix multiplication.
3x3 matrix multiplication is 5x9 = 45 FLOPS.
4x4 matrix multiplication is 7x16 = 112 FLOPS.
So out of 112 TFLOPS of tensor cores A0 can use 45 TFLOPS.
In addition to that it can also use those 14 TFLOPS of real FP32 operations.
In total 60 TFLOPS.
TPU gen2 is 180 TFLOPS.
So basically you need 3 Titan V cards to match performance of 1 TPU gen 2 on A0 CNNs.

Milos
Posts: 3348
Joined: Wed Nov 25, 2009 12:47 am

Re: So Alpha Zero was a hoax?

Post by Milos » Sat Mar 17, 2018 4:24 am

mjlef wrote:
Jouni wrote:I have feeling, that DeepMind is now ashamed of premature report and we get nothing new!
If you are going to call Alpha Zero a hoax, you really should have some evidence. "feelings" are not evidence.
I thought Google is the one that should provide evidence, so far they provided none (well fanboys assume 10 games to be evidence but that just is delusional, since even RomiChess can be trained to have 10 games in which it destroys SF that has almost no randomness in play).
Without any evidence from Google proving A0 is a hoax is like proving the existence of God.

David Xu
Posts: 47
Joined: Mon Oct 31, 2016 8:45 pm

Re: So Alpha Zero was a hoax?

Post by David Xu » Sat Mar 17, 2018 5:28 am

I see the troll has shown up again. Nobody replied until now, though, which is good. :wink: Keep it up, guys.

noobpwnftw
Posts: 293
Joined: Sun Nov 08, 2015 10:10 pm

Re: So Alpha Zero was a hoax?

Post by noobpwnftw » Sat Mar 17, 2018 5:29 am

David Xu wrote:I refer you to the title of this thread. "So Alpha Zero was a hoax?" sounds pretty conspiracy theory-esque to me. If this is not your view, then I apologize for lumping you in with everyone else, but frankly, if you don't want the label "conspiracy theorist", you're going to need to explain why you called AlphaZero an "attention-seeking attempt" in the LCZero thread earlier.
You are contradicting yourself, I had my opinion(as one of the general public and not a part of the almighty shareholders) but what's wrong if it does not favor the A0 side?
If you don't understand my point then I will try to rephrase it all together here instead of you quoting lines here and there without context.
The A0 introduces an approach to use generalized algorithms to play many different games(truth), particularly in chess they released some information that the training worked(truth) and claim that it performed better than SF(inconclusive, doubtful and unscientific) in a 100-game match(truth).

David Xu wrote:I fail to see what is unscientific about the preprint. They described the architecture of the network itself, they described the training procedure, and they described the results against a specific setup for Stockfish--results which are in principle replicable by a third party, given the information they provided. That's what it takes for a result to be "scientific". More varied experiments would have been nice, certainly, but that fact hardly invalidates the results of the experiment they did perform.
Stating those facts does not equal to being scientific.
I can state the fact based on the 10 games that was published and conditions given, it is obvious and convincing that A0 would have a 100% win rate over SF, does it sound scientific to you?
There is a reason why we run those tournament matches instead of playing one on each side and decide the winner, why would this sound irrational to you when it comes to DM should've played more games especially on a short time control and have more variety to be scientific?
David Xu wrote:what is so unbelievable to you about this result that you would sooner postulate deception on the part of Google DeepMind than take them at their word?
If it is so true then why the easy part(as I described above) is not done so to avoid these doubts is unknown to me. I would think of them as if they'd built a boat but sailing in a pool.
David Xu wrote:Monte-Carlo tree search is certainly a well-known technique that has existed for years, but the technique of using it as a policy improvement operator is, as far as I'm aware, a novel one. This technique, of course, the critical aspect of DeepMind's approach, since it's what allowed them to generate such high-quality training data.
What is so different from people are constantly improving alpha-beta search by implementing things like PVS, MTF-d and refining search bounds in many different ways? Does your term "decades-old" include all these too?
David Xu wrote: What I am saying is that the issue of overfitting is well-known, so bringing it up as a specific argument against neural networks in chess is not a particularly strong objection.
This aspect is particularly more important for playing games like Chess as opposed to playing Go, where you are more likely to make one blunder move that would cost you the entire game. There is less tolerance of a wrong "likehood" estimation that a NN would normally give, so this results in less numbers of samples we can gather and correlate and more issues with overfitting.
I have my reasons to support my opinion than calling something I don't agree "disingenuous".

noobpwnftw
Posts: 293
Joined: Sun Nov 08, 2015 10:10 pm

Re: So Alpha Zero was a hoax?

Post by noobpwnftw » Sat Mar 17, 2018 10:29 am

This entire thing is like they are putting a jet engine into a car, describing how to build that awesome jet engine and then say hey this car can take you to 100 places faster than a helicopter.

Then the fanboys all yell "that's quite an awesome jet engine, we must rip ______ and put it on everything, any non-believers are either unfounded or overconfident."...

Yes it's an awesome jet engine but sorry I wouldn't buy that car.

syzygy
Posts: 4426
Joined: Tue Feb 28, 2012 10:56 pm

Re: So Alpha Zero was a hoax?

Post by syzygy » Sat Mar 17, 2018 11:58 am

CheckersGuy wrote:
Vizvezdenec wrote:Anyone who is working in anything remotedly close to science knows that you will never use smth like "it took ONLY 3 hours to..." and statements like that in any scientific-related paper unless you want to get fired.
So seeing statement like this used in pre-print constantly, using old fish, providing cherrypicked games, etc. makes any reasonable person who is not a bandwagoner to think about A0 paper how you really should - good PR project.
Sure, NNs are incredible, but will they make a revolution in chess engine development? My guess is no, at least with current level of NNs. And by the time NNs will need reasonable time to train on pure community enthusiasm, IMHO, alpha-beta engines will anyway play perfect chess.
So if it really did take 3 hours, they should have lied about it ? :D
I guess they then should have left out the word "only", which in fact they did not include :-)

Henk
Posts: 5618
Joined: Mon May 27, 2013 8:31 am

Re: So Alpha Zero was a hoax?

Post by Henk » Sat Mar 17, 2018 12:42 pm

I'm sure they come with a great new report on 1 April so that's pretty soon.

jhellis3
Posts: 373
Joined: Fri Aug 16, 2013 10:36 pm

Re: So Alpha Zero was a hoax?

Post by jhellis3 » Sat Mar 17, 2018 2:53 pm

It is unlikely that NNs can find something systematically wrong and SF can't improve on.
If you genuinely believe this, then there is not much I can say or do to help you. But I will give you a little challenge which you can take on as your courage allows. SF currently has difficulty recognizing and properly penalizing cases of bad/entombed bishops. Your mission is to write a patch which handles this flawlessly for all cases.... Ok, I'll settle for writing a patch, passing [0,5] on fishtest, and getting it merged into master (regardless of overall efficacy). Now, this a pretty easily recognizable and visually trivial pattern for humans to see, so coding up a solution should be no problem. Let me know when your patch gets merged...
You can use anything vs SF and run as many matches as you want until you get a satisfying set of 100 games when the fate is on your side, it is still the fact and you are not forging it.
Sorry, I am not interested in conspiracy theories.
For the former I think training against SF would probably make it weaker.
Not in a match vs SF.
just use them to play as a book already.
No, that would be much weaker... If you don't understand why, think about it.
You can read my edit on why this is not likely to happen.
I'm not really interested in what you think you know.
Then the fanboys all yell "that's quite an awesome jet engine, we must rip ______ and put it on everything, any non-believers are either unfounded or overconfident."...
Ad hominem attacks and car analogies are not a level of discourse I am interested in.

Anyway, here is much more important question:

Is gravity a hoax?

I expect people's opinions and reasonings change the answer to that just about as much as the one posed in the OP.

peter
Posts: 1748
Joined: Sat Feb 16, 2008 6:38 am
Full name: Peter Martan

Re: So Alpha Zero was a hoax?

Post by peter » Sat Mar 17, 2018 4:47 pm

:D
Some more sense of humour would the whole thing fit quite well I guess,
dear Henk
:!:
Peter.

Post Reply