AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
clumma
Posts: 152
Joined: Fri Oct 10, 2014 8:05 pm
Location: Berkeley, CA

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by clumma » Fri Dec 08, 2017 12:50 am

Milos wrote:Y-scale in Fig.2 means Elo without added reference point (that is what that clumsy term google used actually means). It doesn't mean Elo difference between 2 programs, because if it was Elo difference you'd need a single curve, not 2 of them (it's pretty obvious if you just used your head a bit).
The text clearly states the Elo baseline (relative Elo = 0) for Fig. 2 is the respective traditional program with 40ms thinking time.
Beside not understanding much, you really don't seem too bright. Sorry, but I don't want to waste any more time with you, it's pointless.
Actually it's obvious to readers of this thread that while Evgeniy stated simple facts, almost everything you've written is incorrect.

-Carl

Gusev
Posts: 1475
Joined: Mon Jan 28, 2013 1:51 pm

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by Gusev » Fri Dec 08, 2017 1:07 am

So, basically, Stockfish 8 lost to Giraffe on steroids. :lol:
Jesse Gersenson wrote:
clumma wrote:A truly stunning result. Matthew Lai is a coauthor!

https://arxiv.org/pdf/1712.01815.pdf

-Carl
Wa to go Matthew!

Adam Hair
Posts: 3185
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by Adam Hair » Fri Dec 08, 2017 1:40 am

EvgeniyZh wrote: You claimed TPU is 180 TOPS, while it is 180 TFLOPS per pod of four TPU, thus your evaluation 4 times higher than it should be.
From what I can find, Google seems to be calling a motherboard containing 4 ASICs a Cloud TPU, and a TPU pod is 64 Cloud TPUs.

Milos
Posts: 2994
Joined: Wed Nov 25, 2009 12:47 am

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by Milos » Fri Dec 08, 2017 1:48 am

clumma wrote:
Milos wrote:Y-scale in Fig.2 means Elo without added reference point (that is what that clumsy term google used actually means). It doesn't mean Elo difference between 2 programs, because if it was Elo difference you'd need a single curve, not 2 of them (it's pretty obvious if you just used your head a bit).
The text clearly states the Elo baseline (relative Elo = 0) for Fig. 2 is the respective traditional program with 40ms thinking time.
Your comment that has absolutely nothing to do with my actual claim (about obviously wrong scaling for SF in that figure) or Evgeniy's pretty absurd claim (that figure represents basically derivative of Elo performance) but it is instead stating totally irrelevant point regarding Elo baseline (Elo reference point = 0) tells basically that you didn't at all understand the argument, claims or anything that has been discussed but only felt the urge to jump in and demonstrate your "smartness" by "proving" me wrong.
Congratulations man, you got me.
Now you can go stare at the sun, count stars or do whatever "smart" thing you usually do.

Adam Hair
Posts: 3185
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by Adam Hair » Fri Dec 08, 2017 1:59 am

Milos wrote:
EvgeniyZh wrote:
Milos wrote:
EvgeniyZh wrote:
Milos wrote:
EvgeniyZh wrote:64 cores to 1 is not like 4096 to 64. Moreover even if it were, the dependency of strength of play vs TTD is also sublinear, and, certainly, is practically bounded. It is actually demonstrated in paper, page 7. Did you read it?
Figure 2 is completely bogus. If wish Google actually cited that reference that shows SFs Elo performance increase when going from 10s to 1min/move of under 20Elo.
So your are playing expert and even don't understand meaning of relative ELO?
Lol, I missed this pearl.
"Relative Elo" (btw. it's Elo not ELO), you just invented that, did you not? :lol: :lol: :lol: :lol:
Ok, so make a thought experiment. You may even actually perform it to get the point. Take two instances of Stockfish. Run a tournament between them, that'd emerge rating for both, which would be around 0 if you made everything right. Now increase the time control. Repeat. You'll still get zero, if you are still doing things right. Would that mean that there wasn't improvement? Do you get the concept now?
Y-scale in Fig.2 means Elo without added reference point (that is what that clumsy term google used actually means). It doesn't mean Elo difference between 2 programs, because if it was Elo difference you'd need a single curve, not 2 of them (it's pretty obvious if you just used your head a bit).
Beside not understanding much, you really don't seem too bright.
Sorry, but I don't want to waste any more time with you, it's pointless.
If you are going to be exceedingly arrogant, at least make sure you do not make mistakes. The 2 curves represent the Elo difference of SF 8 and of AlphaZero from SF 8 at 40ms/move as the thinking time varies.

Milos
Posts: 2994
Joined: Wed Nov 25, 2009 12:47 am

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by Milos » Fri Dec 08, 2017 1:59 am

Adam Hair wrote:
EvgeniyZh wrote: You claimed TPU is 180 TOPS, while it is 180 TFLOPS per pod of four TPU, thus your evaluation 4 times higher than it should be.
From what I can find, Google seems to be calling a motherboard containing 4 ASICs a Cloud TPU, and a TPU pod is 64 Cloud TPUs.
The guy is just a cocky kiddo with a keyboard faster than his brain. It's a lost cause really to argue anything he says.

Just a quote from https://ai.google/tools/cloud-tpus/:
Each device delivers up to 180 teraflops of floating-point performance, and these new TPUs are designed to be connected into even larger systems. A 64-TPU pod can apply up to 11.5 petaflops of computation to a single ML training task.

Milos
Posts: 2994
Joined: Wed Nov 25, 2009 12:47 am

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by Milos » Fri Dec 08, 2017 2:04 am

Adam Hair wrote:If you are going to be exceedingly arrogant, at least make sure you do not make mistakes. The 2 curves represent the Elo difference of SF 8 and of AlphaZero from SF 8 at 40ms/move as the thinking time varies.
If you are nitpicking they represent Elo performance of Alpha0 and SF vs thinking time with Elo baseline (Elo = 0) taken as Elo of SF8 at 40ms/move which is pretty much equivalent to what I've said up in the thread. So writing something just to write is pretty pointless.
Normally you don't have such a knee-jerk reaction, but hey everyone can have a bad day ;).

P.S. Instead try being constructive. Have you seen the paper, I guess you did? How you explain this Fig. 2 performance of SF gaining 20Elo going from 10s to 1min thinking time? Pretty sure if they are ever gonna publish it in Nature they'd have to change it. ;)
Last edited by Milos on Fri Dec 08, 2017 2:12 am, edited 1 time in total.

Adam Hair
Posts: 3185
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by Adam Hair » Fri Dec 08, 2017 2:11 am

Milos wrote:
clumma wrote:
Milos wrote:Y-scale in Fig.2 means Elo without added reference point (that is what that clumsy term google used actually means). It doesn't mean Elo difference between 2 programs, because if it was Elo difference you'd need a single curve, not 2 of them (it's pretty obvious if you just used your head a bit).
The text clearly states the Elo baseline (relative Elo = 0) for Fig. 2 is the respective traditional program with 40ms thinking time.
Your comment that has absolutely nothing to do with my actual claim (about obviously wrong scaling for SF in that figure) or Evgeniy's pretty absurd claim (that figure represents basically derivative of Elo performance) but it is instead stating totally irrelevant point regarding Elo baseline (Elo reference point = 0) tells basically that you didn't at all understand the argument, claims or anything that has been discussed but only felt the urge to jump in and demonstrate your "smartness" by "proving" me wrong.
Congratulations man, you got me.
Now you can go stare at the sun, count stars or do whatever "smart" thing you usually do.
He commented on your nonsensical statement of "Elo without added reference point", which is obviously not what the y axis represents.

As far as your actual claim, the scaling could be due to the test conditions, specifically the lack of opening positions. I am planning on checking this out to see if I can replicate the scaling.

Milos
Posts: 2994
Joined: Wed Nov 25, 2009 12:47 am

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by Milos » Fri Dec 08, 2017 2:15 am

Adam Hair wrote:He commented on your nonsensical statement of "Elo without added reference point", which is obviously not what the y axis represents.
So reference point of Y-axis of 0 (I am not talking about X-axis which is 40ms) is not equivalent to "without reference point"???
So you have solution of integral and it is something + constant, and I say solution is something without constant and you come and say no solution is something with constant =0.
And you are not nitpicking, gee...
I don't mind you being a jerk from time to time, but you can also be quite a hypocrite when you criticize ppl for being arrogant, and then you behave in exactly same manner...

Milos
Posts: 2994
Joined: Wed Nov 25, 2009 12:47 am

Re: AlphaZero beats AlphaGo Zero, Stockfish, and Elmo

Post by Milos » Fri Dec 08, 2017 2:36 am

Adam Hair wrote:As far as your actual claim, the scaling could be due to the test conditions, specifically the lack of opening positions. I am planning on checking this out to see if I can replicate the scaling.
So you plan to play SF8 against itself always from root position and with fixed time per move? Well I suggest you play on a single core. Maybe you discover something new like that SF's scaling function is a step function, maybe you also rediscover the wheel in the process :lol: :lol: :lol:.

Post Reply