Historic Milestone: AlphaZero

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
MikeGL
Posts: 694
Joined: Thu Sep 01, 2011 12:49 pm

Re: Much weaker than Stockfish

Post by MikeGL » Wed Dec 06, 2017 3:16 pm

Lyudmil Tsvetkov wrote:
Alpha chooses only 1.d4 and 1.Nf3, while Stockfish goes for 1.e4
Judging from this, I can say that Alpha is much weaker than SF in terms of software, and the only reason for the win is the very big hardware advantage.
I think Table 2 [ECO opennings] in the PDF would answer your argument.
All those 12 common opennings (on that Table 2) was played by AlphaZero against SF8, 100 times each. and only a total of 4 losses (out of 300 games) as white starting with 1.e4 (for AlphaZero) as shown on that table.

Milos
Posts: 2994
Joined: Wed Nov 25, 2009 12:47 am

Re: Much weaker than Stockfish

Post by Milos » Wed Dec 06, 2017 3:24 pm

MikeGL wrote:
Lyudmil Tsvetkov wrote:
Alpha chooses only 1.d4 and 1.Nf3, while Stockfish goes for 1.e4
Judging from this, I can say that Alpha is much weaker than SF in terms of software, and the only reason for the win is the very big hardware advantage.
I think Table 2 [ECO opennings] in the PDF would answer your argument.
All those 12 common opennings, on that Table 2, was played by AlphaZero against SF8, 100 times each. and only a total of 4 losses (out of 60 games) as white starting with 1.e4 (for AlphaZero) as shown on that table.
3 in Sicilian and one in Reti, that is pretty indicative. Also by far the worst percentage of Alpha0 vs SF in those openings.

Lyudmil Tsvetkov
Posts: 6033
Joined: Tue Jun 12, 2012 10:41 am

Re: Historic Milestone: AlphaZero

Post by Lyudmil Tsvetkov » Wed Dec 06, 2017 3:28 pm

I just read somewhere that Tesla K80 GPU has 12 cores.
Also that the TPU is 20 times faster than K80, according to Google.

Stockfish used 64 cores, that makes 12x5 K80s.
Alpha used 4 TPUs, so actually, it results that Alpha had 20x4/5=16 times bigger hardware.

Is that true?

In case this is so, of what comparisons and matches we are talking?
By these figures, current Stockfish development is at least 300 elos stronger than Alpha.

Is not it a bit shameless to assert you have achieved something, when in actual fact it is all hardware.

Stockfish on tremendous hardware and suitable SMP algorithms will certainly thrash SF 64 cores 90/10 or so, so why all this hype?

User avatar
MikeGL
Posts: 694
Joined: Thu Sep 01, 2011 12:49 pm

Re: Much weaker than Stockfish

Post by MikeGL » Wed Dec 06, 2017 3:29 pm

Milos wrote:
MikeGL wrote:
Lyudmil Tsvetkov wrote:
Alpha chooses only 1.d4 and 1.Nf3, while Stockfish goes for 1.e4
Judging from this, I can say that Alpha is much weaker than SF in terms of software, and the only reason for the win is the very big hardware advantage.
I think Table 2 [ECO opennings] in the PDF would answer your argument.
All those 12 common opennings, on that Table 2, was played by AlphaZero against SF8, 100 times each. and only a total of 4 losses (out of 60 games) as white starting with 1.e4 (for AlphaZero) as shown on that table.
3 in Sicilian and one in Reti, that is pretty indicative. Also by far the worst percentage of Alpha0 vs SF in those openings.
edit: Not 60 games. 4 losses out of 300 games

50 games as white (and 50 as black) x 6 types of 1.e4 on that Table, if I understood the table correctly.

jdart
Posts: 3510
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Historic Milestone: AlphaZero

Post by jdart » Wed Dec 06, 2017 3:34 pm

Even Stockfish on a Raspberry Pi is a strong chess player, though. So AlphaZero has decent performance, we just need a better comparison. I don't think there is any fundamental reason a NN based system such as AlphaZero couldn't run on commodity hardware, so maybe that can happen.

--Jon

Lyudmil Tsvetkov
Posts: 6033
Joined: Tue Jun 12, 2012 10:41 am

Re: Much weaker than Stockfish

Post by Lyudmil Tsvetkov » Wed Dec 06, 2017 3:34 pm

MikeGL wrote:
Lyudmil Tsvetkov wrote:
Alpha chooses only 1.d4 and 1.Nf3, while Stockfish goes for 1.e4
Judging from this, I can say that Alpha is much weaker than SF in terms of software, and the only reason for the win is the very big hardware advantage.
I think Table 2 [ECO opennings] in the PDF would answer your argument.
All those 12 common opennings (on that Table 2) was played by AlphaZero against SF8, 100 times each. and only a total of 4 losses (out of 300 games) as white starting with 1.e4 (for AlphaZero) as shown on that table.
For some reason, I have no access to that table and the page as a whole.
Very weird.
It seems the page recognises its detractors. :)

User avatar
MikeGL
Posts: 694
Joined: Thu Sep 01, 2011 12:49 pm

Re: Much weaker than Stockfish

Post by MikeGL » Wed Dec 06, 2017 3:47 pm

Lyudmil Tsvetkov wrote:
MikeGL wrote:
Lyudmil Tsvetkov wrote:
Alpha chooses only 1.d4 and 1.Nf3, while Stockfish goes for 1.e4
Judging from this, I can say that Alpha is much weaker than SF in terms of software, and the only reason for the win is the very big hardware advantage.
I think Table 2 [ECO opennings] in the PDF would answer your argument.
All those 12 common opennings (on that Table 2) was played by AlphaZero against SF8, 100 times each. and only a total of 4 losses (out of 300 games) as white starting with 1.e4 (for AlphaZero) as shown on that table.
For some reason, I have no access to that table and the page as a whole.
Very weird.
It seems the page recognises its detractors. :)
Image

The uppermost right diagram is a French Defence [C00], won't fit on my screen.

*
Just download the PDF (right-click then choose download) then upload again at gmail then view it from there.
Otherwise, just download and view it on your PC using any PDF reader.
Last edited by MikeGL on Wed Dec 06, 2017 3:49 pm, edited 1 time in total.

Lyudmil Tsvetkov
Posts: 6033
Joined: Tue Jun 12, 2012 10:41 am

Re: Much weaker than Stockfish

Post by Lyudmil Tsvetkov » Wed Dec 06, 2017 3:48 pm

Finally I opened it, and the only thing I understood was is number 25 of their References list: Stockfish page/code.

Why don't they disclose what their evaluation is: that will be a big step towards knowing the truth.

Lyudmil Tsvetkov
Posts: 6033
Joined: Tue Jun 12, 2012 10:41 am

Re: Much weaker than Stockfish

Post by Lyudmil Tsvetkov » Wed Dec 06, 2017 3:54 pm

MikeGL wrote:
Milos wrote:
MikeGL wrote:
Lyudmil Tsvetkov wrote:
Alpha chooses only 1.d4 and 1.Nf3, while Stockfish goes for 1.e4
Judging from this, I can say that Alpha is much weaker than SF in terms of software, and the only reason for the win is the very big hardware advantage.
I think Table 2 [ECO opennings] in the PDF would answer your argument.
All those 12 common opennings, on that Table 2, was played by AlphaZero against SF8, 100 times each. and only a total of 4 losses (out of 60 games) as white starting with 1.e4 (for AlphaZero) as shown on that table.
3 in Sicilian and one in Reti, that is pretty indicative. Also by far the worst percentage of Alpha0 vs SF in those openings.
edit: Not 60 games. 4 losses out of 300 games

50 games as white (and 50 as black) x 6 types of 1.e4 on that Table, if I understood the table correctly.
Was not it just 100 games?
How can 100 games become 300 games?
Was a book used or not after all?
Too many unclear things.

It seems that hardware advantage was 16/1.

I am skeptical, because you can not just go tuning like that.
It simply does not work. You can improve Stockfish play by 200-300 elos or so with perfect tuning, maybe even 500, but not more.
What is necessary is to get rid of redundancies and introduce very specific evaluation patterns, otherwise it simply does not work.

So, basically, currently Alpha is around 3000 elos or so, maybe even lower.
That is weaker even than Fritz 12.

Henk
Posts: 5105
Joined: Mon May 27, 2013 8:31 am

Re: Historic Milestone: AlphaZero

Post by Henk » Wed Dec 06, 2017 3:57 pm

--
Last edited by Henk on Wed Dec 06, 2017 4:03 pm, edited 1 time in total.

Post Reply