Chess World to Google Deep Mind..Prove You beat Stockfish 8!

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
Ovyron
Posts: 4556
Joined: Tue Jul 03, 2007 4:30 am

Re: Chess World to Google Deep Mind..Prove You beat Stockfis

Post by Ovyron »

zullil wrote:As far as I know, Stockfish has only been defeated by engines that use alpha-beta (and, of course, by Lyudmil and Pablo). :D
Pablo has defeated Stockfish by closing the position and beating it (or rather, a lagging GUI) on the clock.

About Lyudmil's victory, the jury is still out on that one...
pilgrimdan
Posts: 405
Joined: Sat Jul 02, 2011 10:49 pm

Re: Chess World to Google Deep Mind..Prove You beat Stockfis

Post by pilgrimdan »

zullil wrote:
pilgrimdan wrote:
corres wrote:It should have to perceive that the match circumstances during games between A0 and Stockfish is an unimportant thing. The essence is the team of DeepMind have proved they can make a powerful system based on neural networks which can solve such complicated tasks like winning go and chess games against very strong opponents.
and chess games against very strong <crippled> opponents...

that statement is more accurate...
Only if Stockfish was "crippled" to such a degree as to no longer be a very strong opponent. Was that the case?
i really don't care that Stockfish loss in such a maner...

what i care about is why do it with non-optimal conditions...

can somebody explain that...
pilgrimdan
Posts: 405
Joined: Sat Jul 02, 2011 10:49 pm

Re: Chess World to Google Deep Mind..Prove You beat Stockfis

Post by pilgrimdan »

syzygy wrote:
pilgrimdan wrote:
corres wrote:It should have to perceive that the match circumstances during games between A0 and Stockfish is an unimportant thing. The essence is the team of DeepMind have proved they can make a powerful system based on neural networks which can solve such complicated tasks like winning go and chess games against very strong opponents.
and chess games against very strong <crippled> opponents...

that statement is more accurate...
I wonder why you have trouble accepting that the Stockfish that lost the 100-game match is an amazingly strong engine.

I have not seen this attitude among any of the engine developers (it is possible that I missed one).
I do not have trouble accepting that Stockfish lost the 100 game match...

the 'trouble' I am having is why do it under non-optimal conditions...

can you explain that...
syzygy
Posts: 5566
Joined: Tue Feb 28, 2012 11:56 pm

Re: Chess World to Google Deep Mind..Prove You beat Stockfis

Post by syzygy »

pilgrimdan wrote:
syzygy wrote:
pilgrimdan wrote:
corres wrote:It should have to perceive that the match circumstances during games between A0 and Stockfish is an unimportant thing. The essence is the team of DeepMind have proved they can make a powerful system based on neural networks which can solve such complicated tasks like winning go and chess games against very strong opponents.
and chess games against very strong <crippled> opponents...

that statement is more accurate...
I wonder why you have trouble accepting that the Stockfish that lost the 100-game match is an amazingly strong engine.

I have not seen this attitude among any of the engine developers (it is possible that I missed one).
I do not have trouble accepting that Stockfish lost the 100 game match...

the 'trouble' I am having is why do it under non-optimal conditions...

can you explain that...
No, I cannot explain why that is such a big issue for so many people when it is really just small potatoes.

I liked this post by Ed, which explained his earlier "resistance" to the paper very well:
http://talkchess.com/forum/viewtopic.ph ... 634#745634
He is not troubled by minor gripes with the test conditions that only people who have spent many many hours with chess engines could possibly have (a category of people that likely lack representation in the Deep Mind workforce... to answer your real question). But he has trouble believing that it was possible to rival in 4 hours of computation time what mankind took 50 years to do (but it's actually closer to 70 years).
Dann Corbit
Posts: 12542
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Chess World to Google Deep Mind..Prove You beat Stockfis

Post by Dann Corbit »

pilgrimdan wrote:
zullil wrote:
pilgrimdan wrote:
corres wrote:It should have to perceive that the match circumstances during games between A0 and Stockfish is an unimportant thing. The essence is the team of DeepMind have proved they can make a powerful system based on neural networks which can solve such complicated tasks like winning go and chess games against very strong opponents.
and chess games against very strong <crippled> opponents...

that statement is more accurate...
Only if Stockfish was "crippled" to such a degree as to no longer be a very strong opponent. Was that the case?
i really don't care that Stockfish loss in such a maner...

what i care about is why do it with non-optimal conditions...

can somebody explain that...
There was nothing wrong with the conditions.
No book?
Neither engine had a book.
Core count?
The technologies are not commensurate, and giving SF 1000 cores would not add much Elo due to SMP scaling. Set the cores to anything you like and run the experiment. The count is clearly arbitrary.
RAM for hash?
They could have spent hours tuning SF, but the choice they made was a good one. At most it cost a couple Elo and would not have made any significant difference in the outcomes.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
User avatar
MikeB
Posts: 4889
Joined: Thu Mar 09, 2006 6:34 am
Location: Pen Argyl, Pennsylvania

Re: Chess World to Google Deep Mind..Prove You beat Stockfis

Post by MikeB »

pilgrimdan wrote:
corres wrote:It should have to perceive that the match circumstances during games between A0 and Stockfish is an unimportant thing. The essence is the team of DeepMind have proved they can make a powerful system based on neural networks which can solve such complicated tasks like winning go and chess games against very strong opponents.
and chess games against very strong <crippled> opponents...

that statement is more accurate...
people are focussed on the wrong metric - it's not the settings, it's not the Sf version used, it's not the time control, -- just look at the games and the moves played by A0 - some of the move played appeared to come from another planet.., consider what they also did with AGo and it's impressive and its believable, it was clearly unexpected and apparently for many people , not believable - something most have been drastically wrong to the point where SF 8 was crippled - why use the word crippled , because the results were over overwhelmingly in favor AO such that SF had to be crippled... it was not done the way many of us would have to like seen it be done , but it is what it is ..I also understand why people feel the way ...it's called emotion. SF is open source , everybody gets it for free, hundreds of people have volunteered either time and money ( time, computers and electricity are not free) and it was and still is the strongest free open source engine available, SF brings a lot of positives to the chess community and people appreciate that it so much such that it has become the favorite son of many chess enthusiasts and rightfully so and then big bad google comes with big bad money, hires a bunch of really big bad smart people, spends a few million on big bad hardware and then blows away the white knight SF , and people get upset ...

for me personally, I want to see some AO games played the way we would want them to see them play - but not because I think it would then be even and SF would have its revenge - but purely from standpoint that it would think it would be interesting and fun to watch...anyway I feel your pain too as the keeper of SF-McB - lets just hope there are some more games we can get to watch and analyze..
pilgrimdan
Posts: 405
Joined: Sat Jul 02, 2011 10:49 pm

Re: Chess World to Google Deep Mind..Prove You beat Stockfis

Post by pilgrimdan »

MikeB wrote:
pilgrimdan wrote:
corres wrote:It should have to perceive that the match circumstances during games between A0 and Stockfish is an unimportant thing. The essence is the team of DeepMind have proved they can make a powerful system based on neural networks which can solve such complicated tasks like winning go and chess games against very strong opponents.
and chess games against very strong <crippled> opponents...

that statement is more accurate...
people are focussed on the wrong metric - it's not the settings, it's not the Sf version used, it's not the time control, -- just look at the games and the moves played by A0 - some of the move played appeared to come from another planet.., consider what they also did with AGo and it's impressive and its believable, it was clearly unexpected and apparently for many people , not believable - something most have been drastically wrong to the point where SF 8 was crippled - why use the word crippled , because the results were over overwhelmingly in favor AO such that SF had to be crippled... it was not done the way many of us would have to like seen it be done , but it is what it is ..I also understand why people feel the way ...it's called emotion. SF is open source , everybody gets it for free, hundreds of people have volunteered either time and money ( time, computers and electricity are not free) and it was and still is the strongest free open source engine available, SF brings a lot of positives to the chess community and people appreciate that it so much such that it has become the favorite son of many chess enthusiasts and rightfully so and then big bad google comes with big bad money, hires a bunch of really big bad smart people, spends a few million on big bad hardware and then blows away the white knight SF , and people get upset ...

for me personally, I want to see some AO games played the way we would want them to see them play - but not because I think it would then be even and SF would have its revenge - but purely from standpoint that it would think it would be interesting and fun to watch...anyway I feel your pain too as the keeper of SF-McB - lets just hope there are some more games we can get to watch and analyze..
I basically agree with what you are saying...

' ... just look at the games and the moves played by A0 - some of the move played appeared to come from another planet.. '

what I <still> don't understand...

if play looked like it came from another planet...

then why couldn't the 'play' be done against a fully functional Stockfish...

if it still would have produced play from another planet ... then fine ...

that would have been much more satisfying...

what am I missing...
User avatar
M ANSARI
Posts: 3707
Joined: Thu Mar 16, 2006 7:10 pm

Re: Chess World to Google Deep Mind..Prove You beat Stockfis

Post by M ANSARI »

Milos wrote:
Evert wrote:Reviewing the drawn games would be useful, and presumably there is no obstacle to theAlphaZero team making these available on-line.
Ofc that there is an obstacle. A0 has zero variability in search or evaluation. The only variability there comes on 8 threads waiting in the queue of each TPU for evaluation. Since A0 uses UTC and not alpha-beta this has minuscule impact on variability.
All other randomness comes from SMP search of SF. Considering TC was fixed time per move, it is very probable that many games are actually similar if not even identical. That would be the only logical explanation why they didn't publish even all the winning games. Many wins basically repeat and there are many more almost identical boring draws.
If they published all the games ppl would see the scam.

Hmmm ... good point and a valid possibility. But let's wait and see. It is very strange why not all the 100 games were not released. I think we can gain more information regarding real strength with the drawn games. There doesn't seem to be a valid reason why the games have not all been released. I will say one thing though ... chess wise the moves played by A0 are really a huge achievement ... I really don't think any other chess engine today could have reproduced such moves no matter how many cores. But I would like to see all the games to get through all the hype.
User avatar
Leto
Posts: 2071
Joined: Thu May 04, 2006 3:40 am
Location: Dune

Re: Chess World to Google Deep Mind..Prove You beat Stockfis

Post by Leto »

M ANSARI wrote:
Milos wrote:
Evert wrote:Reviewing the drawn games would be useful, and presumably there is no obstacle to theAlphaZero team making these available on-line.
Ofc that there is an obstacle. A0 has zero variability in search or evaluation. The only variability there comes on 8 threads waiting in the queue of each TPU for evaluation. Since A0 uses UTC and not alpha-beta this has minuscule impact on variability.
All other randomness comes from SMP search of SF. Considering TC was fixed time per move, it is very probable that many games are actually similar if not even identical. That would be the only logical explanation why they didn't publish even all the winning games. Many wins basically repeat and there are many more almost identical boring draws.
If they published all the games ppl would see the scam.

Hmmm ... good point and a valid possibility. But let's wait and see. It is very strange why not all the 100 games were not released. I think we can gain more information regarding real strength with the drawn games. There doesn't seem to be a valid reason why the games have not all been released. I will say one thing though ... chess wise the moves played by A0 are really a huge achievement ... I really don't think any other chess engine today could have reproduced such moves no matter how many cores. But I would like to see all the games to get through all the hype.
I would think they'd release all the games when their paper gets published, that will be a while. What we've seen so far is just a preview of their paper.
Dann Corbit
Posts: 12542
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Chess World to Google Deep Mind..Prove You beat Stockfis

Post by Dann Corbit »

pilgrimdan wrote:
MikeB wrote:
pilgrimdan wrote:
corres wrote:It should have to perceive that the match circumstances during games between A0 and Stockfish is an unimportant thing. The essence is the team of DeepMind have proved they can make a powerful system based on neural networks which can solve such complicated tasks like winning go and chess games against very strong opponents.
and chess games against very strong <crippled> opponents...

that statement is more accurate...
people are focussed on the wrong metric - it's not the settings, it's not the Sf version used, it's not the time control, -- just look at the games and the moves played by A0 - some of the move played appeared to come from another planet.., consider what they also did with AGo and it's impressive and its believable, it was clearly unexpected and apparently for many people , not believable - something most have been drastically wrong to the point where SF 8 was crippled - why use the word crippled , because the results were over overwhelmingly in favor AO such that SF had to be crippled... it was not done the way many of us would have to like seen it be done , but it is what it is ..I also understand why people feel the way ...it's called emotion. SF is open source , everybody gets it for free, hundreds of people have volunteered either time and money ( time, computers and electricity are not free) and it was and still is the strongest free open source engine available, SF brings a lot of positives to the chess community and people appreciate that it so much such that it has become the favorite son of many chess enthusiasts and rightfully so and then big bad google comes with big bad money, hires a bunch of really big bad smart people, spends a few million on big bad hardware and then blows away the white knight SF , and people get upset ...

for me personally, I want to see some AO games played the way we would want them to see them play - but not because I think it would then be even and SF would have its revenge - but purely from standpoint that it would think it would be interesting and fun to watch...anyway I feel your pain too as the keeper of SF-McB - lets just hope there are some more games we can get to watch and analyze..
I basically agree with what you are saying...

' ... just look at the games and the moves played by A0 - some of the move played appeared to come from another planet.. '

what I <still> don't understand...

if play looked like it came from another planet...

then why couldn't the 'play' be done against a fully functional Stockfish...

if it still would have produced play from another planet ... then fine ...

that would have been much more satisfying...

what am I missing...
There was nothing wrong with the SF it played against.
That's what you are missing.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.