Chess World to Google Deep Mind..Prove You beat Stockfish 8!

[chessmobile]

Okay I've been following this story since it broke and am open minded on the actual reality of it all. My main gripe with DeepMind/Google is their delay in posting more details about the match (missing games). Secondly they sprang this announcement out of the blue no forewarning of what they were planning where as in the Go experiment they were more forthcoming with information and you could see the planning of matches against human in advance. Why change procedures when it came to chess? Would they have known that there would have been protests from the chess programming community if we knew that Stockfish was handicapped the way it was.
Too many unanswered questions causes doubt in minds of the public.
Until DeepMind give out more information then I remain sceptical.

[zullil]

My main gripe with DeepMind/Google is their delay in posting more details about the match (missing games). Secondly they sprang this announcement out of the blue no forewarning of what they were planning

I haven't followed this story closely. Was anything "announced" at all? I am aware that a group of DeepMind researchers posted a preprint on arXiv.

Preprints are just that, initial versions of manuscripts describing research, Some are more fully formed than others.

[Milos]

This.
As an alternative example, consider any data that comes out of CERN, or from satelites. Sure, anyone can repeat the experiments in principle, but not in practice because you lack the equipment to do so.

You get literary all the possible available raw data from CERN.
From A0 match you get 10 cherry-picked pgn's. You really think that is comparable???

[corres]

It should have to perceive that the match circumstances during games between A0 and Stockfish is an unimportant thing. The essence is the team of DeepMind have proved they can make a powerful system based on neural networks which can solve such complicated tasks like winning go and chess games against very strong opponents.

[Rodolfo Leoni]

It should have to perceive that the match circumstances during games between A0 and Stockfish is an unimportant thing. The essence is the team of DeepMind have proved they can make a powerful system based on neural networks which can solve such complicated tasks like winning go and chess games against very strong opponents.

+1

This is the real substance of the "event". All the discussion has been about subjective opinions and perceptions about the advertising factor, but the really important thing is that there's a new frontier to explore in a moment when engines were trying to fight for half ELO point per patch.

[Evert]

Okay I've been following this story since it broke and am open minded on the actual reality of it all. My main gripe with DeepMind/Google is their delay in posting more details about the match (missing games).

The time to make that data available is when the paper is published and it bevomes clear where and how such data is to be made available. Even then, these things are not always done by default, unless the referee asks for it to be done.
Even without that, it wouldn't be odd for them to share data with other researchers on request. Has anyone asked them, or are people just complaining?

Secondly they sprang this announcement out of the blue no forewarning of what they were planning where as in the Go experiment they were more forthcoming with information and you could see the planning of matches against human in advance. Why change procedures when it came to chess?

It's just a next step in their line of research. I for one don't find it so surprising. Perhaps the rate at which they achieved results, but let's be honest: starting from AlphaGoZero it's not a big step.

Would they have known that there would have been protests from the chess programming community if we knew that Stockfish was handicapped the way it was.

They don't give a rat's ass about what "the chess programming community" thinks, they're not trying to build the best and greatest chess playing program. They're trying to build a general AI that can learn to solve complex puzzles (in this case, play games) by re-inforced learning.

Even that is not such a leap to grasp. It's fairly obvious that a general evaluation function that correlates every bit of information with every other bit is going to be superior to the limited and crude evaluation functions we all use. The difficulty is in getting all the correlation coefficients (weights in the neural net) nailed down and evaluate the net itself in a reasonable time. That was not feasible before, what they show is that it is feasible now and it works. Whether it beats Stockfish or not is, frankly, irrelevant.

[Gary Internet]

I'd be really interested is seeing all the drawn games and understanding how they came to be drawn. Clearly AlphaZero isn't amazing 100% of the time.

The way people talk about this match, you'd think AlphaZero won all 100 games. Clearly Stockfish wasn't that weak because it managed to hold for 72 draws.

[supersharp77]

I'd be really interested is seeing all the drawn games and understanding how they came to be drawn. Clearly AlphaZero isn't amazing 100% of the time.

The way people talk about this match, you'd think AlphaZero won all 100 games. Clearly Stockfish wasn't that weak because it managed to hold for 72 draws.

.....Yes....Yes....Yes!! Bingo!!!!! AR

[Uri Blass]

<snipped>

In the case of AlphaZero, the 100-game match between the fully trained AlphaZero and SF8 can be described in just a few words. SF8 was running on 64 cores at about 70 Mnps. AlphaZero using 4 TPUs beat SF8 at 1 minute per move 28-0-72. What more is there to say?

There is a lot more to say
For example a pgn of all the games including not only the 100 games that AlphaZero played against SF8 but all the games that alphazero played earlier against istelf.

That would all be very interesting, but I don't see how that is necessary to make the preprint "scientific". (And a preprint with 44 million games would cost a lot of trees to print out. We didn't get all that information for AlphaGo, but I suppose all the AlphaGo papers are equally unscientific?)

Preprint with 44 million games is impossible but a pgn of 44 million games to download is clearly possible.

I dislike the fact that they show only 10 games that they won out of 100 games.

It seem biased information to give a wrong impression and maybe if we see that stockfish missed a win in part of the draws or if we see that after a lot of draws they simply repeat using specific openings that gave them good chance to win the result is going to be less impressive.

[zullil]

I dislike the fact that they show only 10 games that they won out of 100 games.

To me, the fact that the AlphaZero algorithm could, tabula rasa, defeat Stockfish even once is an amazing occurrence.

As far as I know, Stockfish has only been defeated by engines that use alpha-beta (and, of course, by Lyudmil and Pablo).