Adam Hair wrote:Perhaps because they felt it was a sufficient sample? It could be that simple.
Sorry, but to me that sounds like extremely naive explanation.
Do you believe they faked AlphaGo? If not, then why so readily believe that the AlphaZero experiment is a sham? After all, it makes more sense to not publicize the experiment if it did not turn out well. Why risk the embarassment?
Not AlphaGo, but for A0 I don't believe they didn't use any prior knowledge beside rules of the game. I believe they also trained it from opening positions that SF8 would reach in the early opening in self-play, i.e. using for example some sample of positions from Cerebellum limited to 8-10 moves as starting positions for A0 self-play.
I also believe they cherry-picked those 100 games. I.e. they ran for example 1000 batches of 100 games and picked up the best one.
What embarrassment are you talking about? How exactly do you think they could be embarrassed when match is virtually non-repeatable for anyone but multi-billion dollar company???
I think we are missing the only relevant point. It's not if they cheated or not, if it was fair of not, if it was scientific or not. It's not matter or believing or not. The "event" gave us an idea about an interesting and innovative approach with a revolutionary hardware. This is impacting a lot on computer chess community. I see (but I could be wrong):
- Some programmers very interested to innovative projects (Gary Linscott is an example);
- The danger of having many demotivated developers about conventional approach.
Those are only my impressions, and I'd like to know programmers ideas about it.
Looks like AlphaZero bashing time !
People who spent 20-30 Years of their Lives living and breathing chess engines based on the old Alpha Beta Search can hardly be expected to go quietly into the Sunset.
I quite understand, AlphaZero literally rocked their World and they are shocked and furious.
My Profound Sympathies.
Adam Hair wrote:Perhaps because they felt it was a sufficient sample? It could be that simple.
Sorry, but to me that sounds like extremely naive explanation.
Do you believe they faked AlphaGo? If not, then why so readily believe that the AlphaZero experiment is a sham? After all, it makes more sense to not publicize the experiment if it did not turn out well. Why risk the embarassment?
Not AlphaGo, but for A0 I don't believe they didn't use any prior knowledge beside rules of the game. I believe they also trained it from opening positions that SF8 would reach in the early opening in self-play, i.e. using for example some sample of positions from Cerebellum limited to 8-10 moves as starting positions for A0 self-play.
I also believe they cherry-picked those 100 games. I.e. they ran for example 1000 batches of 100 games and picked up the best one.
The risk/reward seems very high to me.
What embarrassment are you talking about? How exactly do you think they could be embarrassed when match is virtually non-repeatable for anyone but multi-billion dollar company???
They probably have a goal with this experiment (which is obviously performed for publicity like the Deep Blue trials), such as selling the TPUs at some point.
If the TPUs never become available at any point in time, then I suppose that they could fake it, but then what would they gain?
Faking the results is clearly a mistake in the long run.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
shrapnel wrote:Looks like AlphaZero bashing time !
People who spent 20-30 Years of their Lives living and breathing chess engines based on the old Alpha Beta Search can hardly be expected to go quietly into the Sunset.
I quite understand, AlphaZero literally rocked their World and they are shocked and furious.
My Profound Sympathies.
Not really. Deep Blue produced (at the time) astonishing chess results that nobody (at the time) could replicate.
But the cost of the Deep Blue hardware means that it was not available to ordinary people.
Similarly, I guess that almost nobody will be able to buy a big collection of TPUs and then dedicate them simply to chess. The TPU experiment is actually quite fascinating because the implementation to other problem spaces seems clearly quite feasible.
In the case of Deep Blue, the chess processors could only play chess and the Risc machine that held them together was nothing special without them.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Dann Corbit wrote:
The risk/reward seems very high to me.
What embarrassment are you talking about? How exactly do you think they could be embarrassed when match is virtually non-repeatable for anyone but multi-billion dollar company???
They probably have a goal with this experiment (which is obviously performed for publicity like the Deep Blue trials), such as selling the TPUs at some point.
If the TPUs never become available at any point in time, then I suppose that they could fake it, but then what would they gain?
Faking the results is clearly a mistake in the long run.
Again what risk, they didn't fake all, they probably just faked it enough so that result is good for advertising? In fair conditions a genuine no-domain-knowledge A0 could be 200 Elo weaker than SF, but then no one would pay attention and it would be a total advertising failure.
So again, how do think anyone could prove that A0 is not 100Elo stronger but 200Elo weaker than SF???
Dann Corbit wrote:
The risk/reward seems very high to me.
What embarrassment are you talking about? How exactly do you think they could be embarrassed when match is virtually non-repeatable for anyone but multi-billion dollar company???
They probably have a goal with this experiment (which is obviously performed for publicity like the Deep Blue trials), such as selling the TPUs at some point.
If the TPUs never become available at any point in time, then I suppose that they could fake it, but then what would they gain?
Faking the results is clearly a mistake in the long run.
Again what risk, they didn't fake all, they probably just faked it enough so that result is good for advertising? In fair conditions a genuine no-domain-knowledge A0 could be 200 Elo weaker than SF, but then no one would pay attention and it would be a total advertising failure.
So again, how do think anyone could prove that A0 is not 100Elo stronger but 200Elo weaker than SF???
Let's examine the goals of the experiment.
They either want to sell the TPUs or the services that run on TPUs.
If the TPUs don't measure up, then the experiment is worse than a flop.
Hype may get people to try, but not to buy.
I see no benefit in faking the results.
You might get people interested. But if it does not measure up when they try it, then the interest is simply pointless and the whole exercise will have been for nothing.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Dann Corbit wrote:Let's examine the goals of the experiment.
They either want to sell the TPUs or the services that run on TPUs.
If the TPUs don't measure up, then the experiment is worse than a flop.
Hype may get people to try, but not to buy.
I see no benefit in faking the results.
You might get people interested. But if it does not measure up when they try it, then the interest is simply pointless and the whole exercise will have been for nothing.
How they are gonna try it, and what they are gonna try?
Google is not trying to sell chess AI service but AI expertise as a service to the businesses and partially hardware (TPUs) as a service (renting).
The possibility for someone to rent TPUs doesn't mean that someone is gonna pay for 50k TPU hours to train NN to play chess in order to repeat a match against SF.
If they can look ahead 50-60 moves in GO it will be a lot more in chess.
Two questions remain -
#1. They speak about moves. Is GO terminology the same as chess terminology regarding: move = 2 plies?
#2. AlphaGo contrary to AlphaGo Zero used rollouts, perhaps they meant the average rollout depth? That would be odd. We (in chess) mention the iteration depth, not the QS depth, a rollout basically is some sort of QS.
matejst wrote:Me too, Brendan. I am a scientific worker myself, and there's nothing scientific in the way the experiment was conducted. On one side, you have a known entity that everybody can check, on the other, an unknown quantity, a pseudo-scientific paper, an uncalibrated testing behind closed doors, no possibility to repeat the experiment...
A lot of science cannot be easily repeated. If someone reports about research that took 20 years and a couple of thousand volunteers to complete, there is no practical way to "repeat the experiment". What you can do is check the description of the experiment to see if the methods used were sound.
This.
As an alternative example, consider any data that comes out of CERN, or from satelites. Sure, anyone can repeat the experiments in principle, but not in practice because you lack the equipment to do so.
