EroSennin wrote:You should also know the same human won 2 out of 5 unofficial blitz games.
do you have a link to this?
It's in the paper.
By the way they played in October, now of course Alphago has been improved.
I'd bet on the bot.
To me the most relevant are the pro human comments on the games, it seems regular 8-9p professionals are more respectful towards AlphaGo, while super 9p say it's still weak. These dan ratings are a bit confusing, but tentative Elo-like rating data seem to show that the difference between super 9p and regular 9p is as much as between regular 9p and 2p (Fan Hui). From the comments and my extrapolation, AlphaGo from October on the cluster seems to be at the level of regular 9p, but not super 9p like Lee Sedol, who can still be significantly stronger (although both are "9p" level).
I think we can all agree that it was only a matter of time before something like that happened. I wonder is something can be gleaned to help chess engines cover some weaknesses they have such as understanding fortresses or better understanding "horizon effect". I could see this as a possible daughter card for top chess engines to use in the endgame.
A good example would be the Caruana vs. Mamadeyarov game where both GM's immediately realized the game was drawn due to fortress yet the engines gave a more than 2.00 advantage.
Laskos wrote:Fan Hui is 2p professional. 5-0 suggests at least 4p for AlphaGo. Also, it beats normal MC programs like Crazy Stone (say 5 dan on i7) by 99.8%, which suggests 8-10 more stones, also above 4p. This thing runs on 170 GPU cards and 1,200 standard processors, and assuming quadrupling for a stone, it has 5 stones advantage over normal hardware on KGS (say a cluster of 24 regular cores). Lee Sedol is above regular 9p. Let's see.
I would bet on Alpha Go. Lee Sedol is toast.
Hard to say, my extrapolation seems to give some good 9p for cluster bot, but Lee Sedol is not a regular 9p. In their paper, using normal Elo points, they give 3200 to the cluster bot, 3500 to Sedol, and Sedol would be clear favorite. But I don't like their Elo system used in the context of Go, they assume that the probability of 7d beating 6d is the same as 8k beating 9k, which is wrong by a lot.
If these commenting Chinese super-champions aren't simply showing off for humanity, I think their opinion that the bot is still weak compared to super-top is worth considering.
Seedol has reached his peak, and probably humanity's peak. On the other hand, AlphaGo is a moving target. It continues to improve with reinforced learning. Consider the huge jumps of elo (or dan or whatever you want to use) it has achieved so rapidly. With lots of computing power, and 2 months to go, it will catch-up and overtake Sedol very soon (if it hasn't already).
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.
Laskos wrote:To me the most relevant are the pro human comments on the games, it seems regular 8-9p professionals are more respectful towards AlphaGo, while super 9p say it's still weak. These dan ratings are a bit confusing, but tentative Elo-like rating data seem to show that the difference between super 9p and regular 9p is as much as between regular 9p and 2p (Fan Hui). From the comments and my extrapolation, AlphaGo from October on the cluster seems to be at the level of regular 9p, but not super 9p like Lee Sedol, who can still be significantly stronger (although both are "9p" level).
/
You just can't compare pro titles easily to one another. With Japanese pros the difference between 1p and 9p is greater than with Koreans. To become a pro in Korea is extremely hard. When they finally do get it they are already very strong players. Also one can not drop their title. It can only go up. Ke Jie might now be very slightly stronger than Lee Sedol. I have to say I was very surprised by these news as most go players were. I remembered Zen became 6d on kgs just recently. But actually it happened 4 years ago already. Now a few days after the news of the alphago it got to 7d. So basically the surprise has a lot to do with the slowness of software improvement. Alphago just skipped a few levels.
Laskos wrote:To me the most relevant are the pro human comments on the games, it seems regular 8-9p professionals are more respectful towards AlphaGo, while super 9p say it's still weak. These dan ratings are a bit confusing, but tentative Elo-like rating data seem to show that the difference between super 9p and regular 9p is as much as between regular 9p and 2p (Fan Hui). From the comments and my extrapolation, AlphaGo from October on the cluster seems to be at the level of regular 9p, but not super 9p like Lee Sedol, who can still be significantly stronger (although both are "9p" level).
/
You just can't compare pro titles easily to one another. With Japanese pros the difference between 1p and 9p is greater than with Koreans. To become a pro in Korea is extremely hard. When they finally do get it they are already very strong players. Also one can not drop their title. It can only go up. Ke Jie might now be very slightly stronger than Lee Sedol. I have to say I was very surprised by these news as most go players were. I remembered Zen became 6d on kgs just recently. But actually it happened 4 years ago already. Now a few days after the news of the alphago it got to 7d. So basically the surprise has a lot to do with the slowness of software improvement. Alphago just skipped a few levels.
Yes, professional titles are hard to deal with quantitatively, although in the AlphaGo paper they try to set some Elo values to dan titles. I just wanted to show the difference between players with the same 9p rating. To have some quantitative assessments, I took the calculated Elo rating (WHR) based on database performance available for professional players. No dan arbitrarienss here. The histogram of the distribution of professionals is here:
Based on the extrapolation of AlhpaGo performance and the estimation in authors' paper, I pictured where cluster AlphaGo roughly stands by October 2015. It is already above average Go professional, but still significantly short of Lee Sedol. Crazy Stone and Zen have no place in this plot, they are too much to the left with an Elo of 1900. It is obvious that the leap is enormous. Also, although the diminishing returns of AlphaGo are apparent, it is expected it will improve in one year by at least some 200-300 Elo points, similarly to what most MCTS newly emerged engines did some 10 years ago, they seem to scale similarly. So, I would bet on AlphaGo beating Lee Sedol in 2017, but I am not sure it will happen in March 2016.
Laskos wrote: Crazy Stone and Zen have no place in this plot, they are too much to the left with an Elo of 1900.
Not sure how go elos work but 1900 seems too weak.
Absolute Elo values don't matter, only their differences. Add 500 to every rating, if it feels better. That there is 1000 Elo points difference between Crazy Stone and AlphaGo single machine is confirmed by both 99.8% result and 4 stones handicap result. Then there is 250-300 Elo points gap between cluster and single machine AlphaGo. All in all, about 1300 Elo points, or from 1900 to 3200 both here and in the paper. It seems they got pretty well the Elo values, only assigning them to dan was done carelessly.
Laskos wrote: Crazy Stone and Zen have no place in this plot, they are too much to the left with an Elo of 1900.
Not sure how go elos work but 1900 seems too weak.
Absolute Elo values don't matter, only their differences. Add 500 to every rating, if it feels better. That there is 1000 Elo points difference between Crazy Stone and AlphaGo single machine is confirmed by both 99.8% result and 4 stones handicap result. Then there is 250-300 Elo points gap between cluster and single machine AlphaGo. All in all, about 1300 Elo points, or from 1900 to 3200 both here and in the paper. It seems they got pretty well the Elo values, only assigning them to dan was done carelessly.
But I feel like we get to minus elos real fast if we compare that way all the way to the beginner.
Laskos wrote: Crazy Stone and Zen have no place in this plot, they are too much to the left with an Elo of 1900.
Not sure how go elos work but 1900 seems too weak.
Absolute Elo values don't matter, only their differences. Add 500 to every rating, if it feels better. That there is 1000 Elo points difference between Crazy Stone and AlphaGo single machine is confirmed by both 99.8% result and 4 stones handicap result. Then there is 250-300 Elo points gap between cluster and single machine AlphaGo. All in all, about 1300 Elo points, or from 1900 to 3200 both here and in the paper. It seems they got pretty well the Elo values, only assigning them to dan was done carelessly.
But I feel like we get to minus elos real fast if we compare that way all the way to the beginner.
Yes, we will get, no problem with minus Elo. It simply means that Elo differences in Go are much larger than in Chess for similarly skilled in respective games people. Go is richer and deeper than Chess. The Elo span in Chess from moderate beginner to Carlsen is roughly from say 400 Elo points to 2850 Elo points, or 2450 Elo points. The Elo span in Go is maybe from -1000 Elo points to 3600 Elo points, or 4600 Elo points, almost 2 times the skill span of Chess.
Laskos wrote:
Based on the extrapolation of AlhpaGo performance and the estimation in authors' paper, I pictured where cluster AlphaGo roughly stands by October 2015. It is already above average Go professional, but still significantly short of Lee Sedol. Crazy Stone and Zen have no place in this plot, they are too much to the left with an Elo of 1900. It is obvious that the leap is enormous. Also, although the diminishing returns of AlphaGo are apparent, it is expected it will improve in one year by at least some 200-300 Elo points, similarly to what most MCTS newly emerged engines did some 10 years ago, they seem to scale similarly. So, I would bet on AlphaGo beating Lee Sedol in 2017, but I am not sure it will happen in March 2016.
The October '15 version had 40 search threads, 1202 CPUs and 176 GPUs at its disposal. For a company with Google's resources, this was merely a test run. With this much prestige on the line, expect one or even two orders of magnitude more computing power being thrown at the Lee Sedol match.
I don't know how MCTS scales, but those last couple of hundreds of ELO points should be well within reach. They must have done the math and concluded that they have a very good shot. Otherwise, even with Facebook with a competing project, why else would Google even consider doing the match so soon?
