| View previous topic :: View next topic |
| Author |
Message |
Don Dailey
Joined: 29 Apr 2008
Posts: 4320
|
 Posted: Mon Apr 02, 2012 2:33 pm Post subject: Re: Luke skywalker has done it again. |
  |
|
| gerold wrote: |
| Daniel Shawul wrote: |
http://chessbase.com/newsdetail.asp?newsid=8047
King's gambit "weakly solved" by Vas. Admit it. This guy definately knows what he does  |
Solved is the wrong word. Maybe the best move the computer could come up with is more like it. |
This whole thing comes down to the validity of the assumption that if Rybka scores 5.12 or more, it is a win with 99.99999999% certainty and that it follows that this happens to be the same certainty for the entire results.
I really have a difficult time with both those assumptions and I want to know how he came up with that value. This is about 1 in 10 billion positions! That means if you sacrifice a queen, or a rook plus a pawn or two that you automatically lose (except once every 10,000,000,000 times.)
I have to say that I think this is utter nonsense, my years of experience in computer chess and other games tells me that no matter what the "score" reported by the program (other than Mate) there are holes in the knowledge and search that can make this go wrong.
_________________
"Your superior intellect is no match for our puny weapons." -Kang and Kodos |
|
| Back to top |
|
 |
Miguel A. Ballicora
Joined: 09 Mar 2006
Posts: 4436
Location: Chicago, Illinois, USA
|
| Posted: Mon Apr 02, 2012 2:45 pm Post subject: Re: Luke skywalker has done it again. |
|
|
| Don wrote: |
| gerold wrote: |
| Daniel Shawul wrote: |
http://chessbase.com/newsdetail.asp?newsid=8047
King's gambit "weakly solved" by Vas. Admit it. This guy definately knows what he does |
Solved is the wrong word. Maybe the best move the computer could come up with is more like it. |
This whole thing comes down to the validity of the assumption that if Rybka scores 5.12 or more, it is a win with 99.99999999% certainty and that it follows that this happens to be the same certainty for the entire results.
I really have a difficult time with both those assumptions and I want to know how he came up with that value. This is about 1 in 10 billion positions! That means if you sacrifice a queen, or a rook plus a pawn or two that you automatically lose (except once every 10,000,000,000 times.)
I have to say that I think this is utter nonsense, my years of experience in computer chess and other games tells me that no matter what the "score" reported by the program (other than Mate) there are holes in the knowledge and search that can make this go wrong. |
The number seems to be a poetic license picked in a conversation with a journalist, rather than a number you write in a paper. Like a teen ager saying I am super-hyper-duper-sure, so I am 99.9999.... etc.
The bottom line is, there is a 100% probabilities to reach a position +5.12 in rybka scale for black after 1.e4 e5 2. f4 exf4 3. Nf5 d6.
Or, this system returns
(I hope some people pick my joke)
Miguel
_________________
http://sites.google.com/site/gaviotachessengine/ |
|
| Back to top |
|
|
Julien MARCEL
Joined: 05 May 2008
Posts: 2269
Location: Nantes (France)
|
| Posted: Mon Apr 02, 2012 2:59 pm Post subject: Re: Luke skywalker has done it again. |
|
|
My sixth sense keeps asking me: "How could Vas get access to such a costly cluster? Especially during four month in a row full time?"
The more I think about it the more it looks like an hoax...
_________________
 Author of Prédateur chess engine: http://predateur-chess.blogspot.fr |
|
| Back to top |
|
|
Don Dailey
Joined: 29 Apr 2008
Posts: 4320
|
| Posted: Mon Apr 02, 2012 2:59 pm Post subject: Re: Luke skywalker has done it again. |
|
|
| Daniel Shawul wrote: |
| Ofcourse vas haters will call BS without bothering to read what is claimed |
This is a advertising piece by Chessbase but the study itself is interesting to me. However scientifically it is just BAD and what Vas claims ... nothing to back it up except his belief. I'm not claiming it is (as you say) BS but the other extreme is what Chessbase intended, that it will be hailed as a valid result without any sort of critical thinking applied - as you seem to be doing. You are the type of gullible audience that Chessbase is addressing and sadly most people will just accept this at face value.
I think it's possible that the result could turn out to be correct, but the methodology cannot lead us to this conclusion - it's a probabilistic conclusion without any scientific groundwork or analysis to help us interpret or understand it, just the assurance from Vas that it's "in fact very, very likely, that there is no flaw." I guess this statement is based on credulity?
There needs to be a paper written on this by Vas when can be opened up to scientific scrutiny - explaining how he came to the conclusion that it's very unlikely to be flawed and how he calculated the 1 out of a 10 billion number for 5.12 score and other details.
I have to say that I like Chessbase, their products and the people that work for them, I know some of them personally. But this sort of thing I find embarrassing.
_________________
"Your superior intellect is no match for our puny weapons." -Kang and Kodos |
|
| Back to top |
|
|
Daniel Shawul
Joined: 14 Mar 2006
Posts: 2187
Location: Ethiopia
|
| Posted: Mon Apr 02, 2012 3:00 pm Post subject: Re: Luke skywalker has done it again. |
|
|
I think Skywalker will explain better if and when it is necessare but from what I understood they did some kind of montecarlo to come up with the percentage. Ofcouse they also made it very very clear there could be flaws.
| Quote: |
But Alpha-Beta reduces the search to about the square root of the total number of moves. The square root of 10^100, however…
Yes, I know. But think about it: you do not need to search every variation to mate. We only need to search a tiny fraction of the overall space. Whenever Rybka evaluates a position with a score of +/– 5.12 we don't need to search any further, we have our proof that in the continuation there is going to be a win or loss, and there is a forced mate somewhere deep down in the tree. We tested a random sampling of positions of varying levels of difficulty that were evaluated at above 5.12, and we never saw a solution fail. So it is safe to use this assumption generally in the search.
|
| Quote: |
So this means that the result is not 100% certain, it is just a hypothesis.
That is technically correct, similar to the assertion that a position where one side is more than two pieces down, without any compensation, is considered lost, even if you cannot calculate it to a forced mate against any defence. Sure, there theoretically might be a way to save the game, but if Rybka is displaying +5.12 or more the outcome is 99.99999999% secure. That is approximately the confidence number we give to our King's Gambit results: 99.99999999%. It might be that there is a flaw somewhere, but if there is it will not be discovered in the course of this universe – that would require more computational power than could ever be provided. And of course it is possible, and in fact very, very likely, that there is no flaw.
|
IMO it is a very solid assumption. Better to make baby steps rather than complain "oh chess has gazillion positions so can never be solved...". I wonder if this methodology has been tried before ...
Anyhow a bright way to start the day 
_________________
https://sites.google.com/site/dshawul/
https://github.com/dshawul |
|
| Back to top |
|
|
Ricardo Barreira
Joined: 27 Apr 2010
Posts: 821
|
| Posted: Mon Apr 02, 2012 3:08 pm Post subject: Re: Luke skywalker has done it again. |
|
|
| Another thing that needs to be spelled out is what depth they required Rybka to search with a >= 5.12 score before they assumed the position as solved. |
|
| Back to top |
|
|
Don Dailey
Joined: 29 Apr 2008
Posts: 4320
|
| Posted: Mon Apr 02, 2012 3:09 pm Post subject: Re: Luke skywalker has done it again. |
|
|
| Daniel Shawul wrote: |
I think Skywalker will explain better if and when it is necessare
|
There is a big problem here. As you say they don't have to explain anything. Most people will just accept their conclusions and thus why would have to prove that it's valid?
What they should have done is announced the experiment, and the result and not tried to interpret it. 5.12 issue is a serious flaw and requires some serious theoretical groundwork in order to be able to claim or estimate the certainty that the results are valid.
| Quote: |
but from what I understood they did some kind of montecarlo to come up with the percentage. Ofcouse they also made it very very clear there could be flaws.
Here is what he said:
| Code: |
| It might be that there is a flaw somewhere, but if there is it will not be discovered in the course of this universe – that would require more computational power than could ever be provided. And of course it is possible, and in fact very, very likely, that there is no flaw. |
This reminds me of the "cold fusion" debacle many years ago - a bold claim without scientific basis which dishonored the scientists involved who did not follow appropriate procedure in their eagerness to announce a result.
| Quote: |
But Alpha-Beta reduces the search to about the square root of the total number of moves. The square root of 10^100, however…
Yes, I know. But think about it: you do not need to search every variation to mate. We only need to search a tiny fraction of the overall space. Whenever Rybka evaluates a position with a score of +/– 5.12 we don't need to search any further, we have our proof that in the continuation there is going to be a win or loss, and there is a forced mate somewhere deep down in the tree. We tested a random sampling of positions of varying levels of difficulty that were evaluated at above 5.12, and we never saw a solution fail. So it is safe to use this assumption generally in the search.
|
| Quote: |
So this means that the result is not 100% certain, it is just a hypothesis.
That is technically correct, similar to the assertion that a position where one side is more than two pieces down, without any compensation, is considered lost, even if you cannot calculate it to a forced mate against any defence. Sure, there theoretically might be a way to save the game, but if Rybka is displaying +5.12 or more the outcome is 99.99999999% secure. That is approximately the confidence number we give to our King's Gambit results: 99.99999999%. It might be that there is a flaw somewhere, but if there is it will not be discovered in the course of this universe – that would require more computational power than could ever be provided. And of course it is possible, and in fact very, very likely, that there is no flaw.
|
IMO it is a very solid assumption. Better to make baby steps rather than complain "oh chess has gazillion positions so can never be solved...". I wonder if this methodology has been tried before ...
Anyhow a bright way to start the day |
_________________
"Your superior intellect is no match for our puny weapons." -Kang and Kodos |
|
| Back to top |
|
|
Carl Bicknell
Joined: 18 Sep 2008
Posts: 280
|
| Posted: Mon Apr 02, 2012 3:16 pm Post subject: Re: Luke skywalker has done it again. |
|
|
| It's a joke, a late April Fools. |
|
| Back to top |
|
|
Terry McCracken
Joined: 01 Aug 2007
Posts: 10782
Location: Canada
|
| Posted: Mon Apr 02, 2012 3:30 pm Post subject: Re: Luke skywalker has done it again. |
|
|
| Daniel Shawul wrote: |
| Ofcourse vas haters will call BS without bothering to read what is claimed |
I'm not a Vas Hater but it is B.S. Check the date above the PGN.
There are too many reasons to go into and shouldn't need to either to know this article is false.
_________________
Terry McCracken |
|
| Back to top |
|
|
Don Dailey
Joined: 29 Apr 2008
Posts: 4320
|
| Posted: Mon Apr 02, 2012 3:30 pm Post subject: Re: Luke skywalker has done it again. |
|
|
| rbarreira wrote: |
| Another thing that needs to be spelled out is what depth they required Rybka to search with a >= 5.12 score before they assumed the position as solved. |
Exactly! Vas needs to write a paper on this so that it can be subjected to scientific scrutiny before making a claim that he solved this opening.
The 5.12 rule is badly flawed. I think Vas assumes that when it's wrong (whether he is exaggerating about how often it is wrong or not) those incorrect results are randomly distributed over the entire search tree, but that is not correct in this case. It's true that a few randomly distributed incorrect results are not likely to get propagated to the root if there are not very many of them but that is not going to help here. He also does not make an attempt to quantify this.
However, any miscalculations are going to occur based on systematic error and there will be nothing randomly distributed about them. They won't be randomly distributed in the tree but in some critical branch it will be wrong in every or most position for the same reason.
It's also too simplistic to simply define a score and say that it's a win every time if you achieve that score.
In the book One Jump Ahead, Jonathan Schaeffer at some point thought that it might be good enough to say that if you were N checkers ahead you could write the position off as a win - and much to his surprise this was not a valid assumption even for a fairly large number of checkers, and in checkers a single pawn (or checker) ahead is a huge advantage. It's been my experience that no simplistic rule can be reliably used to stop a search without introducing scalability issues - because you will ALWAYS be able to find a position where it is badly wrong! In this study Vas it treating 5.12 as a forward pruning rule to represent a complete search to the end of the game.
_________________
"Your superior intellect is no match for our puny weapons." -Kang and Kodos |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
