Hart wrote:I don't know if the question is taboo or even appropriate but I ask anyways out of sheer inquisitive curiosity. With the sources of Ippolit/RobboLito, so called "clones" of Rybka 3, are we any closer to answering such questions and if we are is it acceptable to discuss such questions in public or are these discussions to remain in dark corners or behind closed doors?
I don't believe there is anything special in Rybka, it's just the program as a whole that is special. I think you will be disappointed if you believe there is some secret that Vas is keeping in the closet somewhere.
I looked at the sources of IPP and the only thing that stood out to me is that it is probably more selective than most or all programs that are really strong, but I did not see any special mechanism for doing this, just the same type of stuff we are doing anyway. Vas is just doing it better.
Another thing that stands out to me is that it is has an excellent evaluation function. It may be the best evaluation function of any of the programs, yet it is still probably weaker in some areas than some other good programs. I have heard that king safety is one of the weak points, although it is improved over the previous versions.
There is no secret in Rybka that you can just transfer over to your program in a few minutes and then expect to compete with Rybka.
- Don
Someone has pointed out (privately) that there is a subtle singular extension algorithm in the thing that is probably worth something no doubt. I've not looked at it myself, yet, so can't comment on how well that feature works. When a personal email asked questions about specific code and gave the code along with an explanation, it was pretty obvious what it was doing, and it is just another point in favor of an entry in my to-do list being moved up a bit. I plan on re-visiting this again.
I'm not claiming that Rybka doesn't have any little goodies in it, I'm just saying that I don't believe anything in particular is a breakthrough. It is essentially just a very well engineered program.
You have to know I agree with you there. I've said hundreds of times in the past that we have not seen anything "revolutionary" since alpha/beta. It has been a steady path of evolutionary progress, with some jumps that might be a bit bigger than others, but nothing _major_...
Here is what I consider the big "breakthroughs" if you want to call it that:
1. Shanons layout out the basic algorithm
2. stand pat quies search of captures only.
3. check extensions
4. hash tables
5. null move pruning
6. reduction of various sorts.
You could easily include other thing such as MVV/LVA, bitboards, square of the pawn, futility pruning and so on but I don't think they are quite as big as the above. Perhaps check extensions is the most controversial, but it was a huge deal when the idea suddenly busted on the scene.
When you use the word "breakthrough" I'm looking for at least 50 ELO or more.
I'm sure someone else would produce a somewhat different list - but this is what stands out to me as the major jumps in computer chess.
This makes sense and it was the first thing Anthony Cozzie said of Rybka 1.0 when the Strelka sources were released. He stated that Rybka was a search-heavy engine with very clear optimizations for speed and efficiency and contained no new tricks or novel ideas. However, many of the weights in Rybka 1.0 looked strange and calculated as if they were generated through some elaborate mathematical process, or automatic tuning. So, if Vas does have some unique tricks for improving Rybka then I suppose they might not necessarily be found in the program itself. I guess one question that could shed some light on this question is if Ipp/Robo perform much worse with the material tables removed if they are not already. What could be inferred if the evaluation was replaced with something generic and the program performs 200 Elo worse?
Hart wrote:I guess one question that could shed some light on this question is if Ipp/Robo perform much worse with the material tables removed if they are not already. What could be inferred if the evaluation was replaced with something generic and the program performs 200 Elo worse?
Nothing at all. The material tables also catch many basic draws. Programs that don't use material tables will find those in another way.
You could set the material values flat for all game phases and look at what happens. I bet it's less than 10 ELO difference.
Hart wrote:This makes sense and it was the first thing Anthony Cozzie said of Rybka 1.0 when the Strelka sources were released. He stated that Rybka was a search-heavy engine with very clear optimizations for speed and efficiency and contained no new tricks or novel ideas. However, many of the weights in Rybka 1.0 looked strange and calculated as if they were generated through some elaborate mathematical process, or automatic tuning. So, if Vas does have some unique tricks for improving Rybka then I suppose they might not necessarily be found in the program itself. I guess one question that could shed some light on this question is if Ipp/Robo perform much worse with the material tables removed if they are not already. What could be inferred if the evaluation was replaced with something generic and the program performs 200 Elo worse?
Your reaching for straws. There is no special magic in the evaluation function, or tables, it's just a very good evaluation function.
Hart wrote:This makes sense and it was the first thing Anthony Cozzie said of Rybka 1.0 when the Strelka sources were released. He stated that Rybka was a search-heavy engine with very clear optimizations for speed and efficiency and contained no new tricks or novel ideas.
Yes, aim to efficency and speed tricks are very visible everywhere.
I am also very interested in sqeezing the last CPU cycle out of a function. But _not_ if this means to compromise readibility or code flow. For instance there are some tricks that, while I understand they can speed up a bit, I won't do in SF because they compromise clean design that IMHO is more important then 1-2% of speed increase especially in the long term and especially for an open source engine like is SF. Here the aim is to be as much clean and clear as possible because the goal is that the engine is used as a reference, a live book, by the other authors.
And I have also learned, with my surprise, that when I avoided to uglify the code with some speed hacks, not much time later I found a way to rewrite the same function in another way that was also faster then before. So the bottom line is that to leave the code well modularized, flexible and easy for modifications and changes _is_ a speed optimization (although indirect) and the kind of speed optimization that I like most.
Hart wrote:I don't know if the question is taboo or even appropriate but I ask anyways out of sheer inquisitive curiosity. With the sources of Ippolit/RobboLito, so called "clones" of Rybka 3, are we any closer to answering such questions and if we are is it acceptable to discuss such questions in public or are these discussions to remain in dark corners or behind closed doors?
I don't believe there is anything special in Rybka, it's just the program as a whole that is special. I think you will be disappointed if you believe there is some secret that Vas is keeping in the closet somewhere.
I looked at the sources of IPP and the only thing that stood out to me is that it is probably more selective than most or all programs that are really strong, but I did not see any special mechanism for doing this, just the same type of stuff we are doing anyway. Vas is just doing it better.
Another thing that stands out to me is that it is has an excellent evaluation function. It may be the best evaluation function of any of the programs, yet it is still probably weaker in some areas than some other good programs. I have heard that king safety is one of the weak points, although it is improved over the previous versions.
There is no secret in Rybka that you can just transfer over to your program in a few minutes and then expect to compete with Rybka.
- Don
Someone has pointed out (privately) that there is a subtle singular extension algorithm in the thing that is probably worth something no doubt. I've not looked at it myself, yet, so can't comment on how well that feature works. When a personal email asked questions about specific code and gave the code along with an explanation, it was pretty obvious what it was doing, and it is just another point in favor of an entry in my to-do list being moved up a bit. I plan on re-visiting this again.
I'm not claiming that Rybka doesn't have any little goodies in it, I'm just saying that I don't believe anything in particular is a breakthrough. It is essentially just a very well engineered program.
You have to know I agree with you there. I've said hundreds of times in the past that we have not seen anything "revolutionary" since alpha/beta. It has been a steady path of evolutionary progress, with some jumps that might be a bit bigger than others, but nothing _major_...
Here is what I consider the big "breakthroughs" if you want to call it that:
1. Shanons layout out the basic algorithm
2. stand pat quies search of captures only.
3. check extensions
4. hash tables
5. null move pruning
6. reduction of various sorts.
I'm not sure that #2 was ever not done. I had a q-search as far back as I can remember. It was in chess 4.x in 1974 or so also. Null-move pruning is certainly more than 50 Elo, but reductions really are not if you already have null-move included. Or reductions are more than 50, but then null-move is not, assuming you add reductions first. They are somewhat complementary since they overlap with the same underlying idea of reducing the depth.
I would likely replace 2 above with alpha/beta which wasn't in shannon's paper at all, it came from Newall, Simon and Shaw a few years later.
Iterative deepening is also a pretty good idea, coming from Slate/Atkin of course.
You could easily include other thing such as MVV/LVA, bitboards, square of the pawn, futility pruning and so on but I don't think they are quite as big as the above. Perhaps check extensions is the most controversial, but it was a huge deal when the idea suddenly busted on the scene.
When you use the word "breakthrough" I'm looking for at least 50 ELO or more.
I'm sure someone else would produce a somewhat different list - but this is what stands out to me as the major jumps in computer chess.
Hart wrote:I don't know if the question is taboo or even appropriate but I ask anyways out of sheer inquisitive curiosity. With the sources of Ippolit/RobboLito, so called "clones" of Rybka 3, are we any closer to answering such questions and if we are is it acceptable to discuss such questions in public or are these discussions to remain in dark corners or behind closed doors?
I don't believe there is anything special in Rybka, it's just the program as a whole that is special. I think you will be disappointed if you believe there is some secret that Vas is keeping in the closet somewhere.
I looked at the sources of IPP and the only thing that stood out to me is that it is probably more selective than most or all programs that are really strong, but I did not see any special mechanism for doing this, just the same type of stuff we are doing anyway. Vas is just doing it better.
Another thing that stands out to me is that it is has an excellent evaluation function. It may be the best evaluation function of any of the programs, yet it is still probably weaker in some areas than some other good programs. I have heard that king safety is one of the weak points, although it is improved over the previous versions.
There is no secret in Rybka that you can just transfer over to your program in a few minutes and then expect to compete with Rybka.
- Don
Someone has pointed out (privately) that there is a subtle singular extension algorithm in the thing that is probably worth something no doubt. I've not looked at it myself, yet, so can't comment on how well that feature works. When a personal email asked questions about specific code and gave the code along with an explanation, it was pretty obvious what it was doing, and it is just another point in favor of an entry in my to-do list being moved up a bit. I plan on re-visiting this again.
I'm not claiming that Rybka doesn't have any little goodies in it, I'm just saying that I don't believe anything in particular is a breakthrough. It is essentially just a very well engineered program.
You have to know I agree with you there. I've said hundreds of times in the past that we have not seen anything "revolutionary" since alpha/beta. It has been a steady path of evolutionary progress, with some jumps that might be a bit bigger than others, but nothing _major_...
Here is what I consider the big "breakthroughs" if you want to call it that:
1. Shanons layout out the basic algorithm
2. stand pat quies search of captures only.
3. check extensions
4. hash tables
5. null move pruning
6. reduction of various sorts.
I'm not sure that #2 was ever not done. I had a q-search as far back as I can remember. It was in chess 4.x in 1974 or so also. Null-move pruning is certainly more than 50 Elo, but reductions really are not if you already have null-move included. Or reductions are more than 50, but then null-move is not, assuming you add reductions first. They are somewhat complementary since they overlap with the same underlying idea of reducing the depth.
I would likely replace 2 above with alpha/beta which wasn't in shannon's paper at all, it came from Newall, Simon and Shaw a few years later.
Iterative deepening is also a pretty good idea, coming from Slate/Atkin of course.
You could easily include other thing such as MVV/LVA, bitboards, square of the pawn, futility pruning and so on but I don't think they are quite as big as the above. Perhaps check extensions is the most controversial, but it was a huge deal when the idea suddenly busted on the scene.
When you use the word "breakthrough" I'm looking for at least 50 ELO or more.
I'm sure someone else would produce a somewhat different list - but this is what stands out to me as the major jumps in computer chess.
- Don
[/list]
When I turn on/off LMR (even after having null move pruning on) I get at least 100 ELO. LMR is powerful for my program.
I think the example of Fruit has shown that it's typically not one thing but a combination of eval, implementation and search. As others have noted Rybka has benefited from careful and professional tuning of the whole thing, much as Bob is doing with Crafty recently.
That said, a cursory look at the forbidden sources seems to indicate that like many other programs it is doing aggressive forward pruning starting at fairly high depths in the tree (7 plies from the q-search, it appears). That's a good strategy in general but tricky to execute.
Hart wrote:I don't know if the question is taboo or even appropriate but I ask anyways out of sheer inquisitive curiosity. With the sources of Ippolit/RobboLito, so called "clones" of Rybka 3, are we any closer to answering such questions and if we are is it acceptable to discuss such questions in public or are these discussions to remain in dark corners or behind closed doors?
I don't believe there is anything special in Rybka, it's just the program as a whole that is special. I think you will be disappointed if you believe there is some secret that Vas is keeping in the closet somewhere.
I looked at the sources of IPP and the only thing that stood out to me is that it is probably more selective than most or all programs that are really strong, but I did not see any special mechanism for doing this, just the same type of stuff we are doing anyway. Vas is just doing it better.
Another thing that stands out to me is that it is has an excellent evaluation function. It may be the best evaluation function of any of the programs, yet it is still probably weaker in some areas than some other good programs. I have heard that king safety is one of the weak points, although it is improved over the previous versions.
There is no secret in Rybka that you can just transfer over to your program in a few minutes and then expect to compete with Rybka.
- Don
Someone has pointed out (privately) that there is a subtle singular extension algorithm in the thing that is probably worth something no doubt. I've not looked at it myself, yet, so can't comment on how well that feature works. When a personal email asked questions about specific code and gave the code along with an explanation, it was pretty obvious what it was doing, and it is just another point in favor of an entry in my to-do list being moved up a bit. I plan on re-visiting this again.
I'm not claiming that Rybka doesn't have any little goodies in it, I'm just saying that I don't believe anything in particular is a breakthrough. It is essentially just a very well engineered program.
You have to know I agree with you there. I've said hundreds of times in the past that we have not seen anything "revolutionary" since alpha/beta. It has been a steady path of evolutionary progress, with some jumps that might be a bit bigger than others, but nothing _major_...
Here is what I consider the big "breakthroughs" if you want to call it that:
1. Shanons layout out the basic algorithm
2. stand pat quies search of captures only.
3. check extensions
4. hash tables
5. null move pruning
6. reduction of various sorts.
I'm not sure that #2 was ever not done. I had a q-search as far back as I can remember. It was in chess 4.x in 1974 or so also. Null-move pruning is certainly more than 50 Elo, but reductions really are not if you already have null-move included. Or reductions are more than 50, but then null-move is not, assuming you add reductions first. They are somewhat complementary since they overlap with the same underlying idea of reducing the depth.
I would likely replace 2 above with alpha/beta which wasn't in shannon's paper at all, it came from Newall, Simon and Shaw a few years later.
Iterative deepening is also a pretty good idea, coming from Slate/Atkin of course.
You could easily include other thing such as MVV/LVA, bitboards, square of the pawn, futility pruning and so on but I don't think they are quite as big as the above. Perhaps check extensions is the most controversial, but it was a huge deal when the idea suddenly busted on the scene.
When you use the word "breakthrough" I'm looking for at least 50 ELO or more.
I'm sure someone else would produce a somewhat different list - but this is what stands out to me as the major jumps in computer chess.
- Don
[/list]
When I turn on/off LMR (even after having null move pruning on) I get at least 100 ELO. LMR is powerful for my program.
- Don
It's pretty powerful in mine as well. But I ran exhaustive tests last year and measured this carefully, at least on Crafty. Removing either drops about 40 Elo. Removing both drops about 120. Adding either back adds about 80 back. Testing results was very consistent. I've not seen anyone claim over +100 for LMR, assuming they use null-move R=3 everywhere. I only tested one non-crafty program and got numbers pretty close. I believe that when I turned it off on Fruit 2.x, it dropped closer to 50. But Fruit has a worse version of LMR than what I am using so that might make a difference in this measurement as well.
Whose "job" is it to analyze a mangled source program to see what it is doing, and then comment it so anybody can discover this without having to do any work of their own???
It's all of our job.
To make what is called human progress...
One person or group discovers a "secret" that enlarges their
personal largesse, security, wealth, happiness, etc.
In my personal opinion, it is our obligation and due diligence
as a species, for future generations, to uncover and expose
secretive/furtive progress and build upon the shoulders of others,
whether they like it or not.
